Carmichael was born at the Ouseburn, in Newcastle upon Tyne, Northumberland, on 9 June 1800, the son of William Carmichael, a ships carpenter. He went to sea at an early age, and spent three years on board a vessel sailing between ports in Spain and Portugal. On his return, he was apprenticed to a shipbuilding firm. After completing his apprenticeship, he devoted all his spare time to art, and eventually gave up the carpentry business, setting himself up as a drawing-master and miniature painter. His first historical painting to attract public notice was the Fight Between the Shannon and Chesapeake, which sold for 13 guineas (£13.65). He then painted The Bombardment of Algiers for Trinity House, Newcastle, for which he received 40 guineas; it is still at Trinity House, along with The Heroic Exploits of Admiral Lord Collingwood in HMS "Excellent" at the Battle of Cape St. Vincent, painted in collaboration with George Balmer. Another important early commission was for a View of Newcastle for which the city corporation paid him 100 guineas.
His name first appears as an exhibitor in 1838, when he contributed an oil painting, Shipping in the Bay of Naples, to the Society of British Artists. He showed both oil paintings and watercolours at the Royal Academy, his contributions including The Conqueror towing the Africa off the Shoals of Trafalgar (1841) and The Arrival of the Royal Squadron (1843).