Born in Auch, Gascony, he was initially expected to train as a priest but instead travelled to Spain and became a ship’s captain, sailing to Cuba. He was taken prisoner by the British navy towards the end of the 1740s and eventually settled in London in about 1758, where it is believed he trained as a painter in Northamptonshire and later in London under Charles Brooking. If Serres did not settle in London until 1758, however, he could not have studied for long under Charles Brooking, since Brooking was buried on 25 March 1759.
Reflecting his early career, many of his paintings have naval themes. Working for a publisher documenting the events in the Seven Years War (1756–1763), he painted a series of depictions including the capture of Havana (1762). He also painted events in the American War of Independence (1776–1783). In 1780, he was appointed Marine Painter to King George III.