He was born in Paris, the son of engraver and publisher Théodore Vibert, and grandson of the influential rose-breeder Jean-Pierre Vibert. He began his artistic training at a young age under the instruction of his maternal grandfather, engraver Jean-Pierre-Marie Jazet. Vibert was more interested in painting than engraving and entered the studio of Félix-Joseph Barrias and eventually the École des Beaux-Arts when he was sixteen. He remained at the École for six years under the instruction of historic painter François-Edouard Picot.
Vibert debuted at the Salon of 1863 with La Sieste (The Siesta) and Repentir (Repentance).