He was born in Paris, the son of engraver and publisher Théodore Vibert, and grandson of the influential rose-breeder Jean-Pierre Vibert. He began his artistic training at a young age under the instruction of his maternal grandfather, engraver Jean-Pierre-Marie Jazet. Vibert was more interested in painting than engraving and entered the studio of Félix-Joseph Barrias and eventually the École des Beaux-Arts when he was sixteen. He remained at the École for six years under the instruction of historic painter François-Edouard Picot.
Vibert debuted at the Salon of 1863 with La Sieste (The Siesta) and Repentir (Repentance).
During the Franco-Prussian War, Vibert became a sharpshooter and was wounded at the battle of Malmaison in October 1870. He was awarded the Légion d’Honneur and became a Chevalier de la Légion d’Honneur in recognition of his sacrifice. He became an Officier of the Légion d’Honneur in 1882.