Design/methodology/approach Considering EHL, a novel model is proposed to calculate the TVMS of orthogonal face gears using the deformation compatibility condition. First, the tooth surface equations of orthogonal face gears are derived according to the tooth surface generation principle. Then, the oil film thickness on the tooth surface of face gears is obtained by solving the governing equations of EHL. Furthermore, the proposed model is used to calculate the TVMS of face gears along the mesh cycle and is verified. Finally, the effects of module, tooth number of shaper cutter and pressure angle on mesh stiffness are analyzed. Findings The results indicate that when the contact ratio is greater than 1 and less than or equal to 2, the TVMS of face gears exhibits a phenomenon of double-single tooth alternating meshing where sudden changes occur. As the module increases, the overall mesh stiffness of face gears increases, and the magnitude of the sudden change at the moment of single-double tooth alternating meshing gradually increases. As the tooth number of shaper cutter and pressure angle increase, so does the TVMS of face gears. When the effect of oil film is considered, the calculated TVMS of face gears slightly increases overall and the increase in average oil film thickness leads to a rise in the TVMS. This study provides a theoretical basis for understanding the dynamic characteristics of face gear drives. Originality/value This study’s originality and value lie in its comprehensive approach, which includes conducting analysis based on loaded tooth contact, considering the influence of elastohydrodynamic lubrication, proposing a novel analytical–finite–element model, calculating TVMS of face gears, verifying the proposed model and analyzing the effects of typical structural parameters and oil film thickness.