Design/methodology/approach A thermal-fluid-solid multi-field coupling numerical model for RBMS in ESPs is developed using the finite element analysis and influence coefficient method. Based on the contact mechanical characteristics of RBMS, the interactions of multiple physical fields between the sealing rings and lubricating oil are accounted for to assess the liquid lubrication state and sealing performance of RBMS in ESPs. Findings The findings indicate the anti-leakage effects of rubber bellows, the transition of lubrication state of the sealing end face and the evolution law of sealing performance with environmental pressure, axial compression amount and contact widths of rubber bellows. Originality/value This study innovatively proposes a multi-field numerical research method to reveal the impact of the downhole environment and rubber bellows on RBMS in ESPs. These findings contribute to a more comprehensive understanding of the sealing mechanism of RBMS and optimize the sealing design for ESPs in high-pressure environments. Peer review The peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-10-2024-0369/