Selecting an appropriate empirical interatomic potential is essential for accurately describing interatomic interactions and simulating the friction and wear of graphene. Four empirical potentials—Tersoff, REBO, AIREBO, and LCBOP—were employed in molecular dynamics simulations to study the wear process of graphene at the atomic scale. The frictional process of graphene was found to be divisible into three distinct phases: elastic deformation, plastic deformation, and wear. Using a progressively increasing load method, the critical load for each phase of graphene under four different empirical potentials was identified. Furthermore, the formation of Stone–Wales (SW) defects, bond distribution, bond breaking and healing, and wrinkle formation were analyzed in detail. Finally, a comparison was made with previous experimental results regarding friction coefficient and wear morphology.