As one of the three weakest parts of a track structure, a rail welded joint is prone to significant wear during service because of its complex microstructure, which directly affects the structural integrity and reliability of the rail. Rail disc rollers with different structural proportions, including the joint bonding line (BL), heat affected zone (HAZ) and softening zone (SZ), were cut from a U75V quenched flash-butt welded rail joint. The double-disc rolling test between U75V quenched rail disc and CL60 wheel disc was carried out under different cycles to research the damage mechanism of U75V rail welded joint. The results indicated that the proeutectoid ferrite existing at the residual austenite grain boundary in the BL is prone to deform with the plastic deformation line of the rail disc, and subsurface cracks are easily generated in response to the increasing number of cycles, finally resulting in the occurrence of “layered peeling”. Owing to the small lamellar spacing of the pearlitic structures in the HAZ, shallower plastic deformation can be observed in the HAZ, and the surface wear is reduced. The fatigue cracks tend to propagate over much longer lengths and larger angles with increasing numbers of cycles. Moreover, severe plastic deformation clearly occurs in the SZ, a particular wear mechanism of densely pored peeling was observed on the contact surface because of the globular pearlite structure.