Laser cladding (LC) wear-resistant alloy on the surface of pearlite rails has been proven to be able to improve their wear resistance significantly. However, the rapid cooling rate in LC will result in the formation of martensite structures with high hardness but low toughness in heat affected zone (HAZ), making the rails' service safety endangered. In this paper, a novel method was proposed to eliminate the un-tempered martensite in U71Mn rail based on the in-situ gradient tempering (IGT) effect induced by the intrinsic thermal cycles in multilayer LC. The influence mechanisms of interlayer and inter-track thermal cycles on the evolutions of microstructure and mechanical properties of HAZs were studied systematically. The results illustrate that the martensite in HAZs can be gradually tempered within the optimal IGT temperature range, and the tempering processing mechanisms are as follows: firstly, the ε-carbides precipitate from HAZs and then dissolve to form granular cementites. Then, the cementites form colonies of lathlike particles with similar orientations. Finally, a composite structure consisting of fine-tempered sorbite and granular cementites forms, in which the lamellar spacing of sorbite is <1/5 of that of the original pearlite, and the proportion of the high angle grain boundaries has increased by 2.2 times. During the tempering process, the microhardness of HAZs gradually decreases, accompanied by the increase of the bending properties therein. The ultimate bending strength and fracture strain of the specimens with fully tempered HAZs are enhanced to be about 1.6 and 5.5 times as much as those of the specimens with fully quenched HAZs, even much better than the substrate.