Wheel idling could cause the generation of white etching layer (WEL) on rails, posing a threat to the safety of train operation. As a tribological phenomenon, the microstructure of rail materials might have different effects on the WEL formation behavior, which have great importances to been explored. Therefore, the influence of rails with different microstructures (two pearlitic rails and one bainite rail) on the formation mechanism of WEL is studied by using a twin-disk machine tool. The results showed that with the increase in the pearlitic material hardness, the thickness of WELs decreased first and then increased. For bainitic rail, the parent austenite grain boundaries and bainite microstructure growth were observed during the formation of WELs. The existence of grain boundaries would hinder the development of plastic deformation. During scratching, the pearlite grains had better resistance to dynamic recrystallization and deformation than bainite, so WELs thickness of pearlitic rails were lower than bainitic rail. The WEL development was a process based on the "thermo-mechanical coupling" mechanism. With the increase in the WEL thickness, the plastic deformation caused by mechanical action would be weakened until it disappeared. After that, the energy for WELs development downward was from the surface friction heat.