This study aims to investigate tire polishing effect on rubber-texture contact and friction characteristics on different asphalt mixtures using laboratory experiments and physics-based modeling. By utilizing a Dynamic Friction Tester (DFT) to measure friction and a high-resolution laser scanner to capture three-dimensional (3-D) texture, the friction and texture changes at different polishing cycles are evaluated. The chip seal exhibits better skid resistance performance but lower resistance to polishing, while the Hot Mix Asphalt (HMA) and Open Graded Friction Courses (OGFC) show the opposite trend. The negative texture features of HMA and OGFC and the positive texture feature of chip seal explain their different trends with polishing. Persson's contact theory is used to predict friction coefficient from 3-D surface texture and backcalculate the effective contact depth between tire rubber and texture, which shows good correlation with root mean square height of surface texture. The friction contributions from micro- and macro-texture reveal that polishing affects micro-texture components for each surface type but only influences macro-texture component on chip seal.