Seal wear caused by hard particles intruding into the seal surface is one of the main causes of seal failure. In this manuscript, the tribological properties of hydrogenated nitrile rubber (HNBR) in the intrusion of five different sizes of garnet particles into the friction pair were studied through the pin-disk rubbing pair. The experimental results show that the particles change the contact pattern of the friction pairs, thus changing the amount of rubber wear. Different forms of particle motion contribute differently to the friction coefficient, and the particle sliding on the rubber surface contributes the most. As the particle size decreases, the friction coefficient tends to increase and then decrease, which can be explained by energy dissipation theory.