Hardness is an important indicator of the wear resistance of tool materials. The study of the effect of hardness on the wear resistance of tool materials has significant practical applications. In this experimental work, a series of polycrystalline cubic boron nitride (PcBN) and diamond (PCD) compacts with different hardness are sintered under high temperature and pressure, and these samples were fashioned into turning tools, used for edge wear testing in the granite turning process. Based on the measurements above, the relationship between hardness and wear resistance in two materials with identical grain sizes (1–2 µm) was examined, and the results reveal a significant positive correlation between the hardness and wear resistance of the two materials. Remarkably, the relationship between hardness and wear resistance in two materials complies with the same type of fitting function. But incredibly, the wear resistance of the two materials exhibits inconsistency as the hardness increases. Specifically, below 40 GPa Vickers hardness, both materials exhibit comparable wear resistance. However, above 40 GPa, the difference in wear resistance between the two materials becomes more pronounced with increasing hardness. At the Vickers hardness of 70 GPa, the abrasion ratio of the PCD is approximately 3.5 times that of the PcBN, and it is anticipated that the disparity between them will continue to expand within the broader range of hardness. Further analysis reveals that the intrinsic hardness of the crystal grains significantly impacts the wear resistance of polycrystalline materials.