This study investigates the influence of a protruding cylindrical pin (PCP) tool on the microstructural, mechanical and tribological properties of AA5083 aluminum alloy surface composites reinforced with equiatomic NbMoTaTiNi refractory high-entropy alloy (RHEA) particles. Friction stir processing (FSP) was carried out using two tool designs: a protruded cylindrical pin (PCP) tool and conventional cylindrical pin (CCP) tool designed to intensify material flow and promote uniform particle dispersion. The RHEA particles were incorporated into the AA5083 Al matrix via a pre-placed groove before FSP. Microstructural analysis revealed that the PCP tool produced finer equiaxed grains (mean ~3.41   µm) compared to the CCP tool (~5.38   µm) and a higher fraction of high-angle grain boundaries (70.9% vs. 59.9%), indicating enhanced particle-stimulated nucleation (PSN) and continuous dynamic recrystallization (CDRX). TEM observations confirmed that PCP processing generated a high dislocation density, refined subgrain boundaries, and significant lattice distortion, consistent with intense shear deformation and effective grain stabilization via Zener pinning. These microstructural enhancements resulted in an ~17% increase in Vickers microhardness and a ~30% improvement in ultimate shear strength, with ductility retained due to strong particle–matrix bonding. Tribological testing showed a ~22% reduction in wear rate and a lower friction coefficient (0.35 ± 0.01) for the PCP tool in contrast to the CCP tool (0.41 ± 0.01), supported by narrower, shallower wear tracks and finer, more uniform wear debris. The findings confirm that the PCP tool significantly improves grain refinement, particle distribution and tribolayer stability, thereby enhancing both mechanical integrity and wear resistance of RHEA-reinforced AA5083 surface composites.