Design/methodology/approach Before the definition of the concentration to be used in a real engine environment, tests of sample dispersion were conducted using an ultrasound bath. The mixtures were added to storage bottles in concentrations of 0.1% and 0.5%, as observed in Figure 1. The samples were subsequently analyzed using the dynamic light scattering (DLS) technique. There was a reduction in the hydrodynamic size for the sample with 0.5% of hexagonal boron nitride (hBN), possibly due to sedimentation of the powder during the analysis, which supported this work to continue with the use of 0.1% concentration. Findings The behavior of hBN as nano additive in a real diesel engine was problematic when compared with laboratory environment, leading to impact in oil temperature. In addition, it was noticed a high amount of deposit formation at the top dead center (TDC) and mid-stroke (MC) and nonsignificative wear at MC and bottom dead center (BDC) of the cylinder, with unusual formation of products from antiwear additives known as zinc dialkyl-dithiophosphate at MC position with the use of hBN. For this reason, this work provides insights into how hBN nanoparticles may not contribute toward the improvement of tribological performance. Originality/value The findings of this work aimed to provide a better understanding of the impact of hBN nanoparticles used as additives in real engine environment in terms of performance and tribological impacts. The results of this work indicated that hBN as additive gave poor results in terms of performance and wear prevention. Peer review The peer review history for this article is available at: https://publons.com/publon/10.1108/ILT-02-2024-0047/