Schmidt rebound hammer test was employed in this study as a nondestructive test. This test method has been universally utilized due to its non-destructiveness for quick and easy assessment of material strength properties and quality of concrete of an existing structure. Industrial waste materials (air-dried alum sludge, treated alum sludge, limestone dust and quarry dust) were employed as replacement material for fine aggregates in this study. A normal strength concrete was designed to achieve 35 MPa at 28 days, with industrial waste materials replacing fine aggregate at different percentages (0%, 5%, 10% and 15%), and then cured for 7, 28 and 180 days. The compressive strength values and rebound numbers for all the mixes obtained were correlated, and a regression equation was established between compressive strength and Schmidt rebound number. The correlation result showed an excellent relationship between rebound number and compressive strength of concrete produced in this study at all curing ages, with correlation coefficients of R2 = 0.98, R2 = 0.99 and R2 = 0.98. The predicted equation showed a strong relationship with the experimental compressive strength. Therefore, it can be used for the prediction of compressive strength of concrete with industrial waste as a replacement for fine aggregate.