Generally, waste heat is redundantly released into the surrounding by anthropogenic activities without strategized planning. Consequently, urban heat islands and global warming chronically increases over time. Thermophotovoltaic (TPV) systems can be potentially deployed to harvest waste heat and recuperate energy to tackle this global issue with supplementary generation of electrical energy. This paper presents a critical review on two dominant types of semiconductor materials, namely gallium antimonide (GaSb) and indium gallium arsenide (InGaAs), as the potential candidates for TPV cells. The advantages and drawbacks of non-epitaxy and epitaxy growth methods are well-discussed based on different semiconductor materials. In addition, this paper critically examines and summarizes the electrical cell performance of TPV cells made of GaSb, InGaAs and other narrow bandgap semiconductor materials. The cell conversion efficiency improvement in terms of structural design and architectural optimization are also comprehensively analyzed and discussed. Lastly, the practical applications, current issues and challenges of TPV cells are critically reviewed and concluded with recommendations for future research. The highlighted insights of this review will contribute to the increase in effort towards development of future TPV systems with improved cell conversion efficiency.