The construction industry generates a substantial volume of solid waste, often destinated for landfills, causing significant environmental pollution. Waste recycling is decisive in managing waste yet challenging due to labor-intensive sorting processes and the diverse forms of waste. Deep learning (DL) models have made remarkable strides in automating domestic waste recognition and sorting. However, the application of DL models to recognize the waste derived from construction, renovation, and demolition (CRD) activities remains limited due to the context-specific studies conducted in previous research. This paper aims to realistically capture the complexity of waste streams in the CRD context. The study encompasses collecting and annotating CRD waste images in real-world, uncontrolled environments. It then evaluates the performance of state-of-the-art DL models for automatically recognizing CRD waste in-the-wild. Several pre-trained networks are utilized to perform effectual feature extraction and transfer learning during DL model training. The results demonstrated that DL models, whether integrated with larger or lightweight backbone networks can recognize the composition of CRD waste streams in-the-wild which is useful for automated waste sorting. The outcome of the study emphasized the applicability of DL models in recognizing and sorting solid waste across various industrial domains, thereby contributing to resource recovery and encouraging environmental management efforts.