This study investigates the feasibility of using texture radiomics features extracted from mammography images to distinguish between benign and malignant breast lesions and to classify benign lesions into different categories and determine the best machine learning (ML) model to perform the tasks. Six hundred and twenty-two breast lesions from 200 retrospective patient data were segmented and analysed. Three hundred fifty radiomics features were extracted using the Standardized Environment for Radiomics Analysis (SERA) library, one of the radiomics implementations endorsed by the Image Biomarker Standardisation Initiative (IBSI). The radiomics features and selected patient characteristics were used to train selected machine learning models to classify the breast lesions. A fivefold cross-validation was used to evaluate the performance of the ML models and the top 10 most important features were identified. The random forest (RF) ensemble gave the highest accuracy (89.3%) and positive predictive value (66%) and likelihood ratio of 13.5 in categorising benign and malignant lesions. For the classification of benign lesions, the RF model again gave the highest likelihood ratio of 3.4 compared to the other models. Morphological and textural radiomics features were identified as the top 10 most important features from the random forest models. Patient age was also identified as one of the significant features in the RF model. We concluded that machine learning models trained against texture-based radiomics features and patient features give reasonable performance in differentiating benign versus malignant breast lesions. Our study also demonstrated that the radiomics-based machine learning models were able to emulate the visual assessment of mammography lesions, typically used by radiologists, leading to a better understanding of how the machine learning model arrive at their decision.