Mitotic nuclei estimation in breast tumour samples has a prognostic significance in analysing tumour aggressiveness and grading system. The automated assessment of mitotic nuclei is challenging because of their high similarity with non-mitotic nuclei and heteromorphic appearance. In this work, we have proposed a new Deep Convolutional Neural Network (CNN) based Heterogeneous Ensemble technique "DHE-Mit-Classifier" for analysis of mitotic nuclei in breast histopathology images. The proposed technique in the first step detects candidate mitotic patches from the histopathological biopsy regions, whereas, in the second step, these patches are classified into mitotic and non-mitotic nuclei using the proposed DHE-Mit-Classifier. For the development of a heterogeneous ensemble, five different deep CNNs are designed and used as base-classifiers. These deep CNNs have varying architectural designs to capture the structural, textural, and morphological properties of the mitotic nuclei. The developed base-classifiers exploit different ideas, including (i) region homogeneity and feature invariance, (ii) asymmetric split-transform-merge, (iii) dilated convolution based multi-scale transformation, (iv) spatial and channel attention, and (v) residual learning. Multi-layer-perceptron is used as a meta-classifier to develop a robust and accurate classifier for providing the final decision. The performance of the proposed ensemble "DHE-Mit-Classifier" is evaluated against state-of-the-art CNNs. The performance evaluation on the test set suggests the superiority of the proposed ensemble with an F-score (0.77), recall (0.71), precision (0.83), and area under the precision-recall curve (0.80). The good generalisation of the proposed ensemble with a considerably high F-score and precision suggests its potential use in the development of an assistance tool for pathologists.