The presence of a well-trained, mobile CNN model with a high accuracy rate is imperative to build a mobile-based early breast cancer detector. In this study, we propose a mobile neural network model breast cancer mobile network (BreaCNet) and its implementation framework. BreaCNet consists of an effective segmentation algorithm for breast thermograms and a classifier based on the mobile CNN model. The segmentation algorithm employing edge detection and second-order polynomial curve fitting techniques can effectively capture the thermograms' region of interest (ROI), thereby facilitating efficient feature extraction. The classifier was developed based on ShuffleNet by adding one block consisting of a convolutional layer with 1028 filters. The modified Shufflenet demonstrated a good fit learning with 6.1 million parameters and 22 MB size. Simulation results showed that modified ShuffleNet alone resulted in a 72% accuracy rate, but the performance excelled to a 100% accuracy rate when integrated with the proposed segmentation algorithm. In terms of diagnostic accuracy of the normal and abnormal test, BreaCNet significantly improves the sensitivity rate from 43% to 100% and specificity of 100%. We confirmed that feeding only the ROI of the input dataset to the network can improve the classifier's performance. On the implementation aspect of BreaCNet, the on-device inference is recommended to ensure users' data privacy and handle an unreliable network connection.