Deep Residual Networks (ResNets) are prone to overfitting in problems with uncertainty, such as intrusion detection problems. To alleviate this problem, we proposed a method that combines the Adaptive Neuro-fuzzy Inference System (ANFIS) and the ResNet algorithm. This method can make use of the advantages of both the ANFIS and ResNet, and alleviate the overfitting problem of ResNet. Compared with the original ResNet algorithm, the proposed method provides overlapped intervals of continuous attributes and fuzzy rules to ResNet, improving the fuzziness of ResNet. To evaluate the performance of the proposed method, the proposed method is realized and evaluated on the benchmark NSL-KDD dataset. Also, the performance of the proposed method is compared with the original ResNet algorithm and other deep learning-based and ANFIS-based methods. The experimental results demonstrate that the proposed method is better than that of the original ResNet and other existing methods on various metrics, reaching a 98.88% detection rate and 1.11% false alarm rate on the KDDTrain+ dataset.
There is a need to develop an accurate and reliable model for predicting suspended sediment load (SSL) because of its complexity and difficulty in practice. This is due to the fact that sediment transportation is extremely nonlinear and is directed by numerous parameters such as rainfall, sediment supply, and strength of flow. Thus, this study examined two scenarios to investigate the effectiveness of the artificial neural network (ANN) models and determine the sensitivity of the predictive accuracy of the model to specific input parameters. The first scenario proposed three advanced optimisers-whale algorithm (WA), particle swarm optimization (PSO), and bat algorithm (BA)-for the optimisation of the performance of artificial neural network (ANN) in accurately predicting the suspended sediment load rate at the Goorganrood basin, Iran. In total, 5 different input combinations were examined in various lag days of up to 5 days to make a 1-day-ahead SSL prediction. Scenario 2 introduced a multi-objective (MO) optimisation algorithm that utilises the same inputs from scenario 1 as a way of determining the best combination of inputs. Results from scenario 1 revealed that high accuracy levels were achieved upon utilisation of a hybrid ANN-WA model over the ANN-BA with an RMSE value ranging from 1 to 6%. Furthermore, the ANN-WA model performed better than the ANN-PSO with an accuracy improvement value of 5-20%. Scenario 2 achieved the highest R2 when ANN-MOWA was introduced which shows that hybridisation of the multi-objective algorithm with WA and ANN model significantly improves the accuracy of ANN in predicting the daily suspended sediment load.
Evaporation is a crucial component to be established in agriculture management and water engineering. Evaporation prediction is thus an essential issue for modeling researchers. In this study, the multilayer perceptron (MLP) was used for predicting daily evaporation. MLP model is as one of the famous ANN models with multilayers for predicting different target variables. A new strategy was used to enhance the accuracy of the MLP model. Three multi-objective algorithms, namely, the multi-objective salp swarm algorithm (MOSSA), the multi-objective crow algorithm (MOCA), and the multi-objective particle swarm optimization (MOPSO), were respectively and separately coupled to the MLP model for determining the model parameters, the best input combination, and the best activation function. In this study, three stations in Malaysia, namely, the Muadzam Shah (MS), the Kuala Terengganu (KT), and the Kuantan (KU), were selected for the prediction of the respective daily evaporation. The spacing (SP) and maximum spread (MS) indices were used to evaluate the quality of generated Pareto front (PF) by the algorithms. The lower SP and higher MS showed better PF for the models. It was observed that the MOSSA had higher MS and lower SP than the other algorithms, at all stations. The root means square error (RMSE), mean absolute error (MAE), percent bias (PBIAS), and Nash Sutcliffe efficiency (NSE) quantifiers were used to compare the ability of the models with each other. The MLP-MOSSA had reduced RMSE compared to the MLP-MOCA, MLP-MOPSO, and MLP models by 18%, 25%, and 35%, respectively, at the MS station. The MAE of the MLP-MOSSA was 2.7%, 4.1%, and 26%, respectively lower than those of the MLP-MOCA, MLP-MOPSO, and MLP models at the KU station. The MLP-MOSSA showed lower MAE than the MLP-MOCA, MLP-MOPSO, and MLP models by 16%, 18%, and 19%, respectively, at the KT station. An uncertainty analysis was performed based on the input and parameter uncertainty. The results indicated that the MLP-MOSSA had the lowest uncertainty among the models. Also, the input uncertainty was lower than the parameter uncertainty. The general results indicated that the MLP-MOSSA had the high efficiency for predicting evaporation.
This study proposes a novel hybrid computational approach that integrates the artificial dragonfly algorithm (ADA) with the Hopfield neural network (HNN) to achieve an optimal representation of the Exact Boolean kSatisfiability (EBkSAT) logical rule. The primary objective is to investigate the effectiveness and robustness of the ADA algorithm in expediting the training phase of the HNN to attain an optimized EBkSAT logic representation. To assess the performance of the proposed hybrid computational model, a specific Exact Boolean kSatisfiability problem is constructed, and simulated data sets are generated. The evaluation metrics employed include the global minimum ratio (GmR), root mean square error (RMSE), mean absolute percentage error (MAPE), and network computational time (CT) for EBkSAT representation. Comparative analyses are conducted between the results obtained from the proposed model and existing models in the literature. The findings demonstrate that the proposed hybrid model, ADA-HNN-EBkSAT, surpasses existing models in terms of accuracy and computational time. This suggests that the ADA algorithm exhibits effective compatibility with the HNN for achieving an optimal representation of the EBkSAT logical rule. These outcomes carry significant implications for addressing intricate optimization problems across diverse domains, including computer science, engineering, and business.
At present, there are widespread financing difficulties in China's trade circulation industry. Supply chain finance can provide financing for small- and medium-sized enterprises in China's trade circulation industry, but it will produce financing risks such as credit risks. It is necessary to analyze the causes of the risks in the supply chain finance of the trade circulation industry and measure these risks by establishing a credit risk assessment system. In this article, a supply chain financial risk early warning index system is established, including 4 first-level indicators and 29 third-level indicators. Then, on the basis of the supply chain financial risk early warning index system, combined with the method of convolution neural network, the supply chain financial risk early warning model of trade circulation industry is constructed, and the evaluation index is measured by the method of principal component analysis. Finally, the relevant data of trade circulation enterprises are selected to make an empirical analysis of the model. The conclusion shows that the supply chain financial risk early warning model and risk control measures established in this article have certain reference value for the commercial circulation industry to carry out supply chain finance. It also provides guidance for trade circulation enterprises to deal with supply chain financial risks effectively.
The primary responsibility for continuously discharging toxic organic pollutants into water bodies and open environments is the increase in industrial and agricultural activities. Developing economical and suitable methods to continuously remove organic pollutants from wastewater is highly essential. The aim of the present research was to apply response surface methodology (RSM) and artificial neural networks (ANNs) for optimization and modeling of photocatalytic degradation of acid orange 7 (AO7) by commercial TiO2-P25 nanoparticles (TNPs). Dose of TNPs, pH, and AO7 concentration were selected as investigated parameters. RSM results reveal the reflective rate of AO7 removal of ~ 94.974% was obtained at pH 7.599, TNP dose of 0.748 g/L, and AO7 concentration of 28.483 mg/L. The resulting quadratic model is satisfactory with the highest coefficient of determination (R2) between the predicted and experimental data (R2 = 0.98 and adjusted R2 = 0.954). On the other hand, ANNs were successfully employed for modeling of AO7 degradation process. The proposed ANN model was absolutely fitted with experimental results producing the highest R2. Furthermore, root mean square error (RMSE), mean average deviation (MAD), absolute average relative error (AARE), and mean square error (MSE) were examined more to compare the predictive capabilities of ANN and RSM models. The experimental data was well fitted into pseudo-first-order and pseudo-second-order kinetics with more accuracy. Thermodynamic parameters, namely enthalpy, entropy, Gibbs' free energy, and activation energy, were also evaluated to suggest the nature of the degradation process. The increase of temperature was analyzed to be more suitable for the fast removal of AO7 over TNPs. Graphical abstract.
In recent years, there has been an immense amount of research into fall event detection. Generally, a fall event is defined as a situation in which a person unintentionally drops down onto a lower surface. It is crucial to detect the occurrence of fall events as early as possible so that any severe fall consequences can be minimized. Nonetheless, a fall event is a sporadic incidence that occurs seldomly that is falsely detected due to a wide range of fall conditions and situations. Therefore, an automated fall frame detection system, which is referred to as the SmartConvFall is proposed to detect the exact fall frame in a video sequence. It is crucial to know the exact fall frame as it dictates the response time of the system to administer an early treatment to reduce the fall's negative consequences and related injuries. Henceforth, searching for the optimal training configurations is imperative to ensure the main goal of the SmartConvFall is achieved. The proposed SmartConvFall consists of two parts, which are object tracking and instantaneous fall frame detection modules that rely on deep learning representations. The first stage will track the object of interest using a fully convolutional neural network (CNN) tracker. Various training configurations such as optimizer, learning rate, mini-batch size, number of training samples, and region of interest are individually evaluated to determine the best configuration to produce the best tracker model. Meanwhile, the second module goal is to determine the exact instantaneous fall frame by modeling the continuous object trajectories using the Long Short-Term Memory (LSTM) network. Similarly, the LSTM model will undergo various training configurations that cover different types of features selection and the number of stacked layers. The exact instantaneous fall frame is determined using an assumption that a large movement difference with respect to the ground level along the vertical axis can be observed if a fall incident happened. The proposed SmartConvFall is a novel technique as most of the existing methods still relying on detection rather than the tracking module. The SmartConvFall outperforms the state-of-the-art trackers, namely TCNN and MDNET-N trackers, with the highest expected average overlap, robustness, and reliability metrics of 0.1619, 0.6323, and 0.7958, respectively. The SmartConvFall also managed to produce the lowest number of tracking failures with only 43 occasions. Moreover, a three-stack LSTM delivers the lowest mean error with approximately one second delay time in locating the exact instantaneous fall frame. Therefore, the proposed SmartConvFall has demonstrated its potential and suitability to be implemented for a real-time application that could help to avoid any crucial fall consequences such as death and internal bleeding if the early treatment can be administered.
The motivation behind this research is to innovatively combine new methods like wavelet, principal component analysis (PCA), and artificial neural network (ANN) approaches to analyze trade in today's increasingly difficult and volatile financial futures markets. The main focus of this study is to facilitate forecasting by using an enhanced denoising process on market data, taken as a multivariate signal, in order to deduct the same noise from the open-high-low-close signal of a market. This research offers evidence on the predictive ability and the profitability of abnormal returns of a new hybrid forecasting model using Wavelet-PCA denoising and ANN (named WPCA-NN) on futures contracts of Hong Kong's Hang Seng futures, Japan's NIKKEI 225 futures, Singapore's MSCI futures, South Korea's KOSPI 200 futures, and Taiwan's TAIEX futures from 2005 to 2014. Using a host of technical analysis indicators consisting of RSI, MACD, MACD Signal, Stochastic Fast %K, Stochastic Slow %K, Stochastic %D, and Ultimate Oscillator, empirical results show that the annual mean returns of WPCA-NN are more than the threshold buy-and-hold for the validation, test, and evaluation periods; this is inconsistent with the traditional random walk hypothesis, which insists that mechanical rules cannot outperform the threshold buy-and-hold. The findings, however, are consistent with literature that advocates technical analysis.
Urban vegetation mapping is critical in many applications, i.e., preserving biodiversity, maintaining ecological balance, and minimizing the urban heat island effect. It is still challenging to extract accurate vegetation covers from aerial imagery using traditional classification approaches, because urban vegetation categories have complex spatial structures and similar spectral properties. Deep neural networks (DNNs) have shown a significant improvement in remote sensing image classification outcomes during the last few years. These methods are promising in this domain, yet unreliable for various reasons, such as the use of irrelevant descriptor features in the building of the models and lack of quality in the labeled image. Explainable AI (XAI) can help us gain insight into these limits and, as a result, adjust the training dataset and model as needed. Thus, in this work, we explain how an explanation model called Shapley additive explanations (SHAP) can be utilized for interpreting the output of the DNN model that is designed for classifying vegetation covers. We want to not only produce high-quality vegetation maps, but also rank the input parameters and select appropriate features for classification. Therefore, we test our method on vegetation mapping from aerial imagery based on spectral and textural features. Texture features can help overcome the limitations of poor spectral resolution in aerial imagery for vegetation mapping. The model was capable of obtaining an overall accuracy (OA) of 94.44% for vegetation cover mapping. The conclusions derived from SHAP plots demonstrate the high contribution of features, such as Hue, Brightness, GLCM_Dissimilarity, GLCM_Homogeneity, and GLCM_Mean to the output of the proposed model for vegetation mapping. Therefore, the study indicates that existing vegetation mapping strategies based only on spectral characteristics are insufficient to appropriately classify vegetation covers.
Hyperparameter tuning is an important process to maximize the performance of any neural network model. This present study proposed the factorial design of experiment for screening and response surface methodology to optimize the hyperparameter of two artificial neural network algorithms. Feed-forward neural network (FFNN) and radial basis function neural network (RBFNN) are applied to predict the permeate flux of palm oil mill effluent. Permeate pump and transmembrane pressure of the submerge membrane bioreactor system are the input variables. Six hyperparameters of the FFNN model including four numerical factors (neuron numbers, learning rate, momentum, and epoch numbers) and two categorical factors (training and activation function) are used in hyperparameter optimization. RBFNN includes two numerical factors such as a number of neurons and spreads. The conventional method (one-variable-at-a-time) is compared in terms of optimization processing time and the accuracy of the model. The result indicates that the optimal hyperparameters obtained by the proposed approach produce good accuracy with a smaller generalization error. The simulation results show an improvement of more than 65% of training performance, with less repetition and processing time. This proposed methodology can be utilized for any type of neural network application to find the optimum levels of different parameters.
A new numerical method was proposed in this paper to address the nonlinear quadratic optimal control problems, with state and control inequality constraints. This method used the quasilinearization technique and Haar wavelet operational matrix to convert the nonlinear optimal control problem into a sequence of quadratic programming problems. The inequality constraints for trajectory variables were transformed into quadratic programming constraints using the Haar wavelet collocation method. The proposed method was applied to optimize the control of the multi-item inventory model with linear demand rates. By enhancing the resolution of the Haar wavelet, we can improve the accuracy of the states, controls and cost. Simulation results were also compared with other researchers' work.
Classification is one of the most hourly encountered problems in real world. Neural networks have
emerged as one of the tools that can handle the classification problem. Feed-Forward Neural Networks
(FFNN's) have been widely applied in many different fields as a classification tool. Designing an efficient
FFNN structure with the optimum number of hidden layers and minimum number of layer's neurons for
a given specific application or dataset, is an open research problem and more challenging depend on
the input data. The random selections of hidden layers and neurons may cause the problem of either
under fitting or over fitting. Over fitting arises because the network matches the data so closely as to
lose its generalization ability over the test data. In this research, the classification performance using
the Mean Square Error (MSE) of Feed-Forward Neural Network (FFNN) with back-propagation algorithm
with respect to the different number of hidden layers and hidden neurons is computed and analyzed to
find out the optimum number of hidden layers and minimum number of layer's neurons to help the
existing classification concepts by MATLAB version 13a. By this process, firstly the random data has
been generated using an suitable matlab function to prepare the training data as the input and target
vectors as the testing data for the classification purposes of FFNN. The generated input data is passed
on to the output layer through the hidden layers which process these data. From this analysis, it is find
out from the mean square error comparison graphs and regression plots that for getting the best
performance form this network, it is better to use the high number of hidden layers and more neurons in
the hidden layers in the network during designing its classifier but so more neurons in the hidden layers
and the high number of hidden layers in the network makes it complex and takes more time to execute.
So as the result it is suggested that three hidden layers and 26 hidden neurons in each hidden layers
are better for designing the classifier of this network for this type of input data features.
In this paper we present a new line search method known as the HBFGS method, which uses the search direction of the conjugate gradient method with the quasi-Newton updates. The Broyden-Fletcher-Goldfarb-Shanno (BFGS) update is used as approximation of the Hessian for the methods. The new algorithm is compared with the BFGS method in terms of iteration counts and CPU-time. Our numerical analysis provides strong evidence that the proposed HBFGS method is more efficient than the ordinary BFGS method. Besides, we also prove that the new algorithm is globally convergent.
Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.
Insect taxonomy lies at the heart of many aspects of ecology, and identification tasks are challenging due to the enormous inter- and intraspecies variation of insects. Conventional methods used to study insect taxonomy are often tedious, time-consuming, labor intensive, and expensive, and recently, computer vision with deep learning algorithms has offered an alternative way to identify and classify insect images into their taxonomic levels. We designed the classification task according to the taxonomic ranks of insects-order, family, and genus-and compared the generalization of four state-of-the-art deep convolutional neural network (DCNN) architectures. The results show that different taxonomic ranks require different deep learning (DL) algorithms to generate high-performance models, which indicates that the design of an automated systematic classification pipeline requires the integration of different algorithms. The InceptionV3 model has advantages over other models due to its high performance in distinguishing insect order and family, which is having F1-score of 0.75 and 0.79, respectively. Referring to the performance per class, Hemiptera (order), Rhiniidae (family), and Lucilia (genus) had the lowest performance, and we discuss the possible rationale and suggest future works to improve the generalization of a DL model for taxonomic rank classification.
The SARS-CoV-2 virus is responsible for the rapid global spread of the COVID-19 disease. As a result, it is critical to understand and collect primary data on the virus, infection epidemiology, and treatment. Despite the speed with which the virus was detected, studies of its cell biology and architecture at the ultrastructural level are still in their infancy. Therefore, we investigated and analyzed the viral morphometry of SARS-CoV-2 to extract important key points of the virus's characteristics. Then, we proposed a prediction model to identify the real virus levels based on the optimization of a full recurrent neural network (RNN) using transmission electron microscopy (TEM) images. Consequently, identification of virus levels depends on the size of the morphometry of the area (width, height, circularity, roundness, aspect ratio, and solidity). The results of our model were an error score of training network performance 3.216 × 10-11 at 639 epoch, regression of -1.6 × 10-9, momentum gain (Mu) 1 × 10-9, and gradient value of 9.6852 × 10-8, which represent a network with a high ability to predict virus levels. The fully automated system enables virologists to take a high-accuracy approach to virus diagnosis, prevention of mutations, and life cycle and improvement of diagnostic reagents and drugs, adding a point of view to the advancement of medical virology.
Recently, ecological damage and environmental pollution have become increasingly serious. Experts in various fields have started to study related issues from diverse points of view. To prevent the accelerated deterioration of the ecological environment, ecolinguistics emerged. Eco-critical discourse analysis is one of the important parts of ecolinguistics research, that is, it is a critical discourse analysis of the use of language from the perspective of the language's ecological environment. Firstly, an ecological tone and modality system are constructed from an ecological perspective. Under the guidance of the ecological philosophy of "equality, harmony, and symbiosis", this study conducts an ecological discourse analysis on the Sino-US trade friction reports, aiming to present the similarities and differences between the two newspapers' trade friction discourses and to reveal the ecological significance of international ecological factors in the discourse. Secondly, this method establishes a vector expression of abstract words based on emotion dictionary resources and introduces emotion polarity and part-of-speech features of words. Then the word vector is formed into the text feature matrix, which is used as the input of the Convolutional Neural Network (CNN) model, and the Back Propagation algorithm is adopted to train the model. Finally, in the light of the trained CNN model, the unlabeled news is predicted, and the experimental results are analyzed. The results reveal that during the training process of Chinese and English datasets, the accuracy of the training set can reach nearly 100%, and the loss rate can be reduced to 0. On the test set, the classification accuracy of Chinese text can reach 83%, while that of English text can reach 90%, and the experimental results are ideal. This study provides an explanatory approach for ecological discourse analysis on the news reports of Sino-US trade frictions and has certain guiding significance for the comparative research on political news reports under different ideologies between China and the United States.
Investigating the effect of changes in neuronal connectivity on the brain's behavior is of interest in neuroscience studies. Complex network theory is one of the most capable tools to study the effects of these changes on collective brain behavior. By using complex networks, the neural structure, function, and dynamics can be analyzed. In this context, various frameworks can be used to mimic neural networks, among which multi-layer networks are a proper one. Compared to single-layer models, multi-layer networks can provide a more realistic model of the brain due to their high complexity and dimensionality. This paper examines the effect of changes in asymmetry coupling on the behaviors of a multi-layer neuronal network. To this aim, a two-layer network is considered as a minimum model of left and right cerebral hemispheres communicated with the corpus callosum. The chaotic model of Hindmarsh-Rose is taken as the dynamics of the nodes. Only two neurons of each layer connect two layers of the network. In this model, it is assumed that the layers have different coupling strengths, so the effect of each coupling change on network behavior can be analyzed. As a result, the projection of the nodes is plotted for several coupling strengths to investigate how the asymmetry coupling influences the network behaviors. It is observed that although no coexisting attractor is present in the Hindmarsh-Rose model, an asymmetry in couplings causes the emergence of different attractors. The bifurcation diagrams of one node of each layer are presented to show the variation of the dynamics due to coupling changes. For further analysis, the network synchronization is investigated by computing intra-layer and inter-layer errors. Calculating these errors shows that the network can be synchronized only for large enough symmetric coupling.
Deep neural networks have made tremendous strides in the categorization of facial photos in the last several years. Due to the complexity of features, the enormous size of the picture/frame, and the severe inhomogeneity of image data, efficient face image classification using deep convolutional neural networks remains a challenge. Therefore, as data volumes continue to grow, the effective categorization of face photos in a mobile context utilizing advanced deep learning techniques is becoming increasingly important. In the recent past, some Deep Learning (DL) approaches for learning to identify face images have been designed; many of them use convolutional neural networks (CNNs). To address the problem of face mask recognition in facial images, we propose to use a Depthwise Separable Convolution Neural Network based on MobileNet (DWS-based MobileNet). The proposed network utilizes depth-wise separable convolution layers instead of 2D convolution layers. With limited datasets, the DWS-based MobileNet performs exceptionally well. DWS-based MobileNet decreases the number of trainable parameters while enhancing learning performance by adopting a lightweight network. Our technique outperformed the existing state of the art when tested on benchmark datasets. When compared to Full Convolution MobileNet and baseline methods, the results of this study reveal that adopting Depthwise Separable Convolution-based MobileNet significantly improves performance (Acc. = 93.14, Pre. = 92, recall = 92, F-score = 92).
Leukemia is a cancer of white blood cells characterized by immature lymphocytes. Due to blood cancer, many people die every year. Hence, the early detection of these blast cells is necessary for avoiding blood cancer. A novel deep convolutional neural network (CNN) 3SNet that has depth-wise convolution blocks to reduce the computation costs has been developed to aid the diagnosis of leukemia cells. The proposed method includes three inputs to the deep CNN model. These inputs are grayscale and their corresponding histogram of gradient (HOG) and local binary pattern (LBP) images. The HOG image finds the local shape, and the LBP image describes the leukaemia cell's texture pattern. The suggested model was trained and tested with images from the AML-Cytomorphology_LMU dataset. The mean average precision (MAP) for the cell with less than 100 images in the dataset was 84%, whereas for cells with more than 100 images in the dataset was 93.83%. In addition, the ROC curve area for these cells is more than 98%. This confirmed proposed model could be an adjunct tool to provide a second opinion to a doctor.