A new numerical method was proposed in this paper to address the nonlinear quadratic optimal control problems, with state and control inequality constraints. This method used the quasilinearization technique and Haar wavelet operational matrix to convert the nonlinear optimal control problem into a sequence of quadratic programming problems. The inequality constraints for trajectory variables were transformed into quadratic programming constraints using the Haar wavelet collocation method. The proposed method was applied to optimize the control of the multi-item inventory model with linear demand rates. By enhancing the resolution of the Haar wavelet, we can improve the accuracy of the states, controls and cost. Simulation results were also compared with other researchers' work.
Classification is one of the most hourly encountered problems in real world. Neural networks have
emerged as one of the tools that can handle the classification problem. Feed-Forward Neural Networks
(FFNN's) have been widely applied in many different fields as a classification tool. Designing an efficient
FFNN structure with the optimum number of hidden layers and minimum number of layer's neurons for
a given specific application or dataset, is an open research problem and more challenging depend on
the input data. The random selections of hidden layers and neurons may cause the problem of either
under fitting or over fitting. Over fitting arises because the network matches the data so closely as to
lose its generalization ability over the test data. In this research, the classification performance using
the Mean Square Error (MSE) of Feed-Forward Neural Network (FFNN) with back-propagation algorithm
with respect to the different number of hidden layers and hidden neurons is computed and analyzed to
find out the optimum number of hidden layers and minimum number of layer's neurons to help the
existing classification concepts by MATLAB version 13a. By this process, firstly the random data has
been generated using an suitable matlab function to prepare the training data as the input and target
vectors as the testing data for the classification purposes of FFNN. The generated input data is passed
on to the output layer through the hidden layers which process these data. From this analysis, it is find
out from the mean square error comparison graphs and regression plots that for getting the best
performance form this network, it is better to use the high number of hidden layers and more neurons in
the hidden layers in the network during designing its classifier but so more neurons in the hidden layers
and the high number of hidden layers in the network makes it complex and takes more time to execute.
So as the result it is suggested that three hidden layers and 26 hidden neurons in each hidden layers
are better for designing the classifier of this network for this type of input data features.
In this paper we present a new line search method known as the HBFGS method, which uses the search direction of the conjugate gradient method with the quasi-Newton updates. The Broyden-Fletcher-Goldfarb-Shanno (BFGS) update is used as approximation of the Hessian for the methods. The new algorithm is compared with the BFGS method in terms of iteration counts and CPU-time. Our numerical analysis provides strong evidence that the proposed HBFGS method is more efficient than the ordinary BFGS method. Besides, we also prove that the new algorithm is globally convergent.
Osteoarthritis (OA), especially knee OA, is the most common form of arthritis, causing significant disability in patients worldwide. Manual diagnosis, segmentation, and annotations of knee joints remain as the popular method to diagnose OA in clinical practices, although they are tedious and greatly subject to user variation. Therefore, to overcome the limitations of the commonly used method as above, numerous deep learning approaches, especially the convolutional neural network (CNN), have been developed to improve the clinical workflow efficiency. Medical imaging processes, especially those that produce 3-dimensional (3D) images such as MRI, possess ability to reveal hidden structures in a volumetric view. Acknowledging that changes in a knee joint is a 3D complexity, 3D CNN has been employed to analyse the joint problem for a more accurate diagnosis in the recent years. In this review, we provide a broad overview on the current 2D and 3D CNN approaches in the OA research field. We reviewed 74 studies related to classification and segmentation of knee osteoarthritis from the Web of Science database and discussed the various state-of-the-art deep learning approaches proposed. We highlighted the potential and possibility of 3D CNN in the knee osteoarthritis field. We concluded by discussing the possible challenges faced as well as the potential advancements in adopting 3D CNNs in this field.
Insect taxonomy lies at the heart of many aspects of ecology, and identification tasks are challenging due to the enormous inter- and intraspecies variation of insects. Conventional methods used to study insect taxonomy are often tedious, time-consuming, labor intensive, and expensive, and recently, computer vision with deep learning algorithms has offered an alternative way to identify and classify insect images into their taxonomic levels. We designed the classification task according to the taxonomic ranks of insects-order, family, and genus-and compared the generalization of four state-of-the-art deep convolutional neural network (DCNN) architectures. The results show that different taxonomic ranks require different deep learning (DL) algorithms to generate high-performance models, which indicates that the design of an automated systematic classification pipeline requires the integration of different algorithms. The InceptionV3 model has advantages over other models due to its high performance in distinguishing insect order and family, which is having F1-score of 0.75 and 0.79, respectively. Referring to the performance per class, Hemiptera (order), Rhiniidae (family), and Lucilia (genus) had the lowest performance, and we discuss the possible rationale and suggest future works to improve the generalization of a DL model for taxonomic rank classification.
The SARS-CoV-2 virus is responsible for the rapid global spread of the COVID-19 disease. As a result, it is critical to understand and collect primary data on the virus, infection epidemiology, and treatment. Despite the speed with which the virus was detected, studies of its cell biology and architecture at the ultrastructural level are still in their infancy. Therefore, we investigated and analyzed the viral morphometry of SARS-CoV-2 to extract important key points of the virus's characteristics. Then, we proposed a prediction model to identify the real virus levels based on the optimization of a full recurrent neural network (RNN) using transmission electron microscopy (TEM) images. Consequently, identification of virus levels depends on the size of the morphometry of the area (width, height, circularity, roundness, aspect ratio, and solidity). The results of our model were an error score of training network performance 3.216 × 10-11 at 639 epoch, regression of -1.6 × 10-9, momentum gain (Mu) 1 × 10-9, and gradient value of 9.6852 × 10-8, which represent a network with a high ability to predict virus levels. The fully automated system enables virologists to take a high-accuracy approach to virus diagnosis, prevention of mutations, and life cycle and improvement of diagnostic reagents and drugs, adding a point of view to the advancement of medical virology.
Recently, ecological damage and environmental pollution have become increasingly serious. Experts in various fields have started to study related issues from diverse points of view. To prevent the accelerated deterioration of the ecological environment, ecolinguistics emerged. Eco-critical discourse analysis is one of the important parts of ecolinguistics research, that is, it is a critical discourse analysis of the use of language from the perspective of the language's ecological environment. Firstly, an ecological tone and modality system are constructed from an ecological perspective. Under the guidance of the ecological philosophy of "equality, harmony, and symbiosis", this study conducts an ecological discourse analysis on the Sino-US trade friction reports, aiming to present the similarities and differences between the two newspapers' trade friction discourses and to reveal the ecological significance of international ecological factors in the discourse. Secondly, this method establishes a vector expression of abstract words based on emotion dictionary resources and introduces emotion polarity and part-of-speech features of words. Then the word vector is formed into the text feature matrix, which is used as the input of the Convolutional Neural Network (CNN) model, and the Back Propagation algorithm is adopted to train the model. Finally, in the light of the trained CNN model, the unlabeled news is predicted, and the experimental results are analyzed. The results reveal that during the training process of Chinese and English datasets, the accuracy of the training set can reach nearly 100%, and the loss rate can be reduced to 0. On the test set, the classification accuracy of Chinese text can reach 83%, while that of English text can reach 90%, and the experimental results are ideal. This study provides an explanatory approach for ecological discourse analysis on the news reports of Sino-US trade frictions and has certain guiding significance for the comparative research on political news reports under different ideologies between China and the United States.
Investigating the effect of changes in neuronal connectivity on the brain's behavior is of interest in neuroscience studies. Complex network theory is one of the most capable tools to study the effects of these changes on collective brain behavior. By using complex networks, the neural structure, function, and dynamics can be analyzed. In this context, various frameworks can be used to mimic neural networks, among which multi-layer networks are a proper one. Compared to single-layer models, multi-layer networks can provide a more realistic model of the brain due to their high complexity and dimensionality. This paper examines the effect of changes in asymmetry coupling on the behaviors of a multi-layer neuronal network. To this aim, a two-layer network is considered as a minimum model of left and right cerebral hemispheres communicated with the corpus callosum. The chaotic model of Hindmarsh-Rose is taken as the dynamics of the nodes. Only two neurons of each layer connect two layers of the network. In this model, it is assumed that the layers have different coupling strengths, so the effect of each coupling change on network behavior can be analyzed. As a result, the projection of the nodes is plotted for several coupling strengths to investigate how the asymmetry coupling influences the network behaviors. It is observed that although no coexisting attractor is present in the Hindmarsh-Rose model, an asymmetry in couplings causes the emergence of different attractors. The bifurcation diagrams of one node of each layer are presented to show the variation of the dynamics due to coupling changes. For further analysis, the network synchronization is investigated by computing intra-layer and inter-layer errors. Calculating these errors shows that the network can be synchronized only for large enough symmetric coupling.
Deep neural networks have made tremendous strides in the categorization of facial photos in the last several years. Due to the complexity of features, the enormous size of the picture/frame, and the severe inhomogeneity of image data, efficient face image classification using deep convolutional neural networks remains a challenge. Therefore, as data volumes continue to grow, the effective categorization of face photos in a mobile context utilizing advanced deep learning techniques is becoming increasingly important. In the recent past, some Deep Learning (DL) approaches for learning to identify face images have been designed; many of them use convolutional neural networks (CNNs). To address the problem of face mask recognition in facial images, we propose to use a Depthwise Separable Convolution Neural Network based on MobileNet (DWS-based MobileNet). The proposed network utilizes depth-wise separable convolution layers instead of 2D convolution layers. With limited datasets, the DWS-based MobileNet performs exceptionally well. DWS-based MobileNet decreases the number of trainable parameters while enhancing learning performance by adopting a lightweight network. Our technique outperformed the existing state of the art when tested on benchmark datasets. When compared to Full Convolution MobileNet and baseline methods, the results of this study reveal that adopting Depthwise Separable Convolution-based MobileNet significantly improves performance (Acc. = 93.14, Pre. = 92, recall = 92, F-score = 92).
Leukemia is a cancer of white blood cells characterized by immature lymphocytes. Due to blood cancer, many people die every year. Hence, the early detection of these blast cells is necessary for avoiding blood cancer. A novel deep convolutional neural network (CNN) 3SNet that has depth-wise convolution blocks to reduce the computation costs has been developed to aid the diagnosis of leukemia cells. The proposed method includes three inputs to the deep CNN model. These inputs are grayscale and their corresponding histogram of gradient (HOG) and local binary pattern (LBP) images. The HOG image finds the local shape, and the LBP image describes the leukaemia cell's texture pattern. The suggested model was trained and tested with images from the AML-Cytomorphology_LMU dataset. The mean average precision (MAP) for the cell with less than 100 images in the dataset was 84%, whereas for cells with more than 100 images in the dataset was 93.83%. In addition, the ROC curve area for these cells is more than 98%. This confirmed proposed model could be an adjunct tool to provide a second opinion to a doctor.
Recent years have witnessed an in-depth proliferation of the Internet of Things (IoT) and Industrial Internet of Things (IIoT) systems linked to Industry 4.0 technology. The increasing rate of IoT device usage is associated with rising security risks resulting from malicious network flows during data exchange between the connected devices. Various security threats have shown high adverse effects on the availability, functionality, and usability of the devices among which denial of service (DoS) and distributed denial of service (DDoS), which attempt to exhaust the capacity of the IoT network (gateway), thereby causing failure in the functionality of the system have been more pronounced. Various machine learning and deep learning algorithms have been used to propose intelligent intrusion detection systems (IDS) to mitigate the challenging effects of these network threats. One concern is that although deep learning algorithms have shown good accuracy results on tabular data, not all deep learning algorithms can perform well on tabular datasets, which happen to be the most commonly available format of datasets for machine learning tasks. Again, there is also the challenge of model explainability and feature selection, which affect model performance. In this regard, we propose a model for IDS that uses attentive mechanisms to automatically select salient features from a dataset to train the IDS model and provide explainable results, the TabNet-IDS. We implement the proposed model using the TabNet algorithm based on PyTorch which is a deep-learning framework. The results obtained show that the TabNet architecture can be used on tabular datasets for IoT security to achieve good results comparable to those of neural networks, reaching an accuracy of 97% on CIC-IDS2017, 95% on CSE-CICIDS2018 and 98% on CIC-DDoS2019 datasets.
This research proposes a 'temporal attention' addition for long-short term memory (LSTM) models for dengue prediction. The number of monthly dengue cases was collected for each of five Malaysian states i.e. Selangor, Kelantan, Johor, Pulau Pinang, and Melaka from 2011 to 2016. Climatic, demographic, geographic and temporal attributes were used as covariates. The proposed LSTM models with temporal attention was compared with several benchmark models including a linear support vector machine (LSVM), a radial basis function support vector machine (RBFSVM), a decision tree (DT), a shallow neural network (SANN) and a deep neural network (D-ANN). In addition, experiments were conducted to analyze the impact of look-back settings on each model performance. The results showed that the attention LSTM (A-LSTM) model performed best, with the stacked, attention LSTM (SA-LSTM) one in second place. The LSTM and stacked LSTM (S-LSTM) models performed almost identically but with the accuracy improved by the attention mechanism was added. Indeed, they were both found to be superior to the benchmark models mentioned above. The best results were obtained when all attributes were included in the model. The four models (LSTM, S-LSTM, A-LSTM and SA-LSTM) were able to accurately predict dengue presence 1-6 months ahead. Our findings provide a more accurate dengue prediction model than previously used, with the prospect of also applying this approach in other geographic areas.
Crime remains a crucial concern regarding ensuring a safe and secure environment for the public. Numerous efforts have been made to predict crime, emphasizing the importance of employing deep learning approaches for precise predictions. However, sufficient crime data and resources for training state-of-the-art deep learning-based crime prediction systems pose a challenge. To address this issue, this study adopts the transfer learning paradigm. Moreover, this study fine-tunes state-of-the-art statistical and deep learning methods, including Simple Moving Averages (SMA), Weighted Moving Averages (WMA), Exponential Moving Averages (EMA), Long Short Term Memory (LSTM), Bi-directional Long Short Term Memory (BiLSTMs), and Convolutional Neural Networks and Long Short Term Memory (CNN-LSTM) for crime prediction. Primarily, this study proposed a BiLSTM based transfer learning architecture due to its high accuracy in predicting weekly and monthly crime trends. The transfer learning paradigm leverages the fine-tuned BiLSTM model to transfer crime knowledge from one neighbourhood to another. The proposed method is evaluated on Chicago, New York, and Lahore crime datasets. Experimental results demonstrate the superiority of transfer learning with BiLSTM, achieving low error values and reduced execution time. These prediction results can significantly enhance the efficiency of law enforcement agencies in controlling and preventing crime.
The stability of clusters is a serious issue in mobile ad hoc networks. Low stability of clusters may lead to rapid failure of clusters, high energy consumption for reclustering, and decrease in the overall network stability in mobile ad hoc network. In order to improve the stability of clusters, weight-based clustering algorithms are utilized. However, these algorithms only use limited features of the nodes. Thus, they decrease the weight accuracy in determining node's competency and lead to incorrect selection of cluster heads. A new weight-based algorithm presented in this paper not only determines node's weight using its own features, but also considers the direct effect of feature of adjacent nodes. It determines the weight of virtual links between nodes and the effect of the weights on determining node's final weight. By using this strategy, the highest weight is assigned to the best choices for being the cluster heads and the accuracy of nodes selection increases. The performance of new algorithm is analyzed by using computer simulation. The results show that produced clusters have longer lifetime and higher stability. Mathematical simulation shows that this algorithm has high availability in case of failure.
Forecasting naturally occurring phenomena is a common problem in many domains of science, and this has been addressed and investigated by many scientists. The importance of time series prediction stems from the fact that it has wide range of applications, including control systems, engineering processes, environmental systems and economics. From the knowledge of some aspects of the previous behaviour of the system, the aim of the prediction process is to determine or predict its future behaviour. In this paper, we consider a novel application of a higher order polynomial neural network architecture called Dynamic Ridge Polynomial Neural Network that combines the properties of higher order and recurrent neural networks for the prediction of physical time series. In this study, four types of signals have been used, which are; The Lorenz attractor, mean value of the AE index, sunspot number, and heat wave temperature. The simulation results showed good improvements in terms of the signal to noise ratio in comparison to a number of higher order and feedforward neural networks in comparison to the benchmarked techniques.
Landslide is one of the natural disasters that occur in Malaysia. Topographic factors such as elevation, slope angle, slope aspect, general curvature, plan curvature, and profile curvature are considered as the main causes of landslides. In order to determine the dominant topographic factors in landslide mapping analysis, a study was conducted and presented in this paper. There are three main stages involved in this study. The first stage is the extraction of extra topographic factors. Previous landslide studies had identified mainly six topographic factors. Seven new additional factors have been proposed in this study. They are longitude curvature, tangential curvature, cross section curvature, surface area, diagonal line length, surface roughness, and rugosity. The second stage is the specification of the weight of each factor using two methods. The methods are multilayer perceptron (MLP) network classification accuracy and Zhou's algorithm. At the third stage, the factors with higher weights were used to improve the MLP performance. Out of the thirteen factors, eight factors were considered as important factors, which are surface area, longitude curvature, diagonal length, slope angle, elevation, slope aspect, rugosity, and profile curvature. The classification accuracy of multilayer perceptron neural network has increased by 3% after the elimination of five less important factors.
The process involved in the local scour below pipelines is so complex that it makes it difficult to establish a general empirical model to provide accurate estimation for scour. This paper describes the use of artificial neural networks (ANN) to estimate the pipeline scour depth. The data sets of laboratory measurements were collected from published works and used to train the network or evolve the program. The developed networks were validated by using the observations that were not involved in training. The performance of ANN was found to be more effective when compared with the results of regression equations in predicting the scour depth around pipelines.
A membrane sequencing batch reactor (MSBR) treating hypersaline oily wastewater was modeled by artificial neural network (ANN). The MSBR operated at different total dissolved solids (TDSs) (35,000; 50,000; 100,000; 150,000; 200,000; 250,000mg/L), various organic loading rates (OLRs) (0.281, 0.563, 1.124, 2.248, and 3.372kg COD/(m(3)day)) and cyclic time (12, 24, and 48h). A feed-forward neural network trained by batch back propagation algorithm was employed to model the MSBR. A set of 193 operational data from the wastewater treatment with the MSBR was used to train the network. The training, validating and testing procedures for the effluent COD, total organic carbon (TOC) and oil and grease (O&G) concentrations were successful and a good correlation was observed between the measured and predicted values. The results showed that at OLR of 2.44kg COD/(m(3)day), TDS of 78,000mg/L and reaction time (RT) of 40h, the average removal rate of COD was 98%. In these conditions, the average effluent COD concentration was less than 100mg/L and met the discharge limits.
Intelligent Transportation Systems rely on understanding, predicting and affecting the interactions between vehicles. The goal of this paper is to choose a small subset from the larger set so that the resulting regression model is simple, yet have good predictive ability for Vehicle agent speed relative to Vehicle intruder. The method of ANFIS (adaptive neuro fuzzy inference system) was applied to the data resulting from these measurements. The ANFIS process for variable selection was implemented in order to detect the predominant variables affecting the prediction of agent speed relative to intruder. This process includes several ways to discover a subset of the total set of recorded parameters, showing good predictive capability. The ANFIS network was used to perform a variable search. Then, it was used to determine how 9 parameters (Intruder Front sensors active (boolean), Intruder Rear sensors active (boolean), Agent Front sensors active (boolean), Agent Rear sensors active (boolean), RSSI signal intensity/strength (integer), Elapsed time (in seconds), Distance between Agent and Intruder (m), Angle of Agent relative to Intruder (angle between vehicles °), Altitude difference between Agent and Intruder (m)) influence prediction of agent speed relative to intruder. The results indicated that distance between Vehicle agent and Vehicle intruder (m) and angle of Vehicle agent relative to Vehicle Intruder (angle between vehicles °) is the most influential parameters to Vehicle agent speed relative to Vehicle intruder.
Training methodology of the Back Propagation Network (BPN) is well documented. One aspect of BPN that requires investigation is whether or not the BPN would get trained for a given training data set and architecture. In this paper the behavior of the BPN is analyzed during its training phase considering convergent and divergent training data sets. Evolution of the weights during the training phase was monitored for the purpose of analysis. The evolution of weights was plotted as return map and was characterized by means of fractal dimension. This fractal dimensional analysis of the weight evolution trajectories is used to provide a new insight to understand the behavior of BPN and dynamics in the evolution of weights.