Cognitive radio (CR) enables unlicensed users to exploit the underutilized spectrum in licensed spectrum whilst minimizing interference to licensed users. Reinforcement learning (RL), which is an artificial intelligence approach, has been applied to enable each unlicensed user to observe and carry out optimal actions for performance enhancement in a wide range of schemes in CR, such as dynamic channel selection and channel sensing. This paper presents new discussions of RL in the context of CR networks. It provides an extensive review on how most schemes have been approached using the traditional and enhanced RL algorithms through state, action, and reward representations. Examples of the enhancements on RL, which do not appear in the traditional RL approach, are rules and cooperative learning. This paper also reviews performance enhancements brought about by the RL algorithms and open issues. This paper aims to establish a foundation in order to spark new research interests in this area. Our discussion has been presented in a tutorial manner so that it is comprehensive to readers outside the specialty of RL and CR.
The vehicular Internet of Things (IoT) comprises enabling technologies for a large number of important applications including collaborative autonomous driving and advanced transportation systems. Due to the mobility of vehicles, strict application requirements, and limited communication resources, the conventional centralized control fails to provide sufficient quality of service for connected vehicles, so a decentralized approach is required in the vicinity to satisfy the requirements of delay-sensitive and mission-critical applications. A decentralized system is also more resistant to the single point of failure problem and malicious attacks. Blockchain technology has been attracting great interest due to its capability of achieving a decentralized, transparent, and tamper-resistant system. There are many studies focusing on the use of blockchain in managing data and transactions in vehicular environments. However, the application of blockchain in vehicular environments also faces some technical challenges. In this paper, we first explain the fundamentals of blockchain and vehicular IoT. Then, we conduct a literature review on the existing research efforts of the blockchain for vehicular IoT by discussing the research problems and technical issues. After that, we point out some future research issues considering the characteristics of both blockchain and vehicular IoT.
Cognitive radio (CR) enables unlicensed users (or secondary users, SUs) to sense for and exploit underutilized licensed spectrum owned by the licensed users (or primary users, PUs). Reinforcement learning (RL) is an artificial intelligence approach that enables a node to observe, learn, and make appropriate decisions on action selection in order to maximize network performance. Routing enables a source node to search for a least-cost route to its destination node. While there have been increasing efforts to enhance the traditional RL approach for routing in wireless networks, this research area remains largely unexplored in the domain of routing in CR networks. This paper applies RL in routing and investigates the effects of various features of RL (i.e., reward function, exploitation, and exploration, as well as learning rate) through simulation. New approaches and recommendations are proposed to enhance the features in order to improve the network performance brought about by RL to routing. Simulation results show that the RL parameters of the reward function, exploitation, and exploration, as well as learning rate, must be well regulated, and the new approaches proposed in this paper improves SUs' network performance without significantly jeopardizing PUs' network performance, specifically SUs' interference to PUs.