Status update control based on reinforcement learning in energy harvesting sensor networks - Han, Zhihui; Gong, Jie (Frontiers in Communications and Networks, 2022)
With the development of the Internet of Things, more and more sensors are deployed to monitor the environment status. To reduce deployment cost, a large number of sensors need to be deployed without stable grid power supply. Therefore, on the one hand, the wireless sensors need to save energy as much as possible to extend their life time. On the other hand, it needs to sense and transmit timely and accurate information for real-time monitoring. In this paper, based on the spatio-temporal correlation of the environmental status monitored by the sensors, status information estimation is considered so as to effectively reduce the information collection frequency of the sensors and hence, reduce the energy cost. Under an ideal communication model with unlimited and perfect channels, a status update scheduling mechanism based on $Q$-learning algorithm is proposed. With non-ideal channel model, a status update scheduling mechanism based on deep reinforcement learning is proposed. In this scenario, all sensors share a limited number of channels, and channel fading is considered. A finite state Markov chain is adopted to model the channel state transition process. Simulation results based on real data set show that compared with several baseline methods, the proposed mechanisms can well balance the energy cost and the information error, and significantly reduces the update frequency while ensuring the accuracy of information.