Scalarized Q Multi-Objective Reinforcement Learning for Area Coverage Control and Light Control Implementation

Main Article Content

Akkhachai Phuphanin
Wipawee Usaha

Abstract

Coverage control is crucial for the deployment of wireless sensor networks (WSNs). However, most coverage control schemes are based on single objective optimization such as coverage area only, which do not consider other contradicting objectives such as energy consumption, the number of working nodes, wasteful overlapping areas. This paper proposes on a Multi-Objective Optimization (MOO) coverage control called Scalarized Q Multi-Objective Reinforcement Learning (SQMORL). The two objectives are to achieve the maximize area coverage and to minimize the overlapping area to reduce energy consumption. Performance evaluation is conducted for both simulation and multi-agent lighting control testbed experiments. Simulation results show that SQMORL can obtain more efficient area coverage with fewer working nodes than other existing schemes.  The hardware testbed results show that SQMORL algorithm can find the optimal policy with good accuracy from the repeated runs.

Article Details

How to Cite
Phuphanin, A., & Usaha, W. (2018). Scalarized Q Multi-Objective Reinforcement Learning for Area Coverage Control and Light Control Implementation. ECTI Transactions on Electrical Engineering, Electronics, and Communications, 16(2), 72–82. https://doi.org/10.37936/ecti-eec.2018162.171333
Section
Communication Systems

References

[1] H. Zhang and J. C. Hou, "Maintaining Sensing Coverage and Connectivity in Large Sensor Networks," Int. J. Ad Hoc Sensor Wireless Networks, Vol. 1, No. 1-2, pp.89-124, Mar. 2005.

[2] A. A. Kumaar, G. Kiran and T. S. B. Sudarshan, "Intelligent Lighting System using Wireless Sensor Networks," Int. J. Ad hoc, Sensor Ubiquitous Comput., Vol. 1, No. 4, pp. 17-27, Dec. 2010.

[3] T. P. Huynh, Y. K. Tan and K. J. Tseng, Energy Aware, "Wireless Sensor Network with Ambient Intelligencefor Smart LED Lighting System Control, " Proc. Annu. Conference IEEE Ind. Electron. Soc., 2011.

[4] R. Mohamaddoust, A. T. Haghighat, M. J. M. Sharif and N. Capanni, "A Novel Design of an Automatic Lighting Control System for a Wireless Sensor Network with Increased Sensor Lifetime and Reduced Sensor Numbers," J. Sensors, Vol. 11, pp. 8933-8952, Sep. 2011.

[5] P. Meng-Shiuan, Y. Lun-Wu, C. Yen-Ann, L. Yu-Hsuan and T. Yu-Chee, "A WSN-based Intelligent Light Control System Considering User Activities and Proles ," IEEE Sensor J., Vol. 8, No.10, pp. 1710-1721, Sep. 2008.

[6] M. Okada, H. Aida, H. Ichikawa and M. Miki, "Design and Implementation of an Energy Efficient Lighting System Driven by Wireless Sensor Networks," Proc. Int. Conference Mobile Comput. Ubiquitous Networking, Mar. 2015.

[7] M. Iqbal, M. Naeem, A. Anpalagan, N. N. Qadriand M. Imran, "Multi-objective Optimization in Sensor Networks: Optimization Classification, Applications and Solution Approaches," J. Comput. Networks, Vol. 99, pp. 134-161, Apr. 2016.

[8] R. Tharmarasa, T. Kirubarajan, J. Peng and T. Lang, "Optimization-Based Dynamic Sensor Management for Distributed Multitarget Tracking, " IEEE Trans. Syst., Man, Cybernetics Part C, Vol. 39, No. 5, pp. 534-546, Sep. 2009.

[9] M. Iqbal, M. Naeem, A. Anpalagan, A. Ahmed and M. Azam, "Wireless Sensor Network Optimization: Multi-Objective Paradigm," J. Sensors, Vol. 15, No. 7, pp. 17572-17620, Jul. 2015.

[10] Z. Fei, B. Li, S. Yang, C. Xing, H. Chen and L. Hanzo, "A Survey of Multi-Objective Optimization in Wireless Sensor Networks: Metrics, Algorithms, and Open Problems," IEEE Commun. Surveys Tutorials, Vol. 19, No.1, pp. 550-586, Sep. 2016.

[11] V. Singhiv, A. Krause, C. Guestrin, J. H. Garrett and H. Matthews, "Intelligent Light Control using Sensor Networks," Proc. 3rd Int. Conference Embedded Networked Sensor Syst., pp. 218-229,
Nov. 2005.

[12] C. A. C. Coello, G. T. Pulido and M. S. Lechuga, "Handling Multiple Objectives with Particle Swarm Optimization," IEEE Trans. Evol. Comput., Vol. 8, No.3, pp. 256-279, Jun. 2014.

[13] J. Jia, J. Chen, G. Chang and Z. Tan, "Energy efficient Coverage Control in Wireless Sensor Networks based on Multi-Objective Genetic Algorithm," J. Comput. Math. Applicat., Vol. 57, No.11-12, pp. 1756-1766, Jun. 2009.

[14] J. Barbancho, C. Leon, F. J. Molina and A. Barbancho, "Using Artificial Intelligence in Routing Schemes for Wireless Networks," J. Comput. Commun., Vol. 30, No.14-15, pp. 2802-2811, Oct. 2007.

[15] Z. Tafa, "Articial Neural Networks in WSNs Design: Mobility Prediction for Barrier Coverage, " Proc. IEEE Int. Symp. Signal Proc. Inform. Technology, Dec. 2016.

[16] M. Rovcanin, E. D. Poorter, D. Akker, I. Moerman, P. Demeester and C. Blondia, "Experimental Validation of a Reinforcement Learning based Approach for a Service-wise Optimisation of Heterogeneous Wireless Sensor Networks," J. Wireless Networks, Vol. 21, No.3, pp. 931-948, Apr. 2015.

[17] A. Phuphanin and W. Usaha, "A Multi-Agent Scheme for Energy-Ecient Coverage Control in Wireless Sensor Networks," Proc. Int. Conference Inform. Technology Sci., Jun. 2016.

[18] S. M. Jameii, K. Faez and M. Dehghan, "Multi-Objective Optimization for Topology and Coverage Control in Wireless Sensor Networks," Int. J. Distributed Sensor Networks, Vol.11, No. 2, pp.1-11, Feb. 2015.

[19] K. V. Moffaert, M. M. Drugan and A. Nowe, "Scalarized Multi-Objective Reinforcement Learning: Novel Design Techniques," Proc. IEEE Symp. Adaptive Dynamic Programming Reinforcement Learning, Apr. 2013.