AUTHOR=Haugerud Hårek , Sobhie Mohamad , Yazidi Anis 

TITLE=Tuning of Elasticsearch Configuration: Parameter Optimization Through Simultaneous Perturbation Stochastic Approximation

JOURNAL=Frontiers in Big Data

VOLUME=Volume 5 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2022.686416

DOI=10.3389/fdata.2022.686416

ISSN=2624-909X

ABSTRACT=Elasticsearch is currently the most popular full-text database management system search engine. By default, its configuration does not change while it receives data. However, when Elasticsearch stores a large amount of data over time, the default configuration becomes an obstacle for achieving better performance. Additionally, the servers that host  Elasticsearch may have limited resources like internal memory and CPU. A general solution to these  problems is to dynamically tune the configuration parameters of Elasticsearch in order to improve the performance. The sheer number of parameters which is involved in this  configuration makes it a complex task to solve. In this work, we apply the Simultaneous Perturbation Stochastic Approximation method for optimizing Elasticsearch with multiple unknown parameters. Using this algorithm we provide an implementation of optimizing Elasticsearch configuration parameters by observing the performance and automatically changing the configuration to improve the performance. The proposed solution is able to change the configuration parameters of Elasticsearch automatically without the need for restarting the currently running instance of Elasticsearch. The results show a higher than 40% improvement of the combined data insertion capacity and the response time of the system.