Tuning of Elasticsearch Configuration: Parameter Optimization Through Simultaneous Perturbation Stochastic Approximation - Haugerud, Hårek; Sobhie, Mohamad; Yazidi, Anis (Frontiers in Big Data, 2022)
Elasticsearch is currently the most popular full-text database management system search engine. By default, its configuration does not change while it receives data. However, when Elasticsearch stores a large amount of data over time, the default configuration becomes an obstacle for achieving better performance. Additionally, the servers that host Elasticsearch may have limited resources like internal memory and CPU. A general solution to these problems is to dynamically tune the configuration parameters of Elasticsearch in order to improve the performance. The sheer number of parameters which is involved in this configuration makes it a complex task to solve. In this work, we apply the Simultaneous Perturbation Stochastic Approximation method for optimizing Elasticsearch with multiple unknown parameters. Using this algorithm we provide an implementation of optimizing Elasticsearch configuration parameters by observing the performance and automatically changing the configuration to improve the performance. The proposed solution is able to change the configuration parameters of Elasticsearch automatically without the need for restarting the currently running instance of Elasticsearch. The results show a higher than 40% improvement of the combined data insertion capacity and the response time of the system.