The statistical consequences of multicollinearity are well-known in statistics for a linear regression model. Multicollinearity is known as the approximately linear dependency among the columns of the matrix X in the following linear model

where y is an n × 1 vector of the given dependent variable, X is a known n × p matrix of the given explanatory variables, β is an p × 1 vector of given unknown regression parameters, and ε is described as an n × 1 vector of the disturbances. Then, the ordinary least squares (OLS) estimator of β for the model (1) is given as:

The multicollinearity problem effects on the behavior of the OLS estimator make it inefficient. Sometimes, it produces wrong signs [1, 2]. Many studies were conducted to handle this. For example, Hoerl and Kennard [2] proposed the ordinary ridge and the generalized ridge (GR) estimators, while Liu [3] introduced the popular Liu and the generalized Liu (GL), and very recently, Kibria and Lukman [1] proposed a ridge-type estimator called the Kibria–Lukman (KL) estimator which is defined by

This estimator has been extended for use in different generalized linear models, such as Lukman et al. [4, 5], Akram et al. [6], and Abonazel et al. [7].

According to recent papers [8–10], we can say that the efficiency of any bias estimator will increase if the estimator is modified or generalized using bias parameters that vary from observation to observation in the sample (k_i and/or d_i) rather than in fixed bias parameters (k and/or d). Hence, the main purpose of this paper is to develop a general form of the KL estimator to combat the multicollinearity in the linear regression model.

The rest of the discussion in this paper is structured as follows: Section Statistical Methodology presents the statistical methodology. In Section Superiority of the Proposed GKL Estimator, we theoretically compare the proposed general form of the KL estimator with each of the mentioned estimators. In Section The Biasing Parameter Estimator of the GKL Estimator, we give the estimation of the biasing parameter of the proposed estimator. Different scenarios of the Monte Carlo simulation are done in Section A Monte Carlo Simulation Study. A real data is used in Section Empirical Application. Finally, Section Conclusion presents some conclusions.

In this section, we make a comparison of the proposed GKL estimator with each of OLS, GR, GL, and KL estimators. First, we offer some useful lemmas for our comparisons of estimators.

Lemma 1: Wang et al. [11]: Suppose M and N are n × n positive definite matrices, then M > N if and only if (iff) λ − 1_max, where λ − 1_max is the maximum eigenvalue of NM⁻¹ matrix.

Lemma 2: Farebrother [12]: Let S be an n × n positive definite matrix. That is, S > 0 and α be some vector. Then, S − αα′ > 0 iff α′S⁻¹α < 1.

Lemma 3: Trenkler and Toutenburg [13]: Let α_i = U_iw, i = 1, 2 be any two linear estimators of α. Suppose that Q=Cov(α^1)-Cov(α^2)>0, where Cov(α^i),i=1,2 be the covariance matrix of α^i and bi=Bias(α^i)=(UiX-I)α. Then,

iff b2′[σ2Q+b1′b1]-1b2<1 where MMSE(α^i)=Cov(α^i)+bibi′

Theorem 1: α^GKL is superior to α^ iff

Proof: The covariance matrices difference is written as

where G-1-W2M2G-1M2′W2′ becomes positive definite iff (gi+ki)2-(gi-ki)2>0 or (g_i + k_i) − (g_i − k_i) > 0. It is clear that for k_i > 0, i = 1, 2, ..., p, (g_i + k_i) − (g_i − k_i) = 2k_i > 0. Therefore, this is done using Lemma 3.

Theorem 2: When λ − 1_max, α^GKL is superior to α^GR iff

where V1=σ2(W2GW2′-W2M2G-1M2′W2′), N=W2KG-1KW2′, and M= 2W2KKW2′.

Proof:

For k_i > 0, it is obvious that M > 0 and N > 0. Then, M − N > 0 iff λ − 1_max, where λ − 1_max is the maximum eigenvalue of NM⁻¹. So, this is done by Lemma 1.

Theorem 3: α^GKL is superior to α^GL iff

where V2=σ2(F2G-1F2′-W2M2G-1M2′W2′).

Proof: The covariance matrices difference is written as

where F2G-1F2′-W2M2G-1M2′W2′ becomes positive definite iff (gi+ki)2(gi+di)2-(gi-ki)2(gi+1)2>0 or (g_i + k_i)(g_i + d_i) − (g_i − k_i)(g_i + 1) > 0. So, if k_i > 0 and 0 < d_i < 1, (g_i + k_i)(g_i + d_i) − (g_i − k_i)(g_i + 1) = k_i(2g_i + d_i + 1) + g_i(d_i − 1) > 0. So, this is done by Lemma 3.

Theorem 4: α^GKL is superior to α^KL iff

where V3=σ2(W1M1G-1M1′W1′-W2M2G- 1M2′W2′).

Proof: The covariance matrices difference is written as

where W1M1G-1M1′W1′-W2M2G-1M2′W2′ becomes positive definite iff (gi+ki)2(gi-k)2-(gi-ki)2(gi+k)2>0 or (g_i + k_i)(g_i − k) − (g_i − k_i)(g_i + k) > 0. So, if k_i > 0 and k_i > k, (g_i + k_i)(g_i − k) − (g_i − k_i)(g_i + k) = 2g_i(k_i − k) > 0. So, this is done by Lemma 3.

The performance of any estimator depends on its biasing parameter. Therefore, the determination of the biasing parameter of an estimator is an important issue. Different studies analyzed this issue (e.g., [2, 3, 8–10, 14–24]).

Kibria and Lukman [1] proposed the biasing parameter estimator of the KL estimator as follows:

Here, we find the estimation of the optimal values of k_i for the proposed GKL estimator. The optimal values of k_i are obtained by minimizing

Differentiating m(k₁, k₂, ..., k_p) with respect to k_i and setting [∂m(k1,k2,...,kp)∂ki]=0, the optimal values of k_i after replacing σ² and αi2 by their unbiased estimators become as follows:

The explanatory variables are generated as follows [25–27]:

where a_ji are the independent pseudo-random numbers that have the standard normal distribution and ρ is known that the correlation between two given explanatory variables. The dependent variable y are given by:

where ε_j are the i.i.dN(0, σ²). The values of β are given such that β′β = 1 as discussed in Dawoud and Abonazel [28], Algamal and Abonazel [29], Abonazel et al. [7, 30], and Awwad et al. [31]. Also, all factors that used in the simulation are given in Table 1.

In order to see the performance of the OLS, KL, and the proposed GKL estimators with their biasing parameters estimators presented in Section Statistical Methodology, the estimated mean squared error (EMSE) are calculated for each replicate with different values of σ, ρ, n, and p using the following formula:

where αl* is the estimated vector of α at the lth experiment of the simulation.

The EMSE values of the OLS, KL, and GKL estimators are presented in Tables 2, 3. We can conclude the following based on the simulation results:

For clarifying the performance of the proposed GKL estimator, the dataset of the Portland cement that was originally due to Woods et al. [32], which was considered in Kibria and Lukman [1], where the dependent variable is the heat evolved after 180 days of curing and measured in calories per gram of cement. In this study, the first explanatory variable is tricalcium aluminate, the second explanatory variable is tricalcium silicate, the third explanatory variable is tetracalcium aluminoferrite, and the fourth explanatory variable is β-dicalcium silicate. The eigenvalues of X′X matrix are 44,676.21, 5,965.42, 809.95, and 105.42. Then, the condition number is 20.58. Therefore, multicollinearity exists among the predictors. The estimated error variance is σ^2=5.84, which shows high noise in the data. The estimated values of the optimal parameters in the GKL estimator are calculated as derived in Section Statistical Methodology. Also, the equation proposed by Kibria and Lukman [1] for estimating the biasing parameter of the KL estimator is used. Consequently, the mean square error (MSE) of the OLS, KL, and GKL estimators are presented in Table 4. From Table 4, we can note that the KL estimator is better than the OLS estimator, and the GKL estimator is better than the OLS and KL estimators.

In this paper, we proposed the GKL estimator. The performance of the proposed GKL estimator is theoretically compared with the OLS, GR, GL, and KL estimators in terms of known matrix mean squared error. Moreover, the optimal shrinkage parameter of the proposed GKL estimator is presented. A simulation study and the numerical example were performed for comparing the performance of the proposed GKL estimator with the OLS and KL estimators based on the estimated mean squared error criterion. The results indicate that the proposed estimator is better than other estimators, in particular, in the case the standard deviation of the errors was large and when the correlation between the explanatory variables is very high.

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

ID, MA, and FA contributed to conception and structural design of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.