Keywords

Liu Dawoud-Kibria, Multicollinearity, OLS estimator, Ridge regression estimator, Monte carlo simulation, Mean square error

Introduction

One of the multiple linear regression model assumptions is that the explanatory variables are independent of each other. However, according to Frisch [1], this independence assumption is often violated in real-life situations leading to the Multicollinearity problem. The ordinary least squares (OLS) estimator has been regarded as one of the most important ways of estimating the parameters of the general linear model since it has minimum variance. But, in the presence of Multicollinearity, the OLSE is no longer a good estimator. Multicollinearity is generally agreed to be present if there is an approximately linear relationship (i.e., shared variance) among some predictor variables in the data [2]. The term multicollinearity refers to a situation where there is an exact (or nearly exact) linear relation among two or more explanatory variables [31]. One of the tests to ascertain the presence of multicollinearity in a dataset is the variance inflation factor (VIF). The VIF values ranging between one and five are moderate and represent a medium level of collinearity. Values of more than five are said to be highly collinear, while values above ten are considered to be extreme. Pairwise correlation of two independent variables can also be used to detect if multicollinearity is present in a dataset. An absolute correlation value of 0.7 and above indicates the presence of multicollinearity.

However, many methods have been proposed to solve this problem by various researchers (Hoerl and Kennard [4]; Liu [5]; Kibria [6]; Sakallioglu and Kaciranlar [7]; Baye and Parker [8]; Yang and Chang [9]; Wu and Yang [10]; Dorugade [11]; Lukman, et al. [12]); Lukman, et al. [13]; Kibria and Lukman [14]; Dawoud and Kibria [15]) and recently Owolabi, et al. [16,17]. This study proposes a new two-parameter estimator to circumvent the problem of Multicollinearity. Aside from multicollinearity, a few terms are defined in the table 1 below.

The organization of the paper is as follows. The model, the proposed and existing estimators are given in Section 2. A comparison of the proposed estimator with some existing ones is shown in section 3. The biasing parameter was obtained in Section 4, while a Monte-Carlo simulation and a numeric example were conducted in Section 5. Section 6 contains concluding remarks.

The Models Specifications and New Estimator

The models and existing estimators

Consider the following linear regression model

$$y\text{ = }X\beta \text{ + }\epsilon ,\left(1\right)$$

where y is a n × 1 vector of response, X is a n × p full column rank matrix, where n and p refer to the sample size and numbers of explanatory variables., β is a p × 1 vector of unknown parameters, ε is a n × 1 vector of errors supposed to be distributed with mean vector 0 and variance-covariance ${\sigma }^{2}I.$

Based on the Gauss-Markov theorem, the ordinary least squares estimator (OLSE) is given as:

$$\widehat{\beta }\text{ = }{\left(X\text{'}X\right)}^{-1}X\text{'}y\left(2\right)$$

The canonical form of Eq. (1) is rewritten as:

$y=Z\alpha +\epsilon \left(3\right)$

where Z = XQ, α = Q'β and Q is the orthogonal matrix whose columns constitute the eigenvectors of X'X. Then Z'Z = Q'X'XQ = Λ = diag(λ₁,..., λ_p),

where λ₁ ≥ λ2 ≥ ...λ_p > 0 are the ordered eigenvalues X'X. The ordinary least square estimator (OLSE) of equation (2) can be defined as:

$$\widehat{\alpha }\text{ = }{\Lambda }^{-1}X\text{'}y\left(4\right)$$

And the Mean Square Error matrix (MSEM) of $\widehat{\alpha }$ is defined as

$$MSEM\left(\widehat{\alpha }\right)\text{ = }{\Lambda }^{-1}{\sigma }^2\left(5\right)$$

The Ordinary Ridge Regression Estimator by Hoerl and Kennard [4] is given as: ${\widehat{\alpha }}_k={\rm A}X\text{'}y\left(6\right)$

where A = (Λ + kI)^-1 and k is a non-negative biasing parameter. And the MSEM is given as:

$$MSEM\left({\widehat{\alpha }}_k\right)\text{ = }{\sigma }^{2}A\Lambda A\text{ + }\left(A\Lambda \text{ - }I\right)\alpha \alpha \text{'}\left(A\Lambda \text{ - }I\right)\text{'}\left(7\right)$$

The Liu estimator is defined as:

$${\widehat{\alpha }}_d\text{ = E}\widehat{\alpha }\left(8\right)$$

where E = (Λ + I)^-1(Λ + dI) and d is a biasing parameter of Liu Estimator. The MSEM of ${\widehat{\alpha }}_d$ is defined as:

$$MSEM\left({\widehat{\alpha }}_d\right)\text{ = }{\sigma }^{2}E{\Lambda }^{-1}E\text{ + }\left(E\text{ - }I\right)\alpha \alpha \text{'}\left(E\text{ - }I\right)\text{'}\left(9\right)$$

The Kibria-Lukman (KL) estimator is defined as:

$${\widehat{\alpha }}_{KL}\text{ = P}\widehat{\alpha }\left(10\right)$$

where P = (Λ + kI)^-1(Λ - kI) and the MSEM of ${\widehat{\alpha }}_{KL}$ is given as

$$MSEM\left({\widehat{\alpha }}_{KL}\right)\text{ = }{\sigma }^{2}P{\Lambda }^{-1}P\text{ + }\left(P\text{ - }I\right)\alpha \alpha \text{'}\left(P\text{ - }I\right)\text{'}\left(11\right)$$

The Modified Ridge Type (MRT) estimator is defined as:

$${\widehat{\alpha }}_{MRT}\text{ = H}\widehat{\alpha }\left(12\right)$$

where H = Λ(Λ + k(1 + d)I)-1 and the MSEM is given as

$$MSEM({\widehat{\alpha }}_{MRT})\text{ = }{\sigma }^{2}H{\Lambda }^{-1}H\text{ + }\text{}\left(H-I\right)\alpha \alpha \text{'}(H-I)\text{'}\left(13\right)$$

The Dawoud Kibria (DK) estimator is defined as:

$${\widehat{\alpha }}_{DK}\text{ = }KM\widehat{\alpha }\left(14\right)$$

where K = (Λ + k(1 + d))-1, M = (Λ - k(1 + d)) and the MSEM is given as $$MSEM\left({\widehat{\alpha }}_{DK}\right)\text{ = }{\sigma }^{2}KM{\Lambda }^{-1}KM\text{ + }\left(KM-I\right)\alpha \alpha \text{'}\left(KM-I\right)\text{'}\left(15\right)$$

The proposed estimator

In this paper, we proposed a New Biasing Estimator called Liu Dawoud-Kibria Estimator by following a method similar to that proposed by Liu [5]; Kaciranlar, et al. [20]; Yang and Chang [9], and Dawoud, et al. [21]. The proposed New Biasing Liu Dawoud-Kibria estimator for α is obtained by replacing with in the Liu estimator, and it becomes as follows:

$${\widehat{\alpha }}_{LDK}\text{ = }WS\widehat{\alpha }\left(16\right)$$

Where W = (Λ + I)-1(Λ + dI) S = (Λ + k(1 + d)I)-1(Λ – k(1 + d)I), d and k are the biasing parameters.

The properties of the new estimator, namely; the bias vector, covariance, and mean squared error matrix (MSEM) of the proposed estimator, are given as follows:

$$B\left({\widehat{\alpha }}_{LDK}\right)\text{ = }\left(WS-I\right)\alpha \left(17\right)$$

$$D\left({\widehat{\alpha }}_{LDK}\right)\text{ = }{\sigma }^{2}WS{\Lambda }^{-1}WS\left(18\right)$$

$$MSEM\left({\widehat{\alpha }}_{LDK}\right)\text{ = }{\sigma }^{2}WS{\Lambda }^{-1}WS\text{ + (}WS-I)\alpha \alpha \text{'}(WS-I)\left(19\right)$$

The following lemmas will be used to make some theoretical comparisons among estimators in the next section.

Lemma 1: Let n x n matrices M > 0, N > 0 (or N > 0), then N > M if and only if ${\lambda }_1\left(N{M}^{-1}\right)\text{ < 1}$ where ${\lambda }_1\left(N{M}^{-1}\right)\text{ < 1}$ is the largest eigenvalue of matrix NM-1 [22].

Lemma 2: Let M be an n x n positive definite matrix, that is M > 0, and α be some vector, then $M-\alpha \alpha \text{'}\ge 0$ if and only if $\alpha \text{'}{M}^{-1}\alpha \le 1$ [23].

Lemma 3: Let ${\widehat{\alpha }}_i\text{ = }{A}_{i}y$ i = 1, 2 be two linear estimators of α. Suppose that $D\text{ = }\mathrm{cov}\left({\widehat{\alpha }}_1\right)-\mathrm{cov}\left({\alpha }_2\right)\text{ > 0,}$ where $\mathrm{cov}\left({\widehat{\alpha }}_1\right)$ i = 1, 2 denotes the covariance matrix of ${\widehat{\alpha }}_{\text{i}}$ and $b_i\text{ = }Bias\left({\widehat{\alpha }}_i\right)=(A_{i}X-I)\alpha ,$ i = 1, 2. Consequently,

$$\Delta \left({\widehat{\alpha }}_1-{\widehat{\alpha }}_2\right)\text{ = }MSEM\left({\widehat{\alpha }}_1\right)-MSEM\left({\widehat{\alpha }}_2\right)={\sigma }^{2}D+b_1{b}_1^{\text{'}}-b_2{b}_2^{\text{'}}>0\left(20\right)$$

If and only if $b_2^{\text{'}}{\left[{\sigma }^{2}D+b_1{b}_1^{\text{'}}\right]}^{-1}{b}_2<1,$ where $MSEM\left({\widehat{\alpha }}_i\right)=\mathrm{cov}\left({\widehat{\alpha }}_i\right)+b_i{b}_i^{\text{'}}$ [24].

Comparison among the Estimators

Theoretical Comparisons among the Proposed LDK Estimator and the OLSE, Ordinary Ridge Regression (ORR), Liu Estimators, Kibria-Lukman (KL) Estimator, Dawoud-Kibria (DK), and Modified Ridge-Type (MRT) Estimator.

Theorem 1: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator $\widehat{\alpha }$ if and only if $$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left({\Lambda }^{-1}-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(21\right)$$

Proof: The difference between the dispersion matrices is given as $$D\left(\widehat{\alpha }\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left({\Lambda }^{-1}-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{1}{{\lambda }_i}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(22\right)$$

${\Lambda }^{-1}-WS{\Lambda }^{-1}WS$ will be positively definite if and only if $${\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$$

Theorem 2: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator ${\widehat{\alpha }}_k$ if and only if $$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left(A\Lambda A-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(23\right)$$

Proof: The difference between the dispersion matrices is given as

$$D\left({\widehat{\alpha }}_k\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left(A\Lambda A-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{{\lambda }_i}{{\left({\lambda }_i+k\right)}^2}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(24\right)$$

AΛA - WSΛ-1WS will be positively definite if and only if

$${\lambda }_i^2{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i+k\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$$

Theorem 3: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator ${\widehat{\alpha }}_d$ if and only if $$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left(E{\Lambda }^{-1}E-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(25\right)$$

Proof: The difference between the dispersion matrices is given as $$D\left({\widehat{\alpha }}_d\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left(E{\Lambda }^{-1}E-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{{\left({\lambda }_i+d\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(26\right)$$

EΛ-1E - WSΛ-1WS will be positively definite if and only if

$${\left({\lambda }_i+d\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$$

Theorem 4: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator ${\widehat{\alpha }}_{KL}$ if and only if $$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left(P{\Lambda }^{-1}P-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(27\right)$$

Proof: The difference between the dispersion matrices is given as $$D\left({\widehat{\alpha }}_{KL}\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left(P{\Lambda }^{-1}P-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{{\left({\lambda }_i-k\right)}^2}{{\lambda }_i{\left({\lambda }_i+k\right)}^2}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(28\right)$$

PΛ-1P - WSΛ-1WS will be positively definite if and only if

$${\left({\lambda }_i-k\right)}^2{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i+k\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$$

Theorem 5: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator ${\widehat{\alpha }}_{MRT}$ if and only if $$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left(H{\Lambda }^{-1}H-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(29\right)$$

Proof: The difference between the dispersion matrices is given as

$$D\left({\widehat{\alpha }}_{MRT}\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left(H{\Lambda }^{-1}H-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{{\lambda }_i}{{\left({\lambda }_i+k\left(1+d\right)\right)}^2}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(30\right)$$

HΛ-1H - WSΛ-1WS will be positively definite if and only if ${\lambda }_i^2{\left({\lambda }_i+1\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$

Theorem 6: The estimator ${\widehat{\alpha }}_{LDK}$ is superior to the estimator ${\widehat{\alpha }}_{DK}$ if and only if

$$\alpha \text{'}\left(WS-I\right)\text{'}⌊{\sigma }^2\left(P{\Lambda }^{-1}P-WS{\Lambda }^{-1}WS\right)⌋\alpha \left(WS-I\right)\text{ < 1 }\left(31\right)$$

Proof: The difference between the dispersion matrices is given as $$D\left({\widehat{\alpha }}_{DK}\right)-D\left({\widehat{\alpha }}_{LDK}\right)={\sigma }^2\left(KM{\Lambda }^{-1}KM-WS{\Lambda }^{-1}WS\right)$$

$$={\sigma }^{2}diag{\left\{\frac{{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+k\left(1+d\right)\right)}^2}-\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+1\right)}^2{\left({\lambda }_i+k\left(1+d\right)\right)}^2}\right\}}_{i=1}^p\left(32\right)$$

KMΛ-1KM - WSΛ-1WS will be positively definite if and only if

$${\left({\lambda }_i+1\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2-{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2>0.$$

Determination of the Parameters k and d

Various researchers have introduced different estimators of k and d for different kinds of regression models. Some of these authors are Hoerl and Kennard [4]; Dorugade [25]; Lukman and Ayinde [26]; Aslam and Ahmad [27], among others.The optimal values of k and d for the proposed estimator are obtainedthus. In determining the optimal value of k, d is fixed. The optimal value of the k can be considered to be that k that minimizes:

$$MSEM\left({\widehat{\alpha }}_{LDK}\right)\text{ = }{\sigma }^{2}WS{\Lambda }^{-1}WS\text{ + (}WS-I)\alpha \alpha \text{'}(WS-I)$$

$$g\left(k,d\right)=MSEM\left({\widehat{\alpha }}_{LDK}\right)=tr\left[MSEM\left({\widehat{\alpha }}_{LDK}\right)\right]$$

$$g\left(k,d\right)={\sigma }^2{\displaystyle \sum _i^p\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(1+d\right)\right)}^2}{{\lambda }_i{\left({\lambda }_i+k\left(1+d\right)\right)}^2{\left({\lambda }_i+1\right)}^2}}+{\displaystyle \sum _i^p\frac{{\left[2k\left({\lambda }_i+{\lambda }_{i}d+d\right)+k\left(d^2+1\right)+{\lambda }_i\left(1-d\right)\right]}^2{\alpha }_i^2}{{\left({\lambda }_i+k\left(1+d\right)\right)}^2{\left({\lambda }_i+1\right)}^2}}\left(33\right)$$

Taking the partial derivative of the function g(k,d) with respect to k gives $$\begin{array}{l}\frac{\partial g\left(k,d\right)}{\partial k}=2{\sigma }^2{\displaystyle \sum _{i=1}^p\frac{{\left({\lambda }_i+d\right)}^2\left({\lambda }_i-k\left(d+1\right)\right)\left(d+1\right)}{{\lambda }_i{\left({\lambda }_i+k\left(d+1\right)\right)}^2{\left({\lambda }_i+1\right)}^2}}-2{\sigma }^2{\displaystyle \sum _{i=1}^p\frac{{\left({\lambda }_i+d\right)}^2{\left({\lambda }_i-k\left(d+1\right)\right)}^2\left(d+1\right)}{{\lambda }_i{\left({\lambda }_i+k\left(d+1\right)\right)}^3{\left({\lambda }_i+1\right)}^2}}\\ +2{\displaystyle \sum _{i=1}^p\frac{\left[2k\left({\lambda }_i+{\lambda }_{i}d+d\right)-{\lambda }_i\left(d-1\right)+k\left(d^2+1\right)\right]{\alpha }_i^2}{{\left({\lambda }_i+k\left(d+1\right)\right)}^2{\left({\lambda }_i+1\right)}^2}}-2{\displaystyle \sum _{i=1}^p\frac{{\left[2k\left({\lambda }_i+{\lambda }_{i}d+d\right)-{\lambda }_i\left(d-1\right)+k\left(d^2+1\right)\right]}^2\left(d+1\right){\alpha }_i^2}{{\left({\lambda }_i+k\left(d+1\right)\right)}^3{\left({\lambda }_i+1\right)}^2}}\left(34\right)\end{array}$$

Let $\frac{\partial g\left(k,d\right)}{\partial k}=0;$

$$k=\frac{{\sigma }^2{\lambda }_i\left({\lambda }_i+d\right)-{\alpha }_i^2{\lambda }_i^2\left(1-d\right)}{{\sigma }^{2}d\left(d+1\right)+{\sigma }^2{\lambda }_i\left(d+1\right)+{\alpha }_i^2{\lambda }_i\left(d^2+1\right)+2{\alpha }_i^2{\lambda }_i\left({\lambda }_{i}d+d+{\lambda }_i\right)}\left(35\right)$$

For practical purposes, σ² and ${\alpha }_i^2$ are replaced with ${\widehat{\sigma }}^2$ and ${\widehat{\alpha }}_i^2,$ respectively. Consequently, (30) becomes

$$\widehat{k}=\frac{{\widehat{\sigma }}^2{\lambda }_i\left({\lambda }_i+d\right)-{\widehat{\alpha }}_i^2{\lambda }_i^2\left(1-d\right)}{{\widehat{\sigma }}^{2}d\left(d+1\right)+{\widehat{\sigma }}^2{\lambda }_i\left(d+1\right)+{\widehat{\alpha }}_i^2{\lambda }_i\left(d^2+1\right)+2{\widehat{\alpha }}_i^2{\lambda }_i\left({\lambda }_{i}d+d+{\lambda }_i\right)}\left(36\right)$$

and,

$${\widehat{k}}_{\min }=\min {\left[\frac{{\widehat{\sigma }}^2{\lambda }_i\left({\lambda }_i+d\right)-{\widehat{\alpha }}_i^2{\lambda }_i^2\left(1-d\right)}{{\widehat{\sigma }}^{2}d\left(d+1\right)+{\widehat{\sigma }}^2{\lambda }_i\left(d+1\right)+{\widehat{\alpha }}_i^2{\lambda }_i\left(d^2+1\right)+2{\widehat{\alpha }}_i^2{\lambda }_i\left({\lambda }_{i}d+d+{\lambda }_i\right)}\right]}_{i=1}^p\left(37\right)$$

and the biasing parameters d proposed by Liu [5] in equation (38) will be adopted in this study. It is given as follows:

$$d=\frac{{\lambda }_i\left({\alpha }_i^2-{\sigma }^2\right)}{{\sigma }^2+{\lambda }_i{\alpha }_i^2}\left(38\right)$$

For practical purposes, ${\sigma }^2$ and ${\alpha }_i^2$ can be considered as ${\widehat{\sigma }}^2$ and ${\widehat{\alpha }}_i^2,$ respectively. Consequently, (38) becomes

$$d=\frac{{\lambda }_i\left({\widehat{\alpha }}_i^2-{\widehat{\sigma }}^2\right)}{{\widehat{\sigma }}^2+{\lambda }_i{\widehat{\alpha }}_i^2}\left(39\right)$$

Simulation and Application

Simulation technique

The simulation procedure used by McDonald and Galarneau [28]; Wichern and Churchill [29]; Gibbons [30]; Lukman and Ayinde [26] was utilized to generate the predictor variables in this study: This is given as:

$$x_{ij}={\left(1-{\rho }^2\right)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}{z}_{ij}+\rho z_{i,p+1},i=1,2,\dots ,n,j=1,2,\dots ,p.\left(40\right)$$

where z_ij is an independent standard normal distribution with mean zero and unit variance, ρ is the correlation between any two explanatory variables, and p is the number of explanatory variables. For this study, we considered the values of ρ to be 0.8, 0.9, 0.95, and 0.99. Also, explanatory variables (p) were taken to be three (3) and seven (7) for the simulation study. The error terms ut were generated following Firinguetti [31] such that $u_t~N\left(0,{\sigma }^{2}I\right).$ The values of $\beta \text{'}\beta =1$ Newhouse and Oman [32]. The standard deviations in this simulation study were σ = 3, 5, and 10.

Simulation results from discussion

Table 2 and Table 3 show that as σ and ρ increase, the estimated MSE values increase. Likewise, as n increases, the estimated MSE values decrease. As expected and from several simulations and empirical research, when the multicollinearity problem exists, the OLS estimator gives the highest MSE values and performs the worst among all estimators. Additionally, the results show that the proposed LDK estimator performs best and dominates the rest of the estimators in this study. Thus, the findings agree with the theoretical results.

Numerical example

In this section, Portland cement data was used to demonstrate the performance of the proposed estimator. The Portland cement data was originally adopted by Woods, et al. [33] and was later adopted by Li and Yang [34] and Ayinde, et al. [35]. The data set is widely known as the Portland cement dataset. The regression model for these data is defined as:

$$y_i={\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4+{\epsilon }_i\left(41\right)$$

where yi = heat evolved after 180 days of curing measured in calories per gram of cement, X₁ = tricalcium aluminate, X₂ = tricalcium silicate, X₃ = tetracalcium aluminoferrite, and X₄ = β-dicalcium silicate. The variance inflation factors (VIFs) are 38.50, 254.42, 46.87, and 282.51, respectively. Eigenvalues of $X\text{'}X$ matrix are λ₁ = 44676.206, λ₂ = 5965.422, λ₃ = 809.952, and λ₄ = 105.419, and the condition number of $X\text{'}X$ is approximately 424. The VIFs, the eigenvalues, and the condition number indicate severe Multicollinearity. The estimated parameters and the MSE values of the estimators are presented in Table 4.

From Table 4, the proposed estimator (LDK) performs best among other estimators as it gives the smallest MSE value. Just as observed in the simulation study results, the OLS estimator did not perform well in the presence of Multicollinearity as it has the highest MSE.

Concluding Remarks

In this paper, a new two-parameter estimator (LKD) is proposed. Theoretical comparison of the proposed with six other existing estimators shows the superiority of the proposed estimator. Results from the simulation study reveal that the proposed estimator performs better than other existing estimators used in this study under certain conditions, which further strengthens the theoretical study. Application to real-life dataset also reveals the dominance of the proposed estimator (LKD). The newly proposed estimator (LKD) is recommended for parameter estimation in the linear regression model in the presence of Multicollinearity.

Declaration of Competing Interest

The authors declare that they have no known competing interests.

Funding

No financial assistance was received for this study.

References

1. Frisch R (1934) Statistical confluence analysis by means of complete regression analysis. University Institute of Economics, Oslo.
2. Belsley DA, Kuh E, Welsch RE (1980) Regression Diagnostics: Identifying influential data and sources of collinearity. John Wiley & Sons, New York.
3. Hawking RR, Pendleton OJ (1983) The regression dilemma. Communication in Statistics Theory and Method 12: 497-527.
4. Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12: 55-67.
5. Liu K (1993) A new class of biased estimate in linear regression. Communications in Statistics - Theory and Methods 22: 393-402.
6. Kibria BM (2003) Performance of some new ridge regression estimators. Communications in Statistics -Simulation and Computation 32: 419-435.
7. Sakallioglu S, Kaciranlar S (2008) A new biased estimator based on ridge estimation. Statistical Papers 49: 669-689.
8. Baye MR, Parker DF (1984) Combining ridge and principal component regression: A money demand illustration. Communications in Statistics - Theory and Methods 13: 197-205.
9. Yang H, Chang X (2010) A new two-parameter estimator in linear regression. Communications in Statistics - Theory and Methods 39: 923-934.
10. Wu J, Yang H (2011) Efficiency of an almost unbiased two-parameter estimator in linear regression model. Statistics 47: 535-545.
11. Dorugade A (2014) A modified two-parameter estimator in linear regression. Statistics in Transition 15: 23-36.
12. Lukman AF, Ayinde K, Binuomote S, Clement OA (2019a) Modified ridge-type estimator to combat multicollinearity: Application to chemical data. Journal of Chemometrics 33: e3125.
13. Lukman AF, Ayinde K, Sek SK, Adewuyi E (2019b) A modified new two-parameter estimator in a linear regression model. Modelling and Simulation in Engineering 2019: 1-10.
14. Kibria BMG, Lukman AF (2020) A new ridge-type estimator for the linear regression model: Simulations and applications. Scientifica 2020: 1-16.
15. Dawoud I, Kibria BMG (2020) A new biased estimator to combat the multicollinearity of the gaussian linear regression model. Stats 3: 526-541.
16. Owolabi AT, Ayinde K, Alabi OO (2022) A new ridge-type estimator for the linear regression model with correlated regressors. Concurrency Computat Pract Exper 34: e6933.
17. Owolabi AT, Ayinde K, Idowu JI, Oladapo OJ, Lukman AF (2022) A new two-parameter estimator in the linear regression model with correlated regressors. J Stat Appl Pro 11: 499-512.
18. Xin Y, Xiao-Gang S (2009) Linear regression analysis: Theory and computing. World Scientific Publishing Co. Pte. Ltd.
19. Chatterjee S, Hadi AS (2006) Regression analysis by example. (4th edn), John Wiley and Sons, NJ.
20. Kaciranlar S, Sakallioglu S, Akdeniz F, Styan GPH, Werner HJ (1999) A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland Cement. Sankhya: Indian Journal of Statistics 61: 443-459.
21. Dawoud I, Lukman AF, Abdul-Rahaman H (2022) A new biased regression estimator: Theory, simulation and application. Scientific African 15: e01100.
22. Wang SG, Wu MX, Jia ZZ (2006) Matrix inequalities. (2nd edn), Chinese Science Press, Beijing.
23. Farebrother RW (1976) Further results on the mean square error of ridge regression. J R Stat Soc 38: 248‐250.
24. Trenkler G, Toutenburg H (1990) Mean squared error matrix comparisons between biased estimators-an overview of recent results. Stat Pap 31: 165‐179.
25. Dorugade AV, Kashid DN (2010) Alternative method for choosing ridge parameter for regression. International Journal of Applied Mathematical Sciences 4: 447-456.
26. Lukman AF, Ayinde K (2017) Review and classifications of the ridge parameter estimation techniques. Hacettepe Journal of Mathematics and Statistics 46: 953‐967.
27. Ahmad S, Aslam M (2020) Another proposal about the new two-parameter estimator for linear regression model with correlated regressors. Communications in Statistics - Simulation and Computation 51: 3054-3072.
28. McDonald GC, Galarneau DI (1975) A Monte Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association 70: 407-416.
29. Wichern DW, Churchill GA (1978) A comparison of ridge estimators. Technometrics 20: 301-311.
30. Gibbons DG (1981) A simulation study of some ridge estimators. Journal of the American Statistical Association 76: 131-139.
31. Firinguetti LL (1989) A simulation study of ridge regression estimators with autocorrelated errors. Communications in Statistics Simulation and Computation 18: 673-702.
32. Newhouse JP, Oman SD (1971) An evaluation of ridge estimators. A report prepared for United States Air Force Project RAND.
33. Woods H, Steinour HH, Starke HR (1932) Effect of composition of Portland cement on heat evolved during hardening. Ind Eng Chem 24: 1207-1214.
34. Li Y, Yang H (2012) A new Liu-type estimator in linear regression model. Stat 53: 427-437.
35. Ayinde K, Lukman AF, Samuel OO, Ajiboye S (2018) Some new adjusted ridge estimators of linear regression model. Int J Civ Eng Technol 9: 2838‐2852.

---