Keywords

Biased estimator, Liu estimator, Monte Carlo simulation, Multicollinearity, MSE, Ridge regression estimator

Introduction

The ordinary least squares (OLS) estimator is the best linear unbiased estimator and has been used to estimate the parameters of the linear regression model since its inception. One of the important assumptions for the linear regression model is that the regressors (independent variables) are independent. However, in practice the regressors may or may not be independent, which causes the problem of multicollinearity. In the presence of the multicollinearity, the OLS estimator is inefficient and gives wrong sign of the parameters in the multiple linear regression models [1]. To handle these problems, many authors have given different types of estimators, to mention a few, [1-7], and recently [8], for one parameter biased estimator. However, to refer for two parameters mode, the following authors are notable, [9-15], among others.

The main objective of this paper is to propose a new two parameter biased estimator for the regression coefficients and then to compare the performance of the new estimator with the OLS, the ordinary ridge regression (ORR) of Hoerl and Kennard [1], the Liu of Liu [6] and the Kibria-Lukman (KL) of Kibria and Lukman [8] estimators. The rest of the paper is organized as follows: Some estimators and their statistical properties are given in Section 2. In section 3, the theoretical comparisons among the proposed estimator and existing estimators and the biasing parameters k and d are given. A Monte Carlo simulation study is performed in section 4. A real-life data are analyzed in section 5. Finally, Some Conclusions are given in section 6.

The Model and Estimators

Consider the following linear regression model:

$y=X\beta +\epsilon ,\text{ (2}\text{.1)}$

where y is an n×1 vector of the response variable, X is a known n×p full rank matrix of the explanatory variables, β is an p×1 vector of unknown regression coefficients, $\epsilon$ is a n × 1 vector of disturbance assumed to be distributed with mean vector 0 and variance covariance matrix σ²I, and I is an identity matrix of order n × n. To define various estimators, canonical form of the model (2.1) is given by:

$y=Z\alpha +\epsilon ,\epsilon ~N(0,{\sigma }^2{I}_n)\text{ (2}\text{.2)}$

where, $Z=XC,$ $\alpha =C^{\prime }\beta$, and C is an orthogonal matrix such that $Z^{\prime }Z=C^{\prime }{X}^{\prime }XC=U=diag(u_1,u_2,\dots ,u_p).$ Then, the OLS estimator of α is given as:

$\widehat{\alpha }=U^{-1}{Z}^{\prime }y,\text{ (2}\text{.3)}$

and then the mean squared error matrix (MSEM) of $\widehat{\alpha }$ is given by

$MSEM(\widehat{\alpha })={\sigma }^2{U}^{-1}.\text{ (2}\text{.4)}$

The ORR of α [1] is

${\widehat{\alpha }}_k=NU\widehat{\alpha },\text{ (2}\text{.5)}$

Where, N=[U+kI]^-1 and the MSEM of ${\widehat{\alpha }}_k$ would be

$MSEM({\widehat{\alpha }}_k)={\sigma }^{2}NU{N}^{\prime }+(NU-I_p)\alpha {\alpha }^{\prime }(NU-I_p{)}^{\prime }\text{ (2}\text{.6)}$

Hoerl, et al. [16] defined the biasing parameter k for ${\widehat{\alpha }}_k$ as follows:

${\widehat{k}}_{HM}=\frac{p{\widehat{\sigma }}^2}{{\displaystyle \sum _{i=1}^p{\widehat{\alpha }}_i^2}}.\text{ (2}\text{.7)}$

The Liu estimator of α [6] is

${\widehat{\alpha }}_d=F\widehat{\alpha },\text{ (2}\text{.8)}$

Where, $F={[U+I_p]}^{-1}[U+d{I}_p]$ and the biasing parameter d of ${\widehat{\alpha }}_d$ is given by

$${\widehat{d}}_{opt}=1-{\widehat{\sigma }}^2\left[\frac{{\displaystyle \sum _{i=1}^{p}1/(u_i(u_i+1))}}{{\displaystyle \sum _{i=1}^p({\widehat{\alpha }}_i^2/{(u_i+1)}^2)}}\right]\text{ (2}\text{.9)}$$

$MSEM({\widehat{\alpha }}_d)={\sigma }^{2}F{U}^{-1}{F}^{\prime }+{(1-d)}^2{(U+I_p)}^{-1}\alpha {\alpha }^{\prime }{(U+I_p)}^{-1}.\text{ (2}\text{.10)}$

If ${\widehat{d}}_{opt}$ is negative, Ozkale and Kaciranlar [9] adopt the alternative biasing parameter below:

${\widehat{d}}_{alt}=\min {\left[\frac{{\widehat{\alpha }}_i^2}{({\widehat{\sigma }}^2/u_i)+{\widehat{\alpha }}_i^2}\right]}_{i=1}^p.\text{ (2}\text{.11)}$

The recently proposed KL estimator by Kibria and Lukman [8] is defined as,

${\widehat{\alpha }}_{KL}=NM\widehat{\alpha },\text{ (2}\text{.12)}$

Where, $M=(U-k{{\rm I}}_p)$ and the biasing parameter k of the KL estimator is defined by

${\widehat{k}}_{\min }(KL)=\min \left[\frac{{\widehat{\sigma }}^2}{2{\widehat{\alpha }}_i^2+({\widehat{\sigma }}^2/u_i)}\right]\text{ (2}\text{.13)}$

$MSEM({\widehat{\alpha }}_{KL})={\sigma }^{2}NM{U}^{-1}{M}^{\prime }{N}^{\prime }+[NM-I_p]\alpha {\alpha }^{\prime }[NM-I_p{]}^{\prime }\text{ (2}\text{.14)}$

The new biased (NB) estimator of α is obtained by minimizing $(y-Z\alpha {)}^{\prime }(y-Z\alpha )$ subject to

$(\sqrt{k}\alpha +\sqrt{k}(1+d)\widehat{\alpha }{)}^{\prime }(\sqrt{k}\alpha +\sqrt{k}(1+d)\widehat{\alpha })=c$ where c is a constant,

$(y-Z\alpha {)}^{\prime }(y-Z\alpha )+[(\sqrt{k}\alpha +\sqrt{k}(1+d)\widehat{\alpha }{)}^{\prime }(\sqrt{k}\alpha +\sqrt{k}(1+d)\widehat{\alpha })-c].\text{ (2}\text{.15)}$

Here, k and d are the Lagrangian multipliers.

The solution to (2.15) gives the new estimator as follows:

${\widehat{\alpha }}_{NB}={(U+k{{\rm I}}_p)}^{-1}(U-k(1+d){{\rm I}}_p)\widehat{\alpha }=NB\widehat{\alpha }\text{ (2}\text{.16)}$

where $B=(U-k(1+d){{\rm I}}_p),k\ge 0$ and $0<d<1.$

Moreover, the proposed NB estimator is also obtained by augmenting $-\sqrt{k}(1+d)\widehat{\alpha }=\sqrt{k}\alpha +{\epsilon }^{\prime }$ to equation (2.2) and then using the OLS estimate.

The proposed NB estimator is a general estimator which includes the OLS and the KL estimator:

1 If $k=0$ in ${\widehat{\alpha }}_{NB}$ becomes $\widehat{\alpha }$.

2 If $d=0$ in ${\widehat{\alpha }}_{NB}$ becomes ${\widehat{\alpha }}_{KL}$.

The MSEM of the proposed NB estimator of α is given by

$MSEM({\widehat{\alpha }}_{NB})={\sigma }^{2}NB{U}^{-1}{B}^{\prime }{N}^{\prime }+[NB-I_p]\alpha {\alpha }^{\prime }[NB-I_p{]}^{\prime }.\text{ (2}\text{.17)}$

The lemmas below will be used for theoretical comparisons among estimators in the next section.

Lemma 2.1: [17] Let E be an $n\times n$ positive definite matrix, that is $E>0$ and α be some vector; then, $E-\alpha {\alpha }^{\prime }>0$ if and only if ${\alpha }^{\prime }{E}^{-1}\alpha <1.$

Lemma 2.2: [18] Let ${\alpha }_i=A_{i}y,i=1,2$ be two linear estimators of α. Suppose that $D=Cov({\widehat{\alpha }}_1)-Cov({\widehat{\alpha }}_2)>0,$ where $Cov({\widehat{\alpha }}_i)i=1,2$ be the covariance matrix of ${\widehat{\alpha }}_i$ and $b_i=Bias({\widehat{\alpha }}_i)=(A_{i}X-I)\alpha ,$ i = 1,2. Consequently,

$\Delta ({\widehat{\alpha }}_1-{\widehat{\alpha }}_2)=MSEM({\widehat{\alpha }}_1)-MSEM({\widehat{\alpha }}_2)={\sigma }^{2}D+b_1{b^{\prime }}_1-b_2{b^{\prime }}_2>0\text{ (2}\text{.18)}$

if and only if ${b^{\prime }}_2{[{\sigma }^{2}D+{b^{\prime }}_1{b}_1]}^{-1}{b}_2<1$ where $MSEM({\widehat{\alpha }}_i)=Cov({\widehat{\alpha }}_i)+b_i{b^{\prime }}_i.$

Comparison among the Estimators

Comparison between $\widehat{\alpha }$ and ${\widehat{\alpha }}_{NB}$

Theorem 3.1: $MSEM(\widehat{\alpha })-MSEM({\widehat{\alpha }}_{NB})>0$ if and only if

${\alpha }^{\prime }[NB-I_p{]}^{\prime }[{\sigma }^2(U^{-1}-NB{U}^{-1}{B}^{\prime }{N}^{\prime })][NB-I_p]\alpha <1\text{ (3}\text{.1)}$

Proof:

$\begin{array}{l}D(\widehat{\alpha })-D({\widehat{\alpha }}_{NB})={\sigma }^2(U^{-1}-NB{U}^{-1}{B}^{\prime }{N}^{\prime })\\ ={\sigma }^{2}diag{\left\{\left.\frac{1}{u_i}-\frac{{(u_i-k(1+d))}^2}{u_i{(u_i+k)}^2}\right\}\right.}_{i=1}^p(3.2)\end{array}$

Where $U^{-1}-NB{U}^{-1}{B}^{\prime }{N}^{\prime }$ will be positive definite (pd) if and only if ${(u_i+k)}^2-{(u_i-k(1+d))}^2>0$ or $(u_i+k)-(u_i-k(1+d))>0.$ Clearly, for $k>0$ and $0<d<1,$ $(u_i+k)-(u_i-k(1+d))=k(2+d)>0.$ By Lemma 2.2. The proof is completed.

Comparison between ${\widehat{\alpha }}_k$ and ${\widehat{\alpha }}_{NB}$

Theorem 3.2: $MSEM({\widehat{\alpha }}_k)-MSEM({\widehat{\alpha }}_{NB})>0$ if and only if

${\alpha }^{\prime }[NB-I_p{]}^{\prime }[V_1+(NU-I_p)\alpha {\alpha }^{\prime }(NU-I_p{)}^{\prime }][NB-I_p]\alpha <1\text{ (3}\text{.3)}$

Where, $V_1={\sigma }^2(NU{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime })$

Proof:

$\begin{array}{l}{V}_1={\sigma }^2(NU{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime })\\ ={\sigma }^{2}diag{\left\{\left.\frac{u_i}{{(u_i+k)}^2}-\frac{{(u_i-k(1+d))}^2}{u_i{(u_i+k)}^2}\right\}\right.}_{i=1}^p(3.4)\end{array}$

Where, $NU{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime }$ will be pd if and only if $u_i^2-{(u_i-k(1+d))}^2>0$ or $u_i-(u_i-k(1+d))>0.$ Clearly, for $k>0$ and $0<d<1$, $u_i-(u_i-k(1+d))=k(1+d)>0.$ By Lemma 2.2. The proof is completed.

Comparison between ${\widehat{\alpha }}_d$ and ${\widehat{\alpha }}_{NB}$

Theorem 3.3: $MSEM({\widehat{\alpha }}_d)-MSEM({\widehat{\alpha }}_{NB})>0$ if and only if

${\alpha }^{\prime }[NB-I_p{]}^{\prime }[V_2+(F-I_p)\alpha {\alpha }^{\prime }(F-I_p{)}^{\prime }][NB-I_p]\alpha <1\text{ (3}\text{.5)}$

where, $V_2={\sigma }^2(F{U}^{-1}F-NB{U}^{-1}{B}^{\prime }{N}^{\prime })$

Proof:

$\begin{array}{l}{V}_2={\sigma }^2(F{U}^{-1}F-NB{U}^{-1}{B}^{\prime }{N}^{\prime })\\ ={\sigma }^{2}diag{\left\{\left.\frac{{(u_i+d)}^2}{u_i{(u_i+1)}^2}-\frac{{(u_i-k(1+d))}^2}{u_i{(u_i+k)}^2}\right\}\right.}_{i=1}^p(3.6)\end{array}$

Where, $F{U}^{-1}F-NB{U}^{-1}{B}^{\prime }{N}^{\prime }$ will be pd if and only if ${(u_i+d)}^2{(u_i+k)}^2-{(u_i+1)}^2{(u_i-k(1+d))}^2>0$ or $(u_i+d)(u_i+k)-(u_i+1)(u_i-k(1+d))>0.$ Clearly, for $k>0$ and $0<d<1,$ $(u_i+d)(u_i+k)-(u_i+1)(u_i-k(1+d))=u_i(2k+kd+d-1)+k(2d+1)>0.$ By Lemma 2.2. The proof is completed.

Comparison between ${\widehat{\alpha }}_{KL}$ and ${\widehat{\alpha }}_{NB}$

Theorem 3.4: $MSEM({\widehat{\alpha }}_{KL})-MSEM({\widehat{\alpha }}_{NB})>0$ if and only if

${\alpha }^{\prime }[NB-I_p{]}^{\prime }[V_3+(NM-I_p)\alpha {\alpha }^{\prime }(NM-I_p{)}^{\prime }][NB-I_p]\alpha <1\text{ (3}\text{.7)}$

Where, $V_3={\sigma }^2(NM{U}^{-1}{M}^{\prime }{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime })$

Proof:

$\begin{array}{l}{V}_2={\sigma }^2(NM{U}^{-1}{M}^{\prime }{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime })\\ ={\sigma }^{2}diag{\left\{\left.\frac{{(u_i-k)}^2}{u_i{(u_i+k)}^2}-\frac{{(u_i-k(1+d))}^2}{u_i{(u_i+k)}^2}\right\}\right.}_{i=1}^p(3.8)\end{array}$

Where, $NM{U}^{-1}{M}^{\prime }{N}^{\prime }-NB{U}^{-1}{B}^{\prime }{N}^{\prime }$ will be pd if and only if ${(u_i-k)}^2-{(u_i-k(1+d))}^2>0$ or $(u_i-k)-(u_i-k(1+d))>0.$ Clearly, for $k>0$ and $0<d<1,(u_i-k)-(u_i-k(1+d))=kd>0.$ By Lemma 2.2 The proof is completed.

Selection of the parameters k and d

Different biasing parameters estimators of k and d are proposed in different studies, for example, Hoerl and Kennard [1], Liu [6], [16,19-23], among others.

So the optimal values of k and d for the proposed NB estimator is going to be found. At first, by minimizing the equation m, we obtain the optimal value of k when d is fixed as

$MSEM({\widehat{\alpha }}_{NB})=E(({\widehat{\alpha }}_{NB}-\alpha {)}^{\prime }({\widehat{\alpha }}_{NB}-\alpha )),$

$m=tr(MSEM({\widehat{\alpha }}_{NB})),$

$m={\sigma }^2{\displaystyle \sum _{i=1}^p\frac{{(u_i-k(1+d))}^2}{u_i{(u_i+k)}^2}}+k^2{(2+d)}^2{\displaystyle \sum _{i=1}^p\frac{{\alpha }_i^2}{{(u_i+k)}^2}}\text{ (3}\text{.9)}$

Differentiating m with respect to k and setting $(\partial m/\partial k)=0,$ we get

$k=\frac{u_i{\sigma }^2}{(d+1)(u_i{\alpha }_i^2+{\sigma }^2)+u_i{\alpha }_i^2}.\text{ (3}\text{.10)}$

Then, the estimated optimal value of k is given as follows:

$\widehat{k}=\frac{u_i{\widehat{\sigma }}^2}{(d+1)(u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2)+u_i{\widehat{\alpha }}_i^2},\text{ (3}\text{.11)}$

and,

${\widehat{k}}_{\min }(NB)=\min {\left\{\frac{u_i{\widehat{\sigma }}^2}{(d+1)(u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2)+u_i{\widehat{\alpha }}_i^2}\right\}}_{i=1}^p.\text{ (3}\text{.12)}$

Also, the optimal value of d will be found by differentiating m with respect to d when k is fixed and setting $(\partial m/\partial d)=0,$ we get

$d=\frac{u_i{\sigma }^2-(2u_i{\alpha }_i^2+{\sigma }^2)k}{k(u_i{\alpha }_i^2+{\sigma }^2)}.\text{ (3}\text{.13)}$

Then, the estimated optimal d with the unbiased estimators is given by

$\widehat{d}=\frac{u_i{\widehat{\sigma }}^2-(2u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2)\widehat{k}}{\widehat{k}(u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2)},\text{ (3}\text{.14)}$

And

${\widehat{d}}_{\min }(NB)=\min {\left\{\frac{u_i{\widehat{\sigma }}^2-(2u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2){\widehat{k}}_{\min }(NB)}{{\widehat{k}}_{\min }(NB)(u_i{\widehat{\alpha }}_i^2+{\widehat{\sigma }}^2)}\right\}}_{i=1}^p.\text{ (3}\text{.15)}$

The parameters k and d estimators selection in ${\widehat{\alpha }}_{NB}$ are found iteratively as follows:

1. Find an initial estimate of d using $\widehat{d}=\min (\frac{{\widehat{\sigma }}^2}{{\widehat{\alpha }}_i^2}).$

2. Determine ${\widehat{k}}_{\min }(NB)$ from (3.12) using $\widehat{d}$ in step 1.

3. Estimate ${\widehat{d}}_{\min }(NB)$ in (3.15) by using ${\widehat{k}}_{\min }(NB)$ in step 2.

4. If ${\widehat{d}}_{\min }(NB)>1$ or ${\widehat{d}}_{\min }(NB)<0,$ use ${\widehat{d}}_{\min }(NB)=\widehat{d}.$

Simulation Study

A Monte Carlo simulation study is performed to show the performance of the NB estimator over some existing estimators. It contains two parts: (i) Simulation technique and (ii) Results discussion.

Simulation technique

Using the equation below, we generate the explanatory variables (see, Gibbons [24] and Kibria [19]):

$x_{ij}={(1-{\rho }^2)}^{1/2}{z}_{ij}+\rho z_{i,p+1},i=1,2,\mathrm{...},n,j=1,2,\mathrm{...},p\text{ (4}\text{.1)}$

where $z_{ij}$ are independent standard normal pseudo-random numbers and $\rho$ is the correlation between any two explanatory variables and $\rho$ here has two values 0.9 and 0.99. The n observations for the response variable y are gotten by the following equation:

$y_i={\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+{\beta }_3{x}_{i3}+e_i,i=1,2,\dots ,n\text{ (4}\text{.2)}$

where $e_i$ are $i.i.dN(0,{\sigma }^2).$ The values of β are obtained as ${\beta }^{\prime }\beta =1$ [25]. Also, we choose the values of the biasing parameters of the estimators as $k=d=0.1,0.2,\dots ,0.9$ by following the work of Wichern and Churchill [26] and Kan, et al. [27] is that when k lies between 0 and 1, the ORR estimator performs better. The Monte Carlo simulation study replication is 1000 times for n = 50 and 100 and ${\sigma }^2=$ 1, 25, and 100. For each replicate, we calculate the mean square error (MSE) of the estimators using the next equation:

$MSE({\alpha }^{*})=\frac{1}{1000}{\displaystyle \sum _{j=1}^{1000}({\alpha }_{ij}^{*}-{\alpha }_i{)}^{\prime }({\alpha }_{ij}^{*}-{\alpha }_i)}\text{ (4}\text{.3)}$

where ${\alpha }_{ij}^{*}$ is the estimator and ${\alpha }_i$ is the true parameter. The estimated MSEs of the estimators are shown in Table 1, Table 2, Table 3 and Table 4 for (ρ = 0.90 and n = 50), (ρ = 0.99 and n = 50), (ρ = 0.90 and n = 100), and (ρ = 0.99 and n = 100), respectively.

Simulation results discussions

From Table 1, Table 2, Table 3 and Table 4, we observed that, when the factors $\sigma$ and $\rho$ are going to increase, the estimated MSE values are also going to increase, while n is going to increase, the estimated MSE values are going to decrease. Also, the OLS estimator is performing the worst for all cases in presence of the multicollinearity. Moreover, the simulation results show that, the proposed NB estimator is performing better than the other estimators for most of cases. The Liu estimator gives better results in the MSE values when $k=d=0.1,0.2$ i.e. when the biasing parameters are near to zero. For ρ = 0.9: The condition number (CN) is approximately around 5 and the variance inflation factors (VIFs) are around 4 to 6 such that it is observed that a close agreement of the proposed NB (although better results) with the KL estimator for low values of k = d and a better performance as k = d increases for a fixed value of σ. This improvement increases with the increase of the value of σ and an even better improvements is observed if ρ = 0.99 where the CN and the VIFs become larger and are approximately around 15 and around 36 to 61, respectively. So, the proposed NB estimator works better for the strong correlated explanatory variables. So, the performance of the proposed NB estimator almost depends on the value of ρ, σ, the biasing parameters k and d, and the true parameter. Thus, simulation results are consistent with the theoretical results.

Application

To illustrate the theoretical and simulation results of this paper, we consider a real life data in this section. The Portland cement data was originally adopted by Woods, et al. [28]. This data was also analyzed by many researchers, for examples, [29,30]; Lukman, et al. [14] and recently by Kibria and Lukman [8], among others. And this data is analyzed here to explain the performance of the proposed NB estimator and the other existing estimators. The regression model for this data is given by

$y_i={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4+{\epsilon }_i.\text{ (5}\text{.1)}$

For knowing more details about this data, we refer Woods, et al. [28].

To show the existence of the multicollinearity, different measures are calculated as the VIFs which are 38.50, 254.42, 46.87 and 282.51, Eigenvalues of $X\text{'}X$ are 44676.206, 5965.422, 809.952 and 105.419 and the CN of $X\text{'}X$ approximately equals 20.58. The VIFs, the eigenvalues and the CN tell us that there is a severe multicollinearity in the data. Also, $\widehat{\sigma }$ is equal to 2.446. The correlation coefficients matrix of the explanatory variables are presented in Table 5 such that there is a significant and strong relationship among the following explanatory variables: $X_1$ and $X_3,X_2$ and $X_4.$ Then, the estimated parameters and the MSE values of the estimators are presented in Table 6. It appears from Table 6 that the proposed NB estimator is performing the best where it is giving an obvious improvement over all existing estimators and a little improvement over the KL estimator in which this is consistent with the simulation results because $\widehat{\sigma }$ here is small and not all the explanatory variables are significantly or strongly correlated at the same degree of correlation even though the data has high CN and VIFs.

Some Concluding Remarks

In this paper, we proposed a new biased (NB) estimator for handling the multicollinearity problem in the multiple linear regression models. Some existing estimators are the special case of the proposed estimator. The proposed NB estimator is compared theoretically with the Ordinary least squares (OLS) estimator, the Ordinary ridge regression (ORR) estimator, the Liu estimator and the Kibria-Lukman (KL) estimator, and then the biasing parameters d and k of the NB estimator are derived. A Monte Carlo simulation study is performed for comparing the performance of the OLS, ORR, Liu, KL and the proposed NB estimators. The main finding of this simulation is that the proposed NB estimator performed better than the above mentioned estimators under some conditions. A real-life data is analyzed to support the findings of the paper.

Acknowledgments

Authors are grateful to referees for their valuable comments and suggestions, which certainly improved the quality and presentation of the paper. Author B M Golam Kibria would like to dedicate this paper to his Ph. D. supervisor, Late Professor M Safiul Haq for his guidance and constant inspiration during his stay at The University of Westen Ontario and finally to achieve this present position.

References

1. Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12: 55-67.
2. Stein C (1956) Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceedings of the Third Berkley Symposium on Mathematical Statistics and Probability 1: 197-206.
3. Massy WF (1965) Principal components regression in exploratory statistical research. Journal of the American Statistical Association 60: 234-256.
4. Mayer LS, Willke TA (1973) On biased estimation in linear models. Technometrics 15: 497-508.
5. Swindel BF (1976) Good ridge estimators based on prior information. Communications in Statistics-Theory and Methods 5: 1065-1075.
6. Liu K (1993) A new class of biased estimate in linear regression. Communication in Statistics-Theory and Methods 22: 393-402.
7. Akdeniz F, Kaçiranlar S (1995) On the almost unbiased generalized liu estimator and unbiased estimation of the bias and mse. Communications in Statistics-Theory and Methods 24: 1789-1797.
8. Kibria BMG, Lukman AF (2020) A new ridge-type estimator for the linear regression model: Simulations and applications. Hindawi Scienti?ca.
9. Ozkale MR, Kaçiranlar S (2007) The restricted and unrestricted two-parameter estimators. Communications in Statistics-Theory and Methods 36: 2707-2725.
10. Sakallioglu S, Kaçiranlar S (2008) A new biased estimator based on ridge estimation. Statistical Papers 49: 669-689.
11. Yang H, Chang X (2010) A new two-parameter estimator in linear regression. Communications in Statistics-Theory and Methods 39: 923-934.
12. Roozbeh M (2018) Optimal QR-based estimation in partially linear regression models with correlated errors using GCV criterion. Computational Statistics & Data Analysis 117: 45-61.
13. Akdeniz F, Roozbeh M (2019) Generalized difference-based weighted mixed almost unbiased ridge estimator in partially linear models. Statistical Papers 60: 1717-1739.
14. Lukman AF, Ayinde K, Binuomote S, Clement OA (2019) Modified ridge-type estimator to combat multicollinearity: Application to chemical data. Journal of Chemometrics 33: e3125.
15. Lukman AF, Ayinde K, Sek SK, Adewuyi E (2019) A modi?ed new two-parameter estimator in a linear regression model. Modelling and Simulation in Engineering.
16. Hoerl AE, Kannard RW, Baldwin KF (1975) Ridge regression: Some simulations. Communications in Statistics 4: 105-123.
17. Farebrother RW (1976) Further results on the mean square error of ridge regression. Journal of the Royal Statistical Society: Series B (Methodological) 38: 248-250.
18. Trenkler G, Toutenburg H (1990) Mean squared error matrix comparisons between biased estimators-an overview of recent results. Statistical Papers 31: 165-179.
19. Kibria BMG (2003) Performance of some new ridge regression estimators. Communications in Statistics-Simulation and Computation 32: 419-435.
20. Kibria BMG, Banik S (2016) Some ridge regression estimators and their performances. Journal of Modern Applied Statistical Methods 15: 206-238.
21. Lukman AF, Ayinde K (2017) Review and classi?cations of the ridge parameter estimation techniques. Hacettepe Journal of Mathematics and Statistics 46: 953-967.
22. Mansson K, Kibria BMG, Shukur G (2015) Performance of some weighted Liu estimators for logit regression model: An application to Swedish accident data. Communications in Statistics Theory and Methods 44: 363-375.
23. Khalaf G, Shukur G (2005) Choosing ridge parameter for regression problems. Communications in Statistics Theory and Methods 34: 1177-1182.
24. Gibbons DG (1981) A simulation study of some ridge estimators. Journal of the American Statistical Association 76: 131-139.
25. Newhouse JP, Oman SD (1971) An evaluation of ridge estimators. A report prepared for United States air force project RAND.
26. Wichern DW, Churchill GA (1978) A comparison of ridge estimators. Technometrics 20: 301-311.
27. Kan B, Alpu O, Yazici B (2013) Robust ridge and robust liu estimator for regression based on the LTS estimator. Journal of Applied Statistics 40: 644-655.
28. Woods H, Steinour HH, Starke HR (1932) Effect of composition of Portland cement on heat evolved during hardening. Ind Eng Chem 24: 1207-1214.
29. Kaciranlar S, Sakallioglu S, Akdeniz F, Styan GPH, Werner HJ (1999) A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on portland cement. Sankhya: The Indian Journal of Statistics, Series B 61: 443-459.
30. Li Y, Yang H (2012) A new Liu-type estimator in linear regression model. Statistical Papers 53: 427-437.

---