# best linear unbiased estimator proof

I got all the way up to 11.3.18 and then the next part stuck me. \$\begingroup\$ It is the best filter in the sense of minimizing the MSE; However, it is not necessarily unbiased. If all Gauss-Markov assumptions are met than the OLS estimators alpha and beta are BLUE – best linear unbiased estimators: best: variance of the OLS estimator is minimal, smaller than the variance of any other estimator linear: if the relationship is not linear – OLS is not applicable.  Rao, C. Radhakrishna (1967). Now, talking about OLS, OLS estimators have the least variance among the class of all linear unbiased estimators. Best Linear Unbiased Estimators. Properties of Least Squares Estimators Each ^ iis an unbiased estimator of i: E[ ^ i] = i; V( ^ i) = c ii˙2, where c ii is the element in the ith row and ith column of (X0X) 1; Cov( ^ i; ^ i) = c ij˙2; The estimator S2 = SSE n (k+ 1) = Y0Y ^0X0Y n (k+ 1) is an unbiased estimator of ˙2. Proof: E[b0Y] = b0Xβ, which equals a0β for all β if and only if a = X0b. 11 BLUE. Journal of Statistical Planning and Inference, 88, 173--179. The Gauss-Markov theorem states that if your linear regression model satisfies the first six classical assumptions, then ordinary least squares regression produces unbiased estimates that have the smallest variance of all possible linear estimators.. Deﬁnition: A linear combination a0β is estimable if it has a linear unbiased estimate, i.e., E[b0Y] = a0β for some b for all β. Lemma 10.2.1: (i) a0β is estimable if and only if a ∈ R(X0). The proof for this theorem goes way beyond the scope of this blog post. Puntanen, Simo; Styan, George P. H. and Werner, Hans Joachim (2000). In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. Goldsman — ISyE 6739 12.2 Fitting the Regression Line Then, after a little more algebra, we can write βˆ1 = Sxy Sxx Fact: If the εi’s are iid N(0,σ2), it can be shown that βˆ0 and βˆ1 are the MLE’s for βˆ0 and βˆ1, respectively. To show this property, we use the Gauss-Markov Theorem. sometimes called best linear unbiased estimator Estimation 7–21. Proof: An estimator is “best” in a class if it has smaller variance than others estimators in the same class. Restrict estimate to be linear in data x 2. MMSE with linear measurements consider speciﬁc case y = Ax+v, x ∼ N(¯x, ... proof: multiply is the Best Linear Unbiased Estimator (BLUE) if εsatisﬁes (1) and (2). If the estimator has the least variance but is biased – it’s again not the best! Find the best one (i.e. Two matrix-based proofs that the linear estimator Gy is the best linear unbiased estimator. (ii) If a0β is estimable, there is … Restrict estimate to be unbiased 3. (See text for easy proof). We now consider a somewhat specialized problem, but one that fits the general theme of this section. A vector of estimators is BLUE if it is the minimum variance linear unbiased estimator. We are restricting our search for estimators to the class of linear, unbiased ones. A property which is less strict than efficiency, is the so called best, linear unbiased estimator (BLUE) property, which also uses the variance of the estimators. If the estimator is both unbiased and has the least variance – it’s the best estimator. with minimum variance) \$\endgroup\$ – Dovid Apr 23 '18 at 14:47 ... they go on to prove the best linear estimator property for the Kalman filter in Theorem 2.1, and the proof does not … Except for Linear Model case, the optimal MVU estimator might: 1. not even exist 2. be difficult or impossible to find ⇒ Resort to a sub-optimal estimate BLUE is one such sub-optimal estimate Idea for BLUE: 1.