Recall that violations of model assumptions are more likely at remote points, and these violations may be hard to detect from inspection of the ordinary residuals because their residuals will usually be smaller. Points that are outlying in the \(x\)-direction are known as leverage points. Influential points are not only remote in terms of the specific values for the regressors, but the observed response is not consistent with the values that would be predicted based on only the other data points. It is important to find these influential points and assess their impact on the model.
Below gives an example of an influential point. The seventh point in the data set is outlying in the \(x\)-direction, and it’s response value is not consistent with the regression line based on the other six observations:
set.seed(330)
x=c(rnorm(6),2.5)
y=x*2+3
y[7]=y[7]+7
plot(x,y,pch=22,bg=1)
a=lm(y~x)
curve(a$coefficients[1]+x*a$coefficients[2],add=T,lwd=3)
curve(x*2+3,add=T,col=2,lwd=3)
a2=lm(y[-7]~x[-7])
curve(a2$coefficients[1]+x*a2$coefficients[2],add=T,lwd=3,col='blue',lty=2)
a$coefficients(Intercept) x
2.048937 3.979977 Sometimes we find that a regression coefficient may have a sign that does not make engineering or scientific sense, a regressor known to be important may be statistically insignificant, or a model that fits the data well and that is logical from an application – environment perspective may produce poor predictions. These situations may be the result of one or, perhaps, a few influential observations.
Recall the hat matrix \(H=X(X^\top X)^{-1}X^\top\), as well as that it holds that \({\textrm{Var}}\left[\hat\epsilon\right]=\sigma^2(I-H)\) and \({\textrm{Var}}\left[\hat Y\right]=\sigma^2 H\). Note that \(h_{ij}\) can be interpreted as the amount of leverage exerted by the \(ith\) observation \(y_i\) on the \(jth\) fitted value \(\hat y_j\). We usually focus attention on the diagonal elements \(h_{ii}\) of the hat matrix \(H\), which may be written as \[h_{ii}=x_i^\top (X^\top X)^{-1} x_i,\] where \(X_i^\top\) is the \(i\)th row of \(X\). The hat matrix diagonal is a standardized measure of the distance of the \(i\)th observation from the center (or centroid) of the \(x\)-space. Therefore, large values of \(h_{ii}\) implies that \(x_i\) is potentially influential. Furthermore, note that \(rank(H)=p\) since the trace of an idempotent matrix equals its rank, which means that \(\bar h= p/n\). It follows that values well above \(p/n\), say \(h_{ii}>2p/n\), can be called leverage points.
X=as.matrix(cbind(rep(1,length(x)),x))
# or
X=model.matrix(a)
hat=X%*%solve(t(X)%*%X)%*%t(X)
diag(hat) 1 2 3 4 5 6 7
0.2027453 0.2288737 0.2596869 0.1751432 0.1735495 0.3887329 0.5712686 p=2
n=7
diag(hat)>2*p/n 1 2 3 4 5 6 7
FALSE FALSE FALSE FALSE FALSE FALSE FALSE Cook’s Distance is one way to incorporate both the \(X\) and \(Y\) values into an outlyingness measure:
\[ D_i\left(X^{\top} X, p, MSE\right) \equiv D_i=\frac{\left(\hat{\beta}_{(i)}-\hat{\beta}\right)^{\top} X^{\top} X\left(\hat{\beta}_{(i)}-\hat{\beta}\right)}{p MSE}, \ i\in [n], \] where \(\hat{\beta}_{(i)}\) is the OLS estimator with the \(i\)th point removed.Large values of Cook’s distance signal a leverage point.
What do we mean by a large value? We can compare \(D_i\) to the 50th percentile of the \(F_{p,n-p}\) distribution. This gives the interpretation that deleting the \(i\)th point moves the estimate to the boundary of a 50% confidence interval. \(F_{p,n-p}\approx 1\), and so usually take \(D_i\geq 1\) to be large.
Observe that \[ D_i=\frac{r_i^2}{p} \frac{\operatorname{Var}\left(\hat{Y}_i\right)}{\operatorname{Var}\left(\hat\epsilon_i\right)}=\frac{r_i^2}{p} \frac{h_{i i}}{1-h_{i i}}, \quad i=1,2, \ldots, n,\] where it is important to recall that \(r_i\) is the studentized residual. Now, the quantity \(\frac{h_{i i}}{1-h_{i i}}\) can be shown to be the distance from the vector \(x_i\) to the centroid of the remaining data. Therefore, \(D_i\) is the product of outlyingness in both the \(X\) and \(Y\) directions. We may also write \(D_i\) as \[D_i=\frac{\left\lVert\hat{y}_{(i)}-\hat{y}\right\rVert^2}{p MSE},\] which allows for the interpretation: The Cook’s distance of the \(i\)th point is the normalized distance between the fitted value with and without point \(i\).
#cut off
cooks.distance(a) 1 2 3 4 5 6
0.1708029420 0.2516095165 0.0180669722 0.0009569213 0.0011772793 0.2002829110
7
3.3311562309 cooks.distance(a)>1 1 2 3 4 5 6 7
FALSE FALSE FALSE FALSE FALSE FALSE TRUE df=data.frame(cbind(y,x))
df[cooks.distance(a)>1,] y x
7 15 2.5A more modern approach and nonparametric approach to outlier detection is through data depth. A data depth function gives meaning to centrality, order and outlyingness in spaces beyond \(\mathbb{R}\). A data depth function is a function which takes a sample and a point, and returns how central the point is, with respect to the sample. Depth functions can be written as \({\textrm{D}}\colon \mathbb{R}^{d}\times \text{Sample} \rightarrow \mathbb{R}^+\). There are different definitions of depth, so I will give a few.
Let \(S^{d-1}= \{x\in \mathbb{R}^{d}\colon \left\lVert x\right\rVert=1\}\) be the set of unit vectors in \(\mathbb{R}^{d}\), let \(\mathbb{X}_{n}=\{(Y_1,X_{1,1},\ldots,X_{1,p-1}),\ldots, (Y_n,X_{n,1},\ldots,X_{n,p-1})\}\), let \(\mathbb{X}_{n}^\top u\) be \(\mathbb{X}_{n}\) projected onto \(u\in S^{d-1}\) and let \(\widehat F_u\) be the empirical CDF with respect to \(\mathbb{X}_{n}^\top u\).
The halfspace depth \({\textrm{D}}_H\) of a point \(x\in \mathbb{R}^{d}\) with respect to a distribution \(F\) over \(\mathbb{R}^{d}\) is \[ {\textrm{D}}_H(x;F)=\inf_{u\in S^{d-1}} \widehat F_u(x^\top u)\wedge (1-F_u(x^\top u))=\inf_{u\in S^{d-1}} F_u(x^\top u). \]
Given a translation and scale equivariant location estimate \(\mu\) and a translation and scale invariant scale estimate \(\sigma\), the outlyingness at \(x\in\mathbb{R}^{d}\) is defined as \[O(x)=\sup _{u\in S^{d-1}} \frac{\left|x^\top u-\mu(\mathbb{X}_{n}^\top u)\right|}{\sigma(\mathbb{X}_{n}^\top u)}.\] Define projection depth as \[{\textrm{D}}_p(x)=(1+O(x))^{-1}.\]
In order to detect outliers, we look for observations that have low depth. See, continuing our toy example:
# install.packages('ddalpha')
depths=ddalpha::depth.projection(cbind(x,y),cbind(x,y))
depths[1] 0.276409011 0.255272074 0.500000000 0.973754328 0.973046927 0.338954415
[7] 0.001795558depths<0.015[1] FALSE FALSE FALSE FALSE FALSE FALSE TRUEExample 7.1 Recall example Example 6.6. Check for leverage and influential points in the proposed models. Compute all three measures of leverage/influence/outlyingness introduced in this lesson. What do you find?
I will load in the data below:
We can now analyze the data:
# df=df[df$Lotsize<70000,]
custom_palette <- c(
"#1f77b4", "#ff7f0e", "#2ca02c", "#d62728",
"#9467bd", "#8c564b", "#e377c2", "#7f7f7f",
"#bcbd22", "#17becf", "#393b79",
"#8c6d31", "#9c9ede", "#637939", "#eb348f"
)
# Our model from the previous lecture
df=df_clean2[-which.max(df_clean2$Lotsize),]
df=df[df$Lotsize>0,]
df['district_3']=df['District']==3
df['district_4']=df['District']==4
df['district_15']=df['District']==15
model2=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df)
# Compute residuals
student_res2=rstudent(model2)
summ2=summary(model2); summ2
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df)
Residuals:
Min 1Q Median 3Q Max
-303393 -24422 -942 23352 719561
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.129e+06 3.625e+04 -31.143 < 2e-16 ***
District 5.695e+03 7.701e+01 73.950 < 2e-16 ***
ExtwallBlock -3.944e+03 3.688e+03 -1.069 0.284879
ExtwallBrick 6.725e+03 7.245e+02 9.281 < 2e-16 ***
ExtwallFiber-Cement 3.847e+04 3.754e+03 10.246 < 2e-16 ***
ExtwallFrame -4.986e+03 9.786e+02 -5.095 3.51e-07 ***
ExtwallMasonry / Frame 3.751e+03 1.711e+03 2.192 0.028404 *
ExtwallPrem Wood 1.894e+04 5.596e+03 3.385 0.000714 ***
ExtwallStone 1.162e+04 1.531e+03 7.594 3.22e-14 ***
ExtwallStucco 3.514e+03 2.149e+03 1.635 0.102070
Stories1 3.684e+02 1.047e+04 0.035 0.971935
Stories1.5 1.317e+04 1.046e+04 1.259 0.207954
Stories2 1.451e+04 1.042e+04 1.392 0.163802
Year_Built 4.629e+02 1.606e+01 28.823 < 2e-16 ***
Fin_sqft 6.048e+01 1.080e+00 55.986 < 2e-16 ***
Units1 5.778e+04 7.389e+03 7.820 5.48e-15 ***
Units2 -9.774e+03 7.386e+03 -1.323 0.185735
Units3 -4.087e+04 7.940e+03 -5.147 2.67e-07 ***
Bdrms0 1.155e+05 1.854e+04 6.227 4.82e-10 ***
Bdrms1 7.894e+04 1.016e+04 7.766 8.41e-15 ***
Bdrms2 8.331e+04 9.086e+03 9.170 < 2e-16 ***
Bdrms3 8.791e+04 9.033e+03 9.733 < 2e-16 ***
Bdrms4 7.905e+04 9.001e+03 8.782 < 2e-16 ***
Bdrms5 7.507e+04 9.002e+03 8.340 < 2e-16 ***
Bdrms6 6.149e+04 9.006e+03 6.828 8.80e-12 ***
Bdrms7 2.462e+04 9.572e+03 2.573 0.010100 *
Bdrms8 1.242e+04 1.008e+04 1.233 0.217672
Fbath0 -1.936e+04 1.388e+04 -1.395 0.162964
Fbath1 -1.050e+04 9.795e+03 -1.072 0.283695
Fbath2 7.289e+03 9.751e+03 0.748 0.454721
Fbath3 2.935e+04 9.641e+03 3.044 0.002339 **
Fbath4 6.757e+04 1.025e+04 6.595 4.35e-11 ***
Lotsize 1.419e+00 1.131e-01 12.545 < 2e-16 ***
Sale_date 4.839e+00 2.603e-01 18.587 < 2e-16 ***
district_3TRUE 9.189e+05 1.373e+05 6.693 2.23e-11 ***
district_4TRUE 1.088e+06 2.504e+05 4.346 1.39e-05 ***
district_15TRUE 4.475e+05 1.212e+05 3.692 0.000223 ***
Year_Built:district_3TRUE -5.100e+02 7.205e+01 -7.078 1.51e-12 ***
Lotsize:district_3TRUE 1.227e+01 4.803e-01 25.549 < 2e-16 ***
Lotsize:district_4TRUE -3.796e+00 1.569e+00 -2.420 0.015512 *
Year_Built:district_4TRUE -5.558e+02 1.306e+02 -4.255 2.09e-05 ***
Year_Built:district_15TRUE -2.829e+02 6.295e+01 -4.493 7.05e-06 ***
Lotsize:district_15TRUE 3.140e+00 1.213e+00 2.589 0.009620 **
Fin_sqft:district_3TRUE 6.621e+01 1.517e+00 43.639 < 2e-16 ***
Fin_sqft:district_4TRUE -2.049e+01 4.181e+00 -4.901 9.61e-07 ***
Fin_sqft:district_15TRUE -1.161e+01 3.163e+00 -3.672 0.000241 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 43870 on 24429 degrees of freedom
Multiple R-squared: 0.7301, Adjusted R-squared: 0.7296
F-statistic: 1468 on 45 and 24429 DF, p-value: < 2.2e-16summ2$adj.r.squared[1] 0.7295574# Compute residual analysis
MSE2=summ2$sigma^2
qqnorm(student_res2,pch=22,bg=1)
abline(0,1)
hist(student_res2,freq=F,breaks=100)
curve(dnorm(x,0,1),add=T)
# hist(student_res2,freq=F,breaks=100)
# curve(dnorm(x,0,1),add=T)
plot(model2$fitted.values,student_res2,pch=22,bg=1)
abline(h=0)
# First measure
X=model.matrix(model2)
hat=X%*%solve(t(X)%*%X)%*%t(X)
# diag(hat)
p=ncol(X)
n=nrow(X)
out_1=which(diag(hat)>2*p/n)
plot(sort(diag(hat)[out_1]))
abline(h=2*p/n)
# I would still look at those after the elbow
# Cooks distances
CDS=cooks.distance(model2)
plot(sort(CDS,T)[1:100])
which(CDS>1)22532
22398 max(CDS)[1] 1.075144df[CDS>1,] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSE# I would still look at those two values that are far from the other distances
# I would still look at those before the elbow
# We may only look at numeric values for depth functions - so we can either
numer=NULL
for(i in names(df)){
if(!is.factor(df[1,i])){
numer=c(numer,i)
}
}
numer[1] "District" "Year_Built" "Fin_sqft" "Lotsize" "Sale_date"
[6] "Sale_price" "district_3" "district_4" "district_15"df_mat=as.matrix(df[,numer])
depths=ddalpha::depth.projection(df_mat,df_mat)
which(depths<0.1)[1:10] [1] 5 130 326 345 471 473 593 637 669 673plot(sort(depths,F)[1:100])
# Notice there is a crack around 0.035, I would look at those observations
plot(sort(depths,F)[1:1000])
plot(sort(depths,F))
# OR
depths=ddalpha::depth.projection(cbind(X,df$Sale_price),cbind(X,df$Sale_price))
which(depths<0.1)[1:10] [1] 2 3 4 5 7 13 15 16 17 20plot(sort(depths,F)[1:100])
which.max(diag(hat))22532
22398 which.max(CDS)22532
22398 which.min(depths)[1] 22398# Hugely expensive home!
df[which.min(depths),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEdf[which.max(CDS),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEdf[which.max(diag(hat)),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEmodel3=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df[-(order(depths)[1:100]),])
# OR
model4=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df[-(order(CDS,decreasing = T)[1:100]),])
# Compare
s=summary(model3)
summary(model3)
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df[-(order(depths)[1:100]),
])
Residuals:
Min 1Q Median 3Q Max
-290250 -24346 -997 23105 620324
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.065e+06 3.638e+04 -29.283 < 2e-16 ***
District 5.691e+03 7.490e+01 75.985 < 2e-16 ***
ExtwallBlock -2.315e+03 3.593e+03 -0.644 0.519354
ExtwallBrick 6.732e+03 7.076e+02 9.514 < 2e-16 ***
ExtwallFiber-Cement 3.976e+04 3.678e+03 10.810 < 2e-16 ***
ExtwallFrame -4.742e+03 9.543e+02 -4.969 6.76e-07 ***
ExtwallMasonry / Frame 4.915e+03 1.670e+03 2.943 0.003251 **
ExtwallPrem Wood 2.085e+04 5.481e+03 3.805 0.000142 ***
ExtwallStone 9.842e+03 1.494e+03 6.588 4.56e-11 ***
ExtwallStucco 3.573e+03 2.107e+03 1.696 0.089908 .
Stories1 -1.553e+04 1.085e+04 -1.431 0.152483
Stories1.5 -2.675e+03 1.084e+04 -0.247 0.805030
Stories2 -1.333e+01 1.081e+04 -0.001 0.999016
Year_Built 4.582e+02 1.572e+01 29.141 < 2e-16 ***
Fin_sqft 5.845e+01 1.062e+00 55.070 < 2e-16 ***
Units1 5.671e+04 7.210e+03 7.866 3.81e-15 ***
Units2 -9.821e+03 7.206e+03 -1.363 0.172920
Units3 -4.600e+04 7.764e+03 -5.925 3.16e-09 ***
Bdrms0 1.029e+05 1.816e+04 5.663 1.50e-08 ***
Bdrms1 6.353e+04 1.015e+04 6.261 3.88e-10 ***
Bdrms2 6.916e+04 9.114e+03 7.589 3.35e-14 ***
Bdrms3 7.409e+04 9.062e+03 8.176 3.07e-16 ***
Bdrms4 6.567e+04 9.030e+03 7.272 3.64e-13 ***
Bdrms5 6.270e+04 9.027e+03 6.946 3.86e-12 ***
Bdrms6 4.787e+04 9.025e+03 5.304 1.14e-07 ***
Bdrms7 1.841e+04 9.632e+03 1.911 0.056024 .
Bdrms8 4.831e+03 1.021e+04 0.473 0.635941
Fbath0 -4.226e+04 1.498e+04 -2.821 0.004790 **
Fbath1 -3.208e+04 1.153e+04 -2.783 0.005394 **
Fbath2 -1.382e+04 1.150e+04 -1.202 0.229410
Fbath3 1.051e+04 1.143e+04 0.920 0.357557
Fbath4 3.571e+04 1.206e+04 2.960 0.003076 **
Lotsize 1.579e+00 1.149e-01 13.740 < 2e-16 ***
Sale_date 4.779e+00 2.536e-01 18.842 < 2e-16 ***
district_3TRUE 8.389e+05 1.377e+05 6.092 1.13e-09 ***
district_4TRUE 1.081e+06 2.437e+05 4.436 9.20e-06 ***
district_15TRUE 5.778e+05 1.300e+05 4.445 8.84e-06 ***
Year_Built:district_3TRUE -4.640e+02 7.239e+01 -6.410 1.48e-10 ***
Lotsize:district_3TRUE 1.797e+01 7.709e-01 23.316 < 2e-16 ***
Lotsize:district_4TRUE -2.561e+00 1.782e+00 -1.437 0.150679
Year_Built:district_4TRUE -5.590e+02 1.274e+02 -4.389 1.14e-05 ***
Year_Built:district_15TRUE -3.680e+02 6.825e+01 -5.392 7.05e-08 ***
Lotsize:district_15TRUE 1.008e+01 1.915e+00 5.265 1.41e-07 ***
Fin_sqft:district_3TRUE 4.999e+01 1.684e+00 29.680 < 2e-16 ***
Fin_sqft:district_4TRUE -1.762e+01 4.585e+00 -3.844 0.000121 ***
Fin_sqft:district_15TRUE -9.735e+00 3.739e+00 -2.604 0.009230 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 42660 on 24329 degrees of freedom
Multiple R-squared: 0.7029, Adjusted R-squared: 0.7024
F-statistic: 1279 on 45 and 24329 DF, p-value: < 2.2e-16summary(model2)
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df)
Residuals:
Min 1Q Median 3Q Max
-303393 -24422 -942 23352 719561
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.129e+06 3.625e+04 -31.143 < 2e-16 ***
District 5.695e+03 7.701e+01 73.950 < 2e-16 ***
ExtwallBlock -3.944e+03 3.688e+03 -1.069 0.284879
ExtwallBrick 6.725e+03 7.245e+02 9.281 < 2e-16 ***
ExtwallFiber-Cement 3.847e+04 3.754e+03 10.246 < 2e-16 ***
ExtwallFrame -4.986e+03 9.786e+02 -5.095 3.51e-07 ***
ExtwallMasonry / Frame 3.751e+03 1.711e+03 2.192 0.028404 *
ExtwallPrem Wood 1.894e+04 5.596e+03 3.385 0.000714 ***
ExtwallStone 1.162e+04 1.531e+03 7.594 3.22e-14 ***
ExtwallStucco 3.514e+03 2.149e+03 1.635 0.102070
Stories1 3.684e+02 1.047e+04 0.035 0.971935
Stories1.5 1.317e+04 1.046e+04 1.259 0.207954
Stories2 1.451e+04 1.042e+04 1.392 0.163802
Year_Built 4.629e+02 1.606e+01 28.823 < 2e-16 ***
Fin_sqft 6.048e+01 1.080e+00 55.986 < 2e-16 ***
Units1 5.778e+04 7.389e+03 7.820 5.48e-15 ***
Units2 -9.774e+03 7.386e+03 -1.323 0.185735
Units3 -4.087e+04 7.940e+03 -5.147 2.67e-07 ***
Bdrms0 1.155e+05 1.854e+04 6.227 4.82e-10 ***
Bdrms1 7.894e+04 1.016e+04 7.766 8.41e-15 ***
Bdrms2 8.331e+04 9.086e+03 9.170 < 2e-16 ***
Bdrms3 8.791e+04 9.033e+03 9.733 < 2e-16 ***
Bdrms4 7.905e+04 9.001e+03 8.782 < 2e-16 ***
Bdrms5 7.507e+04 9.002e+03 8.340 < 2e-16 ***
Bdrms6 6.149e+04 9.006e+03 6.828 8.80e-12 ***
Bdrms7 2.462e+04 9.572e+03 2.573 0.010100 *
Bdrms8 1.242e+04 1.008e+04 1.233 0.217672
Fbath0 -1.936e+04 1.388e+04 -1.395 0.162964
Fbath1 -1.050e+04 9.795e+03 -1.072 0.283695
Fbath2 7.289e+03 9.751e+03 0.748 0.454721
Fbath3 2.935e+04 9.641e+03 3.044 0.002339 **
Fbath4 6.757e+04 1.025e+04 6.595 4.35e-11 ***
Lotsize 1.419e+00 1.131e-01 12.545 < 2e-16 ***
Sale_date 4.839e+00 2.603e-01 18.587 < 2e-16 ***
district_3TRUE 9.189e+05 1.373e+05 6.693 2.23e-11 ***
district_4TRUE 1.088e+06 2.504e+05 4.346 1.39e-05 ***
district_15TRUE 4.475e+05 1.212e+05 3.692 0.000223 ***
Year_Built:district_3TRUE -5.100e+02 7.205e+01 -7.078 1.51e-12 ***
Lotsize:district_3TRUE 1.227e+01 4.803e-01 25.549 < 2e-16 ***
Lotsize:district_4TRUE -3.796e+00 1.569e+00 -2.420 0.015512 *
Year_Built:district_4TRUE -5.558e+02 1.306e+02 -4.255 2.09e-05 ***
Year_Built:district_15TRUE -2.829e+02 6.295e+01 -4.493 7.05e-06 ***
Lotsize:district_15TRUE 3.140e+00 1.213e+00 2.589 0.009620 **
Fin_sqft:district_3TRUE 6.621e+01 1.517e+00 43.639 < 2e-16 ***
Fin_sqft:district_4TRUE -2.049e+01 4.181e+00 -4.901 9.61e-07 ***
Fin_sqft:district_15TRUE -1.161e+01 3.163e+00 -3.672 0.000241 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 43870 on 24429 degrees of freedom
Multiple R-squared: 0.7301, Adjusted R-squared: 0.7296
F-statistic: 1468 on 45 and 24429 DF, p-value: < 2.2e-16# Notice that some of the coefficients moved several standard errors!! This is a huge change - recall that outside 2 SE is outside the confidence interval.
sort(abs(model3$coefficients-model2$coefficients)/s$coefficients[,2],T) Fin_sqft:district_3TRUE Lotsize:district_3TRUE
9.627451867 7.398289591
Lotsize:district_15TRUE Fbath4
3.625141562 2.642349385
Fin_sqft Fbath1
1.909169272 1.871811508
Fbath2 (Intercept)
1.836032302 1.750708384
Fbath3 Bdrms2
1.648092589 1.553248683
Fbath0 Bdrms3
1.528569449 1.525182764
Bdrms1 Bdrms6
1.519310478 1.509337136
Bdrms4 Stories1
1.481971014 1.464803525
Stories1.5 Lotsize
1.461736494 1.395571747
Bdrms5 Stories2
1.370203968 1.344293830
Year_Built:district_15TRUE ExtwallStone
1.246962320 1.192651347
district_15TRUE Bdrms8
1.002886648 0.743645391
ExtwallMasonry / Frame Bdrms0
0.697108076 0.694800600
Lotsize:district_4TRUE Units3
0.693460029 0.661358970
Bdrms7 Year_Built:district_3TRUE
0.645702298 0.634841414
Fin_sqft:district_4TRUE district_3TRUE
0.625373650 0.580922886
Fin_sqft:district_15TRUE ExtwallBlock
0.502891319 0.453267804
ExtwallFiber-Cement ExtwallPrem Wood
0.350629856 0.349173256
Year_Built ExtwallFrame
0.301049304 0.255547865
Sale_date Units1
0.235454459 0.148580537
District district_4TRUE
0.043599621 0.028319548
ExtwallStucco Year_Built:district_4TRUE
0.027812022 0.024823550
ExtwallBrick Units2
0.010291781 0.006490842 How should we treat influential observations? The easiest course of action is removal. If there are many influential observations, then you might want to try robust model fitting methods, which automatically account for outliers and influential observations.
Complete the Chapter 6 textbook questions.
Exercise 7.1 What are the three methods we have learned for detecting influential/leverage points?
Exercise 7.2 Compute the hat values, Cook’s distances and the depth values for the body weight example. Are there any influential/leverage points/outliers?
Exercise 7.3 Compute the hat values, Cook’s distances and the depth values for the cars example. Are there any outliers/influential/leverage points?
Exercise 7.4 Fit a model without location to the real estate data of your choosing. Compute the hat values, Cook’s distances and the depth values for the cars example. Are there any influential/leverage points/outliers? Print out the influential/leverage points/outliers. Why do you think they are outlying? Should we remove them?