Recall that violations of model assumptions are more likely at remote points, and these violations may be hard to detect from inspection of the ordinary residuals because their residuals will usually be smaller. Points that are outlying in the \(x\)-direction are known as leverage points. Influential points are not only remote in terms of the specific values for the regressors, but the observed response is not consistent with the values that would be predicted based on only the other data points. It is important to find these influential points and assess their impact on the model.
Below gives an example of an influential point. The seventh point in the data set is outlying in the \(x\)-direction, and it’s response value is not consistent with the regression line based on the other six observations:
set.seed(330)
x=c(rnorm(6),2.5)
y=x*2+3
y[7]=y[7]+7
plot(x,y,pch=22,bg=1)
a=lm(y~x)
curve(a$coefficients[1]+x*a$coefficients[2],add=T,lwd=3)
curve(x*2+3,add=T,col=2,lwd=3)
a2=lm(y[-7]~x[-7])
curve(a2$coefficients[1]+x*a2$coefficients[2],add=T,lwd=3,col='blue',lty=2)
a$coefficients(Intercept) x
2.048937 3.979977 Sometimes we find that a regression coefficient may have a sign that does not make engineering or scientific sense, a regressor known to be important may be statistically insignificant, or a model that fits the data well and that is logical from an application – environment perspective may produce poor predictions. These situations may be the result of one or, perhaps, a few influential observations.
Recall the hat matrix \(H=X(X^\top X)^{-1}X^\top\), as well as that it holds that \({\textrm{Var}}\left[\hat\epsilon\right]=\sigma^2(I-H)\) and \({\textrm{Var}}\left[\hat Y\right]=\sigma^2 H\). Note that \(h_{ij}\) can be interpreted as the amount of leverage exerted by the \(ith\) observation \(y_i\) on the \(jth\) fitted value \(\hat y_j\). We usually focus attention on the diagonal elements \(h_{ii}\) of the hat matrix \(H\), which may be written as \[h_{ii}=x_i^\top (X^\top X)^{-1} x_i,\] where \(X_i^\top\) is the \(i\)th row of \(X\). The hat matrix diagonal is a standardized measure of the distance of the \(i\)th observation from the center (or centroid) of the \(x\)-space. Therefore, large values of \(h_{ii}\) implies that \(x_i\) is potentially influential. Furthermore, note that \(rank(H)=p\) since the trace of an idempotent matrix equals its rank, which means that \(\bar h= p/n\). It follows that values well above \(p/n\), say \(h_{ii}>2p/n\), can be called leverage points.
X=as.matrix(cbind(rep(1,length(x)),x))
# or
X=model.matrix(a)
hat=X%*%solve(t(X)%*%X)%*%t(X)
diag(hat) 1 2 3 4 5 6 7
0.2027453 0.2288737 0.2596869 0.1751432 0.1735495 0.3887329 0.5712686 p=2
n=7
diag(hat)>2*p/n 1 2 3 4 5 6 7
FALSE FALSE FALSE FALSE FALSE FALSE FALSE Cook’s Distance is one way to incorporate both the \(X\) and \(Y\) values into an outlyingness measure:
\[ D_i\left(X^{\top} X, p, MSE\right) \equiv D_i=\frac{\left(\hat{\beta}_{(i)}-\hat{\beta}\right)^{\top} X^{\top} X\left(\hat{\beta}_{(i)}-\hat{\beta}\right)}{p MSE}, \ i\in [n], \] where \(\hat{\beta}_{(i)}\) is the OLS estimator with the \(i\)th point removed.Large values of Cook’s distance signal a leverage point.
What do we mean by a large value? We can compare \(D_i\) to the 50th percentile of the \(F_{p,n-p}\) distribution. This gives the interpretation that deleting the \(i\)th point moves the estimate to the boundary of a 50% confidence interval. \(F_{p,n-p}\approx 1\), and so usually take \(D_i\geq 1\) to be large.
Observe that \[ D_i=\frac{r_i^2}{p} \frac{\operatorname{Var}\left(\hat{Y}_i\right)}{\operatorname{Var}\left(\hat\epsilon_i\right)}=\frac{r_i^2}{p} \frac{h_{i i}}{1-h_{i i}}, \quad i=1,2, \ldots, n,\] where it is important to recall that \(r_i\) is the studentized residual. Now, the quantity \(\frac{h_{i i}}{1-h_{i i}}\) can be shown to be the distance from the vector \(x_i\) to the centroid of the remaining data. Therefore, \(D_i\) is the product of outlyingness in both the \(X\) and \(Y\) directions. We may also write \(D_i\) as \[D_i=\frac{\left\lVert\hat{y}_{(i)}-\hat{y}\right\rVert^2}{p MSE},\] which allows for the interpretation: The Cook’s distance of the \(i\)th point is the normalized distance between the fitted value with and without point \(i\).
#cut off
cooks.distance(a) 1 2 3 4 5 6
0.1708029420 0.2516095165 0.0180669722 0.0009569213 0.0011772793 0.2002829110
7
3.3311562309 cooks.distance(a)>1 1 2 3 4 5 6 7
FALSE FALSE FALSE FALSE FALSE FALSE TRUE df=data.frame(cbind(y,x))
df[cooks.distance(a)>1,] y x
7 15 2.5A more modern approach and nonparametric approach to outlier detection is through data depth. A data depth function gives meaning to centrality, order and outlyingness in spaces beyond \(\mathbb{R}\). A data depth function is a function which takes a sample and a point, and returns how central the point is, with respect to the sample. Depth functions can be written as \({\textrm{D}}\colon \mathbb{R}^{d}\times \text{Sample} \rightarrow \mathbb{R}^+\). There are different definitions of depth, so I will give a few.
Let \(S^{d-1}= \{x\in \mathbb{R}^{d}\colon \left\lVert x\right\rVert=1\}\) be the set of unit vectors in \(\mathbb{R}^{d}\), let \(\mathbb{X}_{n}=\{(Y_1,X_{1,1},\ldots,X_{1,p-1}),\ldots, (Y_n,X_{n,1},\ldots,X_{n,p-1})\}\), let \(\mathbb{X}_{n}^\top u\) be \(\mathbb{X}_{n}\) projected onto \(u\in S^{d-1}\) and let \(\widehat F_u\) be the empirical CDF with respect to \(\mathbb{X}_{n}^\top u\).
The halfspace depth \({\textrm{D}}_H\) of a point \(x\in \mathbb{R}^{d}\) with respect to a distribution \(F\) over \(\mathbb{R}^{d}\) is \[ {\textrm{D}}_H(x;F)=\inf_{u\in S^{d-1}} \widehat F_u(x^\top u)\wedge (1-F_u(x^\top u))=\inf_{u\in S^{d-1}} F_u(x^\top u). \]
Given a translation and scale equivariant location estimate \(\mu\) and a translation and scale invariant scale estimate \(\sigma\), the outlyingness at \(x\in\mathbb{R}^{d}\) is defined as \[O(x)=\sup _{u\in S^{d-1}} \frac{\left|x^\top u-\mu(\mathbb{X}_{n}^\top u)\right|}{\sigma(\mathbb{X}_{n}^\top u)}.\] Define projection depth as \[{\textrm{D}}_p(x)=(1+O(x))^{-1}.\]
In order to detect outliers, we look for observations that have low depth. See, continuing our toy example:
# install.packages('ddalpha')
depths=ddalpha::depth.projection(cbind(x,y),cbind(x,y))
depths[1] 0.276409011 0.255272074 0.500000000 0.973754328 0.973046927 0.341922765
[7] 0.005016447depths<0.015[1] FALSE FALSE FALSE FALSE FALSE FALSE TRUEExample 7.1 Recall example Example 6.6. Check for leverage and influential points in the proposed models. Compute all three measures of leverage/influence/outlyingness introduced in this lesson. What do you find?
I will load in the data below:
We can now analyze the data:
# df=df[df$Lotsize<70000,]
custom_palette <- c(
"#1f77b4", "#ff7f0e", "#2ca02c", "#d62728",
"#9467bd", "#8c564b", "#e377c2", "#7f7f7f",
"#bcbd22", "#17becf", "#393b79",
"#8c6d31", "#9c9ede", "#637939", "#eb348f"
)
# Our model from the previous lecture
df=df_clean2[-which.max(df_clean2$Lotsize),]
df=df[df$Lotsize>0,]
df['district_3']=df['District']==3
df['district_4']=df['District']==4
df['district_15']=df['District']==15
model2=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df)
# Compute residuals
student_res2=rstudent(model2)
summ2=summary(model2); summ2
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df)
Residuals:
Min 1Q Median 3Q Max
-303393 -24422 -942 23352 719561
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.129e+06 3.625e+04 -31.143 < 2e-16 ***
District 5.695e+03 7.701e+01 73.950 < 2e-16 ***
ExtwallBlock -3.944e+03 3.688e+03 -1.069 0.284879
ExtwallBrick 6.725e+03 7.245e+02 9.281 < 2e-16 ***
ExtwallFiber-Cement 3.847e+04 3.754e+03 10.246 < 2e-16 ***
ExtwallFrame -4.986e+03 9.786e+02 -5.095 3.51e-07 ***
ExtwallMasonry / Frame 3.751e+03 1.711e+03 2.192 0.028404 *
ExtwallPrem Wood 1.894e+04 5.596e+03 3.385 0.000714 ***
ExtwallStone 1.162e+04 1.531e+03 7.594 3.22e-14 ***
ExtwallStucco 3.514e+03 2.149e+03 1.635 0.102070
Stories1 3.684e+02 1.047e+04 0.035 0.971935
Stories1.5 1.317e+04 1.046e+04 1.259 0.207954
Stories2 1.451e+04 1.042e+04 1.392 0.163802
Year_Built 4.629e+02 1.606e+01 28.823 < 2e-16 ***
Fin_sqft 6.048e+01 1.080e+00 55.986 < 2e-16 ***
Units1 5.778e+04 7.389e+03 7.820 5.48e-15 ***
Units2 -9.774e+03 7.386e+03 -1.323 0.185735
Units3 -4.087e+04 7.940e+03 -5.147 2.67e-07 ***
Bdrms0 1.155e+05 1.854e+04 6.227 4.82e-10 ***
Bdrms1 7.894e+04 1.016e+04 7.766 8.41e-15 ***
Bdrms2 8.331e+04 9.086e+03 9.170 < 2e-16 ***
Bdrms3 8.791e+04 9.033e+03 9.733 < 2e-16 ***
Bdrms4 7.905e+04 9.001e+03 8.782 < 2e-16 ***
Bdrms5 7.507e+04 9.002e+03 8.340 < 2e-16 ***
Bdrms6 6.149e+04 9.006e+03 6.828 8.80e-12 ***
Bdrms7 2.462e+04 9.572e+03 2.573 0.010100 *
Bdrms8 1.242e+04 1.008e+04 1.233 0.217672
Fbath0 -1.936e+04 1.388e+04 -1.395 0.162964
Fbath1 -1.050e+04 9.795e+03 -1.072 0.283695
Fbath2 7.289e+03 9.751e+03 0.748 0.454721
Fbath3 2.935e+04 9.641e+03 3.044 0.002339 **
Fbath4 6.757e+04 1.025e+04 6.595 4.35e-11 ***
Lotsize 1.419e+00 1.131e-01 12.545 < 2e-16 ***
Sale_date 4.839e+00 2.603e-01 18.587 < 2e-16 ***
district_3TRUE 9.189e+05 1.373e+05 6.693 2.23e-11 ***
district_4TRUE 1.088e+06 2.504e+05 4.346 1.39e-05 ***
district_15TRUE 4.475e+05 1.212e+05 3.692 0.000223 ***
Year_Built:district_3TRUE -5.100e+02 7.205e+01 -7.078 1.51e-12 ***
Lotsize:district_3TRUE 1.227e+01 4.803e-01 25.549 < 2e-16 ***
Lotsize:district_4TRUE -3.796e+00 1.569e+00 -2.420 0.015512 *
Year_Built:district_4TRUE -5.558e+02 1.306e+02 -4.255 2.09e-05 ***
Year_Built:district_15TRUE -2.829e+02 6.295e+01 -4.493 7.05e-06 ***
Lotsize:district_15TRUE 3.140e+00 1.213e+00 2.589 0.009620 **
Fin_sqft:district_3TRUE 6.621e+01 1.517e+00 43.639 < 2e-16 ***
Fin_sqft:district_4TRUE -2.049e+01 4.181e+00 -4.901 9.61e-07 ***
Fin_sqft:district_15TRUE -1.161e+01 3.163e+00 -3.672 0.000241 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 43870 on 24429 degrees of freedom
Multiple R-squared: 0.7301, Adjusted R-squared: 0.7296
F-statistic: 1468 on 45 and 24429 DF, p-value: < 2.2e-16summ2$adj.r.squared[1] 0.7295574# Compute residual analysis
MSE2=summ2$sigma^2
qqnorm(student_res2,pch=22,bg=1)
abline(0,1)
hist(student_res2,freq=F,breaks=100)
curve(dnorm(x,0,1),add=T)
# hist(student_res2,freq=F,breaks=100)
# curve(dnorm(x,0,1),add=T)
plot(model2$fitted.values,student_res2,pch=22,bg=1)
abline(h=0)
# First measure
X=model.matrix(model2)
hat=X%*%solve(t(X)%*%X)%*%t(X)
# diag(hat)
p=ncol(X)
n=nrow(X)
out_1=which(diag(hat)>2*p/n)
plot(sort(diag(hat)[out_1]))
abline(h=2*p/n)
# I would still look at those after the elbow
# Cooks distances
CDS=cooks.distance(model2)
plot(sort(CDS,T)[1:100])
which(CDS>1)22532
22398 max(CDS)[1] 1.075144df[CDS>1,] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSE# I would still look at those two values that are far from the other distances
# I would still look at those before the elbow
# We may only look at numeric values for depth functions - so we can either
numer=NULL
for(i in names(df)){
if(!is.factor(df[1,i])){
numer=c(numer,i)
}
}
numer[1] "District" "Year_Built" "Fin_sqft" "Lotsize" "Sale_date"
[6] "Sale_price" "district_3" "district_4" "district_15"df_mat=as.matrix(df[,numer])
depths=ddalpha::depth.projection(df_mat,df_mat)
which(depths<0.1)[1:10] [1] 5 64 130 326 471 473 593 637 669 673plot(sort(depths,F)[1:100])
# Notice there is a crack around 0.035, I would look at those observations
plot(sort(depths,F)[1:1000])
plot(sort(depths,F))
# OR
depths=ddalpha::depth.projection(cbind(X,df$Sale_price),cbind(X,df$Sale_price))
which(depths<0.1)[1:10] [1] 2 3 4 5 7 13 15 17 20 22plot(sort(depths,F)[1:100])
which.max(diag(hat))22532
22398 which.max(CDS)22532
22398 which.min(depths)[1] 22398# Hugely expensive home!
df[which.min(depths),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEdf[which.max(CDS),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEdf[which.max(diag(hat)),] District Extwall Stories Year_Built Fin_sqft Units Bdrms Fbath Lotsize
22532 3 Block 1 1960 4323 1 4 3 72480
Sale_date Sale_price District District District
22532 17713 1250000 TRUE FALSE FALSEmodel3=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df[-(order(depths)[1:100]),])
# OR
model4=lm(Sale_price~.-district_3-district_4-district_15+district_3*Year_Built+district_3*Lotsize+district_4*Lotsize+district_4*Year_Built+district_15*Year_Built+district_15*Lotsize+district_3*Fin_sqft+district_4*Fin_sqft+district_15*Fin_sqft,df[-(order(CDS,decreasing = T)[1:100]),])
# Compare
s=summary(model3)
summary(model3)
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df[-(order(depths)[1:100]),
])
Residuals:
Min 1Q Median 3Q Max
-286875 -24345 -1026 23048 621671
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.060e+06 3.610e+04 -29.373 < 2e-16 ***
District 5.696e+03 7.423e+01 76.724 < 2e-16 ***
ExtwallBlock -2.424e+03 3.561e+03 -0.681 0.496069
ExtwallBrick 6.680e+03 7.014e+02 9.524 < 2e-16 ***
ExtwallFiber-Cement 4.060e+04 3.622e+03 11.207 < 2e-16 ***
ExtwallFrame -4.681e+03 9.443e+02 -4.957 7.20e-07 ***
ExtwallMasonry / Frame 4.856e+03 1.660e+03 2.926 0.003433 **
ExtwallPrem Wood 2.136e+04 5.432e+03 3.933 8.43e-05 ***
ExtwallStone 9.845e+03 1.481e+03 6.649 3.02e-11 ***
ExtwallStucco 3.720e+03 2.094e+03 1.776 0.075702 .
Stories1 -1.506e+04 1.020e+04 -1.476 0.139861
Stories1.5 -2.677e+03 1.019e+04 -0.263 0.792839
Stories2 -1.052e+01 1.016e+04 -0.001 0.999174
Year_Built 4.591e+02 1.558e+01 29.464 < 2e-16 ***
Fin_sqft 5.873e+01 1.050e+00 55.915 < 2e-16 ***
Units1 5.763e+04 7.137e+03 8.075 7.06e-16 ***
Units2 -7.819e+03 7.134e+03 -1.096 0.273045
Units3 -4.350e+04 7.675e+03 -5.668 1.46e-08 ***
Bdrms0 8.917e+04 1.805e+04 4.942 7.80e-07 ***
Bdrms1 4.878e+04 1.015e+04 4.805 1.56e-06 ***
Bdrms2 5.470e+04 9.149e+03 5.979 2.28e-09 ***
Bdrms3 5.953e+04 9.099e+03 6.543 6.16e-11 ***
Bdrms4 5.096e+04 9.069e+03 5.619 1.94e-08 ***
Bdrms5 4.735e+04 9.069e+03 5.221 1.80e-07 ***
Bdrms6 3.282e+04 9.078e+03 3.615 0.000301 ***
Bdrms7 2.798e+03 9.636e+03 0.290 0.771550
Bdrms8 -7.997e+03 1.013e+04 -0.789 0.430012
Fbath0 -3.648e+04 1.517e+04 -2.405 0.016194 *
Fbath1 -2.478e+04 1.185e+04 -2.092 0.036425 *
Fbath2 -6.777e+03 1.181e+04 -0.574 0.566104
Fbath3 1.622e+04 1.176e+04 1.380 0.167704
Fbath4 3.995e+04 1.238e+04 3.226 0.001255 **
Lotsize 1.588e+00 1.139e-01 13.941 < 2e-16 ***
Sale_date 4.720e+00 2.513e-01 18.780 < 2e-16 ***
district_3TRUE 1.179e+06 1.397e+05 8.440 < 2e-16 ***
district_4TRUE 1.076e+06 2.414e+05 4.458 8.30e-06 ***
district_15TRUE 4.478e+05 1.169e+05 3.831 0.000128 ***
Year_Built:district_3TRUE -6.513e+02 7.355e+01 -8.855 < 2e-16 ***
Lotsize:district_3TRUE 2.289e+01 9.772e-01 23.424 < 2e-16 ***
Lotsize:district_4TRUE -2.915e+00 1.612e+00 -1.808 0.070563 .
Year_Built:district_4TRUE -5.533e+02 1.259e+02 -4.393 1.12e-05 ***
Year_Built:district_15TRUE -2.820e+02 6.071e+01 -4.645 3.42e-06 ***
Lotsize:district_15TRUE 3.267e+00 1.170e+00 2.794 0.005216 **
Fin_sqft:district_3TRUE 4.854e+01 1.728e+00 28.083 < 2e-16 ***
Fin_sqft:district_4TRUE -1.951e+01 4.120e+00 -4.734 2.21e-06 ***
Fin_sqft:district_15TRUE -1.319e+01 3.057e+00 -4.314 1.61e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 42280 on 24329 degrees of freedom
Multiple R-squared: 0.6807, Adjusted R-squared: 0.6802
F-statistic: 1153 on 45 and 24329 DF, p-value: < 2.2e-16summary(model2)
Call:
lm(formula = Sale_price ~ . - district_3 - district_4 - district_15 +
district_3 * Year_Built + district_3 * Lotsize + district_4 *
Lotsize + district_4 * Year_Built + district_15 * Year_Built +
district_15 * Lotsize + district_3 * Fin_sqft + district_4 *
Fin_sqft + district_15 * Fin_sqft, data = df)
Residuals:
Min 1Q Median 3Q Max
-303393 -24422 -942 23352 719561
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.129e+06 3.625e+04 -31.143 < 2e-16 ***
District 5.695e+03 7.701e+01 73.950 < 2e-16 ***
ExtwallBlock -3.944e+03 3.688e+03 -1.069 0.284879
ExtwallBrick 6.725e+03 7.245e+02 9.281 < 2e-16 ***
ExtwallFiber-Cement 3.847e+04 3.754e+03 10.246 < 2e-16 ***
ExtwallFrame -4.986e+03 9.786e+02 -5.095 3.51e-07 ***
ExtwallMasonry / Frame 3.751e+03 1.711e+03 2.192 0.028404 *
ExtwallPrem Wood 1.894e+04 5.596e+03 3.385 0.000714 ***
ExtwallStone 1.162e+04 1.531e+03 7.594 3.22e-14 ***
ExtwallStucco 3.514e+03 2.149e+03 1.635 0.102070
Stories1 3.684e+02 1.047e+04 0.035 0.971935
Stories1.5 1.317e+04 1.046e+04 1.259 0.207954
Stories2 1.451e+04 1.042e+04 1.392 0.163802
Year_Built 4.629e+02 1.606e+01 28.823 < 2e-16 ***
Fin_sqft 6.048e+01 1.080e+00 55.986 < 2e-16 ***
Units1 5.778e+04 7.389e+03 7.820 5.48e-15 ***
Units2 -9.774e+03 7.386e+03 -1.323 0.185735
Units3 -4.087e+04 7.940e+03 -5.147 2.67e-07 ***
Bdrms0 1.155e+05 1.854e+04 6.227 4.82e-10 ***
Bdrms1 7.894e+04 1.016e+04 7.766 8.41e-15 ***
Bdrms2 8.331e+04 9.086e+03 9.170 < 2e-16 ***
Bdrms3 8.791e+04 9.033e+03 9.733 < 2e-16 ***
Bdrms4 7.905e+04 9.001e+03 8.782 < 2e-16 ***
Bdrms5 7.507e+04 9.002e+03 8.340 < 2e-16 ***
Bdrms6 6.149e+04 9.006e+03 6.828 8.80e-12 ***
Bdrms7 2.462e+04 9.572e+03 2.573 0.010100 *
Bdrms8 1.242e+04 1.008e+04 1.233 0.217672
Fbath0 -1.936e+04 1.388e+04 -1.395 0.162964
Fbath1 -1.050e+04 9.795e+03 -1.072 0.283695
Fbath2 7.289e+03 9.751e+03 0.748 0.454721
Fbath3 2.935e+04 9.641e+03 3.044 0.002339 **
Fbath4 6.757e+04 1.025e+04 6.595 4.35e-11 ***
Lotsize 1.419e+00 1.131e-01 12.545 < 2e-16 ***
Sale_date 4.839e+00 2.603e-01 18.587 < 2e-16 ***
district_3TRUE 9.189e+05 1.373e+05 6.693 2.23e-11 ***
district_4TRUE 1.088e+06 2.504e+05 4.346 1.39e-05 ***
district_15TRUE 4.475e+05 1.212e+05 3.692 0.000223 ***
Year_Built:district_3TRUE -5.100e+02 7.205e+01 -7.078 1.51e-12 ***
Lotsize:district_3TRUE 1.227e+01 4.803e-01 25.549 < 2e-16 ***
Lotsize:district_4TRUE -3.796e+00 1.569e+00 -2.420 0.015512 *
Year_Built:district_4TRUE -5.558e+02 1.306e+02 -4.255 2.09e-05 ***
Year_Built:district_15TRUE -2.829e+02 6.295e+01 -4.493 7.05e-06 ***
Lotsize:district_15TRUE 3.140e+00 1.213e+00 2.589 0.009620 **
Fin_sqft:district_3TRUE 6.621e+01 1.517e+00 43.639 < 2e-16 ***
Fin_sqft:district_4TRUE -2.049e+01 4.181e+00 -4.901 9.61e-07 ***
Fin_sqft:district_15TRUE -1.161e+01 3.163e+00 -3.672 0.000241 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 43870 on 24429 degrees of freedom
Multiple R-squared: 0.7301, Adjusted R-squared: 0.7296
F-statistic: 1468 on 45 and 24429 DF, p-value: < 2.2e-16# Notice that some of the coefficients moved several standard errors!! This is a huge change - recall that outside 2 SE is outside the confidence interval.
sort(abs(model3$coefficients-model2$coefficients)/s$coefficients[,2],T) Lotsize:district_3TRUE Fin_sqft:district_3TRUE
10.867448748 10.220720724
Bdrms6 Bdrms2
3.158528023 3.128200424
Bdrms3 Bdrms4
3.119547827 3.097666901
Bdrms5 Bdrms1
3.057382712 2.970962442
Bdrms7 Fbath4
2.265160080 2.230957459
Bdrms8 Year_Built:district_3TRUE
2.015004139 1.921347695
(Intercept) district_3TRUE
1.898021932 1.861916025
Fin_sqft Stories1.5
1.669174609 1.554667197
Stories1 Lotsize
1.512458567 1.485113240
Bdrms0 Stories2
1.457421541 1.429906689
Fbath1 ExtwallStone
1.205749437 1.201524687
Fbath2 Fbath0
1.190989660 1.128322832
Fbath3 ExtwallMasonry / Frame
1.115673245 0.666055622
ExtwallFiber-Cement Lotsize:district_4TRUE
0.588037320 0.546634425
Fin_sqft:district_15TRUE Sale_date
0.515061850 0.471299565
ExtwallPrem Wood ExtwallBlock
0.446014933 0.426793177
Units3 ExtwallFrame
0.342662399 0.323043120
Units2 Year_Built
0.274017973 0.247358263
Fin_sqft:district_4TRUE Lotsize:district_15TRUE
0.238701677 0.108839595
ExtwallStucco ExtwallBrick
0.098158720 0.063110328
district_4TRUE Units1
0.049699827 0.021203281
Year_Built:district_4TRUE Year_Built:district_15TRUE
0.020328818 0.014684984
District district_15TRUE
0.012204714 0.002979716 How should we treat influential observations? The easiest course of action is removal. If there are many influential observations, then you might want to try robust model fitting methods, which automatically account for outliers and influential observations.
Complete the Chapter 6 textbook questions.
Exercise 7.1 What are the three methods we have learned for detecting influential/leverage points?
Exercise 7.2 Compute the hat values, Cook’s distances and the depth values for the body weight example. Are there any influential/leverage points/outliers?
Exercise 7.3 Compute the hat values, Cook’s distances and the depth values for the cars example. Are there any outliers/influential/leverage points?
Exercise 7.4 Fit a model without location to the real estate data of your choosing. Compute the hat values, Cook’s distances and the depth values for the cars example. Are there any influential/leverage points/outliers? Print out the influential/leverage points/outliers. Why do you think they are outlying? Should we remove them?