Dealing with multi-collinearity: Pearson’s R and variance inflation factor analysis

A final step concerns only the environmental variables. The choice of the environmental variables to be included as predictors is a very crucial task. The choice should be take into account the ecological requirements of the species under study but it is important to avoid high degrees of collinearity between the selected predictors. Among the main issues related to collinearity there are: inflation of standard errors on estimates, impossibility to separate variables effects and severe errors in extrapolations in space and time (for further details see Dorman et al., 2013). In this section of the tutorial we provide two so-called “threshold-based” methods that remove predictors correlated over a given threshold: Pearson’s r and the variance inflation factor (VIF) (for details, see Dorman et al., 2013 and Zuur et al., 2010 respectively). Pearson’s r is the most common metric used to evaluate the correlation degree between two numerical predictors. VIF is equal to 1 / (1 - R squared), which reflects the degree to which the variance of an estimated regression coefficient (in a model using these variables as predictors) is increased due only to the correlations among covariates.

First we extract values of environmental rasters in 10000 random points within our training area:

sample<-randomPoints(vars, 10000)
vars_tab<-extract(vars, sample)

Then we calculate the Pearson's r and check which variables have a Pearson's r greater than 0.7:

> abs(cor(vars_tab))>0.7

bio1 bio2 bio3 bio4 bio5 bio6 bio7 bio8 bio9 bio10 bio11 bio12 bio13 bio14 bio15 bio16 bio17 bio18 bio19
bio1 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio2 FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio3 FALSE FALSE TRUE TRUE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio4 FALSE FALSE TRUE TRUE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio5 FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio6 FALSE FALSE TRUE TRUE FALSE TRUE TRUE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio7 FALSE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio8 FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio9 TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio10 TRUE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio11 TRUE FALSE TRUE TRUE FALSE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
bio12 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE TRUE FALSE TRUE FALSE
bio13 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE TRUE FALSE FALSE FALSE
bio14 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE
bio15 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE
bio16 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE TRUE FALSE FALSE FALSE
bio17 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE TRUE FALSE FALSE
bio18 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE FALSE
bio19 FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE

From this output we can understand that bio1 is highly correlated with bio5, bio6, bio8, bio9, bio10 and bio11. Other bioclimatic variables show high levels of correlation. Now we need to select predictors (with r>0.7) to be removed. This choice is somehow arbitrary. For instance in the case of bio1-bio10 we can remove bio1 or bio10. Usually one will choose to retain those variables that have a “biological meaning” for the specie of interest, on the basis of your experience. We will choose to retain 7 variables:

>abs(cor(vars_tab[-c(3,4,5,6,8,9,10,11,13,14,16,18), -c(3,4,5,6,8,9,10,11,13,14,16,18)])) > 0.7

bio1 bio2 bio7 bio12 bio15 bio17 bio19
bio1 TRUE FALSE FALSE FALSE FALSE FALSE FALSE
bio2 FALSE TRUE FALSE FALSE FALSE FALSE FALSE
bio7 FALSE FALSE TRUE FALSE FALSE FALSE FALSE
bio12 FALSE FALSE FALSE TRUE FALSE FALSE FALSE
bio15 FALSE FALSE FALSE FALSE TRUE FALSE FALSE
bio17 FALSE FALSE FALSE FALSE FALSE TRUE FALSE
bio19 FALSE FALSE FALSE FALSE FALSE FALSE TRUE

According to Pearson's R we create the new set of rasters including only the bioclimatic variables chosen:

> vars_r <- vars[[-c(3,4,5,6,8,9,10,11,13,14,16,18)]]

Now we try with VIF test, the variance inflation factor analysis (function provided with the tutorial)

> source("script/vif_functions.R")

this function find and remove each predictor reporting a VIF greater than 3

> vars_vif<-vif(vars, thresh=3, trace=F)

We create a new set of rasters according with VIF:

> vars_vif<-r[[which(names(r)%in%vars_vif)]]

and finally we compare raster dataset based respectively on Pearson,s r and VIF:

> names(vars_r)

[1] "bio1" "bio2" "bio7" "bio12" "bio15" "bio17" "bio19"

> names(vars_vif)

[1] "bio2" "bio3" "bio8" "bio13" "bio14" "bio15" "bio18" "bio19"

The two procedures show some differences in the number and type of raster layers selected. Pearson's r seems to be more conservative than the VIF. Furthermore, Pearson's r allow a certain degree of arbitrariness respect to VIF (where you have to choose a threshold value) because the researcher must operate a choice of the variables to retain. To train model in the module 3 we decided to use layeres selected on the basis of the Pearson correlation test. We create a new directory where we will store the selected variables

> dir.create("predictors")
> writeRaster(vars_r, "predictors/", bylayer=T, suffix="names")