Linear Regression with R

In this lab we will learn how to apply what we have learned during the lecture about linear regression.

1. Just to be sure to understand the maths

In this section we will check if the maths related to fitting a linear model are properly understood. To this aim, we will fit a linear model on simulated data (so that the true model is perfectly known) from scratch and using the dedicated builtin R function.

a) Generate some artificial data

Let’s first simulate some data

set.seed(1)##fix the random seed so that we all have the same results
n.obs <- 100

## Build some features
X1 <- cos(seq(0, 2 * pi, length = n.obs))
X2 <- 1:n.obs

## A design matrix
X <- cbind(1, X1, X2)

## The true parameter beta
beta <- c(2, -2, 0.5)

## The true response
Ytheo <- X %*% beta

## The observed response (true + noise)
Y <- Ytheo + rnorm(n.obs, 0, 0.25)

b) Analysis “from scratch”

1. Compute the least square estimate \(\hat{\beta}\) (slide 25).

## Fill in this part

2. Compute the fitted values \(\hat{Y} = X \hat{\beta}\)

## Fill in this part

3. Compare the fitted values \(\hat Y\) to the observed ones \(Y\).

## Fill in this part

c) Analysis using the dedicated R function

1. Read the help file of the lm function.
2. Build a data frame that contains all the features of our linear model.

## Fill in this part

3. Fit the corresponding linear model using the lm function.

## Fill in this part

4. Run the following code and comment

plot(fit)##fit is the output of the previous lm call, I'm here to explain each plot ;-)

5. Use the predict function to make prediction

## Fill in this part

2. Redoing the Galapagos analysis

We move a bit further by analyzing the Galapagos data set of the lecture. The dataset is available from the R package faraway. Try to redo the analysis we did in Slide 26–41.

3. Modelling the monthly total of international airline passengers

The AirPassengers dataset, already included into R, stores the monthly total of international airline passengers from 1949 to 1960.

1. Plot the data and comment what you see.

2. Let \(Y_t\) be the number of passenger at time \(t\). Fit the following linear model: \[\begin{equation*} Y_t = \beta_0 + \beta_1 (t - 1955) / 100 + \varepsilon_t, \qquad \varepsilon_t \sim N(0, \sigma^2), \end{equation*}\] analyze your prediction and give comments about the limitations of your fitted model.

3. Now do exactly the same but on a log-scale, i.e., \[\begin{equation*} \log Y_t = \beta_0 + \beta_1 (t - 1955) / 100 + \varepsilon_t, \qquad \varepsilon_t \sim N(0, \sigma^2). \end{equation*}\] Analyze your prediction and give comments about the limitations of your fitted model.

4. Although we must rather use \(SARIMA\) modelling (beyond the scope of this lecture) to this aim, to model the observed seasonality we can use the following (autoregressive) linear model: \[\begin{equation*} Y_t = \beta_0 + \beta_1 Y_{t-1} + \ldots + \beta_{13} Y_{t-13} + \varepsilon_t, \qquad \varepsilon_t \sim N(0, \sigma^2). \end{equation*}\]

1. Learn how to use the embed function to store all the features into a single data frame.
2. Fit the corresponding linear model
3. Do model selection (see the lecture note for this)
4. Give a plot showing how your model perform and do prediction for years 1961 and 1962.