Exercise 3.1

Suppose that the knots are located at \(t_1\) and \(t_2\). Then, the spline is linear on each of the three intervals: \(f(t)=\alpha_0+b_0t\) (for \(t<t_1\)), \(f(t)=\alpha_1+b_1t\), (for \(t_1\le t<t_2\)), and \(f(t)=\alpha_2+b_2t\), (for \(t\ge t_2\)). The six model parameters are: intercepts, \(\alpha_0, \alpha_1, \alpha_1\) and slopes, \(b_0, b_1, b_2\).

If we have data points at the knots, with function values \(f_1\) at \(t_1\) and \(f_2\) at \(t_2\), then the splines must pass through these coordinates: \(\alpha_0+b_0 t_1 = f_1\), \(\alpha_1+b_1 t_1 = f_1\), and \(\alpha_1+b_1 t_2 = f_2\), \(\alpha_2+b_2 t_2 = f_2\).

For continuity of function at the two knots: \(\alpha_0+b_0 t_1 = \alpha_1+b_1 t_1\), and \(\alpha_1 + b_1 t_2 = \alpha_2+b_2 t_2\). Note, however, that these two constraints will automatically be satisfied if the previous set of four, to force the spline through the data points, are satisfied – they are redundant. But note, that the opposite is not true, that is continuity does not imply that they pass through the data points.

In the outer two intervals we require the spline to be constant, hence: \(b_1=0\) and \(b_2=0\).


Exercise 3.2

If the constraints in Exercises 3.1 are written as a system of linear equations we obtain In the outer two intervals we require the spline to be constant, hence: \[\alpha_0+b_0 t_1 = f_1\] \[\alpha_1+b_1 t_1 = f_1\] \[\alpha_1+b_1 t_2 = f_2\] \[\alpha_2+b_2 t_2 = f_2\] \[b_1=0\] \[b_2=0\]

or in matrix form: \[ \begin{bmatrix} f1 \\ f1 \\ f2\\ f2\\ 0\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & t_1 & 0 & 0 \\ 0 & 1 & 0 & 0 & t_1 & 0 \\ 0 & 1 & 0 & t_2 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & t_2 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} \alpha_0 \\ \alpha_1 \\ \alpha_3\\ b_0\\ b_1 \\ b_2 \end{bmatrix} \] Note that other orderings of rows and column are fine.

This could be solved by direct inversion. Let the system be written \({\mathbf y} = X \boldsymbol{\beta}\) then \({\hat{\boldsymbol{\beta}}} = X^{-1}{\mathbf y}\).


Exercise 3.3

Suppose we have data points at \((1,3)\) and \((2,5)\), then R the problem can be solved as follows, but note hat at most steps there are other ways to organise the calculations.

locations = c(1, 2)
mydata = data.frame(x=c(1,2), y=c(3,5))

plot(mydata, xlim=c(0,3), ylim=c(0,6), pch=16)

y = c(3, 3, 5, 5, 0, 0)

# Design matrix
X= matrix(c(1,0,0,1,0,0,
            0,1,0,0,1,0,
            0,1,0,0,2,0,
            0,0,1,0,0,2,
            0,0,0,1,0,0,
            0,0,0,0,0,1), ncol=6, byrow=T)

hatbeta = solve(X) %*% y

curve(hatbeta[1]+hatbeta[4]*x, 0, 1, add=T)
curve(hatbeta[2]+hatbeta[5]*x, 1, 2, add=T)
curve(hatbeta[3]+hatbeta[6]*x, 2, 3, add=T)


Exercise 3.4

Linear natural splines can be written as: \[ f(t) = a_0 + \sum_{i=1}^m b_i \left| t - t_i \right|; \quad \sum_{i=1}^m b_i = 0 \] and with \(m=2\) these become \[ f(t) = a_0 + b_1 \left| t - t_1 \right|+b_2 \left| t - t_2 \right|; \quad b_1+b_2 = 0. \] In matrix form, for the two data points, this can be written as follows: \[ \begin{bmatrix} f1 \\ f2\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & |t_1-t_1| & |t_1-t_2| \\ 1 & |t_2-t_1| & |t_2-t_2| \\ 0 & 1 & 1 \\ \end{bmatrix} \begin{bmatrix} a_0 \\ b_1 \\ b_2 \end{bmatrix} \]


Exercise 3.5

With the earlier chosen data pints the system becomes: \[ \begin{bmatrix} 3 \\ 5\\ 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 1 \\ \end{bmatrix} \begin{bmatrix} a_0 \\ b_1 \\ b_2 \end{bmatrix} \] Then, solving the system in R:

y2 = c(3,5,0)
X2 = matrix(c(1,0,1,
           1,1,0,
           0,1,1), ncol=3, byrow=T)

hatbeta2 = solve(X2) %*% y2

plot(mydata, xlim=c(0,3), ylim=c(0,6), pch=16)

curve(hatbeta2[1]+hatbeta2[2]*abs(x-1)+hatbeta2[3]*abs(x-2), 0,3, add=T, lwd=2)

# Add the earlier calculated spline
curve(hatbeta[1]+hatbeta[4]*x, 0, 1, add=T, col="red", lty=2)
curve(hatbeta[2]+hatbeta[5]*x, 1, 2, add=T, col="red", lty=2)
curve(hatbeta[3]+hatbeta[6]*x, 2, 3, add=T, col="red", lty=2)

The two splines appear identical, but you could also do the algebra to check.

Perhaps the second approach is preferable as the code is shorter, but the first approach may be more intuitive – the choice is yours!


Exercise 3.6

Make a copy of the R code from earlier and remove the set.seed line. Here, \(\texttt{par(mfrow=c(1,2))}\) has been added to get the graphs side-by-side.

par(mfrow=c(1,2))
par(mar=c(3,3,1,1), mgp=c(2,1,0))

x = 1:9; y = rnorm(9)

# spline deals with points
plot(x, y, xlim=c(0,10), ylim=c(-3,3), pch=4)

mysplinefit1 = spline(x, y, method="natural")
lines(mysplinefit1)


# splinefun produces a function
plot(x, y, xlim=c(0,10), ylim=c(-3,3), pch=4)

mysplinefit2 = splinefun(x, y, method="natural")
curve(mysplinefit2, 0, 10, add=T)


Exercise 3.7

When considering the function in each interval, the important issue is to correctly remove the absolution value. In particular, \(|t|\) for values of \(t\in (-\infty,0)\) can be replaced by \(-t\), and \(|t-1|\) for \(t\in (1,\infty)\) can be replaced by \(t-1\). The full list of replacements are:

Term \((-\infty, 0)\) \((0,1)\) \((1, \infty\))
\(|t|\) \(-t\) \(t\) \(t\)
\(|t-1|\) \(-(t-1)\) \(-(t-1)\) \((t-1)\)

Within the intervals the spline \(f(t) = 3 + 2t + 4|t|^3 + |t-1|^3\) becomes, after expanding and collecting the terms:

Interval Spline function First derivative Second derivative
\((-\infty,0)\) \(f_0(t)=4-t+3t^2-5t^3\) \(f_0'(t)=-1+6t-15t^2\) \(f_0''(t)=6-30t\)
\((0,1)\) \(f_1(t)=4-t+3t^2+3t^3\) \(f_1'(t)=-1+6t+9t^2\) \(f_1''(t)=6+18t\)
\((1,\infty)\) \(f_2(t)=2+5t-3t^2+5t^3\) \(f_2'(t)=5-6t+15t^2\) \(f_2''(t)=-6+30t\)

It is easy to show continuity of function at the knots (\(t_1=0\) and \(t_2=1\)): \(f_0(t_1=0)=f_1(t_1=0)=4\) and \(f_1(t_2=1)=f_2(t_2=1)=9\). Then, continuity of first derivative: \(f_0'(t_1=0)=f_1'(t_1=0)=-1\) and \(f_1'(t_2=1)=f_2'(t_2=1)=14\). Finally, of second derivative: \(f_0''(t_1=0)=f_1''(t_1=0)=6\) and \(f_1''(t_2=1)=f_2''(t_2=1)=24\).

The function \(f\) is a cubic spline, but it is not a natural spline as it is not linear outside the range of the knots, that is for \(t<0\) and \(t>1\).

Another way to prove this result is to write \(f\) in the form \(f(t) = a_0 + a_1 t + \sum_{j=1}^n b_j |t-t_j|^3\), where \(n=2\) and \(b_1=4, b_2 = 1\). Since the function \(|t-t_j|^3\) is twice continuously differentiable at \(t=t_j\) (check) and is cubic for \(t<t_j\) and for \(t>t_j\), it follows immediately that \(f\) is a cubic spline. But since it is not the case that \(\sum b_j = 0, \ \sum b_j t_j = 0\), it follows that the spline is not natural.


Exercise 3.8

The function \(f(t) = 3+|t|-|t-2|\) takes the more explicit form \[\begin{align*} f(t) &= 3-t+(t-2) = 1, \quad t<0,\\ f(t) &= 3+t+(t-2) = 2t+1, \quad 0<t<1,\\ f(t) &= 3+t-(t-2) = 5, \quad t>2. \end{align*}\] Note that \(f\) is a natural linear spline (natural since \(f\) is constant for \(t<0\) and \(t>2\)). Its first derivative takes the form \(f'(t) = 0, \ t<0, f'(t) = 2, \ 0<t<2,\) and \(f'(t) = 0, \ t>2\). Hence \[ \int_{-\infty}^{\infty} \{f'(t)\}^2 \,dt = \int_{0}^{2} 2^2\,dt = 8. \] In terms of the \(2 \times 2\) matrix \(K = (k_{ij}) = (|t_i - t_j|)\), where \(t_1=0, \ t_2=2\) and the coefficient vector \(\mathbf b\) with \(b_1=1, b_2=-1\), we find that the quadratic form \[ -2 \mathbf b^T K \mathbf b = \begin{bmatrix} 1 & -1 \end{bmatrix} \begin{bmatrix} 0 & 2 \\ 2 & 0 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = 8 \] takes the same value as the integral.


End of Solutions to Exercises 3