Formulation of the H-principle

It is recommended to build up the solution in terms of elementary parts. An expression should be computed for the possible improvements of the solution in terms of fit or other some criteria. Also, compute an expression for the precision of the possible solutions.

More precisely, the recommended steps of the H-principle are:

1). Carry out the modelling in steps. You specify how you want to look at the data at this step by formulating how the weights are computed.

2). At each step compute expressions for

improvement in fit, ΔFit

the associated prediction, ΔPrecision

3). Compute the solution that maximizes the product
 
                        ΔFit ´ ΔPrecision

4). In case the computed solution improves the prediction abilities of the model, the solution is accepted. If the solution does not provide this improvement, we stop modelling.

5). The data is adjusted for what has been selected and start again at 1.

1) suggests that weights should be used to define the preferences at each step. The two expressions in 2) should be balanced as shown in 3). The optimisation of 3) gives a solution where equal weight is given to the (absolute value of) relative increase in these terms. The improvement is evaluated as suggested in 4).

The main motivation for this approach is the prediction aspect of the model. The prediction variance for a standard regression model is,

Var(Y(x₀)) = [Y^T(I-X(X^TX)^-1X^T)Y]´x₀^T(X^TX)^-1x₀/(N-K)

Assuming normal distribution for the data the precision, (X^TX)^-1, and residual variation, [Y^T(I-X(X^TX)^-1X^T)Y], are stochastically independent, hence both need to be modelled. The optimization of a product in 3) is due to the symmetry of these two terms in the prediction variance.

Application to linear regression

It is desired to find a score vector t, t=Xw. There are two basic aspects of the modeling task of the score vector:

a) the size of the  improvement in fit, |Y^Tt|²/(t^Tt)

b) the associated precision in terms of the variance, s²/(t^Tt)

Balancing these two terms as suggested in 3) we get

  maximise [|Y^Tt|²/(t^Tt)]´[1/(s²/(t^Tt))]=maximise |Y^Tt|²/s² =maximise w^TX^TYY^TXw/s² for |w|=1.

Assuming s² constant the solution is to choose w as the eigen vector associated with the leading eigen value to

X^TYY^TXw =lw.

This criterion leads to PLS regression. The H-method uses this eigen value system to find the weight vector w at each step. But the H-method identifies the part of X^TY that should be used in the eigen value system.