Decomposition

Decomposition of data, an example

Suppose that S=X^TX+U, with U positive semi-definite matrix. Algorithms have been developed of approximating the exact solution B. In the analysis each term of the decompositions is evaluated. A terms are used, if it is judged that further terms do not improve the prediction ability of the model.

An example of decompositions in linear regression:

S = d₁ p₁ p₁^T + … + d_A p_A p_A^T+ … + d_K p_K p_K^T = PDP^T

S^-1 = d₁ v₁ v₁^T + … + d_A v_A v_A^T + … + d_K v_K v_K^T = VDV^T

X = d₁t₁p₁^T + … + d_A t_A p_A^T +… + d_Kt_Kp_K^T = TDP^T

X^TY = d₁p₁q₁^T + … + d_A p_A q_A^T +… + d_Kp_Kq_K^T = PDQ^T

B = d₁ v₁ q₁^T + … + d_A v_A q_A^T +… + d_Kv_Kq_K^T = VDQ^T

Ŷ=XB= d₁ t₁ q₁^T + … + d_A t_A q_A^T +… + d_Kt_Kq_K^T = TDQ^T

The A-th terms are the result of an optimization of the prediction ability of the model at that step. This is evaluated for the reduced data, i.e., when results from the first (A-1) steps have been subtracted from the data.

S =	d₁ p₁ p₁^T + …	+ d_A p_A p_A^T+ …	+ d_K p_K p_K^T	= PDP^T
S^-1 =	d₁ v₁ v₁^T + …	+ d_A v_A v_A^T + …	+ d_K v_K v_K^T	= VDV^T
X =	d₁t₁p₁^T + …	+ d_A t_A p_A^T +…	+ d_Kt_Kp_K^T	= TDP^T
X^TY =	d₁p₁q₁^T + …	+ d_A p_A q_A^T +…	+ d_Kp_Kq_K^T	= PDQ^T
B =	d₁ v₁ q₁^T + …	+ d_A v_A q_A^T +…	+ d_Kv_Kq_K^T	= VDQ^T
Ŷ=XB=	d₁ t₁ q₁^T + …	+ d_A t_A q_A^T +…	+ d_Kt_Kq_K^T	= TDQ^T