Deming regression

The two-dimensional case of the method of least complete squares (the Deming regression). The red bars show the error in both x and y , which differs from the traditional least squares method, in which the error is measured only along the y axis. The case is shown when the deviation is measured perpendicularly, which occurs when x and y have equal variances.

In the statistics, the Deming regression , named W.C. Deming , is a kind of that tries to find the best smoothing line for a two-dimensional data set. Regression differs from in that it takes into account the in the observation in both the x axis and the y axis. Regression is a special case of the method of least complete squares , which considers any number of indicators and has a more complex structure of errors.

The Deming regression is equivalent to the maximum likelihood estimate for , in which the errors of the two variables are considered independent and have a normal distribution , and the ratio of their variances, δ , is known ^[1] . In practice, this ratio can be estimated from the source data. However, the regression procedure does not take into account possible errors in estimating the variance relations.

Deming's regression is only slightly more complicated than . Most of the statistical packages used in clinical chemistry provide a Deming regression.

The model was originally proposed by Adcock ^[2] , who considered the case δ = 1, and then was considered in a more general form by Kummellem ^[3] with arbitrary δ . However, their ideas remained largely unnoticed for more than 50 years, until Kupmans ^[4] revived them and later distributed Deming ^[5] . The book of the latter has become so popular in clinical chemistry and related fields that the method in these areas was called the Deming regression ^[6] .

Content

Specification

Suppose that the data ( y _i , x _i ) are values obtained during measurements of the "true" values ( y _i * , x _i * ) that lie on the regression line:

{\begin{aligned}y_{i}&=y_{i}^{*}+\varepsilon _{i},\\x_{i}&=x_{i}^{*}+\eta _{i},\end{aligned}}

{\ displaystyle {\ begin {aligned} y_ {i} & = y_ {i} ^ {*} + \ varepsilon _ {i}, \\ x_ {i} & = x_ {i} ^ {*} + \ eta _ {i}, \ end {aligned}}}

where the errors ε and η are independent and the ratio of their variances is known:

\delta ={\frac {\sigma _{\varepsilon }^{2}}{\sigma _{\eta }^{2}}}.

{\ displaystyle \ delta = {\ frac {\ sigma _ {\ varepsilon} ^ {2}} {\ sigma _ {\ eta} ^ {2}}}.}

In practice, the variance of parameters $x$ ${\ displaystyle x}$ and $y$ ${\ displaystyle y}$ often unknown, which complicates the assessment $\delta$ ${\ displaystyle \ delta}$ . Note that when the measurement method $x$ ${\ displaystyle x}$ and $y$ ${\ displaystyle y}$ the same, these dispersions are most likely equal, so in this case $\delta =1$ ${\ displaystyle \ delta = 1}$ .

We are trying to find the direct "best smoothing"

y^{*}=\beta _{0}+\beta _{1}x^{*},

{\ displaystyle y ^ {*} = \ beta _ {0} + \ beta _ {1} x ^ {*},}

such that the weighted sum of squares of residuals is minimal ^[7]

SSR=\sum _{i=1}^{n}{\bigg (}{\frac {\varepsilon _{i}^{2}}{\sigma _{\varepsilon }^{2}}}+{\frac {\eta _{i}^{2}}{\sigma _{\eta }^{2}}}{\bigg )}={\frac {1}{\sigma _{\varepsilon }^{2}}}\sum _{i=1}^{n}{\Big (}(y_{i}-\beta _{0}-\beta _{1}x_{i}^{*})^{2}+\delta (x_{i}-x_{i}^{*})^{2}{\Big )}\ \to \ \min _{\beta _{0},\beta _{1},x_{1}^{*},\ldots ,x_{n}^{*}}SSR

{\ displaystyle SSR = \ sum _ {i = 1} ^ {n} {\ bigg (} {\ frac {\ varepsilon _ {i} ^ {2}} {\ sigma _ {\ varepsilon} ^ {2}} } + {\ frac {\ eta _ {i} ^ {2}} {\ sigma _ {\ eta} ^ {2}}} {\ bigg)} = {\ frac {1} {\ sigma _ {\ varepsilon } ^ {2}}} \ sum _ {i = 1} ^ {n} {\ Big (} (y_ {i} - \ beta _ {0} - \ beta _ {1} x_ {i} ^ {* }) ^ {2} + \ delta (x_ {i} -x_ {i} ^ {*}) ^ {2} {\ Big)} \ \ to \ \ min _ {\ beta _ {0}, \ beta _ {1}, x_ {1} ^ {*}, \ ldots, x_ {n} ^ {*}} SSR}

Solution

The solution can be expressed in terms of second order moments. That is, we first calculate the following values (all sums are taken by i = 1: n ):

{\begin{aligned}&{\overline {x}}={\frac {1}{n}}\sum x_{i},\quad {\overline {y}}={\frac {1}{n}}\sum y_{i},\\&s_{xx}={\tfrac {1}{n-1}}\sum (x_{i}-{\overline {x}})^{2},\\&s_{xy}={\tfrac {1}{n-1}}\sum (x_{i}-{\overline {x}})(y_{i}-{\overline {y}}),\\&s_{yy}={\tfrac {1}{n-1}}\sum (y_{i}-{\overline {y}})^{2}.\end{aligned}}

{\ displaystyle {\ begin {aligned} & {\ overline {x}} = {\ frac {1} {n}} \ sum x_ {i}, \ quad {\ overline {y}} = {\ frac {1 } {n}} \ sum y_ {i}, \\ & s_ {xx} = {\ tfrac {1} {n-1}} \ sum (x_ {i} - {\ overline {x}}) ^ {2 }, \\ & s_ {xy} = {\ tfrac {1} {n-1}} \ sum (x_ {i} - {\ overline {x}}) (y_ {i} - {\ overline {y}} ), \\ & s_ {yy} = {\ tfrac {1} {n-1}} \ sum (y_ {i} - {\ overline {y}}) ^ {2}. \ end {aligned}}}

Finally, the least squares estimation parameters will be ^[8] :

{\begin{aligned}&{\hat {\beta }}_{1}={\frac {s_{yy}-\delta s_{xx}+{\sqrt {(s_{yy}-\delta s_{xx})^{2}+4\delta s_{xy}^{2}}}}{2s_{xy}}},\\&{\hat {\beta }}_{0}={\overline {y}}-{\hat {\beta }}_{1}{\overline {x}},\\&{\hat {x}}_{i}^{*}=x_{i}+{\frac {{\hat {\beta }}_{1}}{{\hat {\beta }}_{1}^{2}+\delta }}(y_{i}-{\hat {\beta }}_{0}-{\hat {\beta }}_{1}x_{i}).\end{aligned}}

{\ displaystyle {\ begin {aligned} & {\ hat {\ beta}} _ {1} = {\ frac {s_ {yy} - \ delta s_ {xx} + {\ sqrt {(s_ {yy} - \ delta s_ {xx}) ^ {2} +4 \ delta s_ {xy} ^ {2}}}} {2s_ {xy}}}, \\ & {\ hat {\ beta}} _ {0} = { \ overline {y}} - {\ hat {\ beta}} _ {1} {\ overline {x}}, \\ & {\ hat {x}} _ {i} ^ {*} = x_ {i} + {\ frac {{\ hat {\ beta}} _ {1}} {{\ hat {\ beta}} _ {1} ^ {2} + \ delta}} (y_ {i} - {\ hat { \ beta}} _ {0} - {\ hat {\ beta}} _ {1} x_ {i}). \ end {aligned}}}

Orthogonal Regression

In case of equality of error variances, i.e. when $\delta =1$ ${\ displaystyle \ delta = 1}$ , the Deming regression becomes an orthogonal regression — it minimizes the sum of squares of . In this case, we denote each sample point z _j on the complex plane (ie, the point ( x _j , y _j ) of the sample is written as z _j = x _j + iy _j , where i is the imaginary unit ). Let Z be the sum of squared differences from the sample points to the center of gravity (also represented in the complex coordinates). The center of gravity is the average of the sampling points. Then ^[9] :

If Z = 0, then any line passing through the center of gravity is a line of the best orthogonal smoothing.
If Z ≠ 0, the line of the best orthogonal smoothing passes through the center of gravity and is parallel to the vector from the origin to ${\sqrt {Z}}$ {\ displaystyle {\ sqrt {Z}}} .

Coolidge gave a trigonometric interpretation of the best orthogonal smoothing line in 1913 ^[10] .

Applications

In the case of three non - collinear points on the plane, the triangle formed by these points has a single Steiner ellipse inscribed that touches the sides of the triangle at mid-points. The main axis of this ellipse will be the orthogonal regression of these three vertices ^[11] .

Notes

↑ Linnet, 1993 .
↑ Adcock, 1878 .
↑ Kummell, 1879 .
↑ Koopmans, 1937 .
↑ Deming, 1943 .
↑ Cornbleet, Gochman, 1979 , p. 432–438.
↑ Fuller, 1987 , p. ch.1.3.3.
↑ Glaister, 2001 , p. 104-107.
↑ Minda, Phelps, 2008 , p. 679–689, Theorem 2.3.
↑ Coolidge, 1913 , p. 187–190.
↑ Minda, Phelps, 2008 , p. 679–689, Corollary 2.4.

Literature

RJ Adcock. A problem in least squares // The Analyst. - Annals of Mathematics, 1878. - Vol. 5 , no. 2 - pp . 53–54 . - DOI : 10.2307 / 2635758 .
JL Coolidge. The American Mathematical Monthly . - 1913. - V. 20 , no. 6 - p . 187–190 . - DOI : 10.2307 / 2973072 .
PJ Cornbleet, N. Gochman. Incorrect Least – Squares Regression Coefficients // Clin. Chem .. - 1979. - V. 25 , no. 3 - p . 432–438 . - PMID 262186 .
WE Deming. Statistical adjustment of data. - Wiley, NY (Dover Publications edition, 1985), 1943. - ISBN 0-486-64685-8 .
Wayne A. Fuller. Measurement error models. - John Wiley & Sons, Inc., 1987. - ISBN 0-471-86187-1 .
P. Glaister. Least squares revisited // The Mathematical Gazette . - 2001. - Vol. 85 March . - p . 104-107 .
TC Koopmans. Linear regression analysis of economic time series. - DeErven F. Bohn, Haarlem, Netherlands, 1937.
CH Kummell. Reduction of Observation Determinations. - Annals of Mathematics, 1879. - Vol. 6 , no. 4 - pp . 97–105 . - DOI : 10.2307 / 2635646 .
K. Linnet. Evaluation of regression procedures for method comparison studies // Clinical Chemistry. - 1993. - V. 39 , no. 3 - p . 424–432 . - PMID 8448852 .
D. Minda, S. Phelps. Triangles, ellipses, and cubic polynomials // American Mathematical Monthly . - 2008. - Vol. 115 , no. 8 - p . 679–689 .

[_4ff572a6be0b29f5-1] Linnet, 1993 .

[_f0fef5b004b5769a-2] Adcock, 1878 .

[_2efe7f5e8233b41d-3] Kummell, 1879 .

[_d2e6d77b0aa8f121-4] Koopmans, 1937 .

[_e6e6d2fa0b36b3f2-5] Deming, 1943 .

[_fb8f114b6ba30f23-6] Cornbleet, Gochman, 1979 , p. 432–438.

[_249c7ae98781b144-7] Fuller, 1987 , p. ch.1.3.3.

[_1842661923c287b1-8] Glaister, 2001 , p. 104-107.

[_79fc54a63194ca43-9] Minda, Phelps, 2008 , p. 679–689, Theorem 2.3.

[_3dcccce40b3e8156-10] Coolidge, 1913 , p. 187–190.

[_92495f2774f7ffcb-11] Minda, Phelps, 2008 , p. 679–689, Corollary 2.4.