Huber Loss Function

The Huber loss function is a loss function used in stable regression , which is less sensitive to outliers than the quadratic error.

Definition

Huber loss function (green,

\delta =1

{\ displaystyle \ delta = 1}

) and the quadratic loss function (blue) as a function of

y-f(x)

{\ displaystyle yf (x)}

The Huber loss function sets a penalty for the evaluation procedure. Huber (1964) described it as a piecewise function of the form: ^[1]

L_{\delta }(a)={\begin{cases}{\frac {1}{2}}{a^{2}}&{\text{для }}|a|\leq \delta ,\\\delta (|a|-{\frac {1}{2}}\delta ),&{\text{иначе.}}\end{cases}}

{\ displaystyle L _ {\ delta} (a) = {\ begin {cases} {\ frac {1} {2}} {a ^ {2}} & {\ text {for}} | a | \ leq \ delta , \\\ delta (| a | - {\ frac {1} {2}} \ delta), & {\ text {otherwise.}} \ end {cases}}}

This function is quadratic for small values of a , and linear for large values, with the same value and slope for different sections of two points where $|a|=\delta$ ${\ displaystyle | a | = \ delta}$ . The variable a is often considered as the remainder, i.e., as the difference between the observed and predicted value $a=y-f(x)$ ${\ displaystyle a = yf (x)}$ , therefore, the original definition can be extended to ^[2] : $L_{\delta }(y,f(x))={\begin{cases}{\frac {1}{2}}(y-f(x))^{2}&{\text{для }}|y-f(x)|\leq \delta ,\\\delta \,|y-f(x)|-{\frac {1}{2}}\delta ^{2}&{\text{иначе.}}\end{cases}}$ ${\ displaystyle L _ {\ delta} (y, f (x)) = {\ begin {cases} {\ frac {1} {2}} (yf (x)) ^ {2} & {\ text {for} } | yf (x) | \ leq \ delta, \\\ delta \, | yf (x) | - {\ frac {1} {2}} \ delta ^ {2} & {\ text {otherwise.}} \ end {cases}}}$

Notes

↑ Huber, Peter J. Robust Estimation of a Location Parameter (English) // Annals of Statistics : journal. - 1964. - Vol. 53 , no. 1 . - P. 73-101 . - DOI : 10.1214 / aoms / 1177703732 .
↑ Hastie, Trevor. The Elements of Statistical Learning / Trevor Hastie, Robert Tibshirani, Jerome Friedman. - 2009. - P. 349. Archived copy of January 26, 2015 by Wayback Machine Compared to Hastie et al. , the loss is scaled by a factor of ½, to be consistent with Huber's original definition given earlier.

[1] Huber, Peter J. Robust Estimation of a Location Parameter (English) // Annals of Statistics : journal. - 1964. - Vol. 53 , no. 1 . - P. 73-101 . - DOI : 10.1214 / aoms / 1177703732 .

[2] Hastie, Trevor. The Elements of Statistical Learning / Trevor Hastie, Robert Tibshirani, Jerome Friedman. - 2009. - P. 349. Archived copy of January 26, 2015 by Wayback Machine Compared to Hastie et al. , the loss is scaled by a factor of ½, to be consistent with Huber's original definition given earlier.

Huber Loss Function

Definition

Notes

More articles: