Semi-invariant

Semi - invariants , or semi-invariants , or cumulants are coefficients in the expansion of the logarithm of the characteristic function of a random variable in the Maclaurin series .

Content

Definition

Through the characteristic function

Semi-invariants, unlike moments, cannot be determined directly through the distribution function p ( x ). They are determined either through the logarithm of the characteristic function G ( u ), or through the moments μ (the second definition, in fact, follows from the first). Formally, semi-invariants are defined as coefficients in the expansion in the MacLoren series of the logarithm of the characteristic function in the same way as the moments for the characteristic function itself are determined:

\ln G(u)\,=\,iu\kappa _{1}+{\frac {(iu)^{2}}{2!}}\kappa _{2}+\ldots +{\frac {(iu)^{n}}{n!}}\kappa _{n}+\ldots =\sum \limits _{n=1}^{\infty }{\frac {(iu)^{n}}{n!}}\kappa _{n}

{\ displaystyle \ ln G (u) \, = \, iu \ kappa _ {1} + {\ frac {(iu) ^ {2}} {2!}} \ kappa _ {2} + \ ldots + { \ frac {(iu) ^ {n}} {n!}} \ kappa _ {n} + \ ldots = \ sum \ limits _ {n = 1} ^ {\ infty} {\ frac {(iu) ^ { n}} {n!}} \ kappa _ {n}}

The only difference is that the first member of this series is assumed to be 0, not 1 as in the case of moments. Thus, the logarithm of the characteristic function is a generating function for semi-invariants, it is sometimes called the second characteristic function and is denoted by:

\varphi (u)\,\equiv \,\ln G(u).

{\ displaystyle \ varphi (u) \, \ equiv \, \ ln G (u).}

The interest in this function is due to the fact that it is additive for independent random variables, that is, for the sum of such quantities it is equal to the sum of the corresponding functions for each quantity:

\ln G(a+b)=\ln G(a)+\ln G(b).

{\ displaystyle \ ln G (a + b) = \ ln G (a) + \ ln G (b).}

This obviously follows from the fact that the characteristic function is multiplicative in independent random variables (equal to the product of the corresponding functions). The same property, as a consequence, is inherent in semi-invariants: in particular, since the mathematical expectation and variance are the first and second semi-invariants of a random variable, then for the sum of independent random variables they are respectively equal to the sum of mathematical expectations or variances of the quantities themselves. (This is also true for the third central moment , which therefore coincides with the third semi-invariant. For fourth and higher centered moments, this equality is no longer fulfilled.) This property simplifies the work with cumulants, since for them, unlike the moments of distribution of the sum of independent random quantities having a rather cumbersome expression through the moments of the quantities themselves, expression through semi-invariant terms is very simple.

From the definition of the MacLoren series, the semi-invariant of order n is defined as:

\kappa _{n}\,=\,(-i)^{n}\left.{\frac {\,\partial ^{n}\varphi }{\,\partial u^{n}}}\right|_{u=0}.

{\ displaystyle \ kappa _ {n} \, = \, (- i) ^ {n} \ left. {\ frac {\, \ partial ^ {n} \ varphi} {\, \ partial u ^ {n} }} \ right | _ {u = 0}.}

In particular, for the first semi-invariant we have:

\kappa _{1}\,=\,-i\left.{\frac {\,\partial \varphi }{\,\partial u}}\right|_{u=0}\,=\,-i\left.{\frac {G'_{u}}{\,G(u)}}\right|_{u=0}.

{\ displaystyle \ kappa _ {1} \, = \, - i \ left. {\ frac {\, \ partial \ varphi} {\, \ partial u}} \ right | _ {u = 0} \, = \, - i \ left. {\ frac {G '_ {u}} {\, G (u)}} \ right | _ {u = 0}.}

Through moments

We now derive an alternative definition of the semi-invariant in terms of moments. Expanding the characteristic function G ( u ) in a MacLoren series in moments, we can rewrite the first formula in the following form:

\ln \left\{1+\sum \limits _{n=1}^{\infty }{\frac {(iu)^{n}}{n!}}\mu _{n}\right\}\,=\,\sum \limits _{n=1}^{\infty }{\frac {(iu)^{n}}{n!}}\kappa _{n}

{\ displaystyle \ ln \ left \ {1+ \ sum \ limits _ {n = 1} ^ {\ infty} {\ frac {(iu) ^ {n}} {n!}} \ mu _ {n} \ right \} \, = \, \ sum \ limits _ {n = 1} ^ {\ infty} {\ frac {(iu) ^ {n}} {n!}} \ kappa _ {n}}

Expanding the logarithm in a MacLoren series and assuming that the conditions for its radius of convergence are satisfied, we get:

\sum \limits _{m=1}^{\infty }(-1)^{m+1}{\frac {\displaystyle \left(\sum \limits _{n=1}^{\infty }{\frac {(iu)^{n}}{n!}}\mu _{n}\right)^{\!\!m}}{m}}\,=\,\sum \limits _{n=1}^{\infty }{\frac {(iu)^{n}}{n!}}\kappa _{n}

{\ displaystyle \ sum \ limits _ {m = 1} ^ {\ infty} (- 1) ^ {m + 1} {\ frac {\ displaystyle \ left (\ sum \ limits _ {n = 1} ^ {\ infty} {\ frac {(iu) ^ {n}} {n!}} \ mu _ {n} \ right) ^ {\! \! m}} {m}} \, = \, \ sum \ limits _ {n = 1} ^ {\ infty} {\ frac {(iu) ^ {n}} {n!}} \ kappa _ {n}}

Equating the coefficients for equal powers of iu in the sums on the left and on the right, we obtain:

{\begin{cases}\kappa _{1}=\mu _{1}\\[1mm]\kappa _{2}=-\mu _{1}^{2}+\mu _{2}\\[1mm]\kappa _{3}=2\mu _{1}^{3}-3\mu _{1}\mu _{2}+\mu _{3}\\[1mm]\;\ldots \end{cases}}

{\ displaystyle {\ begin {cases} \ kappa _ {1} = \ mu _ {1} \\ [1mm] \ kappa _ {2} = - \ mu _ {1} ^ {2} + \ mu _ { 2} \\ [1mm] \ kappa _ {3} = 2 \ mu _ {1} ^ {3} -3 \ mu _ {1} \ mu _ {2} + \ mu _ {3} \\ [1mm ] \; \ ldots \ end {cases}}}

An interesting derivative-based method for finding these relationships more easily, and these expressions for higher orders are described by Kendall. He also gives a general formula for finding moments through semi-invariants and vice versa, the same formula is also found in Shiryaev . By the way, in some literature this general formula is called the Shiryaev – Leontiev formula, although apparently they were not the first to derive it.

History

The semi-invariants were introduced by Danish astronomer and mathematician Torvald Nikolai Thiele in 1889 (according to other sources in 1903). The Russian language also uses the name semi-invariants (from Latin semi-, which means half-, half). Thiele called these statistical quantities semi-invariant, and until the 1930s they were called that way until the English statistician Fisher suggested the name cumulants , because of their cumulative properties, and over time this name was fixed in literature. Nevertheless, in Russian-language literature, preference was always given to the original name, for example, Shiryaev uses only the original Latin name. The Greek letter κ is almost always used to denote the semi-invariant, although, for example, Shiryaev uses ξ.

Despite the fact that the semi-invariants were introduced long ago, very little attention was paid to them; it was only in the late 1930s that Fisher first conducted a systematic study of semi-invariants.

Today, semi-invariants have firmly entered the world of modern statistics and its applications. In particular, they are very widely used in the field of signal processing, which is associated with some of their useful properties: for example, all semi-invariants of the third and higher orders are equal to zero for normal processes, and mixed semi-invariants of all orders of statistically independent quantities are equal to zero. Using the concept of semi-invariants, we can introduce a more general concept of statistical independence of two quantities up to the nth order, implying that all mixed semi-invariants of order up to n (inclusive) are equal to zero.