Matrix rate

The norm of a matrix is the norm in the linear space of matrices, usually related in some way to the corresponding vector norm (consistentor subordinate ).

Content

1 Definition
2 Operator Standards
- 2.1 Examples of operator norms
3 Non-operator matrix norms
- 3.1 Example non-operator norm
4 Examples of norms
- 4.1 Norm L _{p, q}
  - 4.1.1 Vector $p$ ${\ displaystyle p}$ $p$ -norm
  - 4.1.2 Frobenius norm
  - 4.1.3 Module maximum
- 4.2 Norm Shatten
5 Consistency of matrix and vector norms
6 Equivalence of norms
7 Application
8 See also
9 notes
10 Literature
11 Links

Definition

Let K be the main field (usually K = R or K = C ) and $K^{m\times n}$ ${\ displaystyle K ^ {m \ times n}}$ Is the linear space of all matrices with m rows and n columns consisting of elements of K. The norm is given on the matrix space if each matrix $A\in K^{m\times n}$ ${\ displaystyle A \ in K ^ {m \ times n}}$ associates a non-negative real number $\|A\|$ ${\ displaystyle \ | A \ |}$ called its norm, so that

$\|A\|>0$ ${\ displaystyle \ | A \ |> 0}$ , if $A\neq 0$ ${\ displaystyle A \ neq 0}$ , and $\|A\|=0$ ${\ displaystyle \ | A \ | = 0}$ , if $A=0$ ${\ displaystyle A = 0}$ .
$\|A+B\|\leq \|A\|+\|B\|,\quad A,B\in K^{m\times n}$ ${\ displaystyle \ | A + B \ | \ leq \ | A \ | + \ | B \ |, \ quad A, B \ in K ^ {m \ times n}}$ .
$\|\alpha A\|=|\alpha |\|A\|,\quad \alpha \in K,\quad A\in K^{m\times n}$ ${\ displaystyle \ | \ alpha A \ | = | \ alpha | \ | A \ |, \ quad \ alpha \ in K, \ quad A \ in K ^ {m \ times n}}$ ^[1] .

In the case of square matrices (i.e. m = n ), the matrices can be multiplied without leaving the space, and therefore the norms in these spaces usually also satisfy the submultiplicative property

$\|AB\|\leq \|A\|\|B\|$ ${\ displaystyle \ | AB \ | \ leq \ | A \ | \ | | B \ |}$ for all matrices A and B in $K^{n\times n}$ ${\ displaystyle K ^ {n \ times n}}$ .

Submultiplicativity can also be performed for the norms of non-square matrices, but determined for several necessary sizes at once. Namely, if A is a ℓ × m matrix, and B is an m × n matrix, then A B is a ℓ × n matrix.

Operator Norms

An important class of matrix norms is operator norms , also referred to as subordinate or induced . The operator norm is uniquely constructed according to the two norms defined in $K^{n}$ ${\ displaystyle K ^ {n}}$ and $K^{m}$ ${\ displaystyle K ^ {m}}$ based on the fact that every m × n matrix is represented by a linear operator from $K^{n}$ ${\ displaystyle K ^ {n}}$ at $K^{m}$ ${\ displaystyle K ^ {m}}$ . Specifically,

{\begin{aligned}\|A\|&=\sup\{\|Ax\|:x\in K^{n},\ \|x\|=1\}\\&=\sup \left\{{\frac {\|Ax\|}{\|x\|}}:x\in K^{n},\ x\neq 0\right\}.\end{aligned}}

{\ displaystyle {\ begin {aligned} \ | A \ | & = \ sup \ {\ | Ax \ |: x \ in K ^ {n}, \ \ | x \ | = 1 \} \\ & = \ sup \ left \ {{\ frac {\ | Ax \ |} {\ | x \ |}}: x \ in K ^ {n}, \ x \ neq 0 \ right \}. \ end {aligned}}}

^[2]

Provided that the norms on the vector spaces are coordinated, such a norm is submultiplicative (see above ).

Examples of operator norms

Matrix norm $\|A\|_{1}=\max \limits _{1\leq j\leq n}\sum _{i=1}^{m}|a_{ij}|$ ${\ displaystyle \ | A \ | _ {1} = \ max \ limits _ {1 \ leq j \ leq n} \ sum _ {i = 1} ^ {m} | a_ {ij} |}$ subordinate to the vector norm $\|x\|_{1}=\sum _{i=1}^{n}|x_{i}|$ ${\ displaystyle \ | x \ | _ {1} = \ sum _ {i = 1} ^ {n} | x_ {i} |}$ .
Matrix norm $\|A\|_{\infty }=\max \limits _{1\leq i\leq m}\sum _{j=1}^{n}|a_{ij}|$ ${\ displaystyle \ | A \ | _ {\ infty} = \ max \ limits _ {1 \ leq i \ leq m} \ sum _ {j = 1} ^ {n} | a_ {ij} |}$ subordinate to the vector norm $\|x\|_{\infty }=\max \limits _{1\leq i\leq n}|x_{i}|$ ${\ displaystyle \ | x \ | _ {\ infty} = \ max \ limits _ {1 \ leq i \ leq n} | x_ {i} |}$ .
Spectral norm $\|A\|_{2}=\sup \limits _{\|x\|_{2}=1}\|Ax\|_{2}=\sup \limits _{(x,x)=1}{\sqrt {(Ax,Ax)}}={\sqrt {\lambda _{max}(A^{*}A)}}$ ${\ displaystyle \ | A \ | _ {2} = \ sup \ limits _ {\ | x \ | _ {2} = 1} \ | Ax \ | _ {2} = \ sup \ limits _ {(x, x) = 1} {\ sqrt {(Ax, Ax)}} = {\ sqrt {\ lambda _ {max} (A ^ {*} A)}}}$ subordinate to the vector norm $\|x\|_{2}={\sqrt {\sum _{i=1}^{n}|x_{i}|^{2}}}$ ${\ displaystyle \ | x \ | _ {2} = {\ sqrt {\ sum _ {i = 1} ^ {n} | x_ {i} | ^ {2}}}}$ .

Properties of the spectral norm:

The spectral norm of an operator is equal to the maximum singular number of this operator.
The spectral norm of a normal operator is equal to the absolute value of the maximum modulo eigenvalue of this operator.
The spectral norm does not change when the matrix is multiplied by the orthogonal ( unitary ) matrix.

Non-operator matrix norms

There are norms of matrices that are not operator. The concept of non-operator matrix norms was introduced by Yu. I. Lyubich ^[3] and investigated by G. R. Belitsky .

Non-Operative Rate Example

For example, consider two different operator norms $\|A\|_{1}$ ${\ displaystyle \ | A \ | _ {1}}$ and $\|A\|_{2}$ ${\ displaystyle \ | A \ | _ {2}}$ e.g. row and column norms. Form a new norm $\|A\|=\max {(\|A\|_{1},\|A\|_{2})}$ ${\ displaystyle \ | A \ | = \ max {(\ | A \ | _ {1}, \ | A \ | _ {2})}}$ . The new norm has a ring property $\|AB\|\leqslant \|A\|\|B\|$ ${\ displaystyle \ | AB \ | \ leqslant \ | A \ | \ | B \ |}$ preserves unit $\|I\|=1$ ${\ displaystyle \ | I \ | = 1}$ and is not an operator ^[4] .

Norm Examples

Norm L _{p, q}

Let be $(a_{1},\ldots ,a_{n})$ ${\ displaystyle (a_ {1}, \ ldots, a_ {n})}$ Is a vector from the columns of the matrix $A .$ ${\ displaystyle A.}$ Norm $L_{2,1}$ ${\ displaystyle L_ {2,1}}$ by definition, it is equal to the sum of the Euclidean norms of the columns of the matrix:

\Vert A\Vert _{2,1}=\sum _{j=1}^{n}\Vert a_{j}\Vert _{2}=\sum _{j=1}^{n}\left(\sum _{i=1}^{m}|a_{ij}|^{2}\right)^{1/2}

{\ displaystyle \ Vert A \ Vert _ {2,1} = \ sum _ {j = 1} ^ {n} \ Vert a_ {j} \ Vert _ {2} = \ sum _ {j = 1} ^ { n} \ left (\ sum _ {i = 1} ^ {m} | a_ {ij} | ^ {2} \ right) ^ {1/2}}

Norm $L_{2,1}$ ${\ displaystyle L_ {2,1}}$ can be generalized to normal $L_{p,q},\;p,q\geqslant 1:$ ${\ displaystyle L_ {p, q}, \; p, q \ geqslant 1:}$

\Vert A\Vert _{p,q}=\left(\sum _{j=1}^{n}\left(\sum _{i=1}^{m}|a_{ij}|^{p}\right)^{q/p}\right)^{1/q}

{\ displaystyle \ Vert A \ Vert _ {p, q} = \ left (\ sum _ {j = 1} ^ {n} \ left (\ sum _ {i = 1} ^ {m} | a_ {ij} | ^ {p} \ right) ^ {q / p} \ right) ^ {1 / q}}

Vector $p$ ${\ displaystyle p}$ Norma

Can be considered $m\times n$ ${\ displaystyle m \ times n}$ matrix as a size vector $m n$ ${\ displaystyle mn}$ and use standard vector norms. For example, from the norm $L_{p,q}$ ${\ displaystyle L_ {p, q}}$ at $p=q$ ${\ displaystyle p = q}$ it turns out the vector p- norm:

\|A\|_{p}=\|\mathrm {vec} (A)\|_{p}=\left(\sum _{i=1}^{m}\sum _{j=1}^{n}|a_{ij}|^{p}\right)^{1/p}

{\ displaystyle \ | A \ | _ {p} = \ | \ mathrm {vec} (A) \ | _ {p} = \ left (\ sum _ {i = 1} ^ {m} \ sum _ {j = 1} ^ {n} | a_ {ij} | ^ {p} \ right) ^ {1 / p}}

This norm is different from the induced p- norm. $\|A\|_{p}=\sup \limits _{x\neq 0}{\frac {\|Ax\|_{p}}{\|x\|_{p}}}$ ${\ displaystyle \ | A \ | _ {p} = \ sup \ limits _ {x \ neq 0} {\ frac {\ | Ax \ | _ {p}} {\ | x \ | _ {p}}} }$ and from the p- norm of Schatten (see below), although the same notation is used.

Norm Frobenius

The Frobenius norm , or Euclidean norm, is a special case of the p- norm for p = 2 : $\|A\|_{F}={\sqrt {\sum _{i=1}^{m}\sum _{j=1}^{n}a_{ij}^{2}}}$ ${\ displaystyle \ | A \ | _ {F} = {\ sqrt {\ sum _ {i = 1} ^ {m} \ sum _ {j = 1} ^ {n} a_ {ij} ^ {2}} }}$ .

The Frobenius norm is easily calculated (compared, for example, with the spectral norm). It has the following properties:

Coherence : $\|Ax\|_{2}\leq \|A\|_{F}\|x\|_{2}$ ${\ displaystyle \ | Ax \ | _ {2} \ leq \ | A \ | _ {F} \ | x \ | _ {2}}$ , since by virtue of the Cauchy-Bunyakovsky inequality

\|Ax\|_{2}^{2}=\sum _{i=1}^{m}\left|\sum _{j=1}^{n}a_{ij}x_{j}\right|^{2}\leq \sum _{i=1}^{m}\left(\sum _{j=1}^{n}|a_{ij}|^{2}\sum _{j=1}^{n}|x_{j}|^{2}\right)=\sum _{j=1}^{n}|x_{j}|^{2}\|A\|_{F}^{2}=\|A\|_{F}^{2}\|x\|_{2}^{2}.

{\ displaystyle \ | Ax \ | _ {2} ^ {2} = \ sum _ {i = 1} ^ {m} \ left | \ sum _ {j = 1} ^ {n} a_ {ij} x_ { j} \ right | ^ {2} \ leq \ sum _ {i = 1} ^ {m} \ left (\ sum _ {j = 1} ^ {n} | a_ {ij} | ^ {2} \ sum _ {j = 1} ^ {n} | x_ {j} | ^ {2} \ right) = \ sum _ {j = 1} ^ {n} | x_ {j} | ^ {2} \ | A \ | _ {F} ^ {2} = \ | A \ | _ {F} ^ {2} \ | x \ | _ {2} ^ {2}.}

Submultiplicativity : $\|AB\|_{F}\leq \|A\|_{F}\|B\|_{F}$ ${\ displaystyle \ | AB \ | _ {F} \ leq \ | A \ | _ {F} \ | B \ | _ {F}}$ , as $\|AB\|_{F}^{2}=\sum _{i,j}\left|\sum _{k}a_{ik}b_{kj}\right|^{2}\leq \sum _{i,j}\left(\sum _{k}|a_{ik}||b_{kj}|\right)^{2}\leq \sum _{i,j}\left(\sum _{k}|a_{ik}|^{2}\sum _{k}|b_{kj}|^{2}\right)=\sum _{i,k}|a_{ik}|^{2}\sum _{k,j}|b_{kj}|^{2}=\|A\|_{F}^{2}\|B\|_{F}^{2}$ ${\ displaystyle \ | AB \ | _ {F} ^ {2} = \ sum _ {i, j} \ left | \ sum _ {k} a_ {ik} b_ {kj} \ right | ^ {2} \ leq \ sum _ {i, j} \ left (\ sum _ {k} | a_ {ik} || b_ {kj} | \ right) ^ {2} \ leq \ sum _ {i, j} \ left ( \ sum _ {k} | a_ {ik} | ^ {2} \ sum _ {k} | b_ {kj} | ^ {2} \ right) = \ sum _ {i, k} | a_ {ik} | ^ {2} \ sum _ {k, j} | b_ {kj} | ^ {2} = \ | A \ | _ {F} ^ {2} \ | B \ | _ {F} ^ {2}}$ .
$\|A\|_{F}^{2}=\mathop {\rm {tr}} A^{*}A=\mathop {\rm {tr}} AA^{*}$ {\ displaystyle \ | A \ | _ {F} ^ {2} = \ mathop {\ rm {tr}} A ^ {*} A = \ mathop {\ rm {tr}} AA ^ {*}} where $\mathop {\rm {tr}} A$ ${\ displaystyle \ mathop {\ rm {tr}} A}$ - matrix trace $A$ ${\ displaystyle A}$ , $A^{*}$ ${\ displaystyle A ^ {*}}$ Is a Hermitian conjugate matrix .
$\|A\|_{F}^{2}=\rho _{1}+\rho _{2}+\dots +\rho _{n}$ ${\ displaystyle \ | A \ | _ {F} ^ {2} = \ rho _ {1} + \ rho _ {2} + \ dots + \ rho _ {n}}$ where $\rho _{1},\rho _{2},\dots ,\rho _{n}$ ${\ displaystyle \ rho _ {1}, \ rho _ {2}, \ dots, \ rho _ {n}}$ - singular matrix numbers $A$ ${\ displaystyle A}$ .
$\|A\|_{F}$ ${\ displaystyle \ | A \ | _ {F}}$ does not change upon matrix multiplication $A$ ${\ displaystyle A}$ left or right on orthogonal ( unitary ) matrices ^[5] .

Module Maximum

The modulus maximum norm is another special case of the p- norm for p = ∞ .

\|A\|_{\text{max}}=\max\{|a_{ij}|\}.

{\ displaystyle \ | A \ | _ {\ text {max}} = \ max \ {| a_ {ij} | \}.}

Norm Schatten

Coherence of matrix and vector norms

Matrix norm $\|\cdot \|_{ab}$ ${\ displaystyle \ | \ cdot \ | _ {ab}}$ on $K^{m\times n}$ ${\ displaystyle K ^ {m \ times n}}$ called consistent with standards $\|\cdot \|_{a}$ ${\ displaystyle \ | \ cdot \ | _ {a}}$ on $K^{n}$ ${\ displaystyle K ^ {n}}$ and $\|\cdot \|_{b}$ ${\ displaystyle \ | \ cdot \ | _ {b}}$ on $K^{m}$ ${\ displaystyle K ^ {m}}$ , if:

\|Ax\|_{b}\leq \|A\|_{ab}\|x\|_{a}

{\ displaystyle \ | Ax \ | _ {b} \ leq \ | A \ | _ {ab} \ | x \ | _ {a}}

for any $A\in K^{m\times n},x\in K^{n}$ ${\ displaystyle A \ in K ^ {m \ times n}, x \ in K ^ {n}}$ . The operator norm by construction is consistent with the original vector norm.

Examples of consistent, but not subordinate matrix norms:

Euclidean norm $\|A\|_{F}={\sqrt {\sum _{i=1}^{n}\sum _{j=1}^{m}a_{ij}^{2}}}$ ${\ displaystyle \ | A \ | _ {F} = {\ sqrt {\ sum _ {i = 1} ^ {n} \ sum _ {j = 1} ^ {m} a_ {ij} ^ {2}} }}$ consistent with vector norm $\|x\|_{2}={\sqrt {\sum _{i=1}^{n}x_{i}^{2}}}$ ${\ displaystyle \ | x \ | _ {2} = {\ sqrt {\ sum _ {i = 1} ^ {n} x_ {i} ^ {2}}}}$ ^[5] .
Norm $\|A\|=\sum _{i,j=1}^{n}|a_{ij}|$ ${\ displaystyle \ | A \ | = \ sum _ {i, j = 1} ^ {n} | a_ {ij} |}$ consistent with vector norm $\|x\|_{1}=\sum _{i=1}^{n}|x_{i}|$ ${\ displaystyle \ | x \ | _ {1} = \ sum _ {i = 1} ^ {n} | x_ {i} |}$ ^[6] .

Equivalence of norms

All norms in space $K^{m\times n}$ ${\ displaystyle K ^ {m \ times n}}$ are equivalent, that is, for any two norms $\|.\|_{\alpha }$ ${\ displaystyle \ |. \ | _ {\ alpha}}$ and $\|.\|_{\beta }$ ${\ displaystyle \ |. \ | _ {\ beta}}$ and for any matrix $A\in K^{m\times n}$ ${\ displaystyle A \ in K ^ {m \ times n}}$ double inequality is true:

C_{1}\|A\|_{\alpha }\leq \|A\|_{\beta }\leq C_{2}\|A\|_{\alpha },

{\ displaystyle C_ {1} \ | A \ | _ {\ alpha} \ leq \ | A \ | _ {\ beta} \ leq C_ {2} \ | A \ | _ {\ alpha},}

where are the constants $C_{1}$ ${\ displaystyle C_ {1}}$ and $C_{2}$ ${\ displaystyle C_ {2}}$ independent of matrix $A$ ${\ displaystyle A}$ .

For $A\in \mathbb {R} ^{m\times n}$ ${\ displaystyle A \ in \ mathbb {R} ^ {m \ times n}}$ inequalities are true:

$\|A\|_{2}\leq \|A\|_{F}\leq {\sqrt {n}}\|A\|_{2}$ ${\ displaystyle \ | A \ | _ {2} \ leq \ | A \ | _ {F} \ leq {\ sqrt {n}} \ | A \ | _ {2}}$ ,
$\|A\|_{\text{max}}\leq \|A\|_{2}\leq {\sqrt {mn}}\|A\|_{\text{max}}$ ${\ displaystyle \ | A \ | _ {\ text {max}} \ leq \ | A \ | _ {2} \ leq {\ sqrt {mn}} \ | A \ | _ {\ text {max}}}$ ,
${\frac {1}{\sqrt {n}}}\|A\|_{\infty }\leq \|A\|_{2}\leq {\sqrt {m}}\|A\|_{\infty }$ ${\ displaystyle {\ frac {1} {\ sqrt {n}}} \ | A \ | _ {\ infty} \ leq \ | A \ | _ {2} \ leq {\ sqrt {m}} \ | A \ | _ {\ infty}}$ ,
${\frac {1}{\sqrt {m}}}\|A\|_{1}\leq \|A\|_{2}\leq {\sqrt {n}}\|A\|_{1}$ ${\ displaystyle {\ frac {1} {\ sqrt {m}}} \ | A \ | _ {1} \ leq \ | A \ | _ {2} \ leq {\ sqrt {n}} \ | A \ | _ {1}}$ ,

Where $\|A\|_{1}$ ${\ displaystyle \ | A \ | _ {1}}$ , $\|A\|_{2}$ ${\ displaystyle \ | A \ | _ {2}}$ and $\|A\|_{\infty }$ ${\ displaystyle \ | A \ | _ {\ infty}}$ - operator norms ^[7] .

Application

Matrix norms are often used in the analysis of computational methods of linear algebra. For example, a program for solving systems of linear algebraic equations may give an inaccurate result if the coefficient matrix is poorly conditioned (“almost degenerate ”). To quantify the proximity to degeneracy, one must be able to measure the distance in the matrix space. This possibility is provided by matrix norms ^[8] .

Notes

↑ Gantmakher, 1988 , p. 410.
↑ Prasolov, 1996 , p. 210.
↑ Lyubich Yu. I. On the operator norms of matrices // Uspekhi Mat . - 1963. - N. 18. Issue. 4 (112) - S. 161-164. - URL: http://mi.mathnet.ru/rus/umn/v18/i4/p161
↑ Belitsky, 1984 , p. 99.
↑ ¹ ² Ilyin, Kim, 1998 , p. 311.
↑ Bellman, 1969 , p. 196.
↑ Golub, Van Lone, 1999 , p. 63.
↑ Golub, Van Lone, 1999 , p. 61.

Literature

Ilyin V.A. , Kim G. D. Linear algebra and analytic geometry. - M .: Publishing house Mosk. University, 1998 .-- 320 p. - ISBN 5-211-03814-2 .
Gantmakher F.R. Matrix Theory. - M .: Science, 1988.
Bellman R. Introduction to matrix theory. - M .: Science, 1969.
Prasolov V.V. Problems and theorems of linear algebra. - M .: Nauka, 1996 .-- 304 p. - ISBN 5-02-014727-3 .
Golub J., Van Lone C. Matrix calculations: Per. from English .. - M .: Mir, 1999 .-- 548 p. - ISBN 5-03-002406-9 .
Belitsky G.R. , Lubich Yu. I. Norms of matrices and their applications. - Kiev: Naukova Dumka, 1984. - 160 p.

Links

Exponenta.ru. Educational mathematical portal (neopr.) . Date of treatment November 15, 2016.
World of Mathematics. Matrix norm (neopr.) . Date of treatment December 3, 2016.

[_1dda23b86b45a8ad-1] Gantmakher, 1988 , p. 410.

[_5abfb64efaa8e0a0-2] Prasolov, 1996 , p. 210.

[3] Lyubich Yu. I. On the operator norms of matrices // Uspekhi Mat . - 1963. - N. 18. Issue. 4 (112) - S. 161-164. - URL: http://mi.mathnet.ru/rus/umn/v18/i4/p161

[_0d098601f0673302-4] Belitsky, 1984 , p. 99.

[_1fd51192d8747556-5] ¹ ² Ilyin, Kim, 1998 , p. 311.

[_0eb02295da43692d-6] Bellman, 1969 , p. 196.

[_9e1990c56f913a4a-7] Golub, Van Lone, 1999 , p. 63.

[_9e1990c56f913a48-8] Golub, Van Lone, 1999 , p. 61.