Generalized inverse explained

A matrix

A^g\inRⁿ

is a generalized inverse of a matrix

A\inR^m

AA^gA=A.

A generalized inverse exists for an arbitrary matrix, and when a matrix has a regular inverse, this inverse is its unique generalized inverse.

Motivation

Consider the linear system

Ax=y

where

is an

m x n

matrix and

y\inlC(A),

the column space of

. If

m=n

and

is nonsingular then

x=A^-1y

will be the solution of the system. Note that, if

is nonsingular, then

AA^-1A=A.

Now suppose

is rectangular (

m ≠ n

), or square and singular. Then we need a right candidate

of order

n x m

such that for all

y\inlC(A),

AGy=y.

That is,

x=Gy

is a solution of the linear system

Ax=y

. Equivalently, we need a matrix

of order

n x m

such that

AGA=A.

Hence we can define the generalized inverse as follows: Given an

m x n

matrix

, an

n x m

matrix

is said to be a generalized inverse of

AGA=A.

The matrix

A^-1

has been termed a regular inverse of

by some authors.

The problem is how to choose an

as the output of

for every

when the map

x\mapstoy=Ax

is not bijective.

is not surjective, then not all

's in its codomain have corresponding

's via

. To circumvent it, we just let

map those

's to arbitrary values.

For example, decompose the codomain of

as the direct sum of the column space

lC(A)

and a complement subspace, and construct

as follows.

's in the former subspace, let

map back to the corresponding

's.

's in the latter subspace, let

map them all to zero (as there're no corresponding

's).

For other

's, decompose them as the sum of the above two components, apply

respectively, then take the sum.

is not injective, then some

's correspond to multiple

's via

. To circumvent it, we let

map every

to one of the

's according an algorithm.

For example, decompose the domain of

as the direct sum of

\kerA

and a complement subspace. For every possible

, its preimage must be parallel to

\kerA

and intersect the chosen complement subspace at a single point. Let

map the

to this point.

is neither surjective nor injective, we combine the above two tricks.

The picture on the right is an example.

Types

Important types of generalized inverse include:

One-sided inverse (right inverse or left inverse)
- Right inverse: If the matrix

has dimensions

m x n

and

rm{rank}(A)=m

, then there exists an

n x m

matrix

	-1
A
	R

called the right inverse of

such that

	-1
A
	R

=I_m

, where

I_m

is the

m x m

identity matrix.

- Left inverse: If the matrix

has dimensions

m x n

and

rm{rank}(A)=n

, then there exists an

n x m

matrix

	-1
A
	L

called the left inverse of

such that

	-1
A
	L

A=I_n

, where

I_n

is the

n x n

identity matrix.

Some generalized inverses are defined and classified based on the Penrose conditions:

AA^gA=A

A^gAA^g=A^g

(AA^g)^*=AA^g

(A^gA)^*=A^gA,

where

{}^*

denotes conjugate transpose. If

A^g

satisfies the first condition, then it is a generalized inverse of

. If it satisfies the first two conditions, then it is a reflexive generalized inverse of

. If it satisfies all four conditions, then it is the pseudoinverse of

, which is denoted by

A⁺

and also known as the Moore–Penrose inverse, after the pioneering works by E. H. Moore and Roger Penrose. It is convenient to define an I

-inverse of

as an inverse that satisfies the subset

I\subset\{1,2,3,4\}

of the Penrose conditions listed above. Relations, such as

A^(1,AA^(1,=A⁺

, can be established between these different classes of

-inverses.

When

is non-singular, any generalized inverse

A^g

is equal to

A^-1

and is therefore unique. For a singular

, some generalized inverses, such as the Drazin inverse and the Moore–Penrose inverse, are unique, while others are not necessarily uniquely defined.

Examples

Non-reflexive generalized inverse

Let

A=\begin{bmatrix} 1&0&3\\ 2&0&6\\ 0&0&0 \end{bmatrix}, G=\begin{bmatrix} 1&0&0\\ 0&0&1\\ 0&0&0 \end{bmatrix}.

Obviously,

is singular.

and

satisfy Penrose conditions (1), but not the other there. Hence,

is a non-reflexive generalized inverse of

The first column of

spans

\operatorname{im}A

, and

maps it to

(1,0,0)

, which does not lie in

\kerA

. Additionally,

maps

(0,0,1)

(0,1,0)

, which lies in

\kerA

. The relationship is summarized in the picture on the right.

Reflexive generalized inverse

Let

A=\begin{bmatrix} 1&2&3\\ 4&5&6\\ 7&8&9 \end{bmatrix}, G=\begin{bmatrix} -

	5
	3

	2
	3

&0\\[4pt]

	4
	3

	1
	3

&0\\[4pt] 0&0&0 \end{bmatrix}.

Since

\det(A)=0

is singular and has no regular inverse. However,

and

satisfy Penrose conditions (1) and (2), but not (3) or (4). Hence,

is a reflexive generalized inverse of

One-sided inverse

Let

A=\begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix},

	-1
A
	R

=\begin{bmatrix} -

	17
	18

	8
	18

\\[4pt] -

	2
	18

	2
	18

\\[4pt]

	13
	18

	4
	18

\end{bmatrix}.

Since

is not square,

has no regular inverse. However,

	-1
A
	R

is a right inverse of

. The matrix

has no left inverse.

Inverse of other semigroups (or rings)

The element b is a generalized inverse of an element a if and only if

a ⋅ b ⋅ a=a

, in any semigroup (or ring, since the multiplication function in any ring is a semigroup).

The generalized inverses of the element 3 in the ring

Z/12Z

are 3, 7, and 11, since in the ring

Z/12Z

3 ⋅ 3 ⋅ 3=3

3 ⋅ 7 ⋅ 3=3

3 ⋅ 11 ⋅ 3=3

The generalized inverses of the element 4 in the ring

Z/12Z

are 1, 4, 7, and 10, since in the ring

Z/12Z

4 ⋅ 1 ⋅ 4=4

4 ⋅ 4 ⋅ 4=4

4 ⋅ 7 ⋅ 4=4

4 ⋅ 10 ⋅ 4=4

If an element a in a semigroup (or ring) has an inverse, the inverse must be the only generalized inverse of this element, like the elements 1, 5, 7, and 11 in the ring

Z/12Z

In the ring

Z/12Z

any element is a generalized inverse of 0; however 2 has no generalized inverse, since there is no b in

Z/12Z

such that

2 ⋅ b ⋅ 2=2

Construction

The following characterizations are easy to verify:

is given by

	-1
A
	R

=A^\intercal\left(AA^\intercal\right)^-1

, provided

has full row rank.

A left inverse of a non-square matrix

is given by

	-1
A
	L

=\left(A^\intercalA\right)^-1A^\intercal

, provided

has full column rank.

A=BC

is a rank factorization, then

	-1
C
	R

	-1
B
	L

is a g-inverse of

, where

	-1
C
	R

is a right inverse of

and

	-1
B
	L

is left inverse of

A=P\begin{bmatrix}I_r&0\ 0&0\end{bmatrix}Q

for any non-singular matrices

and

, then

G=Q^-1\begin{bmatrix}I_r&U\ W&V\end{bmatrix}P^-1

is a generalized inverse of

for arbitrary

U,V

and

be of rank

. Without loss of generality, let

A = \beginB & C\\ D & E\end,

where

B_r

is the non-singular submatrix of

. Then,

G = \begin B^ & 0\\ 0 & 0 \end

is a generalized inverse of

if and only if

E=DB^-1C

Uses

Any generalized inverse can be used to determine whether a system of linear equations has any solutions, and if so to give all of them. If any solutions exist for the n × m linear system

Ax=b,

with vector

of unknowns and vector

of constants, all solutions are given by

x=A^gb+\left[I-A^gA\right]w,

parametric on the arbitrary vector

, where

A^g

is any generalized inverse of

. Solutions exist if and only if

A^gb

is a solution, that is, if and only if

AA^gb=b

. If A has full column rank, the bracketed expression in this equation is the zero matrix and so the solution is unique.

Generalized inverses of matrices

The generalized inverses of matrices can be characterized as follows. Let

A\inR^m

, and

$A = U \begin \Sigma_1 & 0 \\ 0 & 0 \end V^\operatorname$

be its singular-value decomposition. Then for any generalized inverse

A^g

, there exist matrices

, and

such that

$A^g = V \begin \Sigma_1^ & X \\ Y & Z \end U^\operatorname.$

Conversely, any choice of

, and

for matrix of this form is a generalized inverse of

. The

\{1,2\}

-inverses are exactly those for which

Z=Y\Sigma₁X

, the

\{1,3\}

-inverses are exactly those for which

X=0

, and the

\{1,4\}

-inverses are exactly those for which

Y=0

. In particular, the pseudoinverse is given by

X=Y=Z=0

$A^+ = V \begin \Sigma_1^ & 0 \\ 0 & 0 \end U^\operatorname.$

Transformation consistency properties

In practical applications it is necessary to identify the class of matrix transformations that must be preserved by a generalized inverse. For example, the Moore–Penrose inverse,

A^+,

satisfies the following definition of consistency with respect to transformations involving unitary matrices U and V:

(UAV)⁺=V^*A⁺U^*

The Drazin inverse,

A^D

satisfies the following definition of consistency with respect to similarity transformations involving a nonsingular matrix S:

\left(SAS^-1\right)^D=SA^DS^-1

The unit-consistent (UC) inverse,

A^U,

satisfies the following definition of consistency with respect to transformations involving nonsingular diagonal matrices D and E:

(DAE)^U=E^-1A^UD^-1

The fact that the Moore–Penrose inverse provides consistency with respect to rotations (which are orthonormal transformations) explains its widespread use in physics and other applications in which Euclidean distances must be preserved. The UC inverse, by contrast, is applicable when system behavior is expected to be invariant with respect to the choice of units on different state variables, e.g., miles versus kilometers.

Sources

Textbook

Book: Adi. Ben-Israel. Thomas Nall Eden. Greville . Generalized Inverses: Theory and Applications. 2nd. New York, NY. Springer. 2003. 978-0-387-00293-4 . 10.1007/b97366. Thomas N. E. Greville. Adi Ben-Israel.
Book: Stephen L.. Campbell. Carl D.. Meyer. Generalized Inverses of Linear Transformations. registration. Dover. 1991. 978-0-486-66693-8.
Book: Horn. Roger Alan. Matrix Analysis. 1985. Cambridge University Press. 978-0-521-38632-6. Johnson. Charles Royal. Roger Horn. Charles Royal Johnson.
Book: Nakamura, Yoshihiko. Advanced Robotics: Redundancy and Optimization. Addison-Wesley. 1991. 978-0201151985.
Book: C. Radhakrishna. Rao. Sujit Kumar. Mitra. Generalized Inverse of Matrices and its Applications. registration. John Wiley & Sons. New York. 1971. 240. 978-0-471-70821-6 .

Publication

James. M.. The generalised inverse. The Mathematical Gazette. 62. 420. June 1978. 109–114. 10.2307/3617665. 3617665.
Uhlmann. Jeffrey K.. Jeffrey Uhlmann. 2018. A Generalized Matrix Inverse that is Consistent with Respect to Diagonal Transformations. SIAM Journal on Matrix Analysis and Applications. 239. 2. 781–800. 10.1137/17M113890X.
Zheng. Bing. Bapat. Ravindra. Generalized inverse A(2)T,S and a rank equation. Applied Mathematics and Computation. 155. 2. 407–415. 2004. 10.1016/S0096-3003(03)00786-0.

Generalized inverse explained

Motivation

Types

Examples

Non-reflexive generalized inverse

Reflexive generalized inverse

One-sided inverse

Inverse of other semigroups (or rings)

Construction

Uses

Generalized inverses of matrices

Transformation consistency properties

See also

Sources

Textbook

Publication