Block matrix explained

In mathematics, a block matrix or a partitioned matrix is a matrix that is interpreted as having been broken into sections called blocks or submatrices.^[1] ^[2]

Intuitively, a matrix interpreted as a block matrix can be visualized as the original matrix with a collection of horizontal and vertical lines, which break it up, or partition it, into a collection of smaller matrices.^[3] For example, the 3×4 matrix presented below is divided by horizontal and vertical lines into four blocks: the top-left 2×3 block, the top-right 2×1 block, the bottom-left 1×3 block, and the bottom-right 1×1 block.

$\left[\begin{array}{ccc|c}
a_{11} & a_{12} & a_{13} & b_{1} \\
a_{21} & a_{22} & a_{23} & b_{2} \\
\hline
c_{1} & c_{2} & c_{3} & d
\end{array}
\right]$

Any matrix may be interpreted as a block matrix in one or more ways, with each interpretation defined by how its rows and columns are partitioned.

This notion can be made more precise for an

matrix

by partitioning

into a collection

rowgroups

, and then partitioning

into a collection

colgroups

. The original matrix is then considered as the "total" of these groups, in the sense that the

(i,j)

entry of the original matrix corresponds in a 1-to-1 way with some

(s,t)

offset entry of some

(x,y)

, where

x\inrowgroups

and

y\incolgroups

.^[4]

Block matrix algebra arises in general from biproducts in categories of matrices.^[5]

Example

The matrix

$\mathbf = \begin 1 & 2 & 2 & 7 \\ 1 & 5 & 6 & 2 \\ 3 & 3 & 4 & 5 \\ 3 & 3 & 6 & 7\end$

can be visualized as divided into four blocks, as

$\mathbf = \left[\begin{array}{cc|cc}
1 & 2 & 2 & 7 \\
1 & 5 & 6 & 2 \\
\hline
3 & 3 & 4 & 5 \\
3 & 3 & 6 & 7
\end{array}
\right].$

The horizontal and vertical lines have no special mathematical meaning,^[6] but are a common way to visualize a partition. By this partition,

is partitioned into four 2×2 blocks, as

$\begin\mathbf_ &= \begin 1 & 2 \\ 1 & 5 \end, & \mathbf_ &= \begin 2 & 7\\ 6 & 2 \end, \\[1ex] \mathbf_ &= \begin 3 & 3 \\ 3 & 3 \end,& \mathbf_ &= \begin 4 & 5 \\ 6 & 7 \end.\end$

The partitioned matrix can then be written as^[7]

$\mathbf = \begin \mathbf_ & \mathbf_ \\ \mathbf_ & \mathbf_\end.$

Formal definition

Let

A\inC^m

. A partitioning of

is a representation of

in the form

$A = \beginA_ & A_ & \cdots & A_ \\A_ & A_ & \cdots & A_ \\\vdots & \vdots & \ddots & \vdots \\A_ & A_ & \cdots & A_\end,$

where

A_ij\in

	m_i x n_j
C

are contiguous submatrices,

\sum_^ m_i = m

, and

\sum_^ n_j = n

.^[8] The elements

A_ij

of the partition are called blocks.

By this definition, the blocks in any one column must all have the same number of columns. Similarly, the blocks in any one row must have the same number of rows.

Partitioning methods

A matrix can be partitioned in many ways. For example, a matrix

is said to be partitioned by columns if it is written as

$A = (a_1 \ a_2 \ \cdots \ a_n),$

where

a_j

is the

th column of

. A matrix can also be partitioned by rows:

$A = \begina_1^T \\a_2^T \\\vdots \\a_m^T\end,$

where

	T
a
	i

is the row of

Common partitions

Often, we encounter the 2×2 partition

$A = \beginA_ & A_ \\A_ & A_\end,$

particularly in the form where

A₁₁

is a scalar:

$A = \begina_ & a_^T \\a_ & A_\end.$

Block matrix operations

Transpose

Let

$A = \beginA_ & A_ & \cdots & A_ \\A_ & A_ & \cdots & A_ \\\vdots & \vdots & \ddots & \vdots \\A_ & A_ & \cdots & A_\end$

where (This matrix

will be reused in and .) Then its transpose is

$A^T = \beginA_^T & A_^T & \cdots & A_^T \\A_^T & A_^T & \cdots & A_^T \\\vdots & \vdots & \ddots & \vdots \\A_^T & A_^T & \cdots & A_^T\end,$

and the same equation holds with the transpose replaced by the conjugate transpose.

Block transpose

A special form of matrix transpose can also be defined for block matrices, where individual blocks are reordered but not transposed. Let

A=(B_ij)

be a

k x l

block matrix with

m x n

blocks

B_ij

, the block transpose of

is the

l x k

block matrix

A^l{B}

with

m x n

blocks

	l{B}\right)
\left(A
	ij

=B_ji

.^[9] As with the conventional trace operator, the block transpose is a linear mapping such that

(A+C)^l{B}=A^l{B}+C^l{B}

. However, in general the property

(AC)^l{B}=C^l{B}A^l{B}

does not hold unless the blocks of

and

commute.

Addition

Let

$B = \beginB_ & B_ & \cdots & B_ \\B_ & B_ & \cdots & B_ \\\vdots & \vdots & \ddots & \vdots \\B_ & B_ & \cdots & B_\end,$

where

B_ij\in

	m_i x n_j
C

, and let

be the matrix defined in . (This matrix

will be reused in .) Then if

p=r

q=s

k_i=m_i

, and

\ell_j=n_j

, then

$A + B = \beginA_ + B_ & A_ + B_ & \cdots & A_ + B_ \\A_ + B_ & A_ + B_ & \cdots & A_ + B_ \\\vdots & \vdots & \ddots & \vdots \\A_ + B_ & A_ + B_ & \cdots & A_ + B_\end.$

Multiplication

It is possible to use a block partitioned matrix product that involves only algebra on submatrices of the factors. The partitioning of the factors is not arbitrary, however, and requires "conformable partitions"^[10] between two matrices

and

such that all submatrix products that will be used are defined.^[11]

Let

be the matrix defined in, and let

be the matrix defined in . Then the matrix product

$C = AB$

can be performed blockwise, yielding

as an

(p x s)

matrix. The matrices in the resulting matrix

are calculated by multiplying:^[12]

$C_ = \sum_^ A_B_.$

Or, using the Einstein notation that implicitly sums over repeated indices:

$C_ = A_B_.$

Depicting

as a matrix, we have

$C = AB = \begin\sum_^ A_B_ & \sum_^ A_B_ & \cdots & \sum_^ A_B_ \\\sum_^ A_B_ & \sum_^ A_B_ & \cdots & \sum_^ A_B_ \\\vdots & \vdots & \ddots & \vdots \\\sum_^ A_B_ & \sum_^ A_B_ & \cdots & \sum_^ A_B_\end.$

Inversion

Computing submatrix inverses from the full inverse

By the symmetry between a matrix and its inverse in the block inversion formula, if a matrix P and its inverse P⁻¹ are partitioned conformally:

$P = \begin & \\ & \end, \quad P^ = \begin & \\ & \end$

then the inverse of any principal submatrix can be computed from the corresponding blocks of P⁻¹:

$^ = - ^$ $^ = - ^$

This relationship follows from recognizing that E⁻¹ = A − BD⁻¹C (the Schur complement), and applying the same block inversion formula with the roles of P and P⁻¹ reversed.^[14] ^[15]

Determinant

The formula for the determinant of a

2 x 2

-matrix above continues to hold, under appropriate further assumptions, for a matrix composed of four submatrices

A,B,C,D

with

and

square. The easiest such formula, which can be proven using either the Leibniz formula or a factorization involving the Schur complement, is

\det\beginA& 0\\ C& D\end = \det(A) \det(D) = \det\beginA& B\\ 0& D\end.

Using this formula, we can derive that characteristic polynomials of

\begin{bmatrix}A&0\ C&D\end{bmatrix}

and

\begin{bmatrix}A&B\ 0&D\end{bmatrix}

are same and equal to the product of characteristic polynomials of

and

. Furthermore, If

\begin{bmatrix}A&0\ C&D\end{bmatrix}

\begin{bmatrix}A&B\ 0&D\end{bmatrix}

is diagonalizable, then

and

are diagonalizable too. The converse is false; simply check

\begin{bmatrix}1&1\ 0&1\end{bmatrix}

is invertible, one has

$\det\beginA& B\\ C& D\end = \det(A) \det\left(D - C A^ B\right),$

and if

is invertible, one has^[16]

$\det\beginA& B\\ C& D\end = \det(D) \det\left(A - B D^ C\right) .$

If the blocks are square matrices of the same size further formulas hold. For example, if

and

commute (i.e.,

CD=DC

), then^[17]

\det\beginA& B\\ C& D\end = \det(AD - BC).

Similar statements hold when

AB=BA

AC=CA

, or . Namely, if

AC=CA

, then

\det\beginA& B\\ C& D\end = \det(AD - CB).

Note the change in order of

and

(we have

instead of

). Similarly, if

BD=DB

, then

should be replaced with

(i.e. we get

\det(DA-BC)

) and if

AB=BA

, then we should have

\det(DA-CB)

. Note for the last two results, you have to use commutativity of the underlying ring, but not for the first two.

This formula has been generalized to matrices composed of more than

2 x 2

blocks, again under appropriate commutativity conditions among the individual blocks.^[18]

For

A=D

and

B=C

, the following formula holds (even if

and

do not commute)

\det\beginA& B\\ B& A\end = \det(A - B) \det(A + B).

Special types of block matrices

Direct sums and block diagonal matrices

Direct sum

Block diagonal matrices

See also: Diagonal matrix. A block diagonal matrix is a block matrix that is a square matrix such that the main-diagonal blocks are square matrices and all off-diagonal blocks are zero matrices.^[19] That is, a block diagonal matrix A has the form

$= \begin A_1 & 0 & \cdots & 0 \\ 0 & A_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & A_n\end$

where A_k is a square matrix for all k = 1, ..., n. In other words, matrix A is the direct sum of A₁, ..., A_n. It can also be indicated as A₁ ⊕ A₂ ⊕ ... ⊕ A_n or diag(A₁, A₂, ..., A_n)^[20] (the latter being the same formalism used for a diagonal matrix). Any square matrix can trivially be considered a block diagonal matrix with only one block.

For the determinant and trace, the following properties hold:

\begin{align} \det{A}&=\det{A}₁ x … x \det{A}_{n,
\end{align}}

^[21] ^[22] and

\begin{align} \operatorname{tr}{A}&=\operatorname{tr}{A}₁+ … +\operatorname{tr}{A}_{n.\end{align}}

A block diagonal matrix is invertible if and only if each of its main-diagonal blocks are invertible, and in this case its inverse is another block diagonal matrix given by^[23] $\begin _ & & \cdots & \\ & _ & \cdots & \\ \vdots & \vdots & \ddots & \vdots \\ & & \cdots & _ \end^ = \begin _^ & & \cdots & \\ & _^ & \cdots & \\ \vdots & \vdots & \ddots & \vdots \\ & & \cdots & _^ \end.$

The eigenvalues and eigenvectors of

{A}

are simply those of the

{A}_k

s combined.

Block tridiagonal matrices

See also: Tridiagonal matrix. A block tridiagonal matrix is another special block matrix, which is just like the block diagonal matrix a square matrix, having square matrices (blocks) in the lower diagonal, main diagonal and upper diagonal, with all other blocks being zero matrices. It is essentially a tridiagonal matrix but has submatrices in places of scalars. A block tridiagonal matrix

has the form

$= \begin B_1 & C_1 & & & \cdots & & 0 \\ A_2 & B_2 & C_2 & & & & \\ & \ddots & \ddots & \ddots & & & \vdots \\ & & A_k & B_k & C_k & & \\ \vdots & & & \ddots & \ddots & \ddots & \\ & & & & A_ & B_ & C_ \\ 0 & & \cdots & & & A_n & B_n\end$

where

{A}_k

{B}_k

and

{C}_k

are square sub-matrices of the lower, main and upper diagonal respectively.^[24] ^[25]

Block tridiagonal matrices are often encountered in numerical solutions of engineering problems (e.g., computational fluid dynamics). Optimized numerical methods for LU factorization are available^[26] and hence efficient solution algorithms for equation systems with a block tridiagonal matrix as coefficient matrix. The Thomas algorithm, used for efficient solution of equation systems involving a tridiagonal matrix can also be applied using matrix operations to block tridiagonal matrices (see also Block LU decomposition).

Block triangular matrices

Block Toeplitz matrices

See also: Toeplitz matrix. A block Toeplitz matrix is another special block matrix, which contains blocks that are repeated down the diagonals of the matrix, as a Toeplitz matrix has elements repeated down the diagonal.

A matrix

is block Toeplitz if

A_(i,j)=A_(k,l)

for all

k-i=l-j

, that is,

$A = \beginA_1 & A_2 & A_3 & \cdots \\A_4 & A_1 & A_2 & \cdots \\A_5 & A_4 & A_1 & \cdots \\\vdots & \vdots & \vdots & \ddots\end,$

where

A_i\in

	n_i x m_i
F

Block Hankel matrices

References

Web site: Strang . Gilbert . Gilbert Strang . Lecture 3: Multiplication and inverse matrices . MIT Open Course ware . 18:30–21:10 . 1999.

Notes and References

Book: Eves, Howard . Howard Eves . Elementary Matrix Theory . 1980 . Dover . New York . 0-486-63946-0 . 37 . registration . reprint . 24 April 2013 . We shall find that it is sometimes convenient to subdivide a matrix into rectangular blocks of elements. This leads us to consider so-called partitioned, or block, matrices..
Web site: Dobrushkin . Vladimir . Partition Matrices . 2024-03-24 . Linear Algebra with Mathematica.
Book: Anton, Howard . Elementary Linear Algebra . 1994 . John Wiley . New York . 0-471-58742-7 . 30 . 7th . A matrix can be subdivided or partitioned into smaller matrices by inserting horizontal and vertical rules between selected rows and columns..
Indhumathi . D. . Sarala . S. . 2014-05-16 . Fragment Analysis and Test Case Generation using F-Measure for Adaptive Random Testing and Partitioned Block based Adaptive Random Testing . International Journal of Computer Applications . 93 . 6 . 13 . 10.5120/16218-5662. 2014IJCA...93f..11I .
Macedo . H.D. . Oliveira . J.N. . 2013 . Typing linear algebra: A biproduct-oriented approach . 10.1016/j.scico.2012.07.012 . Science of Computer Programming . 78 . 11. 2160–2191 . 1312.4818 .
Book: Johnston, Nathaniel . Advanced linear and matrix algebra . 2021 . Springer Nature . 978-3-030-52814-0 . Cham, Switzerland . 298.
Book: Jeffrey, Alan . Matrix operations for engineers and scientists: an essential guide in linear algebra . 2010 . Springer . 978-90-481-9273-1 . Dordrecht [Netherlands] ; New York . 54 . 639165077.
Book: Stewart, Gilbert W. . Matrix algorithms. 1: Basic decompositions . 1998 . Soc. for Industrial and Applied Mathematics . 978-0-89871-414-2 . Philadelphia, PA . 18–20.
Mackey . D. Steven . 2006 . Structured linearizations for matrix polynomials . University of Manchester . 1749-9097 . 930686781 .
Book: Eves, Howard . Howard Eves . Elementary Matrix Theory . 1980 . Dover . New York . 0-486-63946-0 . 37 . registration . reprint . 24 April 2013 . A partitioning as in Theorem 1.9.4 is called a conformable partition of A and B..
Book: Anton, Howard . Elementary Linear Algebra . 1994 . John Wiley . New York . 0-471-58742-7 . 36 . 7th . ...provided the sizes of the submatrices of A and B are such that the indicated operations can be performed..
Book: Johnston, Nathaniel . Introduction to linear and matrix algebra . 2021 . Springer Nature . 978-3-030-52811-9 . Cham, Switzerland . 30,425.
Book: Bernstein, Dennis . Matrix Mathematics . Princeton University Press . 2005 . 44 . 0-691-11802-7 .
Web site: Is this formula for a matrix block inverse in terms of the entire matrix inverse known?. MathOverflow.
Escalante-B.. Alberto N.. Wiskott. Laurenz. Improved graph-based SFA: Information preservation complements the slowness principle. Machine Learning. 2016. 10.1007/s10994-016-5563-y. 1412.4679.
Taboga, Marco (2021). "Determinant of a block matrix", Lectures on matrix algebra.
J. R.. Silvester. Determinants of Block Matrices. Math. Gaz.. 84. 501. 2000. 460–467. 3620776. 10.2307/3620776. 2021-06-25. 2015-03-18. https://web.archive.org/web/20150318222335/http://www.ee.iisc.ernet.in/new/people/faculty/prasantg/downloads/blocks.pdf. dead.
Sothanaphan. Nat. Determinants of block matrices with noncommuting blocks. Linear Algebra and Its Applications. January 2017. 512. 202–218. 10.1016/j.laa.2016.10.004. 1805.06027. 119272194.
Book: Abadir . Karim M. . Matrix Algebra . Magnus . Jan R. . Cambridge University Press . 2005 . 9781139443647 . 97,100,106,111,114,118 . en.
Book: Gentle, James E. . Matrix Algebra: Theory, Computations, and Applications in Statistics . 2007 . Springer New York Springer e-books . 978-0-387-70873-7 . Springer Texts in Statistics . New York, NY . 47,487.
Book: Quarteroni . Alfio . Numerical mathematics . Sacco . Riccardo . Saleri . Fausto . 2000 . Springer . 978-0-387-98959-4 . Texts in applied mathematics . New York . 10,13.
George . Raju K. . Ajayakumar . Abhijith . 2024 . A Course in Linear Algebra . University Texts in the Mathematical Sciences . en . 35,407 . 10.1007/978-981-99-8680-4 . 978-981-99-8679-8 . 2731-9318. subscription .
Book: Prince, Simon J. D. . Computer vision: models, learning, and inference . 2012 . Cambridge university press . 978-1-107-01179-3 . New York . 531.
Book: Dietl, Guido K. E. . Linear estimation and detection in Krylov subspaces . 2007 . Springer . 978-3-540-68478-7 . Foundations in signal processing, communications and networking . Berlin ; New York . 85,87 . en . ocm85898525.
Book: Horn . Roger A. . Matrix analysis . Johnson . Charles R. . 2017 . Cambridge University Press . 978-0-521-83940-2 . Second edition, corrected reprint . New York, NY . 36 . en.
Book: Datta, Biswa Nath . Numerical linear algebra and applications . 2010 . SIAM . 978-0-89871-685-6 . 2 . Philadelphia, Pa . 168.
Book: Stewart, Gilbert W. . Matrix algorithms. 2: Eigensystems . 2001 . Soc. for Industrial and Applied Mathematics . 978-0-89871-503-3 . Philadelphia, Pa . 5.
Book: Bernstein, Dennis S. . Matrix mathematics: theory, facts, and formulas . Princeton University Press . 2009 . 978-0-691-14039-1 . 2 . Princeton, NJ . 168,298 . en.