G-test explained

In statistics, G-tests are likelihood-ratio or maximum likelihood statistical significance tests that are increasingly being used in situations where chi-squared tests were previously recommended.^[1]

Formulation

The general formula for test statistics of the G-test is

G=2\sum_i{O_i ⋅ ln\left(

	O_i
	E_i

\right)},

where

O_i\geq0

is the observed count in a cell,

E_i>0

is the expected count under the null hypothesis,

denotes the natural logarithm, and the sum is taken over all non-empty cells. The resulting

is asymptotically chi-squared distributed as the total number of observations tends to infinity (convergence in distribution^[2]).

Furthermore, the total observed count must be equal to the total expected count:

\sum_iO_i=\sum_iE_i=N,

where

is the total number of observations.

Both, the G-test statistics

and the chi-square test statistics

\chi²

are special cases of a general family of power divergence statistics by Cressie and Read^[2] . For

λ\notin\{0,-1\}

set

\operatorname{CR}_λ=

	2
	λ(λ+1)

\sum_iO_i\left(\left(

	O_i
	E_i

\right)^λ-1\right).

Then,

G=\lim_λ\to0\operatorname{CR}_λ, \chi²=\operatorname{CR}_1.

Derivation

We can derive the value of the G-test from the log-likelihood ratio test where the underlying model is a multinomial model.

Suppose we had a sample

O=(O_1,\ldots,O_m)

where each

O_i

is the number of times that an object of type

was observed. Furthermore, let

	m
\sum
	i=1

O_i

be the total number of observations. If we assume that the underlying model is multinomial, then the test statistic is defined by

ln\left(

	L(\tildep\|O)
	L(\hatp\|O)

\right) =ln\left(

\prod

\tilde

	O_i
p
	i

i=1

	m
\prod		\hat{p
	i=1

	O_i

	i

} \right),where

\tildep=(\tildep_{1,\ldots,\tilde}p_m)

is the null hypothesis and

\hatp=(\hatp_{1,\ldots,\hat}p_m)

is the maximum likelihood estimate (MLE) of the parameters given the data. Recall that for the multinomial model, the MLE of

\hatp_i

given some data is given by

\hatp_i=

	O_i
	N

Furthermore, we may represent each null hypothesis parameter

\tildep_i

\tildep_i=

	E_i
	N

where

E_i

is the expected count of objects of type

under the null hypothesis. Thus, by substituting the representations of

\tildep_i

and

\hatp_i

in the log-likelihood ratio, the equation simplifies to

ln\left(

	L(\tildep\|O)
	L(\hatp\|O)

\right) =ln\left(

	m
\prod		\left(
	i=1

	E_i
	O_i

	O_i
\right)

\right) =

	m
\sum
	i=1

O_iln\left(

	E_i
	O_i

\right)

Finally, multiply by a factor of

-2

(used to make the G-test formula asymptotically equivalent to the Pearson's chi-squared test statistics) to achieve the form

G =-2

	m
\sum
	i=1

O_iln\left(

	E_i
	O_i

\right) =2

	m
\sum
	i=1

O_iln\left(

	O_i
	E_i

\right)

Heuristically, one can imagine

O_i

as continuous and approaching zero, in which case

O_ilnO_i\to0

, and terms with zero observations can simply be dropped. However the expected count in each cell must be strictly greater than zero for each cell (

E_i>0

for all

) to apply the method.

Distribution and use

Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of the test statistics

is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.

For very small samples the multinomial test for goodness of fit, and Fisher's exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the G-test.^[3] McDonald recommends to always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1 000 .

There is nothing magical about a sample size of 1 000, it's just a nice round number that is well within the range where an exact test, chi-square test, and G–test will give almost identical values. Spreadsheets, web-page calculators, and SAS shouldn't have any problem doing an exact test on a sample size of 1 000 .

— John H. McDonald (2014)^[3]

G-tests have been recommended at least since the 1981 edition of Biometry, a statistics textbook by Robert R. Sokal and F. James Rohlf.^[4]

Relation to other metrics

Relation to the chi-squared test

The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the G-tests are based.^[5]

The general formula for Pearson's chi-squared test statistic is

\chi²=\sum_i{

\left(O

	2
E
	i\right)

E_i

The approximation of the G-test statistics by chi-squared test statistics is obtained by a second order Taylor expansion of the natural logarithm around 1 (see the derivation below).We have

G ≈ \chi²

when the observed counts

O_i

are close to the expected counts

E_i

. When this difference is large, however, the approximation by the chi-squared test statistics begins to break down. Here, the effects of outliers in data will be more pronounced, and this explains the why chi-squared tests fail in situations with little data.

For samples of a reasonable size, the G-test and the chi-squared test will lead to the same conclusions. However, the approximation to the theoretical chi-squared distribution for the G-test is better than for the Pearson's chi-squared test.^[6] In cases where

O_i>2 ⋅ E_i

for some cell case the G-test is always better than the chi-squared test.

For testing goodness-of-fit the G-test is infinitely more efficient than the chi-squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodges and Lehmann.^[7] ^[8]

Derivation (chi-squared)

Consider

G=2\sum_i{O_iln\left(

	O_i
	E_i

\right)},

and let

O_i=E_i+\delta_i

with

style\sum_i\delta_i=0

, so that the total number of counts remains the same. Assume that

\delta_i=O_i-E_i

is small in comparison to

E_i

for all

. To be more precise, notice that

E_i=\Theta(n)

using big Θ notation. If

O_i=E_i+l{O}(n^1/2)

using big O notation for large

, which should be true under the null hypothesis because of the central limit theorem, then

\delta_i=l{O}(n^1/2)

and

	3
\delta
	i

	2
E
	i

l{O}\left(	n^3/2
	n²

\right)

=l{O}(n^-1/2)

follow, which will be used later.

Upon substitution we find,

G=2\sum_i(E_i+\delta_i)ln\left(1+

	\delta_i
	E_i

\right).

Using the Taylor expansion

ln(1+x)=x-\tfrac{1}{2}x²+l{O}(x³⁾

yields

G=2\sum_i(E_i+\delta_i)\left(

	\delta_i
	E_i

	1
	2

	2
\delta
	i

	2
E
	i

l{O}\left(

	3
\delta
	i

	3
E
	i

\right)

\right),

and distributing terms we find,

G=2\sum_i\left(\delta_i+

	1
	2

	2
\delta
	i

E_i

l{O}\left(

	3
\delta
	i

	2
E
	i

\right)

\right).

Now, using

style\sum_i\delta_i=0

and

\delta_i=O_i-E_i

and

	2)=l{O}(n
l{O}(\delta
	i

^-1/2)

for large

, we can write the result,

G ≈ \sum_i

\left(O

	2

	i\right)

i-E

E_i

Relation to Kullback–Leibler divergence

The G-test statistic is proportional to the Kullback–Leibler divergence of the theoretical distribution

\tildep=(\tildep_{1,\ldots,\tilde}p_m)

of the null hypothesis from the empirical distribution

\hatp=(\hatp_{1,\ldots,\hat}p_m)

of the observed data:

\begin{align} G &=2\sum_i{O_i ⋅ ln\left(

	O_i
	E_i

\right)}=2N\sum_i{\hatp_i ⋅ ln\left(

	\hatp_i
	\tildep_i

\right)}\\ &=2ND_KL(\hatp\|\tildep), \end{align}

where

is the total number of observations and

\tildep_i=\tfrac{E_i}{N}

and

\hatp_i=\tfrac{O_i}{N}

are the theoretical and empirical probabilities of objects of type

, respectively.

Relation to mutual information

For analysis of contingency tables the value of the G-test statistics can also be expressed in terms of mutual information.

In this case objects with two-dimensional types

(i,j)

are considered. Let

O_ij

be the count of objects of type

(i,j)

, i.e.,

O_ij

is the entry in the contingency table in row

and column

. Set

N=\sum_ijO_ij, \hatp_ij=

	O_ij
	N

, \hatp_i=

	\sum_jO_ij
	N

, \hatp_\bullet=

	\sum_iO_ij
	N

Then the estimated expected count of objects of type

(i,j)

assuming independence is given by

E_ij=N\hatp_i\hatp_\bullet.

Finally, the G-test statistics in this case is given by

G=2\sum_ijO_ijln\left(

	O_ij
	E_ij

\right)

Let

X,Y

be random variables with joint distribution given by the empirical distribution

\hatp_ij

of the contingency table, i.e.,

P(X=i,Y=j)=\hatp_ij, P(X=i)=\hatp_i, P(Y=j)=\hatp_\bullet.

Then the G-test statistics can be expressed in several alternative forms:

\begin{align} G &=2N ⋅ \sum_ij{\hatp_ij\left(ln(\hatp_ij)-ln(\hatp_i)-ln(\hatp_\bullet)\right)}\\ &=2N ⋅ l(H(X)+H(Y)-H(X,Y)r)\\ &=2N ⋅ \operatorname{MI}(X,Y), \end{align}

where the entropies

H(X)

and

H(Y)

are given

H(X)=-\sum_i\hatp_iln(\hatp_i), H(Y)=-\sum_j\hatp_\bulletln(\hatp_\bullet)

and the joint entropy

H(X,Y)

is given by

H(X,Y)=-\sum_ij\hatp_ijln(\hatp_ij)

and the mutual information of

and

\operatorname{MI}(X,Y)=H(X)+H(Y)-H(X,Y).

It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of G applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the G-test statistic.

Application

The McDonald–Kreitman test in statistical genetics is an application of the G-test.
Dunning^[9] introduced the test to the computational linguistics community where it is now widely used.
The R-scape program (used by Rfam) uses G-test to detect co-variation between RNA sequence alignment positions.^[10]

Statistical software

In R fast implementations can be found in the AMR and Rfast packages. For the AMR package, the command is g.test which works exactly like chisq.test from base R. R also has the likelihood.test function in the Deducer package. Note: Fisher's G-test in the GeneCycle Package of the R programming language (fisher.g.test) does not implement the G-test as described in this article, but rather Fisher's exact test of Gaussian white-noise in a time series.^[11]
Another R implementation to compute the G-test statistic and corresponding p-values is provided by the R package entropy. The commands are Gstat for the standard G statistic and the associated p-value and Gstatindep for the G statistic applied to comparing joint and product distributions to test independence.
In SAS, one can conduct G-test by applying the /chisq option after the proc freq.^[12]
In Stata, one can conduct a G-test by applying the lr option after the tabulate command.
In Java, use org.apache.commons.math3.stat.inference.GTest.^[13]
In Python, use scipy.stats.power_divergence with lambda_=0.^[14]

External links

G²/Log-likelihood calculator

Notes and References

Book: McDonald, J.H.. 2014. Handbook of Biological Statistics. Baltimore, Maryland. Sparky House Publishing. Third. G–test of goodness-of-fit. http://www.biostathandbook.com/gtestgof.html. 53–58.
Cressie . Noel . Read . Timothy R. C. . Multinomial goodness-of-fit tests . Journal of the Royal Statistical Society. Series B (Methodological). 1984 . 46 . 3 . 440–464 . 10.1111/j.2517-6161.1984.tb01318.x . 2345686 . 14 January 2026.
Book: McDonald, John H. . 2014 . Handbook of Biological Statistics . Baltimore, MD . Sparky House Publishing . 3rd . Small numbers in chi-square and G–tests . http://www.biostathandbook.com/small.html . 86–89.
Book: Sokal . R. R. . Rohlf . F. J. . 1981 . Biometry: The Principles and Practice of Statistics in Biological Research . New York . Freeman . Second . 978-0-7167-2411-7 . registration .
Hoey . J. . 2012 . 1206.4881. The Two-Way Likelihood Ratio (G) Test and Comparison to Two-Way Chi-Squared Test . stat.ME .
Book: Harremoës . P. . Tusnády . G. . 2012 . 1202.1125 . Information divergence is more chi squared distributed than the chi squared statistic . Proceedings ISIT 2012 . 538–543 . 2012arXiv1202.1125H .
Quine . M. P. . Robinson . J. . 1985 . Efficiencies of chi-square and likelihood ratio goodness-of-fit tests . . 13 . 2. 727–742 . 10.1214/aos/1176349550. free .
Harremoës . P. . Vajda . I. . 2008 . On the Bahadur-efficient testing of uniformity by means of the entropy . . 54 . 1 . 321–331 . 10.1109/tit.2007.911155. 2008ITIT...54..321H . 10.1.1.226.8051 . 2258586 .
Dunning . Ted . Accurate Methods for the Statistics of Surprise and Coincidence . . 1993 . 19 . 1 . 61–74 .
Rivas . Elena . RNA structure prediction using positive and negative evolutionary information . PLOS Computational Biology . 30 October 2020 . 16 . 10 . e1008387 . 10.1371/journal.pcbi.1008387. 33125376 . free . 7657543 . 2020PLSCB..16E8387R .
Fisher . R. A. . 1929 . Tests of significance in harmonic analysis . Proceedings of the Royal Society of London A . 125 . 796. 54–59 . 10.1098/rspa.1929.0151. 1929RSPSA.125...54F. free . 2440/15201 . free .
http://www.biostathandbook.com/gtestind.html G-test of independence
Web site: org.apache.commons.math3.stat.inference.GTest . 2018-07-11 . 2018-07-26 . https://web.archive.org/web/20180726222159/http://commons.apache.org/proper/commons-math/javadocs/api-3.3/org/apache/commons/math3/stat/inference/GTest.html .
Web site: Scipy.stats.power_divergence — SciPy v1.7.1 Manual.