Like what you see?Follow Adrian on Twitter to be notified of new content.Follow
The moments of a statistical distribution are a family of quantities that describe the distribution’s shape. Moments include the familiar quantities mean, variance, skewness and kurtosis.
Moments are broken down into raw, central and standardised moments. They are also ordered. For example, variance is the second central moment. The first 4 sets of moments are:
Order
Raw moments
Central moments
Standardised moments
1
E[X]=μ
E[X−μ]=0
E[σX−μ]=0
2
E[X2]
E[(X−μ)2]=σ2
E[(σX−μ)2]=1
3
E[X3]
E[(X−μ)3]
E[(σX−μ)3]=s
4
E[X4]
E[(X−μ)4]
E[(σX−μ)4]=κ
where μ is the mean, σ2 is the variance, s is the skewness, and κ is the kurtosis.
Often, you can calculate the moments of a function of a random value based on the moments of that random variable. I frequently find myself calculating the moments of functions of Gaussian random variables. Here, I list out the moments of the Gaussian distribution for reference and describe a method for calculating co-moments.
Gaussian moments
If X is a Gaussian distribution, X∼N(μ,σ2), then the moments are 12:
Raw moments
Central moments
Standardised moments
E[X]=μ
E[X−μ]=0
E[σX−μ]=0
E[X2]=μ2+σ2
E[(X−μ)2]=σ2
E[(σX−μ)2]=1
E[X3]=μ3+3μσ2
E[(X−μ)3]=0
E[(σX−μ)3]=0=s
E[X4]=μ4+6μ2σ2+3σ4
E[(X−μ)4]=3σ4
E[(σX−μ)4]=3=κ
Gaussian co-moments
Co-moments involve multiple random variables. For example, E[XY] is a co-moment of X and Y. These can be fairly tedious to derive and require a trick.
How to derive
The trick to deriving Gaussian co-moments is to write the two variables as linear combinations of three standard normal distributions (X^,Y^,Z^∼N(0,1)). Then, expand out the expected value into combinations of E[X^iY^jZ^k] which resolve to E[X^i]E[Y^j]E[Z^k]3.
Say we have two Gaussian random variables X and Y with means μX and μY, variances σX2 and σY2 and covariance σXY2. We can write these two variables as functions of three uncorrelated standard normals X^, Y^ and Z^:
σ^Xσ^YXY=σX2−σXY2=σY2−σXY2=μX+σ^XX^+σXYZ^=μY+σ^YY^+σXYZ^
We can check these formulas by checking that the variance and covariance resolve to σX2, σY2 and σXY2. The variance of summed independent Gaussians is the sum of the variances, and so:
var[X]=σ^X2+σXY2=σX2Y resolves the same. And the covariance:
cov[X,Y]=E[(σ^XX^+σXYZ^)(σ^YY^+σXYZ^)]=E[σ^Xσ^YX^Y^+σ^XσXYX^Z^+σXYσ^YY^Z^+σXY2Z^2]=σ^Xσ^YE[X^Y^]+σ^XσXYE[X^Z^]+σXYσ^YE[Y^Z^]+σXY2E[Z^2]=σXY2
Derivations
We can use this method of rewriting into a combination of 3 standard normals to derive various co-moments. First expand out the two Gaussians:
E[XY]=E[(μX+σ^XX^+σXYZ^)(μY+σ^YY^+σXYZ^)]=μXμY+μXσ^YE[Y^]+μXσXYE[Z^]+σ^XμYE[X^]+σ^Xσ^YE[X^Y^]+σ^XσXYE[X^Z^]+σXYμYE[Z^]+σXYσ^YE[Y^Z^]+σXY2E[Z^2]
Then, expand the monomials from E[X^iY^jZ^k] to E[X^i]E[Y^j]E[Z^k]:
=μXμY+μXσ^YE[Y^]+μXσXYE[Z^]+σ^XμYE[X^]+σ^Xσ^YE[X^]E[Y^]+σ^XσXYE[X^]E[Z^]+σXYμYE[Z^]+σXYσ^YE[Y^]E[Z^]+σXY2E[Z^2]
And replace all the Gaussian moments with their values from the table above (for example E[Y^]=0):
E[XY]=μXμY+σXY2(1)
Two other co-moments that come up often are:
E[X2Y]=E[(μX+σ^XX^+σXYZ^)2(μY+σ^YY^+σXYZ^)]=μX2μY+2μXσXY2+μYσX2(2)