EE3901/EE5901 Sensor Technologies

Week 2 Notes

Measurement uncertainty

Measurement is a statistical estimation problem

Recall that the central problem of sensing is to use measurements to obtain information about the measurand. This is fundamentally a question of statistical estimation. If the measurand is not changing, then each measurement is a random variable drawn from the same distribution. The randomness represents the measurement noise. (We will consider the case of a time-varying measurand next week.)

Review of probability and statistics

We will typically use a Bayesian interpretation of probability, i.e. probability represents a degree of belief. The probability reflects the degree to which a statement is supported by the available evidence.

Expected value

The expected value is the most likely outcome. In the context of measurement, it is the value that we believe best represents the true measurand based upon the available evidence.

Mathematically, it is defined as follows. Let $X$ be a random variable defined by its probability density $f(x)$. The expected value of $X$ is

Adjust the limits of integration if the probability density is defined over a different range. We will often write the expected value with a bar, e.g. $\bar{X}=E[X]$.

Sometimes we want the expected value of some calculated result instead of the expected value of the measurement itself. In this case the expected value of some function of X is given by:

For a finite sample of measurements, the expected value is just the average computed in the usual way:

Variance and standard deviation

The variance represents how far samples are from the mean. There are two related quantities: $\sigma_{X}^{2}$ is called the variance, and has units of the measurement squared. The standard deviation (with the same units as the measurement) is called $\sigma_{X}$. The standard deviation is the square root of the variance.

The definition of variance is:

Translated into words, this indicates the “average of the squared distance from the mean”.

For an entire population the variance can be calculated using

Note that the above definition applies to an entire population. Most often in statistics we deal with a finite sample drawn from the larger population, in which case the correct (“unbiased”) estimator of variance is

The proof of this estimator is outside the scope of this class, so refer to a statistics book for more details. The idea of the proof is to treat $\sigma_{X}^{2}$ as a random variable (since it depends upon a random sampling of the broader population), then find the expected value of that random variable. The resulting algebra gives the correction factor of $1/(N-1)$.

Visual illustration of the mean and standard deviation

The impact of the mean and standard deviation can be explored using interactive Figure 1. This figure plots the normal distribution (also called the Gaussian distribution), which has probability density function

where $\bar{x}$ is the expected value and $\sigma$ is the standard deviation.

Drag the sliders to adjust the mean and standard deviation.

Correlation

Correlation is the tendency of two variables to exhibit a linear relationship. Mathematically:

Correlation is always in the range $-1 \leq \rho \leq 1$. If $\rho \approx 0$ then the variables are said to be uncorrelated.

It is best explained visually (Figure 2). Drag the slider to see different levels of correlation.

Covariance

Covariance is similar to correlation but not normalised. While correlation is a dimensionless number between $-1$ and $+1$, covariance has units and there is no upper or lower limit.

Let $\sigma_{X,Y}^2$ be the covariance between variables $X$ and $Y$. Covariance is also written as $\text{cov}(X,Y)$. The definition of covariance is as follows:

Equation (10) shows that covariance is equivalent to the correlation multiplied by the standard deviation of each variable. The relationship to correlation provides an intuitive explanation, as shown in Figure 3. When two variables have nonzero covariance, the area of overlap is reduced, overall allowing for better precision in the joint measurement of the entire system.

Note that the covariance of a variable with itself is the variance:

Since the covariance is defined between pairs of variables, it is convenient to list all pairwise combinations in a matrix. For instance, given two variables $X$ and $Y$, the covariance matrix is of dimensions $2 \times 2$ and is given by:

This matrix is symmetric because $\sigma_{X,Y}^{2}=\sigma_{Y,Z}^{2}$.

In the general case of an arbitrary number of random variables, we can form the covariance matrix as follows. Firstly define a vector $\mathbf{V}$ containing the variables:

and then the covariance matrix is

Visual illustration of the covariance matrix for a 2D Gaussian distribution

Figure 4 shows a two-dimensional Gaussian with the specified correlation coefficient $\rho$ and standard deviations $\sigma_x$ and $\sigma_y$.

Error propagation

Defining error

There are two ways that we can define error:

1. Absolute error giving a range of possible values, e.g. $v_{1}=0.5\pm0.1$ V, without specifying the likelihood of values within that range. The limits can be thought of as giving worst case scenarios.
2. Error characterised through a probability distribution, for example, as a normal distribution with a given standard deviation. The probability distribution gives the relative likelihood of each amount of error. We will typically use this approach.

Introduction to error propagation

Suppose that you perform a measurement and obtain a result $x$, however, this result is subject to uncertainty. You would like to use that measurement in some arbitrary calculation $f(x)$. The problem of error propagation is to calculate the uncertainty in the calculated result $f(x)$ given the uncertainty in $x$.

An intuitive picture is shown in Figure 5. The measurement $x$ has error bars shown by the standard deviation $\sigma_x$. Our task is to calculate $\sigma_y$.

Derivation of the variance formula

Let $x$ be a random variable with expected value $\bar{x}$ and variance $\sigma_{x}^{2}$. Let $y = f(x)$ be a calculated value based on $x$. Notice that $y$ is also a random variable.We seek an expression for the variance $\sigma_{y}^{2}$.

Suppose we perform a measurement and obtain the result $x_0$. Take a Taylor series expansion of $f$ about $x_0$.

The notation $\left.\frac{\partial f}{\partial x}\right|_{x_{0}}$ means take the derivative then substitute $x_{0}$. In other words, this quantity is just a number. Let $J=\left.\frac{\partial f}{\partial x}\right|_{x_{0}}$ so that $y \approx f(x_0) + J\,(x - x_0)$.

Our goal is to find $\sigma_{y}^{2}=E\left[(y-\bar{y})^{2}\right]$. We need an expression for $\bar{y}$.

Consequently,

Therefore

This is the formula for propagating variance.

Variance propagation with multiple variables

Let $x_{1},x_{2},...,x_{n}$ be random variables (e.g. from different sensors).

Represent these variables as a column vector $\boldsymbol{x}=\left[\begin{array}{ccc} x_{1} & x_{2} & \cdots\end{array}\right]^{T}$.

Let the expected values be $\bar{\boldsymbol{x}}=\left[\begin{array}{ccc} \bar{x}_{1} & \bar{x}_{2} & \cdots\end{array}\right]^{T}$.

Let the covariance matrix of these variables be $\Sigma_{x}$.

Define some calculation $\boldsymbol{y}=f(x_{1},x_{2},...,x_{n})=f(\boldsymbol{x})$.

If $\boldsymbol{y}$ is a vector then we seek the covariance matrix $\Sigma_{y}$. If $y$ is a scalar then $\Sigma_{y}$ is a 1x1 (i.e. a scalar) giving the variance $\sigma_{y}^{2}$.

Derivation

Let $\boldsymbol{x}_{0}$ be a measurement result.

Define the Jacobian as:

where the measurement result $\boldsymbol{x}_{0}$ is substituted into the Jacobian, so this is a matrix of numbers. If $\boldsymbol{y}$ is a scalar then the Jacobian is a row vector.

Then the Taylor series expansion of $f$ about the point $\boldsymbol{x}_{0}$ is

Notice that the matrix product $J\times(\boldsymbol{x}-\boldsymbol{x}_{0})$ produces all the first order terms in the multivariate Taylor series.

Proceeding as above, we find

Hence

Hence the covariance matrix is

To summarise, the variance propagation law for a multivariable function $\boldsymbol{y}=f(x_{1},x_{2},...,x_{n})$ is

where $J$ is the Jacobian matrix of the calculation evaluated at the measurement result, and $\Sigma_x$ is the covariance matrix of the measurements.

Root sum of squares error vs absolute error

There is a simplification to the above method in the case that the variables are uncorrelated. The simplification (which you will prove in the tutorial questions) is as follows:

If $y=f(x_{1},x_{2},...)$ then the uncertainty in $y$ is given by:

This is sometimes called the root sum of squares (RSS) method.

However, if the uncertainties are treated as absolute errors (instead of standard deviations) then another formula is sometimes used. This is the absolute error combination formula:

Use the absolute error method only if the uncertainties are not derived from a standard deviation.

Introduction to calibration

Calibration is the process of estimating a sensor’s transfer function. Often there is prior knowledge of the functional form but some coefficients need to be adjusted to account for manufacturing variability, change in sensor properties over time, etc.

One point calibration

One point calibration is performed using one value of the measurand. Hence it is the simplest calibration method. Its role is to estimate bias (so it can be subtracted away).

This method assumes that the sensitivity and functional form of the transfer function are correctly known, and the only issue is with bias. Specifically this method addresses miscalibrations of the form $y_\text{raw}=y_\text{reference}+b,$ where $y_\text{raw}$ is the actual measurement, $y_\text{reference}$ is the sensor response in the absence of bias, and $b$ is the bias. Refer to Figure 6 for an illustration.

We calibrate the sensor by finding the bias $b$. If you have raw measurements obtained from your sensor, and reference measurements of the same measurand that are known to be accurate, then the bias is

The calibration for the sensor is then given by

Two point calibration

Two point calibration is appropriate for linear sensors. The idea is to measure the sensor response at two known points, and fit a straight line between the points (Figure 7).

The measurand can be determined using another sensor, or by measuring particular calibration samples whose characteristics are already known.

The calibrated transfer function is the equation of a straight line that passes through these two points:

Sensor noise can be handled by repeatedly sampling at the reference points. In this case, replace $y_1$ and $y_2$ with $E[y_1]$ and $E[y_2]$.

Conclusion

Systematic analysis of a sensor system requires characterisation of its measurement uncertainty. In this subject we will specify uncertainty with a covariance matrix, which represents the variance of each variable as well as information about linear correlations between variables.

The central result of this week is the method of propagating variance through calculations. Given the covariance of the measurements ($\Sigma_x$), the covariance of any calculated result $\mathbf{y} = f(\mathbf{x})$, provided that $f$ is differentiable, is

where $J$ is the Jacobian of the function $f$.

References

Clarence W. de Silva, Sensor Systems: Fundamental and Applications, CRC Press, 2017, Sections 6.4, 6.5 and Appendix C.