EE3901/EE5901 Sensor Technologies

Week 2 Notes
Measurement uncertainty

College of Science and Engineering, James Cook University

Last updated 14 January 2022

In practice it is often impossible to know the true value of the measurand. If you knew the true value then you wouldn't bother doing a measurement! Every measurement is always subject to some level of uncertainty. Our goal this week is to learn how to rigorously handle measurement error. These concepts are essential in their own right, but also form the foundation of the sensor fusion methods that we will begin work on next week.

Measurement is a statistical estimation problem

Recall that the central problem of sensing is to use measurements to obtain information about the measurand. This is fundamentally a question of statistical estimation. If the measurand is not changing, then each measurement is a random variable drawn from the same distribution. The randomness represents the measurement noise. (We will consider the case of a time-varying measurand next week.)

Review of probability and statistics

We will typically use a Bayesian interpretation of probability, i.e. probability represents a degree of belief. The probability reflects the degree to which a statement is supported by the available evidence.

Expected value

The expected value is the most likely outcome. In the context of measurement, it is the value that we believe best represents the true measurand based upon the available evidence.

Mathematically, it is defined as follows. Let be a random variable defined by its probability density . The expected value of is

Adjust the limits of integration if the probability density is defined over a different range. We will often write the expected value with a bar, e.g. .

Sometimes we want the expected value of some calculated result instead of the expected value of the measurement itself. In this case the expected value of some function of X is given by:

For a finite sample of measurements, the expected value is just the average computed in the usual way:

Variance and standard deviation

The variance represents how far samples are from the mean. There are two related quantities: is called the variance, and has units of the measurement squared. The standard deviation (with the same units as the measurement) is called . The standard deviation is the square root of the variance.

The definition of variance is:

Translated into words, this indicates the “average of the squared distance from the mean”.

For an entire population the variance can be calculated using

Note that the above definition applies to an entire population. Most often in statistics we deal with a finite sample drawn from the larger population, in which case the correct (“unbiased”) estimator of variance is

The proof of this estimator is outside the scope of this class, so refer to a statistics book for more details. The idea of the proof is to treat as a random variable (since it depends upon a random sampling of the broader population), then find the expected value of that random variable. The resulting algebra gives the correction factor of .

Visual illustration of the mean and standard deviation

The impact of the mean and standard deviation can be explored using interactive Figure 1. This figure plots the normal distribution (also called the Gaussian distribution), which has probability density function

where is the expected value and is the standard deviation.

Drag the sliders to adjust the mean and standard deviation.

Figure 1:
The probability density function of a one dimensional Gaussian distribution, for various values of mean and standard deviation.
Zoom:

= -5

= 0.2

Correlation

Correlation is the tendency of two variables to exhibit a linear relationship. Mathematically:

Correlation is always in the range . If then the variables are said to be uncorrelated.

It is best explained visually (Figure 2). Drag the slider to see different levels of correlation.

Figure 2:
Random variables and with the specified correlation.
Zoom:

= -1

Covariance

Covariance is similar to correlation but not normalised. While correlation is a dimensionless number between and , covariance has units and there is no upper or lower limit.

Let be the covariance between variables and . Covariance is also written as . The definition of covariance is as follows:

Equation (10) shows that covariance is equivalent to the correlation multiplied by the standard deviation of each variable. The relationship to correlation provides an intuitive explanation, as shown in Figure 3. When two variables have nonzero covariance, the area of overlap is reduced, overall allowing for better precision in the joint measurement of the entire system.

Figure 3:
Storing information about covariance allows for more precise understanding of interactions between variables. The two variables and each have some uncertainty according to their standard deviations and . **(a)** In the case of no correlation (and hence zero covariance), the entire area of and overlap must be considered as a plausible state of the system. **(b)** In contrast, when there is positive correlation, a high implies a higher and vice-versa. Therefore, the top-left and lower-right corners of the overlap region can be excluded as improbable. Knowledge of the covariance relationship allows for better state estimation by reducing the joint uncertainty. The covariance matrix is labelled .
Zoom:

Note that the covariance of a variable with itself is the variance:

Since the covariance is defined between pairs of variables, it is convenient to list all pairwise combinations in a matrix. For instance, given two variables and , the covariance matrix is of dimensions and is given by:

This matrix is symmetric because .

In the general case of an arbitrary number of random variables, we can form the covariance matrix as follows. Firstly define a vector containing the variables:

and then the covariance matrix is

Visual illustration of the covariance matrix for a 2D Gaussian distribution

Figure 4 shows a two-dimensional Gaussian with the specified correlation coefficient and standard deviations and .

Figure 4:
Two dimensional Gaussian distribution with the specific covariance matrix. Experiment with this figure to gain an understanding of the meaning of correlation, variance, and covariance.
Zoom:

= -0.75

= 1

= 1

Error propagation

Defining error

There are two ways that we can define error:

Absolute error giving a range of possible values, e.g. V, without specifying the likelihood of values within that range. The limits can be thought of as giving worst case scenarios.
Error characterised through a probability distribution, for example, as a normal distribution with a given standard deviation. The probability distribution gives the relative likelihood of each amount of error. We will typically use this approach.

Introduction to error propagation

Suppose that you perform a measurement and obtain a result , however, this result is subject to uncertainty. You would like to use that measurement in some arbitrary calculation . The problem of error propagation is to calculate the uncertainty in the calculated result given the uncertainty in .

An intuitive picture is shown in Figure 5. The measurement has error bars shown by the standard deviation . Our task is to calculate .

Figure 5:
An intuitive explanation of why error propagation depends upon the derivative of the function. Notice that the error bars on are larger than the error bars of due to the steep slope of the function in this region.
Zoom:

Derivation of the variance formula

Let be a random variable with expected value and variance . Let be a calculated value based on . Notice that is also a random variable.We seek an expression for the variance .

Suppose we perform a measurement and obtain the result . Take a Taylor series expansion of about .

The notation means take the derivative then substitute . In other words, this quantity is just a number. Let so that .

Our goal is to find . We need an expression for .

Consequently,

Therefore

This is the formula for propagating variance.

Example of variance propagation

The energy stored in a 1 μF capacitor is . You measure the voltage to be 3.6 V but your measurement is noisy. You know that your voltmeter’s inherent noise has a standard deviation of 0.1 V. What is the standard deviation of the energy?

Solution: the energy is

Calculate the derivative and substitute the known values:

Now use the variance propagation law

μ

Variance propagation with multiple variables

Let be random variables (e.g. from different sensors).

Represent these variables as a column vector .

Let the expected values be .

Let the covariance matrix of these variables be .

Define some calculation .

If is a vector then we seek the covariance matrix . If is a scalar then is a 1x1 (i.e. a scalar) giving the variance .

Derivation

Let be a measurement result.

Define the Jacobian as:

where the measurement result is substituted into the Jacobian, so this is a matrix of numbers. If is a scalar then the Jacobian is a row vector.

Then the Taylor series expansion of about the point is

Notice that the matrix product produces all the first order terms in the multivariate Taylor series.

Proceeding as above, we find

Hence

Hence the covariance matrix is

To summarise, the variance propagation law for a multivariable function is

where is the Jacobian matrix of the calculation evaluated at the measurement result, and is the covariance matrix of the measurements.

Example of multivariate error propagation

Given the function

find the standard deviation in for the following conditions.

Solution

The state vector is

The covariance matrix is

Given that we have , the Jacobian is

Hence the variance in is

Root sum of squares error vs absolute error

There is a simplification to the above method in the case that the variables are uncorrelated. The simplification (which you will prove in the tutorial questions) is as follows:

If then the uncertainty in is given by:

This is sometimes called the root sum of squares (RSS) method.

However, if the uncertainties are treated as absolute errors (instead of standard deviations) then another formula is sometimes used. This is the absolute error combination formula:

Use the absolute error method only if the uncertainties are not derived from a standard deviation.

Introduction to calibration

Calibration is the process of estimating a sensor’s transfer function. Often there is prior knowledge of the functional form but some coefficients need to be adjusted to account for manufacturing variability, change in sensor properties over time, etc.

One point calibration

One point calibration is performed using one value of the measurand. Hence it is the simplest calibration method. Its role is to estimate bias (so it can be subtracted away).

This method assumes that the sensitivity and functional form of the transfer function are correctly known, and the only issue is with bias. Specifically this method addresses miscalibrations of the form where is the actual measurement, is the sensor response in the absence of bias, and is the bias. Refer to Figure 6 for an illustration.

We calibrate the sensor by finding the bias . If you have raw measurements obtained from your sensor, and reference measurements of the same measurand that are known to be accurate, then the bias is

The calibration for the sensor is then given by

Two point calibration

Two point calibration is appropriate for linear sensors. The idea is to measure the sensor response at two known points, and fit a straight line between the points (Figure 7).

The measurand can be determined using another sensor, or by measuring particular calibration samples whose characteristics are already known.

The calibrated transfer function is the equation of a straight line that passes through these two points:

Sensor noise can be handled by repeatedly sampling at the reference points. In this case, replace and with and .

Conclusion

Systematic analysis of a sensor system requires characterisation of its measurement uncertainty. In this subject we will specify uncertainty with a covariance matrix, which represents the variance of each variable as well as information about linear correlations between variables.

The central result of this week is the method of propagating variance through calculations. Given the covariance of the measurements (), the covariance of any calculated result , provided that is differentiable, is

where is the Jacobian of the function .

References

Clarence W. de Silva, Sensor Systems: Fundamental and Applications, CRC Press, 2017, Sections 6.4, 6.5 and Appendix C.