|
|
As a developer and marketer of uncertainty analysis tools, it is
important for Integrated Sciences Group to periodically assess the
capabilities of similar software applications. A
summary of our software assessment is
provided below. Detailed results of our
software assessment were presented at the 2004 Measurement Sciences
Conference in a paper titled "A Comprehensive Comparison of Uncertainty
Analysis Tools." An updated copy of this paper in Adobe Acrobat pdf format
can be downloaded via the hypertext link below.
Software
Comparison Paper
Software
Comparison Summary
The focus of this comparison was limited to
applications or tools that apply the methods contained in ISO/TAG4/WG3
(the GUM) and ANSI/NCSL Z540-2-1997 (the U.S. Version of the GUM).
Software using Monte Carlo simulation, or other
alternative analysis methods, were not included in this assessment.
The software applications and tools evaluated herein were also limited
to those readily available via Internet download (e.g., demos or
freeware) or currently available for purchase. In all, two
freeware applications and five commercial products were evaluated.
Basic software information is summarized in the
Software Summary Table.
In order to assess the features and capabilities of individual software
applications, we must consider how they address the basic steps in
conducting an uncertainty analysis. The general uncertainty
analysis steps are outlined below along with hypertext links to
corresponding software comparison tables.
1.
Define the Measurement Process
|
The
first step in any uncertainty analysis is to identify the
physical quantity whose value is estimated via measurement.
This quantity may be a directly measured value, such as the
weight of a 1 gm mass or the output of a voltage reference.
Alternatively, the quantity may be indirectly determined
through the measurement of other variables, as in the case of
estimating the volume of a cylinder by measuring its length
and diameter. The former type of measurement are called
"direct measurement," while the latter are call "multivariate
measurements."
For multivariate measurements, it is important to
develop an equation that defines the mathematical relationship
between the quantity of interest and the measured variables.
At this stage of the analysis, it is also useful to briefly
describe the test setup, environmental conditions, technical
information about the instruments, reference standards, or other
equipment used and the procedure for obtaining the measurement(s).
This information will help identify the measurement process
errors.
A overview of the main software features and
capabilities, including the entry and display of analysis
information is provided in the
Main Software Features Table. |
2.
Identify the Error Sources and Distributions
|
Measurement process errors are the basic elements of uncertainty
analysis. Once these fundamental error sources have been
identified, we can begin to develop uncertainty estimates.
The errors most often encountered in making measurements
include, but are not limited to the following:
-
Bias
in the measuring device and/or quantity being measured.
-
The
error associated with repeat measurements.
-
The
error resulting from the finite resolution of the measuring
device and/or quantity being measured.
-
The
error introduced by variations in environmental conditions or by
correcting for environmental conditions.
-
The
error introduced by digitizing an analog signal.
-
The
error introduced by the person making the measurements.
Another important aspect of the uncertainty analysis process is
the fact that measurement errors can be characterized by
probability distributions. The distribution for a type of
measurement error is a mathematical description that relates the
frequency of occurrence of values with the values themselves.
A comparison of how the various software applications assist the
user in identifying and describing measurement process errors
can be found in the
Errors and Distributions Table. |
3. Estimate Uncertainties
|
Our
lack of knowledge about the sign and magnitude of measurement
error is called measurement uncertainty. And, since
measurement errors follow probability distributions, they can
be described in such a way that their sign and magnitude have
some definable probability of occurrence. In this
context,
uncertainty is defined as the square root of the
variance
of the measurement error distribution. The variance is the mean square
dispersion of the error distribution.
There are two approaches to estimating variance and
uncertainty. Type A estimates involve data sampling and
analysis. Type B estimates use engineering knowledge or
recollected experience of measurement processes. Given the marked
difference in these approaches, it is best to separately evaluate
the software capabilities for each type of uncertainty estimate.
A comparison of software features and capabilities
for conducting Type A uncertainty estimates is given in the
Type A Estimates
Table. A comparison
of software features and capabilities for conducting Type B
uncertainty estimates is given in the
Type B Estimates Table. |
4. Combine Uncertainties
|
The next step in the analysis procedure is to
combine the uncertainty estimates for the measurement process
errors. To do this, we invoke the variance addition rule,
which requires that we account for any correlations between
measurement process errors or any cross-correlations between measurement
components. For multivariate analyses, it is also important
that the uncertainties are weighted or multiplied by the
appropriate sensitivity coefficients. We must also estimate
the effective degrees of freedom for the combined
uncertainty. A comparison of software features and
capabilities for combining uncertainties is given in the
Combining Uncertainties Table. |
5. Report the Analysis Results
|
The
last step in the uncertainty analysis procedure is to report
the results. When reporting the results of an uncertainty
analysis, Section 7 of the GUM recommends that the following
information be included:
-
The estimated value
of the quantity of interest (measurand) and its combined
uncertainty and degrees of freedom.
-
The functional
relationship between the quantity of interest and the measured
components, along with the sensitivity coefficients.
-
The value of each
measurement component and its combined uncertainty and degrees
of freedom.
-
A list of the
measurement process uncertainties and associated degrees of
freedom for each component, along with a description of how
they were estimated.
-
A list of
applicable correlations coefficients, including any
cross-correlations between component uncertainties.
It
is also a good practice to provide a brief description of the
measurement process, including the procedures and
instrumentation used, and additional data, tables and plots
that help clarify the analysis results. A comparison of
the software reporting features and capabilities is given in
the
Reporting the Analysis Results Table.
|
|
|
