What you should know about analog technology
2.2 Precision/resolution
Analog value processing
16 Function Manual, 06/2014, A5E03461439-AC
Relation between resolution and precision
A certain resolution must be given to reach a specific precision (operational error).
Example
Measuring error as a result of digitization at a resolution of 8 bits and 14 bits
An analog module has a measuring range of 0 V to 10 V. A resolution of 8 bits represents a
total of 256 values. This is equivalent to a minimum possible voltage step of 39 mV or 0.4%
of the measuring range end value. A resolution of 14 bits represents a total of 16384 values.
This is equivalent to a minimum possible voltage step of 0.61 mV or 0.006% of the
measuring range end value.
The percentages derived in this way also represent the theoretical best-case values for the
operational limit. With a resolution of 8 bits and a measuring range of 0 to 10 V, it is therefore
impossible to achieve a precision of more than 0.4 %. In practical life, this value would be
clearly worse, depending on the implemented circuitry.
Calculating the maximum measuring error
The maximum measuring error of a 0 to 10 V analog module with an operational limit of
±0.5 % across the entire temperature range is calculated as follows:
10 V x 0.5/100 = 50 mV
This means the maximum measuring error is approximately ±50 mV. This also means that
each analog voltage input is subject to a distortion of ±50 mV across the entire input range.
A voltage of 3.5 V that is to be measured can therefore be visualized by any value between
3.45 V and 3.55 V.
For more information on the operational/basic error limits, refer to the Operational and basic
error limits (Page 23) section.
To Next Page
Loading...
