Table 2-3 - Extreme Range Error Terms For Impedance and Resistance, K
h
and K
l
Frequency
K
l
K
h
100 Hz, 120 Hz, 1 kHz
(1 mΩ/Z
m
) (Z
m
/2 GΩ)
10 kHz
(1 mΩ/Z
m
) (Z
m
/1.5 GΩ)
100 kHz
(4 mΩ/Z
m
) (Z
m
/50 MΩ)
Accuracy Equations for Specific Measurement Modes
R + Q Accuracy
In the R + Q measurement mode, the basic impedance accuracy, Az, in equation (1), may be read
from Figure 2-1 directly while interpreting the “impedance” as “resistance.” The resistance accuracy is
calculated from equation (2a), with the additional stipulation that if the measured Q has an absolute
value greater than 0.1, then the basic resistance accuracy factor should be multiplied by the factor (1
+ |Q|).
A
r(measured)
% = ± [A
r
× K
i
× K
v
+ 100 × (K
h
+ K
l
) (2a)
Where A
r(measured)
% is the measured or published resistance accuracy and A
r
is the basic
resistance accuracy. The basic resistance accuracy, A
r
, which can be taken from Figure 2-1 by
substituting impedance, Z, with resistance, R.
With the basic resistance accuracy factor denoted A
r
. The accuracy of the measurement of Q is given
by:
Accuracy of Q = ± [(A
r
/100) × (1+Q
2
)] (2b)
Note that the accuracy of Q is specified as a magnitude, not as a percentage.
L+Q Accuracy
The basic impedance accuracy depicted in Figure 2-1 applies to inductance measurements when the
impedance is interpreted to be 2π f L, where f is the test frequency in Hz and L is the inductance in
Henrys. For convenience, Figure 2-1 is redrawn as Figure 2-2 with lines of constant inductance
superimposed. Also, Table 2-3 is recreated for inductive impedances and named as Table 2-4. Note
from Table 2-4 that the range error factor K
l
is negligible for inductances above 15.9/f H and K
h
is
negligible for inductances below 159/f H.
The accuracy of the inductance measurement, A
l (measured)
, is calculated by applying equation (3a),
with the additional stipulation that if the measured Q has an absolute value less than 10, then the
basic inductance accuracy factor, A
l
, should be multiplied by the factor (1 + |1/Q|).
A
l (measured)
% = ± [ A
l
× K
i
× K
v
+ 100 × (K
h
+ K
l
)] (3a)