6240B DC Voltage Current Source/Monitor Operation Manual
9.5.2 How to Calculate Accuracy for Resistance Measurement
9-15
CMV item digit value
CMV = A Vo / 1 V
= 30 nA 3 V / 1 V (*1)
= 90 nA
The source resolution in the 3 mA range for current source is 100 nA according to the specification.
Thus, the CMV item digit value is 0.90 digits.
From , and
Full-scale (FS) item for current source = 8 digits + 12 digits + 0.9 digits
= 20.9 digits
From the above results,
Accuracy =
0.05% 20.9 digits
Next, the accuracy in current source of 3 mA and voltage measurement of 3 V is calculated.
Measurement resistance value R = V / I = 3 V / 3 mA = 1 k (Display: 1.0000 k)
Accuracy = (1 k 0.05%) 20.9 digits
=
0.5 20.9 digits
Thus, the accuracy is specified between 0.9974 k to 1.0026 k
*1: In accuracy calculation for resistance measurement, the measurement voltage value is used as output
voltage Vo.
To Next Page
Loading...
