6253 DC Voltage Current Source/Monitor Operation Manual
8.6.2 How to Calculate Accuracy for Resistance Measurement
8-16
8.6.2 How to Calculate Accuracy for Resistance Measurement
Example) Calculating the accuracy in current source of 3 mA (3 mA range) and voltage measurement of
3 V (3 V range)
Accuracies for resistance measurement must be calculated separately.
The accuracy formula for resistance measurement is as follows:
The accuracies in the 3 mA range for current source and in the 3 V range for voltage measurement are
calculated from the specifications as follows:
Accuracy in the 3 mA range for current source: 0.025 + 350 nA + 30 nA
Accuracy in the 3 V range for voltage measurement: 0.02 + 120 V
Here, the reading error for current source is as follows:
Reading error for current source = Current source setting error + Voltage measurement reading error
= 0.025 + 0.02 = 0.045 %
Next, the full-scale error for current source is as follows:
Full-scale error for current source = Current source full-scale error digit value- - - - - - - - - - - -
+ Voltage measurement full-scale error digit value - - - - - -
+ CMV error digit value - - - - - - - - - - - - - - - - - - - - - - -
Current source full-scale error digit value
The source resolution in the 3 mA range for current source is 50 nA according to the specification.
The offset error accuracy is 350 nA.
Thus, the current source full-scale error digit value is 35 digits.
Condition
Accuracy
(% of reading) (digits + digits + digits)
Voltage source
Reading error: (Voltage source setting error + Current measurement reading error)
Full-scale error: (Voltage source full-scale error digit value + current measurement full-scale error
digit value + CMV error digit value)
Current source
Reading error: (Current source setting error + Voltage measurement reading error)
Full-scale error: (Current source full-scale error digit value + Voltage measurement full-scale error
digit value + CMV error digit value)
To Next Page
