6240B DC Voltage Current Source/Monitor Operation Manual
9.5.2 How to Calculate Accuracy for Resistance Measurement
9-14
9.5.2 How to Calculate Accuracy for Resistance Measurement
Example)Calculating the accuracy in current source of 3 mA (3 mA range) and voltage measurement of
3 V (3 V range)
The accuracies for resistance measurement must be calculated separately.
The accuracy formula for resistance measurement is as follows:
The accuracies in the 3 mA range for current source and in the 3 V range for voltage measurement are
calculated from the specifications as follows:
Accuracy in the 3 mA range for current source: 0.03 + 800 nA + 30 nA
Accuracy in the 3 V range for voltage measurement: 0.02 + 120 V
Here, the reading item for current source is as follows:
Reading item for current source = Current source setting item + Voltage measurement reading item
= 0.03 + 0.02 = 0.05 %
Next, the full-scale (FS) item for current source is as follows:
Full-scale item for current source =Current source full-scale item digit value- - - - - - - - -
+ Voltage measurement full-scale item digit value - - -
+ CMV item digit value- - - - - - - - - - - - - - - - - - - - -
Current source full-scale item digit value
The source resolution in the 3 mA range for current source is 100 nA according to the specification.
The offset item accuracy is 800 nA.
Thus, the current source full-scale item digit value is 8 digits
.
Voltage measurement full-scale item digit value
The measurement resolution in the 3 V range for voltage measurement is 10 V according to the
specification.
The offset item accuracy is 120 V.
Thus, the voltage measurement full-scale item digit value is 12 digits
.
Condition
Accuracy
(% of reading) (digits + digits + digits)
Voltage source
Reading item: (Voltage source setting item + Current measurement reading item)
Full-scale item: (Voltage source full-scale item digit value + current measurement full-scale item
digit value + CMV item digit value)
Current source
Reading item: (Current source setting item + Voltage measurement reading item)
Full-scale item: (Current source full-scale item digit value + Voltage measurement full-scale
item digit value + CMV item digit value)
To Next Page
Loading...
Need help?
General
