6240B DC Voltage Current Source/Monitor Operation Manual
9.5.3 How to Calculate Accuracy for Low-Resistance Measurement
9-17
Example)
Calculating the accuracy in current source of 1 A (1 A range) with a resistance measured value
of 10 m and in the voltage limiter range 300 mV
1. Reading item calculation
Resistance measurement reading item
=
Current source setting item + Voltage measurement reading item
= 0.05 % + 0.02 % = 0.07 %
Setting item in the current source 1 A range: 0.05 %
Reading item in the voltage measurement 300 mV: 0.02 %
2. Full-scale item calculation
Full-sale item = (A + Rm B)/Is
= (10 V + 10 m 300 A)/1 A = 13
From *2,
A: 10 V (Current limiter range 300 mV)
Rm: 10 m (Resistance measured value)
B: 300 A (The current source resolution in the 1A range is 100 A. Thus,
3 digits 100 A = 300 A)
Is: 1 A (Current source setting value)
From the above results,
Accuracy =
0.07 % 13
= (10 m 0.07 %) 13
= 7 13 =
20
Thus, the accuracy is specified between 9.980 m to 10.020 m.
To Next Page
Loading...
