6240B DC Voltage Current Source/Monitor Operation Manual
9.5.3 How to Calculate Accuracy for Low-Resistance Measurement
9-16
9.5.3 How to Calculate Accuracy for Low-Resistance Measurement
The accuracy for low-resistance measurement is calculated by using the following formulas:
*2: Full-scale item = (A + Rm B)/Is
A: Voltage limiter range tolerance
Typical value of full scale item tolerance
Low-resistance measurement
(4-wire connection)
Reading item:
(Current source setting item + Voltage measurement reading item)
Full-scale item:
*2
Voltage limiter range A
300 mV 10 V
3 V 50 V
15 V 1 mV
Rm: Resistance measured value
B: Current value tolerance due to current source linearity
3 digits worth (5 digits worth in the 4 A range)
Is: Current source setting value
Voltage
source range
Voltage limiter range A Rm B Is Tolerance
30 mA 300 mV 10 V
10 m
3 A10 mA1 m
300 mA 300 mV 10 V30 A 100 mA 103
1 A 300 mV 10 V 300 A1 A 13
30 mA 300 mV 10 V
100 m
3 A10 mA1 m
300 mA 300 mV 10 V30 A 100 mA 130
1 A 300 mV 10 V 300 A1 A 40
30 mA 300 mV 10 V
1
3 A 10 mA 1.3 m
300 mA 300 mV 10 V30 A 100 mA 400
1 A 3 V 50 V 300 A1 A350