4.6 Correcting Measured Values and Displaying Physical Properties Other than Resistance
77
4
This function applies a correction to measured values. It can be used to cancel the effects
of the probing position or differences between measuring instruments, or to apply a user-
specified offset as an alternative to zero-adjustment. In addition, units can be specified,
allowing it to be used to convert measured values to physical properties other than resis-
tance (for example, length).
Scaling is performed by means of the following equations:
R
S
= A × R + B
R
S
: Resistance value after scaling
R : Measured value after zero-adjustment and temperature correction
A : Gain coefficient Setting range: 0.2000 × 10
-3
to 1.9999 × 10
3
B : Offset Setting range: 0 to ±9 × 10
9
(maximum resolution: 1 n)
Displayed and sent/received measured values as well as the printer output format vary with
the gain coefficient.
4.6 Correcting Measured Values and Displaying
Physical Properties Other than Resistance
Values (Scaling Function)
Low-Power: OFF
* When high-precision mode is off in the 100 M range, 4 digits are displayed.
Low-Power: ON
Range
Gain coefficient
(0.2000 to
1.9999)
×10
-3
(0.2000 to
1.9999)
×10
-2
(0.2000 to
1.9999)
×10
-1
(0.2000 to
1.9999)
×1(10
0
)
(0.2000 to
1.9999)
×10(10
1
)
(0.2000 to
1.9999)
×10
2
(0.2000 to
1.9999)
×10
3
10 m 00.000 000.000 0000.000 00.000 00 m 000.000 0 m 0000.000 m 00.000 00
100 m 000.000 0000.000 00.000 00 m 000.000 0 m 0000.000 m 00.000 00 000.000 0
1000 m 0000.000 00.000 00 m 000.000 0 m 0000.000 m 00.000 00 000.000 0 0000.000
10 00.00 000 m 000.000 0 m 0000.000 m 00.000 00 000.000 0 0000.000 00.000 00 k
100 000.000 0 m 0000.000 m 00.000 00 000.0000 0000.000 00.000 00 k 000.000 0 k
1000 0000.000 m 00.000 00 000.000 0 0000.000 00.000 00 k 000.000 0 k 0000.000 k
10 k 00.000 00 000.000 0 0000.000 00.000 00 k 000.000 0 k 0000.000 k 00.000 00 M
100 k 000.000 0 0000.000 00.000 00 k 000.000 0 k 0000.000 k 00.000 00 M 000.000 0 M
1000 k 0000.000 00.000 00 k 000.000 0 k 0000.000 k 00.000 00 M 000.000 0 M 0000.000 M
10 M 00.000 00 k 000.000 0 k 0000.000 k 00.000 00 M 000.000 0 M 0000.000 M 00.000 00 G
100 M* 000.000 0 k 0000.000 k 00.000 00 M 000.000 0 M 0000.000 M 00.000 00 G 000.000 0 G
1000 M 0000.0 k 00.000 M 000.00 M 0000.0 M 00.000 G 000.00 G 0000.0 G
Range
Gain coefficient
(0.2000 to
1.9999)
×10
-3
(0.2000 to
1.9999)
×10
-2
(0.2000 to
1.9999)
×10
-1
(0.2000 to
1.9999)
×1(10
0
)
(0.2000 to
1.9999)
×10(10
1
)
(0.2000 to
1.9999)
×10
2
(0.2000 to
1.9999)
×10
3
1000 m 0000.00 00.000 0 m 000.000 m 0000.00 m 00.000 0 000.000 0000.00
10 00.000 0 m 000.000 m 0000.00 m 00.000 0 000.000 0000.00 00.000 0 k
100 000.000 m 0000.00 m 00.000 0 000.000 0000.00 00.000 0 k 000.000 k
1000 0000.00 m 00.000 0 000.000 0000.00 00.000 0 k 000.000 k 0000.00 k