124
Comparison
Tolerance:
OCT
o
The value of this variable determines
the
maximum tolerance (how different
the
two
numbers must be
to
be considered unequal) when using any relational function and
at
least one argument
is
a noninteger. For example,
two
numbers are considered un-
equal
if
the
relative difference between
the
two
numbers exceeds
the
comparison
tolerance value. The following illustration shows how
the
comparison tolerance
works with
the
relational functions:
•
-----
A>B
±OCTxA'
~
Value of argument A
Real number line
The relationship of
any value (argument
B)
to
argument A
A'
is
the
next lower integer power
of
16 for
the
largest argument. For example:
:1.
6
*·0
:1.
;.~
~~
I.,
....
----The
first five integer powers of 16.
i
16
256
4096
65536
[leT
~-
I
:I.
1.
'4·::::
1.!)
I
o
:1.
!):::::I, 6 I
1
o
1
Next lower integer power (A')
is
1;
therefore,
the
difference between
the
arguments exceeds
±OCTxA'
(.1x1 = .1).
Next lower integer power (A')
is
16; therefore,
the difference between
the
arguments does
not
exceed ±.OCTxA' (.1x16 = 1.6).
Next lower integer power (A')
is
16; therefore,
the
difference between
the
arguments exceeds
±.OCTxA' (.1x16
= 1.6).
Next lower integer power (A')
is
256; therefore,
the
difference between
the
arguments does
not
exceed ± DCTxA'
(.1
x256
= 25.6).
Note: The OCT function considers any number
in
decimal form a non integer. For
example, 1000
is
an integer and 1000.
is
a noninteger.
The value of
the
comparison tolerance variable also affects the floor and ceiling
functions. The comparison tolerance
is
added
to
the
argument for
the
floor function
and subtracted for
the
ceiling function. For example:
3
I.~
[]CT~-
,0:3
L 2 I
<7B
r~5
10
1
+
2.98
+
.03
= 3.01 (The integer 3
is
in
the
range of
2.98
+ .03.)
2.96
+ .03 = 2.99
3.03
- .03 = 3 (The integer 3
is
in
the
range
of
3.03
- .03.)
3.04 - .03
= 3.01
To Next Page
Loading...
