Sets a tolerance such that if two numbers differ by less than the precision (exclusive), they will be considered equal. Specifically, n1 == n2 if Abs (n1 - n2) < p, where p is the precision.
SetPrecision (rPrecision)
Variable | Description |
---|---|
rPrecision | The tolerance to which real numbers are compared. REAL. |
The default precision is 0.000001.
n1 == n2 iff abs(n1-n2) + DBL_EPSILON <= precisionwhere DBL_EPSILON is the smallest number such that 1.0+DBL_EPSILON != 1.0. It is typically defined as: 2.2204460492503131e-016.
To normalize the comparison, apply a suitable multiplier to the precision and to the values being compared. The multiplier should be based on 1e-15, which is the first power of 10 greater than DBL_EPSILON. The example below demonstrates the use of such a multiplier.
[ ] const MIN_PRECISION = 1e-15 // first power of 10 greater than DBL_EPSILON [ ] [-] testcase Example () appstate none [-] LIST OF NUMBER ln1 = {...} [ ] 1 [ ] .045 [ ] .04 [ ] 1.0e-20 [ ] 1.0e-20 [-] LIST OF NUMBER ln2 = {...} [ ] 4 [ ] .04 [ ] .05 [ ] 1.1e-20 [ ] 1.1e-20 [-] LIST OF REAL lrPrecisions = {...} [ ] 10 [ ] .01 [ ] .01 [ ] 0.00000000000000000001 [ ] 1.0e-21 [ ] REAL r1, r2, rPrecision, rMultiplier [ ] INTEGER i [ ] [-] for i = 1 to ListCount (ln1) [-] if i <= 3 [ ] Print ("Comparing {ln1[i]} to {ln2[i]} with precision {lrPrecisions[i]}") [-] else [ ] Printf ("Comparing %1.24f to %1.24f with precision %1.24f\n", ln1[i], ln2[i], lrPrecisions[i]) [ ] r1 = ln1[i] // will implicitly convert INTEGER to REAL [ ] r2 = ln2[i] // will implicitly convert INTEGER to REAL [ ] rPrecision = lrPrecisions[i] [ ] // If necessary, normalize the comparison. [ ] // If omitted, the fourth comparison (1.0e-20 to 1.1e-20) will fail. [ ] rMultiplier = MIN_PRECISION / rPrecision [-] if rMultiplier > 1.0 [ ] r1 *= rMultiplier [ ] r2 *= rMultiplier [ ] rPrecision *= rMultiplier [ ] SetPrecision (rPrecision) [ ] // The following verification should raise an exception [ ] // for the third comparison (.04 to .05 with precision .01) [ ] // and fifth comparison (1.0e-20 to 1.1e-20 with precision 1.0e-21) [-] do [ ] Verify (r1, r2, "the numbers match within the precision") [ ] // Could also use: [ ] // Verify (r1 == r2, TRUE, "the numbers match within the precision") [-] except [ ] LogError (ExceptData ())
The results file is:
Script OnLineHelpExample[1].t - 2 errors Machine: (local) Started: 03:49:33PM on 08-Nov-2006 Elapsed: 0:00:06 Passed: 0 tests (0%) Failed: 1 test (100%) Totals: 1 test, 2 errors, 0 warnings Testcase Example - 2 errors Comparing 1 to 4 with precision 10.000000 Comparing 0.045000 to 0.040000 with precision 0.010000 Comparing 0.040000 to 0.050000 with precision 0.010000 *** Error: Verify the numbers match within the precision failed - got 0.040000, expected 0.050000 Comparing 0.000000000000000000010000 to 0.000000000000000000011000 with precision 0.000000000000000000010000 Comparing 0.000000000000000000010000 to 0.000000000000000000011000 with precision 0.000000000000000000001000 *** Error: Verify the numbers match within the precision failed - got 0.000000, expected 0.000000