About Precision and Accuracy for Numeric DataBusiness Rules - trillium_discovery - trillium_quality - Latest

Inline Quality and Discovery

Product type
Software
Portfolio
Verify
Product family
Trillium
Product
Trillium > Trillium Quality
Trillium > Trillium Discovery
Version
Latest
Language
English
Product name
Trillium Quality and Discovery
Title
Inline Quality and Discovery
Copyright
2024
First publish date
2008
Last updated
2024-10-18
Published on
2024-10-18T15:10:12.949492

For information about the functions that support this implementation, see Expression Builder Functions.

For numeric operations, a fixed point decimal arithmetic implementation is available. This allows for accurate arithmetic calculations by-passing previous floating point implementations and converting all numbers into integers. Since calculations are effectively done using integer arithmetic giving accurate results, you do not have to factor in floating point rounding errors. For example, you can simply add up 20000 + 20000 to get 40000, not 40000.00001.

For numeric operations, a fixed point decimal arithmetic implementation is available. This allows for accurate arithmetic calculations by-passing previous floating point implementations and converting all numbers into integers. Since calculations are effectively done using integer arithmetic giving accurate results, you do not have to factor in floating point rounding errors. For example, you can simply add up 20000 + 20000 to get 40000, not 40000.00001.

Note:

The Expression Builder supports 20 after the decimal place precision.

For example:

  • d[1].d[49] (result would be rounded to d[1].d[20])
  • 0.d[50] (leading zero are not significant; result would be rounded to 0.d[20])
  • d[50].0 (trailing zero are not significant)
  • d[10].d[40] (result would be rounded to d[10].d[20])
  • d[40].d[10]

where d = a digit from 0 to 9 and the [] indicates the number of digits in the number.

Note:

Significant digits are defined as the total digits in a number, excluding leading and trailing zeroes, before and after the decimal place. Precision is defined as the number of digits after the decimal place.