Precision and Accuracy for Numeric Data - trillium_discovery - 17.2

Trillium Expression Builder

Product type
Software
Portfolio
Verify
Product family
Trillium
Product
Trillium > Trillium Quality
Trillium > Trillium Discovery
Version
17.2
Language
English
Product name
Trillium Quality and Discovery
Title
Trillium Expression Builder
Topic type
Administration
Overview
Installation
Reference
How Do I
Configuration
First publish date
2008
ft:lastEdition
2024-07-01
ft:lastPublication
2024-07-01T08:45:29.606589

For numeric operations, a fixed point decimal arithmetic implementation is available. This allows for accurate arithmetic calculations by-passing previous floating point implementations and converting all numbers into integers. Since calculations are effectively done using integer arithmetic giving accurate results, you do not have to factor in floating point rounding errors. For example, you can simply add up 20000 + 20000 to get 40000, not 40000.00001.

Note: The Expression Builder supports 20 after the decimal place precision.

For example:

  • d[1].d[49] (result would be rounded to d[1].d[20])
  • 0.d[50] (leading zero are not significant; result would be rounded to 0.d[20])
  • d[50].0 (trailing zero are not significant)
  • d[10].d[40] (result would be rounded to d[10].d[20])
  • d[40].d[10]

where d = a digit from 0 to 9 and the [] indicates the number of digits in the number.

Note: Significant digits are defined as the total digits in a number, excluding leading and trailing zeroes, before and after the decimal place. Precision is defined as the number of digits after the decimal place.

For information about the functions that support this implementation, see Expression Builder Functions.