Each component, developed on a common code base, functions seamlessly with each other across environments, providing the consistency and reliability that is expected in an enterprise integration tool. At the core of the product are three components:
- Controller Daemons - Authenticate all connection requests, managing secure communication between Capture/Publisher Agents, Engines often running on other platforms as well as Utilities.
- Data Capture Agents - The data capture layer consists of near-real-time and asynchronous data capture agents designed to be high performance and best of breed for their respective datastore types. Each Capture Agent is tightly coupled to a Publisher providing the second and third layers of the framework which provide fully configurable Transient Storage and forward Transport of the captured data.
- Engines - The multi-function Apply Engine performs, based on a SQL like scripting language, all necessary data filtering, transformation and augmentation required to apply the changed data to a target datastore. The Replicator Engine streamlines change data replication for selected sources and provides a streaming utility function to Kafka and HDFS including automatic generation of the target schema and optionally, schema evolution for the target schema based on changes made to the source schema.
These components, together with a highly efficient transient storage and TCP/IP based transport that uses NaCL libraries for authentication and optional encryption, provide a single high performance enterprise class solution".
Other components that supplement the core product include:
- Utilities that configure, manage and control the Capture and Publisher Agents and Engines.