Defining Targets - ironstream_for_kafka - ironstream_for_elastic - Ironstream_Hub - ironstream_for_splunk - 1.3

Ironstream Hub Administration
Product type
Software
Portfolio
Integrate
Product family
Ironstream
Product
Ironstream > Ironstream for Elastic®
Ironstream > Ironstream for Kafka®
Ironstream > Ironstream for Splunk®
Ironstream > Ironstream Hub
Version
1.3
Language
English
ContentType
Administration
Product name
Ironstream Hub
Title
Ironstream Hub Administration
First publish date
2022

Targets are represented as green rectangles on the Pipeline Canvas.

A Pipeline must have at least one Target. Once a Pipeline is initially created, it will have a single, undefined Target. A single Pipeline can be used to forward data to multiple Targets which can be of the same type (e.g., different Splunk indexes) or different types (e.g., QRadar and Splunk) as required.

To configure a Target:
  1. Click the undefined Target box, which is added to the Pipeline by default when it is created.
  2. Click the icon next to the Source or Process and select Add Target to add an additional Target to the Pipeline Canvas.

When you select an undefined Target or add a new Target, you will be presented a list of all available Target types. Some may be disabled as your license determines the Targets you can select.

For example, consider the situation where you have a license to send z/OS data to Splunk. If you have not yet defined a Source, all Target types will be available, as Mimix HealthLink data can always be sent from an IBM i platform to any destination without a license. If you define the Source as coming from z/OS, then only the HTTP[S] Target will be selectable. Your license only permits you to send the data from z/OS to Splunk by using an HTTP[S] Target.

After selecting the Target type, a configuration panel will be displayed at the bottom of the screen for the entry of details such as name, IP address(es), credentials, and so on.

You can configure following types of targets:

When a new Target is created, it is automatically added to the Canvas and linked to the Source or Process.

If you want to change the configuration details of a previously configured Target, click into the appropriate Target Box on the Canvas to display the Target Configuration Panel.

Datadog Log/Datadog Metric

To add a Datadog Log or a Datadog Metric Target:
  1. Click anywhere in the Undefined Target box to configure the Target.
  2. In the Add Target tab, click either Datadog Log or Datadog Metric, depending on the type of data that is being forwarded.
  3. Complete the fields in both the General and HTTPS tabs of the Datadog Log Configuration panel.
    Note: There is a relationship between the Site field on the General tab and the URL field on the HTTP[S] tab. Essentially the Site field is a drop-down of all current Datadog sites and the selected site defines the URL where data is forwarded to. In the event that Datadog add further sites in-between Hub releases, the URL field is editable to ensure that data can get to your required destination.

File

The File Target configuration panel has two tabs: General and File Management. The General tab contains important information required to configure a File Target. The File Management tab can be treated as an advanced setting for the Target configuration.

To add a File Target:
  1. Click anywhere in the Untitled Target box to configure the Target.
  2. In the Add Target tab, click File.
  3. Complete the fields for the General tab in the bottom panel of the canvas.
    Note: The file name generated by the Hub is path + hostname + file identifier + datetime + file extension. The total length of the generated name must not exceed 260 characters for a Windows system or 4096 for a Linux system.
  4. Click the File Management tab and complete the fields.

Google Chronicle

Note: This feature is applicable only in v1.3.6 or later versions.

Google Chronicle is a cloud-based service built on Google’s infrastructure for enterprises to capture, search, and analyze security and network telemetry.

To add a Google Chronicle target:
  1. Click on the Untitled Target node.
  2. Select Google Chronicle from the Add Target panel.
  3. Complete the fields in the Edit Google Chronicle Configuration panel.
  4. Field Description
    URL This is the API endpoint URL. The URL must end with googleapis.com and be preceded with the region-specific sub-domain name. These URLs are listed on Google's Chronicle Ingestion API documentation web page.
    Customer ID This is a unique identifier (UUID) corresponding to a particular Chronicle instance, which is assigned by your Chronicle representative.
    Ingestion API Key The JSON file which is uploaded from the UI and contains the OAuth credentials that allow access to Google Cloud. This Ingestion API Key JSON file can be downloaded from the Google Cloud Credentials page.
    Authorization Scope
    The authorization scope determines the level of access that is being requested during authentication.
    Note: The default value of https://www.googleapis.com/auth/malachite-ingestion will request sufficient privileges to send data to Chronicle's ingestion endpoint. The field must contain the string malachite.
    Initial Timeout Number of seconds to wait for the initial response. The default time is 60 seconds.
    Retry Interval If a connection to a URL fails, the Hub waits for the initial timeout and then will retry the connection based on the drop-down. By default, No Retry is selected.

HTTP[S]

Note: This feature is applicable only in v1.3.2 or later versions.

The HTTP[S] Target configuration panel has two tabs: General and Headers. The General tab contains important information required to configure an HTTP[S] Target. The Headers tab can be treated as an advanced setting for the Target configuration.

To add an HTTP[S] Target:
  1. Click anywhere in the Untitled Target box to configure the Target.
  2. In the Add Target tab, click HTTP[S].
  3. Complete these fields for the General tab in the bottom panel of the canvas:
    Field Description
    Target name Specify the name of the target. The character limit is from 1 to 50.
    Description Provide a short description of the target. This will be visible on the canvas. The character limit is 200.
    URL Enter the API endpoint URL.
    Note: The specified URL will be validated in accordance with the licenses in place. For example, if you only have a Splunk license, then the URL will be validated to ensure that it is referencing a Splunk address, similarly, if you only have a DataDog or BigPanda license, then the URL will be validated to ensure that the correct destination is specified.
    Verb or Method Send data to an external URL via an HTTP POST or GET request. POST is selected by default.
    Initial Timeout Number of seconds to wait for the initial response. The default time is 60 seconds.
    Retry Interval If a connection to an URL fails, the Hub waits for the initial timeout, then will retry the connection based on the drop-down. By default, No Retry is selected.
  4. Add any necessary Headers on the Headers tab of the HTTP[S] Target configuration panel. An example of a Header might be the need to set a content type like this:

    Content-Type: application/json

  5. Here you can use the New Header button, enter a key of Content-Type without the trailing ':' character as this is added automatically, and then enter application/json as the value. You can enter a key without a value, and you can enter multiple Header keys.

    If you need a Header key or value pair to be secure because it contains sensitive information, you can use the New Secure Header button. The key remains visible in the UI, but any data entered into the value field is obfuscated as you type and encrypted when saved in the configuration file. It is only decrypted when it is used in a connection to the Target URL. If you create a Secure Header, you must enter a value. You can enter multiple keys with the same value.

IBM QRadar

Note: This feature is available from Hub v1.3.7 or later.

The IBM QRadar configuration panel allows you to configure a Target to send SYSLOG data directly to the QRadar SIEM for analysis and alerting.

To add an IBM QRadar Target:
  1. Click anywhere in the Untitled Target box to configure the Target.
  2. In the Add Target tab, click IBM QRadar.
  3. Complete the fields in the Edit IBM QRadar Configuration panel.

Kafka

Note: This feature is applicable only for Ironstream Hub version 1.3.5 and later.

The Kafka configuration panel allows you to configure a Target to send data into Kafka.

To add a Kafka Target:
  1. Click on the Untitled Target node.
  2. Select Kafka from the Add Target panel.
  3. Complete the fields in the Edit Kafka Target Configuration panel.
    Field Description
    Name The name of the Target. The character limit is 1 to 50.
    Description An optional description of the Target. The character limit is 1 to 200.
    Topic The name of the Kafka topic into which data will be inserted, with a maximum length of 249 characters. Only the following characters are permitted in a topic name:
    • Letters
    • Numbers
    • Periods
    • Underscores
    • Hyphens
    Maximum Cache Size

    The maximum amount of data to hold in memory if a connection to a Kafka target drops. When the target becomes available again, the data in the cache will be transmitted, avoiding data loss. If the cache fills before the connection is re-established, the oldest data is progressively discarded.

    The minimum value you can set is 8 MB, and the maximum is 4096 MB.
    Retry Interval If a connection to the Kafka target fails, the Hub will attempt a reconnection based on this drop-down. The default value is No Retry.
    Enable TLS Encryption Indicate if TLS encryption is to be implemented for the connection between Hub and the Kafka Broker.
    PKCS #12 File Path

    The path to a file containing the PKCS #12 archive for TLS to use.

    This must be a valid path for the operating system holding the file. The file does not need to exist for you to be able to save a Pipeline, but it must be present to run it.
    PKCS #12 Password

    The password associated with the PKCS #12 archive.

    Requires a minimum of 6 characters and a maximum of 255.
    Brokers See the table below.

A Kafka target must have one or more Brokers associated with it. Each Broker must have a unique combination of hostname/port, and at least one must be enabled.

For each broker, complete the following fields:
Field Description
Hostname The hostname of the Broker. Must be a valid hostname.
Port The port number for the Broker. For a non-TLS connection, the default is 9092 and for a TLS connection, it is 9093. You may set any value from 0 to 65535.
Enable/Disable Allow or disallow the sending of data to this Broker. This is useful if a Broker is unavailable but you do not wish to remove it from the Target.

ServiceNow Discovery

The ServiceNow Discovery configuration panel allows you to configure a ServiceNow Discovery Target.
Important: This feature is still being completed, so although it shows as a Target type it is currently not fully functional. No licenses will be provided to enable it to be operational. This is intended to be accomplished in the future release of the Hub.

TCP/IP

Note: This feature is applicable only in v1.3.4 or later versions.

The TCP/IP target configuration panel allows you to configure a TCP/IP target.

To add a TCP/IP target:
  1. Click anywhere in the Untitled Target box.
  2. Select TCP/IP from the Add Target panel.
  3. Complete the fields in the Edit TCP/IP Target Configuration panel.
    Field Description
    Name The name of the target. The character limit is 1 to 50.
    Description A short optional description of the target. This will be visible on the canvas. The character limit is 200.
    Hostname or IP address The IP address or fully qualified domain name (FQDN) of the TCP/IP server.
    Port The port number of the TCP/IP server.
    Initial timeout

    The timeout when initially establishing a connection to the TCP/IP server.

    A value of 0 results in a connection attempt taking place indefinitely.
    I/O timeout

    The amount of time a connection will wait for input or output operations to complete.

    A value of 0 results in the connection being dropped as soon as data is no longer being sent.
    Enable Keepalive Enable Keepalive to help ensure that connections stay active and are reliable. It sends small test messages between connected devices to achieve this.
    Keepalive interval

    The interval between Keepalive packets being sent over the network.

    This can only be configured when Enable Keepalive is ticked.