Configuring Payload Wrapper - Ironstream_Hub - 1.3.2

Ironstream Hub Administration

Product type
Software
Portfolio
Integrate
Product family
Ironstream
Product
Ironstream > Ironstream Hub
Version
1.3.2
Language
English
ContentType
Administration
Product name
Ironstream Hub
Title
Ironstream Hub Administration
Topic type
Administration
First publish date
2022

This process allows you to batch JSON records and include one or more fixed JSON key/value pairs along with other data. The purpose of this process is twofold. One is to enable the sending of data in batches as a single 'payload', rather than individual records. The other is to allow the incorporation of information in a user-chosen location within that batch, which may be necessary to support data destined for specific targets. The records in any given batch replace the mandatory <<PAYLOAD>> string.

When ProcessType is set to PayloadWrapper, a parent field named PayloadWrapperConfiguration must be populated with sets of fields as shown below. If you do not enter sufficient properties or if they contain invalid values, the process will be invalid and the pipeline will not start.

You can connect the output of this process to further JSON wrapper processes if necessary.

Fields Description Valid Values
Wrapper Describes additional data added to a batch of records, and instructions as to where those records should be inserted - they will replace the <<PAYLOAD>> string. Any valid JSON plus it must include the string <<PAYLOAD>>
ThresholdInBytesForCreatingANewBatch The size in bytes beyond which the batch of records will be finished. This setting is not a guarantee of an exact size of batch. It is used as a threshold to inform the process where to complete the current batch, and restart batching with new records. An integer number with a minimum value of 100 and a maximum of 1000000
Example:
{
  "Name": "F_PayloadWrapper",
  "Id": "f12bedb8-08f4-11ed-861d-0242ac12000d",
  "PayloadWrapperConfiguration":

{     "Wrapper": "

{\"app_key\": \"e6fed41e7334a914d23495d2112a3f23\", \"alerts\": [<<PAYLOAD>>]}
",
    "ThresholdInBytesForCreatingANewBatch":1000,
  },
  "ProcessType": "PayloadWrapper"
}
For the example parameters above, given two JSON records of:
{"DATETIME":"2021-05-19T21:58:22","JobNameJobId":"QCLNUSRPGM|496355",
"host":"Test7","isourcetype":"msgq","job_id":"496355","job_name":"QCLNUSRPGM",
"msg_id":"CPI1E81","msg_text":"","primary_property":"host",
"secondary_property":"JobNameJobId","status":"critical"}
{"DATETIME":"2021-05-19T21:58:24","JobNameJobId":"QCLNUSRPGM|496355",
"host":"Test7","isourcetype":"msgq","job_id":"496357","job_name":"QCLNUSRPGN",
"msg_id":"CPI1E82","msg_text":"","primary_property":"host",
"secondary_property":"JobNameJobId","status":"critical"}
A single batch of data would occur with both records inside it where the <<PAYLOAD>> string was in the Wrapper - the text has additional line feeds incorporated for easier reading - in reality this would be all one continuous string of data:,
{ "app_key": "e56ed41e7174a914d23495d2112a3f23", "alerts": [ 
{"DATETIME":"2021-05-19T21:58:22"," JobNameJobId":"QCLNUSRPGM|496355"," host":"Test7"," 
isourcetype":"msgq"," job_id":"496355"," job_name":"QCLNUSRPGM"," msg_id":"CPI1E81","msg_text":"", 
"primary_property":"host", "secondary_property":"JobNameJobId", "status":"critical"}
}
,
{

{"DATETIME":"2021-05-19T21:58:24"," JobNameJobId":"QCLNUSRPGM|496355", "host":"Test7", 
"isourcetype":"msgq", "job_id":"496357" ,"job_name":"QCLNUSRPGN", "msg_id":"CPI1E82", "msg_text":"", 
"primary_property":"host" ,"secondary_property":"JobNameJobId" ,"status":"critical"}
}

]  <-- this is the final bracket of the Wrapper: property from the example above

}