Micro-batch processing is a technique where you include more than one record in a single service request. By including multiple records in a request instead of issuing separate requests for each record, you can significantly improve performance when processing a large collection of records through a service. Spectrum OnDemand supports micro-batch processing for REST and SOAP web services.
Micro-batch end point
For AMER, use amer-microbatch.precisely.com. It has a 5 minute timeout allowing larger micro-batch sizes.Micro-Batch Size
For AMER and EMEA using the special micro-batch endpoint, you may put as many records as can fit within a 5 minute timeout. You are charged for each record in the request even if it times out, so it is recommended to choose the number of records that can be processed in 4 minutes in case processing takes longer.
For APAC using the standard endpoint, the maximum number of records allowed in a request depends on the service's category.
| Service Category | Maximum Input Entities |
|---|---|
| Addressing | 25 |
| Analytic | 10 |
| Data Quality | 10 |
| Geocoding | 25 |
| Consumer Insight | 10 |
| PSAP | 25 |
| Routing | 1 |
| Spatial | 1 |
| Tax Jurisdiction | 25 |
| Telco | 1 |
Each micro-batch call must be synchronous (1 at a time.) Send the request and wait for the response before sending another request. Asynchronous micro-batch calls can flood Spectrum OnDemand which may result in decreased performance for other customers. If you use asynchronous calls or micro-batches that are too large, your account may be disabled to ensure other customers use of Spectrum OnDemand is not impacted.
Using a Record ID
You may find it helpful to assign an ID to each record in a micro-batch so that you can correlate the records in the request with the records returned in the response. Use user fields to do this. For information about user fields, see The SOAP Interface.
Micro-Batch Processing in SOAP
To perform micro-batch processing in a SOAP web service, include two or more records in the SOAP request. For example, this request contains two records:
POST http://spectrum.example.com:8080/soap/ValidateAddress HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 782
Host: config813vm0:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:val="http://www.pb.com/spectrum/services/ValidateAddress">
<soapenv:Header/>
<soapenv:Body>
<val:ValidateAddressRequest>
<val:input_port>
<val:Address>
<val:AddressLine1>1 N. State St.</val:AddressLine1>
<val:City>Chicago</val:City>
<val:StateProvince>IL</val:StateProvince>
</val:Address>
<val:Address>
<val:AddressLine1>3001 summer</val:AddressLine1>
<val:City>stamford</val:City>
<val:StateProvince>ct</val:StateProvince>
</val:Address>
</val:input_port>
</val:ValidateAddressRequest>
</soapenv:Body>
</soapenv:Envelope>