Micro-batch Processing - Spectrum_OnDemand - spectrum_platform - web_services - Latest

Spectrum OnDemand Web Services (REST)

Product type
Software
Portfolio
Verify
Product family
Spectrum
Product
Spectrum > Geocoding > Enterprise Tax
Spectrum > Web Services
Spectrum > Spatial > Spectrum Spatial
Spectrum > Spectrum Platform
Spectrum > Quality > Addressing
Spectrum > Quality > Spectrum Quality
Spectrum > Spatial > Spectrum Spatial Insights
Spectrum > Discovery
Spectrum OnDemand
Spectrum > Dataflow Designer
Spectrum > Spatial > Spectrum Spatial Analyst
Spectrum > Quality > Context Graph
Version
Latest
Language
English
Product name
Spectrum OnDemand
Title
Spectrum OnDemand Web Services (REST)
Copyright
2024
First publish date
2006
ft:lastEdition
2024-07-17
ft:lastPublication
2024-07-17T23:23:16.481321

Micro-batch processing is a technique where you include more than one record in a single service request. By including multiple records in a request instead of issuing separate requests for each record, you can significantly improve performance when processing a large collection of records through a service. Spectrum OnDemand supports micro-batch processing for REST and SOAP web services.

Micro-batch end point

For AMER, use amer-microbatch.precisely.com. It has a 5 minute timeout allowing larger micro-batch sizes.
For EMEA, use emea-microbatch.precisely.com. It has a 5 minute timeout allowing larger micro-batch sizes.
For APAC, use the standard endpoint as it does not has a special micro-batch endpoint. This endpoint has a 30 second timeout, so the number of records in a micro-batch will need to be smaller.

Micro-Batch Size

For AMER and EMEA using the special micro-batch endpoint, you may put as many records as can fit within a 5 minute timeout. You are charged for each record in the request even if it times out, so it is recommended to choose the number of records that can be processed in 4 minutes in case processing takes longer.

For APAC using the standard endpoint, the maximum number of records allowed in a request depends on the service's category.

Service Category Maximum Input Entities
Addressing 25
Analytic 10
Data Quality 10
Geocoding 25
Consumer Insight 10
PSAP 25
Routing 1
Spatial 1
Tax Jurisdiction 25
Telco 1

Each micro-batch call must be synchronous (1 at a time.) Send the request and wait for the response before sending another request. Asynchronous micro-batch calls can flood Spectrum OnDemand which may result in decreased performance for other customers. If you use asynchronous calls or micro-batches that are too large, your account may be disabled to ensure other customers use of Spectrum OnDemand is not impacted.

Using a Record ID

You may find it helpful to assign an ID to each record in a micro-batch so that you can correlate the records in the request with the records returned in the response. Use user fields to do this. For information about user fields, see The REST Interface.

Micro-Batch Processing in REST

To perform micro-batch processing with a REST web service, include two or more records as XML or JSON in the body of the request and send the request using the POST method. For more information about sending a POST request to a Spectrum OnDemand web service, see JSON POST Request and XML POST Request.

For example, this request includes two records as XML in the body of the request:

POST https://spectrum.precisely.com/rest/ValidateAddressPOST/results.xml
HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: application/xml
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 533
Host: spectrum.precisely.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)

<ValidateAddressPOSTRequest xmlns:svc="http://www.pb.com/spectrum/services/ValidateAddressPOST">
    <svc:Input>
       <svc:Row>
          <svc:AddressLine1>3001 Summer</svc:AddressLine1>
          <svc:City>Stamford</svc:City>
          <svc:StateProvince>CT</svc:StateProvince>
       </svc:Row>
       <svc:Row>
          <svc:AddressLine1>33 west monroe</svc:AddressLine1>
          <svc:City>Chicago</svc:City>
          <svc:StateProvince>IL</svc:StateProvince>
       </svc:Row>
    </svc:Input>
</ValidateAddressPOSTRequest>
Note: Services do not have POST support enabled by default. To perform micro-batch processing with these services you must enable POST support. For more information, see Adding POST Support to a REST Web Service.