Where Connect for Big Data Sort Fits into MapReduce - Connect_ETL - 9.13

Connect ETL for Big Data Sort User Guide

Product type
Software
Portfolio
Integrate
Product family
Connect
Product
Connect > Connect (ETL, Sort, AppMod, Big Data)
Version
9.13
Language
English
Product name
Connect ETL
Title
Connect ETL for Big Data Sort User Guide
Copyright
2023
First publish date
2003
Last updated
2023-09-11
Published on
2023-09-11T19:03:59.237517

When a MapReduce job is invoked with Connect for Big Data sort acceleration, Connect for Big Data Sort replaces the following stages of the job processing:

  • Sort (on the map side), if one or both of the following conditions is true:
  • the property dmx.map.sort.a is set to ‘d’
  • the map output compression is set to bzip2
  • Otherwise, the “null sort” is invoked, in which case Connect for Big Data partitions but does not sort the data on the map side.
  • Merge (on the reduce side)