Now that the SDK is installed and the routing reference data is configured the reference
data must be distributed around the cluster.
For the purposes of this guide, we will:
-
Create an install directory on hdfs and give ownership to sdkuser.
sudo -u hdfs hadoop fs -mkdir hdfs:///precisely
sudo -u hdfs hadoop fs -chown sdkuser:sdkuser hdfs:///precisely
-
Upload the reference data into HDFS.
hadoop fs -mkdir hdfs:///precisely/routing
hadoop fs -copyFromLocal /precisely/routing/data hdfs:///precisely/routing/data
-
When the data node performs routing tasks, the node will download the reference data
from HDFS and onto the local file system. This means a local directory needs to be set up
on all data nodes. Perform the following commands on all data nodes and HiverServer
nodes.
sudo mkdir /precisely/downloads
sudo chown sdkuser:hadoop /precisely/downloads
sudo chmod 775 /precisely/downloads