Advanced configuration of execution sandboxing - Data360_Analyze - Latest

For the steps to enable execution sandboxing, see Enabling execution sandboxing using Docker. This topic provides information on advanced configuration options.

As per the configuration instructions in the Enabling execution sandboxing using Docker topic, all of the detailed configuration work is done in the call to laeConfig executionContainers enable. For advanced configurations, you can provide additional arguments with this call.

The last argument to laeConfig executionContainers enable is always the name of the network within which the Docker containers are to run. This does not need an argument name to be provided.

The other possible arguments are:

• --user - The username of the Data360 Analyze user under which the configuration is to be performed. This must correspond to an admin user in the Data360 Analyze application. If not provided, you will be prompted for this.
• --password - The password of the Data360 Analyze user under which the configuration is to be performed. If not provided, you will be prompted for this.
• --dockerFile - Optionally specify the Dockerfile which is to be used to create the Data360 Analyze image. On Linux, this option can generally be ignored. On Windows, this option should be provided as the template Dockerfile needs to be modified prior to being used. Note that before modifying the Dockerfile, it is recommended that you make a copy of the original template. If no dockerFile is specified, the dockerFile under <installDir>/lib/Dockerfile will be used.
  Note: Only one of --dockerFile and --imageId arguments can be provided.
• --imageId - Optionally specify the ID of the docker image to use. If you have built the image separately, you can provide the imageId instead of using a Dockerfile. If an imageId is provided, the Docker image is not built.
• --tags - Optionally specify any additional tags to apply on the container when it is built. If a container is being built it will always be tagged with analyze-server-<port> and any provided tags will be used in addition to (not in place of) this tag. Has no effect if an imageId is provided.
• --maxContainers - Optionally specify the maximum number of Data360 Analyze Docker containers that are able to run concurrently. The default value is 3.

  As each user's executions will be in their own sandboxed Docker execution container, this number should be set to the maximum number of users that would be expected to be concurrently executing data flows. For any given user, note that all of their executions can occur in the one Docker execution container.

  You can change this value at any time by modifying the value of the property ls.brain.server.container.maxNumberContainers in the cust.prop file in your <site>/conf/ directory. After changing the property, restart Data360 Analyze.

  If the value set in the ls.brain.server.container.maxNumberContainers property is exceeded when an additional user attempts to run a node, the application will repeatedly check if any executions have completed in order to start a new container for the new user to run the node. After each check, the application will wait a brief period of time. By default the wait time is 2s (2000ms). You can configure the number of milliseconds to wait before checking if any containers can be stopped by modifying the value of the property ls.lae.execution.container.resourceWaitSleepTimeMs in the cust.prop file in your <site>/conf/ directory. After changing the property, restart Data360 Analyze.

  The application will check a maximum of ls.lae.execution.container.resourceWaitMaxRetryCount times. The default value for ls.lae.execution.container.resourceWaitMaxRetryCount is set to 10. If this property has the value set to -1, the application will never stop waiting until there are resources free and the execution it is trying to run can proceed (that is, when an execution container has been stopped, or there is nothing running on an execution container anymore and it can be stopped). You can edit this property in the cust.prop file in your <site>/conf/ directory. After changing the property, restart Data360 Analyze.

• --containerExpiryMinutes - Optionally specify the number of minutes that the Data360 Analyze Docker containers are to be active for. The Data360 Analyze Docker execution containers are said to "expire" when the specified number of minutes have passed without an active execution in the container.

  When the container cleanup task runs, any containers which have not had any active executions running in the past <containerExpiryMinutes> minutes will be stopped.

  Note that containers can be stopped prior to this expiry time elapsing if they do not have any active executions running, the maximum number of containers has been reached and there is a pending execution request waiting for an available container.

  You can change this value at any time by modifying the value of the property ls.lae.execution.container.executionContainerExpirySeconds in the cust.prop file in your <site>/conf/ directory. After changing the property, restart Data360 Analyze.

  Note: Note that the property in the configuration file is in seconds whereas the value provided to the laeConfig is in minutes.

  The default value is 60 (minutes)

• --containerCleanupFrequencyMinutes - Optionally specify the frequency with which the container cleanup task runs. This effectively determines how frequently expired containers will be stopped. The value of this argument should be less than 60.

  You can change this value at any time by modifying the value of the property ls.lae.execution.container.containerCleanupCronSchedule in the cust.prop file in your <site>/conf/ directory. After changing the property, restart Data360 Analyze.

  Tip: Note that the property in the configuration file is a cron schedule for when the task is run. Therefore, if you require a different frequency which is not simply something less than 60 minutes, you can leave the default when running the laeConfig command and enter a different cron schedule for the property in the properties file.

  The default value is 30 (minutes).

• --createNetwork - Optionally specify that you want to create the network within which the networks are to run. If the --createNetwork flag is not provided, it is assumed that the specified network already exists in Docker and will be used for the Data360 Analyze Docker containers.
  Note: The name of the network must be provided as the last argument to the laeConfig executionContainers enable command.
• --processingUser - Optionally specify the name of the user that will run the Data360 Analyze server.
  Note: This argument should not be provided on Windows or if you have configured a multi-user installation. It should also not be provided if you are specifying a custom dockerFile. For Linux installations which are not configured for a multi-user installation, and using the default dockerFile, then this argument must be provided.
• --trustedMode - Optionally specify how the trust is to be configured such that the web application can communicate with the controlling Data360 Analyze server and how the controlling Data360 Analyze server is to communicate with execution servers. The options are gateway or host. The gateway option will be used by default except when the network specified at the time execution sandboxing is enabled is an l2Bridge network.

  If gateway is specified, the controlling server will be configured to accept communications from the web application over the gateway IP address. Similarly, if the controlling server is not in a Docker container, execution servers will accept communications from the controlling server over the gateway IP address.

  If host is specified, the controlling server will be configured to accept communications from the web application over the host IP address, as will execution servers from the controlling Data360 Analyze server.