Configuring Scheduled Jobs to Run Concurrently - trillium_discovery - trillium_quality - Latest

Trillium DQ Repository Administrator Guide

Product type
Software
Portfolio
Verify
Product family
Trillium
Product
Trillium > Trillium Discovery
Version
Latest
Language
English
Product name
Trillium Quality and Discovery
Title
Trillium DQ Repository Administrator Guide
Copyright
2024
First publish date
2008
Last updated
2024-10-18
Published on
2024-10-18T15:31:00.219841

In Control Center and Discovery Center, scheduled jobs can run consecutively or concurrently, depending on whether you run the jobs now or at later times.

How jobs run also depends on the maximumWeight parameter set in the server configuration (config.txt) file:

  • If maximumWeight is 1, scheduled jobs run consecutively in sequential order.

  • If maximumWeight is greater than 1, scheduled jobs run concurrently up to the set value. For example, if maximumWeight = 10, up to ten scheduled jobs can run at the same time.

Note: Out of the box, maximumWeight is set to 2.

Note the following guidelines:

  • If you are using Trillium Discovery for Big Data, the value of the maximumWeight parameter must be 2 or greater to support multiple jobs running simultaneously. Otherwise, when you profile HDFS data sources on the edge node, the process will fail.

  • When you analyze business rules using the Group By option and the maximumWeight parameter is greater than 1, business rule analysis runs concurrently on multiple rules at a time. This can lead to performance issues including a slow down (or hanging) of your system. For more information, see Configuring Business Group Limit Setting.

To configure scheduled jobs to run concurrently

  1. On the Trillium server system, stop the Trillium scheduler service.

  2. Open the config.txt file for editing. The file is located in the following location:

    • On Windows: C:\Program Files\Trillium Software\MBSW\17\etc 

    • On UNIX: …install path/metabase/etc 

  3. Go to the key scheduler section.

  4. Modify the value of the maximumWeight parameter as needed. For example,

    key scheduler { 
        value port [get {public ports scheduler}] 
        value timeout [expr {5*60*1000}]   
        value maximumWeight 10
  5. Save and close the file.

  6. Start the Trillium scheduler service.