Logging and handling Python errors - Latest

Data360 Analyze Server Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 Analyze
Version
Latest
Language
English
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2024
First publish date
2016
Last updated
2024-11-28
Published on
2024-11-28T15:26:57.181000

Within the Python node, you have access to four different methods for logging. These methods all perform the same operation, but use a different log level. The methods are as follows:

  • self.logInfo(msg)
  • self.logLow(msg)
  • self.logMedium(msg)
  • self.logHigh(msg)

The severity of the log level with which the message is logged, in comparison to the LogLevel property on the Python node determine whether or not the message is actually written to the log.

These methods and corresponding log levels (as will be reported in the node’s log) are outlined in the following table:

Method LogLevel When to use When written to log? Example

logInfo(msg)

DEBUG

Used when logging information unrelated to any errors, just debugging information which may be useful to identify what is going on in the processing logic. The contents of these debug statements may only make sense to the node developer for troubleshooting purposes.

Node LogLevel = 0

self.logInfo("Beginning to process records")
rec = self.inputs[0].read()
if rec is None:
    self.logInfo("Finished processing records")
    return False
else:
    #process record

logLow(msg)

INFO

Generally useful if an error occurs, but is handled.

Node LogLevel <= 1

rec = self.inputs[0].read()
if rec is None:
    return False
else:
    self.recordNum = self.recordNum+1   optionalField = rec.field["optional"]
    if optionalField is None:
        self.logLow("field 'optional' is not set on record %d" % self.recordNum)
    else:
        #continue processing

logMedium(msg)

WARN

Generally used when an error occurs, but the code is able to make some assumptions & continue. Is possible/likely to lead to an error later.

Node LogLevel <= 2

rec = self.inputs[0].read()if rec is None:
return False
else:
    self.recordNum = self.recordNum+1
    requiredField = rec.field["required"]
    if requiredField is None:
        if self.continueOnRecordError:
            self.logMedium("field 'required' is not set on record %d" % self.recordNum)
        else:
            #continue processing

logHigh(msg)

ERROR

Used when the node is about to fail. Often, an exception will be raised instead and the message will be in the exception details. Can be used in places where a single error can cause the node to fail, but you want to log all of these errors prior to failing the node.

ALWAYS

errors = False
self.dataIdx = inputs[0].metadata.find("Data")
self.lookupIdx = inputs[0].metadata.find("Lookup")
if self.dataIdx == -1:
    self.logHigh("Expected node input 'Data' does not exist")
errors = True

if self.lookupIdx == -1:
    self.logHigh("Expected node input 'Lookup' does not exist")
    errors = True
    
if errors:
    raise braininfo.BrainNodeException, "Required fields did not exist, unable to process node"

Any exception can be raised from the code within the initialize, finalize and pump methods and the node will handle them, failing the node appropriately.

In general, the easiest mechanism for forcing node failure, and reporting a message back to the node log is to raise a BrainNodeException. The following example shows where the required field specified in the InputFieldName property does not exist on the metadata:

idx = input.metadata.find(self.inputFieldName)
if idx == -1:
    raise braininfo.BrainNodeException, "Specified InputFieldName: %s does not exist on input: %s " % (self.inputFieldName, input.name)