Deletes files from an Azure Datalake Storage location.
Azure Datalake Storage nodes enable you to access data lakes on Azure storage, so that you can integrate your data flows accordingly. See:
Deleting a single file from Azure
-
Drag an Azure Datalake Storage Delete node onto the canvas, then in the RemotePath property, specify the path to the file that you want to delete.
-
Provide your Azure AccountName, together with the AccountKey, or the following properties combined, in the relevant field/s:
ClientID, together with the ClientSecret and the TenantID.
- Run the Azure Datalake Storage Delete node to delete the specified file from your Azure location
Deleting multiple files from Azure
-
Drag an Azure Datalake Storage List node onto the canvas and connect the listed files output to an Azure Datalake Storage Delete node.
-
In the RemotePath property of the Azure Datalake Storage List node, type the path to the Azure Datalake Storage location where the files that you want to delete are located.
-
Provide your Azure AccountName, together with the AccountKey, or the following properties combined, in the relevant field/s:
ClientID, together with the ClientSecret and the TenantID.
- Run the Azure Datalake Storage List node to generate a list of files in the specified location.
- On the Azure Datalake Storage Delete node, select the (from Field) variant of the RemotePath property, and specify the fieldname from the input dataset that references the Azure location.
- Repeat Step 3 above, in the Azure Datalake Storage Delete node.
- Run the Azure Datalake Storage Delete node to remove the files in the specified location.
Properties
FileSystem
Specify the file system of the Azure Datalake Storage.
A value is required for this property.
RemotePath
Specify the path to the Azure Datalake Storage objects.
A value is required for this property.
Recurse
Optionally specify whether to recursively delete the files under RemotePath.
The default value is False.
AccountName
Specify the Azure Account Name.
A value is required for this property.
One of the following should be entered:
-
AccountKey
The Azure Secret Key.
Or the combination of:
-
ClientID
The Client ID for the registered app.
-
ClientSecret
The Client Secret for the registered app.
-
TenantID
The Tenant ID (directory) for the registered app.
FailureBehavior
Optionally specify what to do when a file fails to be removed. Choose from:
- Error - Report error and stop further processing.
- Log - Log a warning message and skip the file.
- Ignore - Skip the file.
The default value is Log.
Enabled
Optionally specify whether the node is enabled or disabled.
You can either choose True or False, or reference another property (see the Using derived property values topic) which will be evaluated to a true or false value.
Disabled nodes are not executed, even if they are selected to run.
The default value is True.
LogLevel
Optionally specify the level at which non-fatal messages are logged.
The lower the level, the more information will be recorded in the log file. Choose from:
- 0 - Information
- 1 - Low
- 2 - Medium
- 3 - High
- 4 - Fatal
The default value is 2 (Medium), which can be changed in the ls_brain_node.prop configuration file, by modifying the property ls.brain.node.logLevel.
Inputs and outputs
Inputs: 1 optional.
Outputs: errors.