AzureBlob Get - Data360_Analyze - Latest

Data360 Analyze Server Help

Product type
Software
Portfolio
Verify
Product family
Data360
Product
Data360 Analyze
Version
Latest
Language
English
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2024
First publish date
2016
Last updated
2024-11-28
Published on
2024-11-28T15:26:57.181000

Downloads files from a server using the Azure Blob protocol.

The AzureBlob nodes enable you to access data from Azure containers so that you can integrate your data flows with Azure. See:

Downloading a single file from an Azure Blob container

  1. Drag an AzureBlob Get node onto the canvas. Then, in the Container property, specify the name of an Azure Blob container where the file that you want to download is located.
  2. In the Object property, specify the name of the Azure object that you want to download, including the file type e.g. example_file.txt.
  3. Provide your Azure account name and account key details in the AccountName and AccountKey properties.
  4. In the Directory property, specify where to store the file that is downloaded.

Downloading multiple files from an Azure Blob container

  1. Drag an AzureBlob List node onto the canvas and connect the listed files output to an AzureBlob Get node.
  2. In the Container property of the AzureBlob List node, type the name of the Azure Blob container where the files that you want to download are stored.
  3. Provide your Azure account name and account key details in the AccountName and AccountKey properties. Then, run the AzureBlob List node to generate a list of files in the specified container.
  4. On the AzureBlob Get node, select the (from Field) variant of the Container property and specify the name of the input field that references the container name e.g. Container.
  5. Select the (from Field) variant of the Object property, then specify the name of the input field that references the files that you want to download e.g. Object.
  6. Provide your Azure account name and account key details in the AccountName and AccountKey properties.
  7. In the Directory property, specify where to store the downloaded files. Then, run the AzureBlob Get node to download the files from the specified Azure container.

Example

You want to download all files with the "Product _A" prefix from an Azure container:

  1. Drag an AzureBlob List node onto the canvas. In the Container property, type the name of the Azure container for which you want to list files, e.g. my-example-container.
  2. In the ObjectPrefix property, type Product_A.
  3. Provide your Azure account name and account key details in the AccountName and AccountKey properties.
  4. Run the AzureBlob List node.

    The output of the AzureBlob List node is as follows:

    Container

    unicode

    Object

    unicode

    BlobType

    unicode

    Size

    long

    Modified

    datetime

    Created

    datetime

    ETag

    unicode

    ContentType

    unicode

    ContentMD5

    unicode

    Metadata

    unicode

    my-example-container Product_A_1.txt BLOCK_BLOB 20 2020-10-26T10:00:00-0400 2020-10-25T17:01:45-0400

    "0x8D8713F56839283"

    text/plain 9e53e712984923778166af== {}
    my-example-container Product_A_2.txt BLOCK_BLOB 20 2020-10-25T10:14:20-0400 2020-10-25T09:11:12-0400 "0x8D8713F780376EB" text/plain F95EWR129866731230655a== {}
    Tip: Only the objects with the specified prefix are listed.

  5. Connect the listed files output of the AzureBlob List node to an AzureBlob Get node. On the Get node, select the (fromField) variant of the Container property and type the name of the input field that references the Azure container, in this case Container.
  6. Select the (from Field) variant of the Object property, then specify the name of the input field that references the files that you want to download, in this case Object.
  7. In the Directory property, specify where to store the downloaded files, e.g. C:\Users\<username>\Documents\AzureBlob_download.
  8. Run the AzureBlob Get node to download the files from the specified Azure Blob container.
Tip: For additional information on Azure Blob storage, see the Microsoft Azure online documentation.

Properties

Container

Specify the name of the Azure Blob Container.

A value is required for this property.

Object

Specify the name of the Azure Object.

A value is required for this property.

AccountName

Specify the Azure Account Name.

A value is required for this property.

AccountKey

Specify the Azure Secret Key.

A value is required for this property.

FileName

Optionally specify the location of the file to store the retrieved object.

The default value is the name of the object.

Choose the (from Field) variant of this property to look up the value from an input field with the name specified.

Directory

Specify the location of a directory as the root to store all retrieved objects.

The default value is unset.

RangeStart

Optionally specify the initial byte offset of the contents to be retrieved. If this is set, the RangeEnd must also be set.

RangeEnd

Optionally specify the last byte offset of the contents to be retrieved.

FailureBehavior

Optionally specify what to do when a file fails to download. Choose from:

  • Error - Report error and stop further processing.
  • Log - Log a warning message and skip the file.
  • Ignore - Skip the file.

The default value is Log.

Inputs and outputs

Inputs: 1 optional.

Outputs: downloaded files, errors.