Google Cloud Services Get - Data360_Analyze - 3.12

Data360 Analyze Server Help

Product
Data360 Analyze
Version
3.12
Language
English
Portfolio
Verify
Product family
Data360
Product name
Data360 Analyze
Title
Data360 Analyze Server Help
Copyright
2023
First publish date
2016

Downloads files from a server using the Google Cloud Services protocol.

The Google Cloud Services nodes enable you to access data from Google Cloud Services buckets, so that you can integrate your data flows with Google. See:

Downloading a single file from Google Cloud Services

  1. In the Bucket property, specify the name of a Google Cloud Services bucket that contains the file that you want to download.
  2. In the Object property, specify the name of the Google Cloud Services file that you want to download.
  3. In the Directory property, specify where to store the file that is downloaded.

Downloading multiple files from Google Cloud Services

  1. Drag a Google Cloud Services List node onto the canvas and connect it to a Google Cloud Services Get node.
  2. In the Bucket property of the Google Cloud Services List node, type the name of the Google Cloud Services bucket that contains the files that you want to download. Run the Google Cloud Services List node to generate a list of files in the specified bucket.
  3. On the Google Cloud Services Get node, select the (from Field) variant of the Bucket property and specify the name of the input field that contains the Google Cloud Services bucket name.
  4. Select the (from Field) variant of the Object property, then specify the name of the input field that references the files that you want to download.
  5. In the Directory property, specify where to store the downloaded files.

Example

You want to download all files with the "Product" prefix from a Google Cloud Services bucket:

  1. Drag a Google Cloud Services List node onto the canvas.
  2. In the Bucket property, type the name of the Google Cloud Services bucket for which you want to list files, e.g. my-example-bucket
  3. In the ObjectPrefix property, type Product.
  4. Run the Google Cloud Services List node and connect it to a Google Cloud Services Get node.

    The output of the Google Cloud Services List node is as follows:

    Bucket

    unicode

    Object

    unicode

    Size

    long

    IsDirectory

    boolean

    Owner

    unicode

    Modified

    datetime

    StorageClass

    unicode

    Md5

    unicode

    my-example-bucket Product_A.txt 25 NULL owner@account 2017-10-27T08:04:58-0400 STANDARD md5
    my-example-bucket Product_B.txt 25 NULL owner@account 2017-10-27T08:04:58-0400 STANDARD md5
    my-example-bucket Product_C.txt 25 NULL owner@account 2017-10-27T08:04:58-0400 STANDARD md5
    my-example-bucket Product_D.txt 25 NULL owner@account 2017-10-27T08:04:58-0400 STANDARD md5

  5. On the Google Cloud Services Get node, select the (fromField) variant of the Bucket property and type the name of the input field that contains the Google Cloud Services bucket name, in this case Bucket.
  6. Select the (from Field) variant of the Object property, then specify the name of the input field that references the files that you want to download, in this case Object.
  7. In the Directory property, specify where to store the downloaded files, e.g. C:\Users\<username>\Documents\Google Cloud Services download.
  8. Run the Google Cloud Services Get node to download the files from Google Cloud Services.
Tip: For additional information on interacting with Google Cloud Services, see the Google Cloud Services Documentation.

Properties

Bucket

Specify the name of the Google Cloud Services Bucket.

Choose the (from Field) variant of this property to look up the value from an input field with the name specified.

A value is required for this property.

Object

Specify the name of the Google Cloud Services Object.

Choose the (from Field) variant of this property to look up the value from an input field with the name specified.

A value is required for this property.

Credentials

Optionally specify the file containing the Google Cloud Services credentials.

If the Credentials are not set then the Google Cloud Services default strategy will be used.

Directory

Specify the location of a directory as the root to store all retrieved objects.

A value is required for this property.

FileName

Optionally specify the location of the file to store the retrieved object.

Choose the (from Field) variant of this property to look up the value from an input field with the name specified.

The default value is the name of the object.

FailureBehavior

Optionally specify what to do when a file fails to download. Choose from:

  • Error - Report error and stop further processing.
  • Log - Log a warning message and skip the file.
  • Ignore - Skip the file.

The default value is Log.

Inputs and outputs

Inputs: 1 optional.

Outputs: downloaded files, errors.