The Read from File stage specifies an input file for a job or subflow. It is not available for services.
- Create a connection to these file servers using Spectrum Management Console or Discovery. For details, see section Defining Connections.
- Select the file using the File name field in File Properties tab (described below).
File Properties Tab
Field Name | Description |
---|---|
Server name | Indicates whether the file you select as input is located on the computer running Spectrum Enterprise Designer or on the Spectrum Technology Platform server. If you select a file on the local computer, the server name will be My Computer. If you select a file on the server the server name will be Spectrum Technology Platform. |
File name | Specifies the path to the file. Click the ellipses
button (...) to go to the file you
want. You can read multiple files by using a wild card character to read data from multiple files in the directory. The wild card characters * and ? are supported. For example, you could specify *.csv to read in all files with a While reading a file from an HDFS file server, the compression formats supported are:
Note: The extension of the file indicates the compression
format to be used to decompress the file.
Attention: If the Spectrum Technology Platform server
is running on Linux, remember that file names and paths on these
platforms are case sensitive.
|
Record type | The format of the records in the file. Select one
of:
|
Character encoding | The text file encoding. For more information, see Character Encodings. |
Field separator | Specifies the character used to separate fields in a delimited
file. For example, this record uses a pipe (|) as a field
separator:
These characters available to define as field separators are:
If the file uses a different character as a field separator, click the ellipses button to select another character as a delimiter. |
Text qualifier |
The character used to surround text values in a delimited file. For example, this record uses double quotes (") as a text qualifier.
The characters available to define as text qualifiers are:
If the file uses a different text qualifier, click the ellipses button to select another character as a text qualifier. |
Record separator |
Specifies the character used to separate records in line a sequential or delimited file. This field is not available if you check the Use default EOL check box. The record separator settings available are:
If your file uses a different record separator, click the ellipses button to select another character as a record separator. |
Use default EOL |
Specifies that the file's record separator is the default end of line (EOL) character used on the operating system on which the Spectrum Technology Platform server is running. Do not select this option if the file uses an EOL character that is different from the default EOL character used on the server's operating system. For example, if the file uses a Windows EOL but the server is running on Linux, do not check this option. Instead, select the Windows option in the Record separator field. |
Record length |
For fixed width files, specifies the exact number of characters in each record. For line sequential files, specifies the length, in characters, of the longest record in the file. |
First row is header record |
Specifies whether the first record in a delimited file contains header information and not data. For example, this file snippet shows a header row in the first record.
|
Treat records with fewer fields than defined as malformed |
Delimited file records containing fewer fields than are defined on the Fields tab will be treated as malformed. |
Import |
Imports the file layout definition, encoding setting, and sort options from a settings file. The settings file is created by exporting settings from another Read from File or Write to File stage that used the same input file or a file that has the same layout as the file you are working with. |
Export |
Saves the file layout definition, encoding setting, and sort options to a settings file. You can then import these settings into other Read from File or Write to File stages that use the same input file or a file that has the same traits as the file you are working with now. You can also use the settings file with job executor to specify file settings at runtime. For information about the settings file, see The File Definition Settings File. |
Fields Tab
The Fields tab defines the names, positions, and, for fixed width and line sequential files, lengths of fields in the file. For more information, see these topics:
Sort Fields Tab
The Sort Fields tab defines fields by which to sort the input records before they are sent into the dataflow. Sorting is optional. For more information, see Sorting Input Records.
Runtime Tab
Field Name | Description |
---|---|
File name |
Displays the file name selected in the first tab. |
Starting record |
If you want to skip records at the beginning of the file when reading records into the dataflow, specify the first record you want to read. For example, if you want to skip the first 50 records, in a file, specify 51. The 51st record will be the first record read into the dataflow. |
All records |
Select this option if you want to read all records starting from the record specified in the Starting record field to the end of the file. |
Max records |
Select this option if you want to only read in a certain number of records starting from the record specified in the Starting record field. For example, if you want to read the first 100 records, select this option and enter 100. |