GZIP CSV Input
Description
GZIP CSV Input reads data from GZIP-compressed delimited files directly, without requiring a separate decompression step. It works similarly to CSV File Input but is designed specifically for compressed source files, making it useful when processing large data feeds or archived datasets delivered in .gz format. The step supports custom delimiters, lazy conversion for performance, and the ability to accept filenames from a previous step for dynamic file processing.
Configurations
| Field Name | Description |
|---|---|
| Step Name | Specify the name of the step. This name has to be unique in a single workflow. |
| Filename OR The filename field (data from previous steps) | Specify the name of the GZIP CSV file to read from. Or Select the fieldname that will contain the filename(s) to read from. If this step receives data from a previous step, this option is enabled as well as the option to include the filename in the output. |
| Delimiter | Specify the file delimiter character used in the target file. Special characters (e.g. CHAR HEX01) can be set with the format $[value], e.g. $[01] or $[6F,FF,00,1F]. |
| Enclosure | Specify the enclosure character used in the target file. Special characters (e.g. CHAR HEX01) can be set with the format $[value], e.g. $[01] or $[6F,FF,00,1F]. |
| NIO buffer size | Specify the size of the read buffer. It is the number of bytes read from the disk at one time. |
| Lazy conversion | The lazy conversion algorithm tries to avoid unnecessary data type conversions. This could result in significant performance improvements if possible. |
| Header row present? | Enable this option if the target file contains a header row containing column names. |
| Add filename to result | Enable checkbox to adds the filename(s) read to the result of the workflow. Filenames in result is a unique list being kept in memory that can be used in the next steps or parent workflow steps or parent process entries. |
| The row number field name (optional) | The name of the Integer field that will contain the row number in the output of this step. |
| Running in parallel? | Check this box if you will have multiple instances of this step running (step copies) and if you want each instance to read a separate part of the file(s). |
| File Encoding | Specify the encoding of the file being read. |
| Fields Table | This table contains an ordered list of fields to be read from the file. |
| Preview button | Click to preview the data coming from the target file. |
| Get Fields button | Click to get all the fields from the file based on the current settings (i.e. Delimiter, Enclosure, etc.). All fields identified will be added to the Fields Table. |