Text File Input
Description
Text File Input is a step in the Input Plugin for Process Studio Workflows. The Text File Input step reads data from text (txt) file types. The other most used format includes Comma Separated Values (CSV files) generated by spreadsheets and fixed width flat files.
The step reads the text input file based on any one of the following two ways:
Column index: Columns are mapped based on their index position in the text file. For example, if the text file contains columns A, B, and C, column index 0 maps to A, index 1 maps to B, and index 2 maps to C.
OR
Column name: Columns are mapped using their header names, for example, A or C instead of index positions. This approach provides flexibility when the column order changes.
Configurations
| Field Name | Description |
|---|---|
| File tab | |
| File or directory | Specify or browse the path of the input text file. Note: Press the "add" button to add the file/directory/wildcard combination to the list of selected files (grid) below. |
| Regular expression* | Specify the regular expression you want to use to select the files in the directory specified in the previous field. For example, you want to process all files that have a .txt extension. (See below "Selecting file using Regular Expressions") |
| Exclude Regular Expression | Specify a regular expression that you want to use to exclude the files in the directory specified in the previous field. For example, specify agent in the field if you want to exclude files containing the word in its filename. |
| Selected Files | This table contains a list of selected files (or wildcard selections) along with a property specifying if file is required or not. If a file is required and it isn't found, an error is generated. Otherwise, the filename is skipped. |
| Delete | Click to delete rows from the Selected Files table. |
| Edit | Click to modify fields to be edited the Selected Fields table. |
| Accept filenames from previous steps | Text File Input step can accept filenames from a previous step enabling dynamic filename handling. Enable this checkbox to get filenames from previous steps. |
| Pass through fields from previous step | Enable this checkbox to add all previous fields coming into the step to the step output. This behaves like a join option. |
| Step to read filenames from | Step from which to read the filenames |
| Field in the input to use as filename | Text File Input looks in this step to determine which filenames to use |
| Button: Show filenames | Click button to display a list of all the files selected. Note that if the workflow is to be run on a separate server, the result might be incorrect. |
| Button: Show file content | Click button to display the first lines of the text-file. In case of any error make sure that the file-format is correct. When in doubt, try both DOS and UNIX formats. |
| Show content from first data line | Helps you position the data lines in complex text files with multiple header lines and more. It shows the first data line excluding header row. |
* Regular expression, searching files using Regular Expressions
The Text File Input step also provides the ability to specify a list of files to read, or a list of directories with wild cards in the form of regular expressions to filter files. Regular expressions are more sophisticated than using '*' and '?' wildcards. Below are a few examples of regular expressions:
| Filename | Regular Expression | Files selected |
|---|---|---|
| /dirA/ | .userdata.\.txt | Find all files in /dirA/ with names containing user data and ending with .txt |
| /dirB/ | AAA.* | Find all files in /dirB/ with names that start with AAA |
| /dirC/ | [ENG:A-Z][ENG:0-9].* | Find all files in /dirC/ with names that start with a capital and followed by a digit (A0-Z9) |
*Accepting filenames from a previous step
This option allows even more flexibility in combination with other steps such as "Get Filenames". You can construct your filename and pass it to this step. This way the filename can come from any source: text file, database table, etc.
| Field Name | Description |
|---|---|
| Content Tab | |
| File type | Select from CSV or Fixed length. |
| Separator | One or more characters that separate the fields in a single line of text. Typically, this is; or a tab. Special character (e.g. CHAR ASCII HEX01) can be set with the format $[value], e.g. $[01] or $[6F, FF,00,1F]. |
| Enclosure | Some fields can be enclosed by a pair of strings to allow separator characters in fields. The enclosure string is optional. If you use repeat enclosures allow text line 'Not the nine o''clock news.'. With ' the enclosure string, this gets parsed as Not the nine o'clock news. Special characters (e.g. CHAR ASCII HEX01) can be set with the format $[value], e.g. $[01] or $[6F, FF,00,1F]. |
| Allow breaks in enclosed fields? | This field is Disabled. |
| Escape | Specify an escape character (or characters) if you have these types of characters in your data. If you have \ as an escape character, the text 'Not the nine o'clock news' (with ' the enclosure) gets parsed as Not the nine o'clock news. Special characters (e.g. CHAR HEX01) can be set with the format $[hex value], e.g. $[01] or $[6F,FF,00,1F]. |
| Header & number of header lines | Enable if your text file has a header row (first lines in the file); you can specify the number of times the header lines appear. |
| Footer & number of footer lines | Enable if your text file has a footer row (last lines in the file); you can specify the number of times the footer row appears. |
| Wrapped lines and number of wraps | Enable checkbox if you deal with data lines that have wrapped beyond a specific page limit; note that headers and footers are never considered wrapped. Specify the number of times wrapped. |
| Paged layout and page size and doc header | Use these options as a last resort when dealing with texts meant for printing on a line printer; use the number of document header lines to skip introductory texts and the number of lines per page to position the data lines. |
| Compression | Enable if your text file is placed in a Zip or GZip archive. Note: Currently, only the first file in the archive is read. |
| No empty rows | Enable if you do not want to send empty rows to the next steps. |
| Include filename in output | Enable if you want the filename to be part of the output. |
| Filename field name | Name of the field that contains the filename. |
| Rownum in output? | Enable if you want the row number to be part of the output. |
| Row number field name | Name of the field that contains the row number. |
| Rownum by file? | Enable to reset the row number per file. |
| Format | Can be either DOS, UNIX or mixed. UNIX files have lines that are terminated by line feeds. DOS files have lines separated by carriage returns and line feeds. If you specify mixed, no verification is done. |
| Encoding | Specify the text file encoding to use; leave blank to use the default encoding on your system. To use Unicode, specify UTF-8 or UTF-16. On first use, Process Studio searches your system for available encodings. |
| Limit | Sets the number of lines that is read from the file; 0 means read all lines. |
| Be lenient when parsing dates? | Disable if you want strict parsing of data fields; if case-lenient parsing is enabled, dates like Jan 32nd will become Feb 1st. |
| The date format Locale | Specify the locale is used to parse dates that have been written in full such as "February 2nd, 2006;" |
| Add filenames to result | Enable checkbox to add the filenames to the internal filename result set. This internal result set can be used later, e.g. to process all read files. |
| Field Name | Description |
|---|---|
| Error Handling Tab | |
| Ignore errors? | Enable if you want to ignore errors during parsing |
| Skip error lines | Enable if you want to skip those lines that contain errors. You can generate an extra file that contains the line numbers on which the errors occurred. Lines with errors are not skipped, the fields that have parsing errors, will be empty (null) |
| Error File Field Name | Specify a field name to contain the error file name. |
| File error message field name | Specify a field name to contain the message of the error in file. |
| Error count field name | Specify a field name to contains the number of errors on the line |
| Error fields field name | Add a field to the output stream rows; this field contains the field names on which an error occurred |
| Error text field name | Add a field to the output stream rows; this field contains the descriptions of the parsing errors that have occurred |
| Warnings file directory | When warnings are generated, they are placed in this directory. The name of that file is <warning dir>/filename.<date_time>.<warning extension> |
| Error files directory | When errors occur related to non-existing or non-accessible files, they are placed in this directory. The name of the file is <errorfile_dir>/filename.<date_time>.<errorfile_extension> |
| Failing line numbers files directory | When a parsing error occurs on a line, the line number is placed in this directory. The name of that file is <errorline dir>/filename.<date_time>.<errorline extension> |
| Field Name | Description |
|---|---|
| Filters Tab | |
| Filter string | Specify the string for which to search. |
| Filter position | The position where the filter string must be at in the line. Zero (0) is the first position in the line. If you specify a value below zero (0) here, the filter string is searched for in the entire string. |
| Stop on filter | Specify Y here if you want to stop processing the current text file when the filter string is encountered. |
| Positive match | Specify Y here if you want to process lines that match the filter, or N if you want to ignore such lines. |
| Field Name | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Fields Tab | Use the Fields tab to define the specific text file columns to extract and configure their output properties. You can map columns by name or by index, and set data types, formatting, and text processing rules. This configuration determines the final structure of the data flowing to the next workflow step. | ||||||||
| Index Based Reading (read based on column index) | Select the checkbox to map input fields based on their numerical index (position) rather than the column header name. Get fields: Click to scan the first row of the Excel file. The step automatically detects the headers and populates the Index Based Reading table with the corresponding names and default settings. | ||||||||
| Name | Specify the name of the column as it appears in the output rows. | ||||||||
| Type | Select the data type for the field to ensure correct processing. Available data types: - Number - String - Date - Boolean - Integer - BigNumber - Binary - Timestamp - Internet Address | ||||||||
| Format | Specify the format to parse the incoming data correctly. The format is used to read or display field value based on the selected type. For example, Select MM/dd/yyyy to read a date value like 12/25/2025 correctly. | ||||||||
| Position | This is needed when processing the 'Fixed' file type. It is zero based, so the first character is starting with position 0. | ||||||||
| Length | Specify the maximum allowable length for the field. For strings, this represents the character count; for numbers, it represents the total number of significant digits. For example, Set Length to 10 to allow a maximum of 10 characters, such as INV1234567. | ||||||||
| Precision | Specify the number of decimal places allowed for numeric values. For example, Set Precision to 2 to store values like 123.45 | ||||||||
| Currency | Specify the currency symbol used in the input data, such as ₹, $, or €. For example, enter ₹ to display amounts as ₹5000. | ||||||||
| Decimal | Specify the character used as the decimal separator, for example, a dot( . ) or a comma (,). | ||||||||
| Group | Specify the character used to separate thousands or other distinct groups within large numbers. For example, a dot (.) or a comma (,). | ||||||||
| Null if | Treat the value as NULL. | ||||||||
| Default | Default value in case the field in the text file was not specified (empty). | ||||||||
| Trim Type | Select a trimming method to remove unwanted whitespace from the data: - None - Left - Right - Both | ||||||||
| Repeat | Select Y (Yes) to enable the repeat function. When enabled, the step fills empty cells with the value from the immediately preceding row. This is essential for processing vertically grouped data. For Example:
If Apple appears once and the next rows are blank, if Repeat is set to Y, the step will repeat Apple for those rows. | ||||||||
| Secret | Select Y to hide the incoming data for the field. The step masks the value (shows ********) in all downstream steps, previews, and logs. To protect the data, the system completely drops this field by default when sending data to a final output file or database. Select N to keep the data visible as plain text. The step displays the actual value in previews and logs and writes it normally to any final output file or database. Important: Combining a secret field with plain text in an upstream step (for example, using a Calculator step to build "Token: " + secretField) causes the system to treat the result as a new field. | ||||||||
| Column Based Reading (read based on column name) | Select checkbox Read Based on Column Name to get inputs based on column name. The fields are mapped using their Column Name or Alias Name property instead of column position. If not selected, columns are read using their index (0, 1, 2...) and Index Based Reading table is available. When you click Get fields, the header names are automatically copied into the Column Based Reading table. Advantages: - Column order changes do not break the workflow. - Fields are easier to understand and maintain. Note: If this checkbox is not selected, the table uses column index instead of names. | ||||||||
| Fail if column name not found | Select the checkbox to stop the workflow and report an error if a specified column is missing from the input file. If the checkbox is cleared, the step continues execution and assigns null value to the missing column, ensuring the workflow does not break due to minor schema changes. | ||||||||
| Case insensitive comparison | Select the checkbox to match column names regardless of capitalization. This allows the step to identify columns even if the header case in the file differs from the configuration (for example, matching ID with id). | ||||||||
| Column Name | Specify the name of column. The column name is the header text from the Excel file. You can specify the column name in different ways, such as static value, field, parameter, or global/local PS variable. The column name must follow the format: SheetName.ColumnName For example: If the sheet name is Country and the column name is Capital: Create a parameter Param1 = Country.Capital Then use ${Param1} in the Get Fields Based on Column Name table. If the value is coming from a field, simply select that field in the table. | ||||||||
| Name | Specify internal name used for the defined column in the workflow output. | ||||||||
| Alias Name | Specify an alternate column name that the system uses when the specified column name is not available in the input file. During execution, the system attempts to match the field using the alias name if the primary column name is missing. Note: You can configure a maximum of five alias names per field. Example: If the specified column name is customerID and it is not present in the input file, the system uses the alias name (for example, customer_id, customerid, CustomerId) to locate and read the corresponding column. | ||||||||
| Type | Select the data type for the field to ensure correct processing. Available data types: - Number - String - Date - Boolean - Integer - BigNumber - Binary - Timestamp - Internet Address | ||||||||
| Format | Specify the format to parse the incoming data correctly. The format is used to read or display field value based on the selected type. For example, Select MM/dd/yyyy to read a date value like 12/25/2025 correctly. | ||||||||
| Length | Specify the maximum allowable length for the field. For strings, this represents the character count; for numbers, it represents the total number of significant digits. For example, Set Length to 10 to allow a maximum of 10 characters, such as INV1234567. | ||||||||
| Precision | Specify the number of decimal places allowed for numeric values. For example, Set Precision to 2 to store values like 123.45 | ||||||||
| Currency | Specify the currency symbol used in the input data, such as ₹, $, or €. For example, enter ₹ to display amounts as ₹5000. | ||||||||
| Decimal | Specify the character used as the decimal separator, for example, a dot( . ) or a comma (,). | ||||||||
| Group | Specify the character used to separate thousands or other distinct groups within large numbers. For example, a dot (.) or a comma (,). | ||||||||
| Null if | Treat this value as NULL. | ||||||||
| Default | Default value in case the field in the text file was not specified (empty). | ||||||||
| Trim Type | Select a trimming method to remove unwanted whitespace from the data: - None - Left - Right - Both | ||||||||
| Repeat | Select Y (Yes) to enable the repeat function. When enabled, the step fills empty cells with the value from the immediately preceding row. This is essential for processing vertically grouped data. For Example:
If Apple appears once and the next rows are blank, if Repeat is set to Y, the step will repeat Apple for those rows. | ||||||||
| Secret | Select Y to hide the incoming data for the field. The system masks the value (shows ********) in all downstream steps, previews, and logs. To protect the data, the system completely drops this field by default when sending data to a final output file or database. Select N to keep the data visible as plain text. The system displays the actual value in previews and logs, and writes it normally to any final output file or database. Important: - Combining a secret field with plain text in an upstream step (for example, using a Calculator step to build "Token: " + secretField) causes the system to treat the result as a brand-new field. This new field is not protected automatically. Mark the new derived field as a secret manually. - Renaming a secret column using the Rename Fields step removes the secret status from the new column and exposes the data as plain text. |
*Number Formats: For information on valid numeric formats used in this step, view the Number Formatting Table as below.
| Symbol | Location | Localized | Meaning |
|---|---|---|---|
| 0 | Number | Yes | Digit |
| # | Number | Yes | Digit, zero shows as absent |
| . | Number | Yes | Decimal separator or monetary decimal separator |
| - | Number | Yes | Minus sign |
| , | Number | Yes | Grouping separator |
| E | Number | Yes | Separates mantissa and exponent in scientific notation; need not be quoted in prefix or suffix |
| ; | Sub pattern boundary | Yes | Separates positive and negative sub patterns |
| % | Prefix or suffix | Yes | Multiply by 100 and show as percentage |
| ‰ | Prefix or suffix | Yes | Multiply by 1000 and show as per mille |
| € (¤) | Prefix or suffix | No | Currency sign, replaced by currency symbol. If doubled, replaced by international currency symbol. If present in a pattern, the monetary decimal separator is used instead of the decimal separator. |
| ' | Prefix or suffix | No | Used to quote special characters in a prefix or suffix, for example, "'#'#" formats 123 to "#123". To create a single quote itself, use two in a row: "# o''clock". |
*Scientific Notation
In a pattern, the exponent character immediately followed by one or more-digit characters indicates scientific notation (for example, "0.###E0" formats the number 1234 as "1.234E3".
*Date formats
| Letter | Date or Time Component | Presentation | Examples |
|---|---|---|---|
| G | Era designator | Text | AD |
| y | Year | Year | 1996; 96 |
| M | Month in year | Month | July; Jul; 07 |
| w | Week in year | Number | 27 |
| W | Week in month | Number | 2 |
| D | Day in year | Number | 189 |
| d | Day in month | Number | 10 |
| F | Day of week in month | Number | 2 |
| E | Day in week | Text | Tuesday; Tue |
| a | Am/pm marker | Text | PM |
| H | Hour in day (0-23) | Number 0 | |
| k | Hour in day (1-24) | Number 24 | |
| K | Hour in am/pm (0-11) | Number 0 | |
| h | Hour in am/pm (1-12) | Number 12 | |
| m | Minute in hour | Number 30 | |
| s | Second in minute | Number 55 | |
| S | Millisecond | Number 978 | |
| z | Time zone | General time zone | Pacific Standard Time; PST; GMT-08:00 |
| Z | Time zone | RFC 822 time zone | -0800 |
| Field | Description |
|---|---|
| Additional Output Fields Tab | |
| Short filename field | Specify the field name that contains the filename without path information but with an extension. |
| Extension field | Specify the field name that contains the extension of the filename. |
| Path field | Specify the field name that contains the path in operating system format. |
| Size field | Specify the field name that contains the size of the field. |
| Is hidden field | Specify the field name that contains if the file is hidden or not (boolean). |
| Uri field | Specify the field name that contains the URI. |
| Root uri field | Specify the field name that contains only the root part of the URI. |
| Function/Button | Description |
|---|---|
| Show filenames | Click button to display a list of all the files selected. Note that if the workflow is to be run on a separate server, the result might be incorrect. |
| Show file content | Click button to display the first lines of the text-file. In case of any error make sure that the file-format is correct. When in doubt, try both DOS and UNIX formats. |
| Show content from first data line | Helps you position the data lines in complex text files with multiple header lines and more. It shows the first data line excluding header row. |
| Get fields | Allows you to guess the layout of the file. In case of a CSV file, this is performed almost automatically. In case you select a file with fixed length fields, you must specify the field boundaries using a wizard. |
| Minimal Width | Click to set all field lengths to the minimum required width, removing any padding. This optimizes memory usage by trimming unnecessary field length allocations. |
| Preview rows | Preview the rows generated by this step. |