Gen AI
Description
This plugin step is utilized to call LLM APIs, enabling the use of various models from different providers, such as OpenAI, based on the configuration.
Reference Links:
OpenAI API Refrence link: https://platform.openai.com/docs/api-reference/chat
Example – Invoice Field Extraction:
Goal
Extract the invoice number, date, total amount, and line items from an invoice, and obtain the bounding box coordinates for each extracted field.
Input:
- On the LLM Settings tab, select OpenAI. Enter your endpoint (https://api.openai.com/v1/chat/completions) and your key (
${OPENAI_KEY}). - Set the Model to gpt-4o, Temperature 0.1 or 0.2 to , Max Tokens to 4000, Top P to 0.9, and Request Timeout to 60.
- On the Input tab, enter the following System Prompt: "You are an expert Invoice AI system for invoice data extraction."
- Enter the following User Prompt: "Extract the invoice fields invoice_number, invoice_date, total_amount, line_items, description, quantity, unit_price and line items from the attached document."
- Set the Input File Or Directory to
${invoicePath}and the Max File Count Limit to 1. - Enable Structured Output Definition and add the following rows to the schema table:
Invoice Schema
| Field Name | Type | Parent | Description |
|---|---|---|---|
| invoice_number | string | Invoice identifier as printed on the document. | |
| invoice_date | string | Invoice date in ISO 8601 format (YYYY-MM-DD). | |
| total_amount | number | Grand total amount with no currency symbol. | |
| line_items | json array | All invoice line items. | |
| description | string | line_items | Item description text. |
| quantity | number | line_items | Quantity as a number. |
| unit_price | number | line_items | Unit price with no currency symbol. |
- On the Output Fields tab, set the Response Field to LLMResponse.
- Select Include Metadata Fields In Output to receive the bounding box data.
What You Can See:
After a successful execution, the step generates the following output row columns:
- LLMResponse (String): The complete JSON response from the model.
- invoice_number, invoice_date, total_amount (String): The promoted top-level extracted fields.
- invoice_number_metadata, invoice_date_metadata, total_amount_metadata (String): JSON objects containing the spatial data for each field, including page_number, coordinates (an 8-point polygon), page_width, page_height, confidence, and value.
- line_items (String): The JSON array representing the table line items. Each item includes the table columns such as description, quantity, and unit price, along with a row_metadata object containing per-cell bounding box details.
Configurations
| Field Name | Description |
|---|---|
| Step Name | Name of the step. This must be unique in a single workflow. The field is mandatory. |
| LLM Settings: | |
| Connection Details: | |
| LLM Provider | LLM Provider name. Default value is OpenAI. Note: Changing the provider automatically updates the underlying API request template. It also disables the Model field for Azure and Google. |
| API EndPoint | LLM provider API endpoint for accessing the service. The field is mandatory. |
| API/LLM Key | API token key for authentication. If the checkbox Accept Value as variable/static is selected, then the password field appears as a text box and accepts static or variable values. OR If the checkbox Accept Value as variable/static is not selected, then the password field appears as a dropdown in which you can select a field from the previous steps. |
| Request Timeout (seconds) | API request timeout in seconds Default value is 60 seconds The field is mandatory. |
| AE Gateway Connection: | |
| Use AE Gateway | Select to route requests through the AE Gateway. Enabling this option ignores the API/LLM Key and activates the Gateway fields. Notes: • If selected, the Token Key field is disabled. • Supports Google Vertex AI and Azure OpenAI LLM providers. |
| Gateway Endpoint | Specify the AE Gateway URL provided by your administrator. The field accepts variable/static values |
| Gateway Token | Specify the token issued by the AE Gateway. The system automatically injects this token as the Authorization header in the API request. If the checkbox Accept Value as variable/static is selected, then the password field appears as a text box and accepts static or variable values. OR If the checkbox Accept Value as variable/static is not selected, then the password field appears as a dropdown in which you can select a field from the previous steps. |
| Test Connection | Click Test to verify the connection is established successfully by checking the provided credentials and connection details. |
| Model Configuration: | Define the parameters that control the model's behavior, response length, and creativity. |
| Model | Specify the model ID recognized by the provider (for example, gpt-4o-mini). The field is unavailable for Azure and Google providers. Note: The field is available if you select LLM provider as OpenAI. Default value: gpt-4o-mini The field is mandatory. |
| Temperature | Temperature value range is between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Default value: 0.4 The field is mandatory. |
| Max Tokens | The maximum number of tokens that can be generated. Default value: 2000 The field is mandatory. |
| Top P | top_p value range is between 0.1 and 1. This parameter used to control the diversity of the generated text. Default value: 0.9 The field is mandatory. |
| Thinking Budget | Specify a positive integer to allocate tokens for the model's internal reasoning process before it generates the final response. Note: The field is available only if you select LLM provider is set as Google Vertex AI or Google AI Default Value: 300 |
| Request Headers: | Use to add custom HTTP headers to include with every API call. |
| Header Key | Specify a unique HTTP header key (for example, api-version, api-key, or x-tenant-id). Important: Do not add an Authorization key if you use AE Gateway mode. The system automatically sets the Authorization: Bearer <gatewayToken> header. Adding it manually causes a duplicate-header error. Note: Header key must not be null/empty. And duplicate header keys are rejected. |
| Header Value | Specify the corresponding value for the header key. Note: Header value must not be null/empty. |
| Input Tab: | |
| Input File Or Directory | Input File Or Directory path. Note: All LLM providers support the following file extensions: .jpeg, .jpg, .png, and .pdf. |
| Max File Count Limit | Specify a positive number to set the maximum number of files to process from the input directory. Allowed range: 1 to 60 Default value: 1 |
| Prompt Configuration Tab: | |
| System_Prompt | Overall behavior, tone, or role of the model for the conversation. It provides context or guidelines that influence how the model should respond. The field is mandatory. |
| User Prompt | Input or questions provided by the user. This is the text that the model responds to. The field is mandatory. |
| Few-Shot Examples (Optional) | Add pairs of user inputs and expected model responses to train the LLM on your preferred output format. The system sends these pairs as a conversation history to guide the model's behavior. Note: If you add a row, you must provide a value in the User Example column. Expected Response is optional. |
| Google Vertex AI Context Cache: | Use to store large, reused inputs (including attached files and system prompts) directly on Google Vertex AI. Re-uploading identical content across multiple LLM calls consumes unnecessary tokens and increases execution time. Context caching allows subsequent executions to reference the stored data, reducing billed input tokens and accelerating workflows that share documents or instructions. Reference: Context caching Best Practices: Enable caching strictly for workflows that send the exact same large documents or instructions across multiple calls within the active Cache TTL window. Note: • The caching is not recommended if the same file is processed only once. • Using the feature to process the same file only once increases API costs. • The context cache configuration fields appear only when the LLM Provider is set to Google Vertex AI. Example: Invoice Processing To classify an invoice document and subsequently extract specific fields using multiple LLM calls: 1. Enable the context cache on the first GenAI step and specify a cache name (for example, InvoiceBill). The system uploads and caches the input file. 2. Specify the exact same cache name in subsequent GenAI steps. The downstream steps reference the stored invoice to perform classification and data extraction without re-uploading the file. 3. Select Delete cache after this step completes checkbox to delete the cache in last LLM calls. |
| Enable Context Cache | Select the checkbox to activate context caching. Note: • The system supports context caching exclusively for Google Vertex AI provider. Selecting any other provider disables the configuration fields. • Google Vertex AI requires a minimum file or prompt size of approximately 1024 tokens and rejects cache creation for smaller inputs. • If an older version of the plugin opens a workflow with context caching enabled, manually reset the API Request Body to the standard template to ensure proper execution. |
| Cache Name | Specify a name to create a cache. To reuse the cache, specify the same cache name in every GenAI step in the workflow. Note: To prevent conflicts when multiple workflows run at the same time: • When running from an Agent, the step appends the request ID to the cache name. • When running from Process Studio, the step appends the project to the cache name. The field accepts static or environment variable value. Notes: • Use the exact same name across multiple steps to share a single cache. • Shared caches require identical model, project, and location configurations. • Execution searches by name and model, applying the newest match to resolve duplicate display names. • Caches remain linked to the specific model and project/location. Steps sharing a cache must utilize the exact same model. • Missing or expired caches trigger automatic recreation and a single call retry to prevent workflow failure. • Step logs track lifecycle events (Created, Reused, Recreated, Deleted) and the unique resource ID for troubleshooting. |
| Cache TTL (seconds) | Specify the lifespan for a newly created cache in seconds. Specify a value between 60 and 1800 (1 to 30 minutes). The field accepts variable/static values Default Value: 600 Note: The step applies the time limit only during initial cache creation. Reusing an existing cache does not reset or add time to the original expiration limit. |
| Include System Prompt in Cache | Select the checkbox to store the system prompt inside the cache along with the attached files. When a step reuses the cache, execution utilizes the cached instructions instead of re-sending the text. Note: • Activation requires a configured system prompt in the Input tab. • If a step reuses a cache but the checkbox is unchecked, the step ignores the configured system prompt. Google Vertex AI ignores provided system instructions when using cached content. |
| Delete cache after this step completes | Select the checkbox to delete the cache immediately after a successful LLM call, bypassing the standard time limit. Note: If the deletion process fails, the step continues running without interruption. The cache remains in the repository and expires naturally based on the Cache TTL. |
| Structured Output Definition | Use the Structured Output tab to force the LLM to return data using a strict JSON schema, allowing you to parse the response directly into individual workflow fields instead of receiving a single block of free-form text. |
| Structured Output Definition | Select the checkbox to enable structured output extraction. The system generates a schema from your table and injects it into the API request. Clear this checkbox to receive free-form text. Note: If you select this option, you must configure at least one row in the grid. |
| Field Name | Output Field name The field is mandatory. |
| Type | Select Type of Output Field i.e. string, json array, integer, Boolean, number The field is mandatory. |
| Parent | Parent of Output Field. Parent Field type should be json array |
| Description | Provide a description of the Output Field to retrieve its value in the response. |
| API Request Body: | Shows the raw JSON payload sent to the LLM. Default placeholders are auto-populated from the LLM Settings and Input tabs. You may add parameters or modify values. Supported placeholders: • #{model_name} – value from Model field. • #{temperature} – value from Temperature field. • #{max_tokens} – value from Max Tokens field. • #{top_p} – value from Top P field. • #{system_prompt} – value from System Prompt. • #{user_and_assistant_prompt} – generated from User Prompt + Few-Shot rows. • #{output_fields} – generated from Structured Output Definition. • #{thinking_budget}} – Value from Thinking Budget field. |
| Reset Payload | Click to discard custom edits and restore the default template for your currently selected LLM Provider. |
| Output Fields Tab: | |
| Response Field | Response text from the LLM API. Default Response field name : OutputText |
| Include Metadata Fields In Output | Select this checkbox to capture metadata (such as coordinates and confidence level) for each extracted value. The system creates an additional field named <field_name>_metadata to hold this information. For example, { "page_number": 4, "confidence": "High", "coordinates": "539,346,602,346,602,356,539,356", "value": "$51,903.10", "page_width": 841.68, "page_height": 595.2 } |