LLM Configuration: Advanced Properties
The following table lists additional configuration options for your LLM. Use these properties to fine-tune the model's behaviour and control how it processes your requests:
| Properties | Description |
|---|---|
| baseUrl | Specify the endpoint address the step used to send requests to the LLM provider. The property controls the service endpoint that handles API calls, such as a regional or custom endpoint. Notes: - This property can also override the provider's default API base URL. - For AWS Bedrock and Azure OpenAI providers, it sets the AE Gateway base URL when you set isUseAEGateway to true. |
| apiEndpoint | Specify the API endpoint used to connect to the LLM provider. Use this property to define the regional service URL (for example, us-central1-aiplatform.googleapis.com) so that requests are routed to the correct location and meet data residency requirements. Note: This property can also override the default LLM API base URL or configure the AE Gateway base URL when isUseAEGateway is set to true for Vertex AI |
AE Tool Property
| Properties | Description |
|---|---|
| aeToolExecutionTimeout | Specify the maximum time (in seconds) the system waits for a workflow (AE tool) to complete during asynchronous execution. The system checks the workflow status every 5 seconds until completion or timeout. Minimum allowed value is 30 seconds and maximum is 600 seconds (10 minutes) If not provided the step use default that is 600 seconds as timeout value. |
AE Gateway Properties
Use the following properties to connect to the AE Gateway. When enabled, configure AE Gateway details instead of LLM provider details.
Note
- These properties are supported only for the following LLM providers: Google Vertex AI, AWS Bedrock, and Azure OpenAI.
- Configure the AE Gateway base URL using the baseUrl or apiEndpoint property, depending on the LLM provider.
| Properties | Description |
|---|---|
| isUseAEGateway | Allow to enable and disable AE gateway use in LLM configuration. Allowed Values: - true - false |
| aeGatewayToken | Specify the authentication token that your application uses to connect to the AE gateway. Use this property to pass the required token for secure access. It ensures that only authorized requests can access the gateway services. Get this token by calling the AE gateway authentication API provided. |
Proxy Details Property
| Properties | Description |
|---|---|
| llmProxyHost | Specifies the host name or IP address of the proxy server used to route LLM requests. |
| llmProxyPort | Specifies the port number of the proxy server used for LLM communication. |
| llmProxyUsername | Specifies the username required to authenticate with the proxy server, if proxy authentication is enabled. |
| llmProxyPassword | Specifies the password used to authenticate with the proxy server. Store this value securely. |
Time-out related settings
| Properties | Description |
|---|---|
| timeout | Specify the maximum time, in seconds, the step waits for a response from the LLM provider before it fails the request. This timer covers the full response, set a higher value for long or complex generations. Default value: 120 |
| connectTimeout | Specify the maximum time, in seconds, the step waits to establish a connection with the LLM provider. If the connection cannot be established within this time, the request fails. Default value: 10 |
Transport type for vertex ai
| Properties | Description |
|---|---|
| vertexAITransportType | Specifies the transport mechanism used to communicate with Vertex AI, such as REST or gRPC. Note: Property is available only for LLM provider type Google Vertex AI. |
Anthropic Properties
| Properties | Description |
|---|---|
| stopSequences | Specify custom text strings in JSON format to control exactly where the model stops generating output. During text generation, if the model encounters any of the provided strings, it immediately stops and returns the response up to that point. Example: ["END"] or ["\n\nUser:"] |
| completionsPath | Specify the API endpoint path the step uses to route requests to Anthropic. This path appends to the base URL to send data to the correct service. Recommendation: Keep the default value. Default Value: /v1/messages |
| anthropicVersion | Specify the Anthropic API version the connector uses. The connector sends this value in the anthropic-version request header so Anthropic applies the correct API behavior. Note: Keep the default unless you need a specific version. Default Value: 2023-06-01 |