Generic Options
llm-exe attempts to normalize the inputs for various llm vendors, providing a single interface that can be used to interact with different models. While this is not always possible, since certain vendors may implement seatures that others don't support, either way only the allowed options make it to the respective api calls.
Options
| Option | Type | Default | Description |
|---|---|---|---|
| timeout | number | 30000 | Max execution time of API call to the LLM, in milliseconds. |
| maxDelay | number | 5000 | Used for retry back-off. Max time to wait between attempts when timeout has been reached, in milliseconds. |
| numOfAttempts | number | 0 | Used for retry. How many attempts should be made before throwing error |
| jitter | "none" | "full" | none | Used for retry back-off. |
| temperature | number | 0 | Used by model. |
| maxTokens | number | 500 | Used by model. |
| stream | boolean | null | null | Note: Not supported |
NOTE
Different vendors will allow (and may require) additional options.