Meta

When using Meta models via AWS Bedrock, llm-exe will make POST requests to https://bedrock-runtime.us-west-2.amazonaws.com/model/{MODEL_ID}/invoke.

Setup

Meta LLama

const llm = useLlm("amazon:meta.chat.v1", {
    model: "llama3-8b-instruct-v1:0", // This is the model id from Bedrock
});

LLama-Specific Options

In addition to the generic options, the following options are available for Meta LLama models on Bedrock.

Option	Type	Default	Description
model	string	—	The Bedrock model id. Must be specified. See AWS Bedrock Docs
temperature	number	undefined	Maps to temperature.
maxTokens	number	2048	Maps to max_gen_len. See AWS Bedrock Docs
topP	number	undefined	Maps to top_p. See AWS Bedrock Docs
awsRegion	string	undefined	AWS Region. Can be set via AWS_REGION environment variable
awsSecretKey	string	undefined	AWS Secret Key. Can be set via AWS_SECRET_ACCESS_KEY environment variable
awsAccessKey	string	undefined	AWS Access Key. Can be set via AWS_ACCESS_KEY_ID environment variable