Language Model Parameters

Constructs an object containing parameters for generating inferences from a model. The included parameters seem to be those held in common across most LLM implementations. Leaving a parameter at the default (NULL) inherits the default of the model (backend).

LanguageModelParams(temperature = NULL, top_p = NULL, top_k = NULL, max_tokens = NULL, presence_penalty = NULL, frequency_penalty = NULL, stop = NULL)

Arguments

temperature

Sampling temperature, with higher being more random. Set to 0 for stable outputs.

top_p

Limit token selection to those contributing to the top_p fraction of probability mass.

top_k

Limit token selection to the top_k most probable.

max_tokens

Maximum limit on the total number of generated tokens.

presence_penalty

Positive numbers penalize tokens already appearing in the context, while negative numbers do the opposite. Typical range is between -2 and 2.

frequency_penalty

Positive numbers penalize tokens that appear frequently in the context, while negative numbers do the opposite. Typical range is between -2 and 2.

stop

A character vector of strings that, when generated, cause the model to stop generating.

See also

Typically the above arguments are passed to chat and predict. Constructing an instance of LanguageModelParams is more for advanced use.

Value

A LanguageModelParams object

Author

Michael Lawrence