Generation

The following objects can be passed to a step

ChatGeneration

@dataclass
class ChatGeneration(BaseGeneration):
    messages: List[GenerationMessage] = Field(default_factory=list)
    type = GenerationType.CHAT

messages · List[Dict]

List of messages sent to the LLM. Following the openai chat format.

provider · str

The provider of the LLM, like openai.

model · str

The model used for the generation, like gpt-4.

error · str

The error message if the generation failed.

settings · Dict

The settings of the LLM, like temperature…

variables · Dict

The variables used to format the prompt.

tags · List[str]

Optional tags to add to the generation.

prompt · str

The prompt used for the generation.

tools · Dict

The tools used to generate the completion, following the openai format.

message_completion · Dict

The message returned by the LLM, following the openai chat format.

token_count · int

The token count of the completion.

input_token_count · int

The token count of the input.

output_token_count · int

The token count of the output.

tt_first_token · float

The time it took to generate the first token. Only available when streaming.

token_throughput_in_s · float

The token throughput in tokens per second. Only available when streaming.

duration · float

The duration of the generation in ms.

CompletionGeneration

provider · str

The provider of the LLM, like openai.

model · str

The model used for the generation, like gpt-4.

error · str

The error message if the generation failed.

settings · Dict

The settings of the LLM, like temperature…

variables · Dict

The variables used to format the prompt.

tags · List[str]

Optional tags to add to the generation.

prompt · str

The prompt used for the generation.

completion · str

The completion returned by the LLM.

token_count · int

The token count of the completion.

input_token_count · int

The token count of the input.

output_token_count · int

The token count of the output.

tt_first_token · float

The time it took to generate the first token. Only available when streaming.

token_throughput_in_s · float

The token throughput in tokens per second. Only available when streaming.

duration · float

The duration of the generation in ms.