Completions

Foundations

Completions are the basic unit of response with an LLM. Given instructions, context, and optionally embedded resources via a prompt, a completion is the response that is generated by the large language model. Within re-factor, completions can be generated in two formats: text and object. The text format is a human-readable string that can be used for human consumption. The object format is a JSON object that can be used for programmatic access and can be constrained using JSONSchema.

Structure

Fields

string

required

The unique identifier for the runnable. This is created automatically by re-factor.

string

required

The type of the runnable. For a completion, this should be "completion".

string

required

The name of the runnable. This is a human-readable label that helps you identify and manage runnables in your system.

string

A description of the runnable’s purpose and intended use. This can provide additional context to users and help them understand the purpose and functionality of the runnable.

CompletionPrompt

required

The prompt template for the completion. This is a string that can contain placeholders for dynamic values and can be used to generate the final prompt text.See the CompletionPrompt type for this object’s structure.

array<Resource>

required

An array of resources that are expected to be embedded into the prompt at runtime. Each resource has a name, type, mime types, description, and optional metadata.Array items must be of type Resource. This field is optional if the prompt does not require embedded resources.

array<LLMConfiguration>

required

An array of LLM configurations that are exposed within the prompt. Each LLM configuration has a name, provider, model, and optional metadata.Array items must be of type LLMConfiguration. At least one LLM configuration with default: true must be defined. Exactly one LLM configuration can have default: true.

Features

Template Variables

Template variables make your completion runnables dynamic and reusable. You can define variables using the {{ variable_name }} syntax, which can be replaced with actual values when the completion is run. This is particularly useful for creating completion runnables that can be used across different contexts or with different inputs.

Resource Embedding

Resources can be seamlessly transformed and embedded into your prompts using our resources system. This allows you to reference external data, documents, or other content directly within your completion prompts. The LLM will have access to this information during its response generation. For example, assuming we have a Word document resource defined with the name document, we can embed it as a PDF into our prompt user message as follows:

Given the following document, please analyze its main points and
provide recommendations:

{{ document.transform({ type: 'pdf' }) }}

Under the hood, this will convert the Word document resource into a PDF and include it in the user message content in the manner required by the underlying LLM provider.

Versioning

As you modify your completion runnables in re-factor, they are automatically versioned and previous versions are available in a prompt history, allowing you to:

Track changes over time
Roll back to previous versions if needed
Compare performance across different versions

Multiple Generations

It’s often advantageous to use multiple LLM generations in pursuit of a correct answer. With re-factor, you can define a script consisting of multiple role: "assistant" messages that will be executed in sequence.

Example

name: document_analyzer
type: completion
description: Analyzes documents for key information

# The LLM configurations available to use when generating the completion(s)
llms:
- name: gpt-4o
  default: true
  provider: openai
  model: gpt-4o

# Declaration of the resources that are expected to be embedded into
# the completion prompt(s) at runtime
resources:
- name: document
  type: file
  mime_types:
    - application/pdf
    - application/vnd.openxmlformats-officedocument.wordprocessingml.document
  description: The document to analyze
  required: true

# The prompt which specifies the completions to generate and the output
# names to assign them
prompt:
  system: You are a document analysis expert.
  messages:
  - role: user
    content: |
      Please provide a summary of the following document, focusing on the
      following aspects:

      {{ document }} # embed a resource
  - role: assistant
    generate: true # mark this message for generation at runtime
    format: text
    set_output: summary_first_draft
  # Leverage a follow up completion to promote greater accuracy
  - role: user
    content: |
      Please review your summary for accuracy and completeness. If there are
      any issues or missing details, please provide an updated version.
  - role: assistant
    generate: true
    format: text
    llm:
      model: gpt-4o-mini # use a smaller model for follow up completion
    set_output: summary_final

import { Completion } from '@re-factor/sdk/runnable';
import { Resource } from '@re-factor/sdk/resource';

const document = await Resource.fromPath(
  './document.pdf',
  { mimeType: 'application/pdf' },
  { store: true }
);

const completion = await Completion.load({ path: './completion.yaml' });

const result = await completion.run({
    resources: [document.embedAs('document')]
});

from refactor.runnable import Completion
from refactor.resource import Resource

completion = Completion.load('./completion.yaml')

document = Resource.from_path(
    path='./document.pdf',
    settings=dict(
        mime_type='application/pdf'
    ),
    store=True
)

result = completion.run(
    resources=[document.embed_as('document')]
)

Best Practices

When creating prompts in re-factor:

Provide as much detail as possible in your system message
Use template variables for dynamic elements
Version your prompts appropriately
Test prompts across different scenarios
Document expected inputs and outputs
Consider resource limitations and costs

For more information on implementing integrations with these features in your application, refer to our API Reference and SDK documentation.

Get Started

Guides

Foundations

Structure

Fields

Features

Template Variables

Resource Embedding

Versioning

Multiple Generations

Example

Best Practices

​Foundations

​Structure

​Fields

​Features

​Template Variables

​Resource Embedding

​Versioning

​Multiple Generations

​Example

​Best Practices

Foundations

Structure

Fields

Features

Template Variables

Resource Embedding

Versioning

Multiple Generations

Example

Best Practices