Skip to main content

​
Foundations

Completions are the basic unit of response with an LLM. Given instructions, context, and optionally embedded resources via a prompt, a completion is the response that is generated by the large language model. Within re-factor, completions can be generated in two formats: text and object. The text format is a human-readable string that can be used for human consumption. The object format is a JSON object that can be used for programmatic access and can be constrained using JSONSchema.

​
Structure

​
Fields

​
id
string
required
The unique identifier for the runnable. This is created automatically by re-factor.
​
type
string
required
The type of the runnable. For a completion, this should be "completion".
​
name
string
required
The name of the runnable. This is a human-readable label that helps you identify and manage runnables in your system.
​
description
string
A description of the runnable’s purpose and intended use. This can provide additional context to users and help them understand the purpose and functionality of the runnable.
​
prompt
CompletionPrompt
required
The prompt template for the completion. This is a string that can contain placeholders for dynamic values and can be used to generate the final prompt text.See the CompletionPrompt type for this object’s structure.
​
resources
array<Resource>
required
An array of resources that are expected to be embedded into the prompt at runtime. Each resource has a name, type, mime types, description, and optional metadata.Array items must be of type Resource. This field is optional if the prompt does not require embedded resources.
​
llms
array<LLMConfiguration>
required
An array of LLM configurations that are exposed within the prompt. Each LLM configuration has a name, provider, model, and optional metadata.Array items must be of type LLMConfiguration. At least one LLM configuration with default: true must be defined. Exactly one LLM configuration can have default: true.

​
Features

​
Template Variables

Template variables make your completion runnables dynamic and reusable. You can define variables using the {{ variable_name }} syntax, which can be replaced with actual values when the completion is run. This is particularly useful for creating completion runnables that can be used across different contexts or with different inputs.

​
Resource Embedding

Resources can be seamlessly transformed and embedded into your prompts using our resources system. This allows you to reference external data, documents, or other content directly within your completion prompts. The LLM will have access to this information during its response generation. For example, assuming we have a Word document resource defined with the name document, we can embed it as a PDF into our prompt user message as follows: 
Given the following document, please analyze its main points and
provide recommendations:

{{ document.transform({ type: 'pdf' }) }}
Under the hood, this will convert the Word document resource into a PDF and include it in the user message content in the manner required by the underlying LLM provider.

​
Versioning

As you modify your completion runnables in re-factor, they are automatically versioned and previous versions are available in a prompt history, allowing you to:
  • Track changes over time
  • Roll back to previous versions if needed
  • Compare performance across different versions

​
Multiple Generations

It’s often advantageous to use multiple LLM generations in pursuit of a correct answer. With re-factor, you can define a script consisting of multiple role: "assistant" messages that will be executed in sequence.

​
Example


name: document_analyzer
type: completion
description: Analyzes documents for key information

# The LLM configurations available to use when generating the completion(s)
llms:
- name: gpt-4o
  default: true
  provider: openai
  model: gpt-4o

# Declaration of the resources that are expected to be embedded into
# the completion prompt(s) at runtime
resources:
- name: document
  type: file
  mime_types:
    - application/pdf
    - application/vnd.openxmlformats-officedocument.wordprocessingml.document
  description: The document to analyze
  required: true

# The prompt which specifies the completions to generate and the output
# names to assign them
prompt:
  system: You are a document analysis expert.
  messages:
  - role: user
    content: |
      Please provide a summary of the following document, focusing on the
      following aspects:

      {{ document }} # embed a resource
  - role: assistant
    generate: true # mark this message for generation at runtime
    format: text
    set_output: summary_first_draft
  # Leverage a follow up completion to promote greater accuracy
  - role: user
    content: |
      Please review your summary for accuracy and completeness. If there are
      any issues or missing details, please provide an updated version.
  - role: assistant
    generate: true
    format: text
    llm:
      model: gpt-4o-mini # use a smaller model for follow up completion
    set_output: summary_final

​
Best Practices

When creating prompts in re-factor:
  1. Provide as much detail as possible in your system message
  2. Use template variables for dynamic elements
  3. Version your prompts appropriately
  4. Test prompts across different scenarios
  5. Document expected inputs and outputs
  6. Consider resource limitations and costs
For more information on implementing integrations with these features in your application, refer to our API Reference and SDK documentation.
⌘I