Skip to content
TheChatArchiveThe Chat Archive
o1-preview-2024-09-12· OpenAIAi2 WildChat-4.8M (AI2)

this is the readme from the outlines: <div align="center" style="margin-bottom: 1em;">…

  1. User

    this is the readme from the outlines:

    [image: .txt Twitter] open [image: Outlines Twitter] open

    [image: Documentation] open [image: Contributors] open [image: Downloads] open [image: Discord] open

    Robust (structured) text generation.

    Made with ❤👷️ by the team at .txt.

    pip install outlines
    

    First time here? Go to our setup guide

    Features

    Outlines has new releases and features coming every week. Make sure to ⭐ star and 👀 watch this repository, follow @dottxtai to stay up to date!

    Why should I use structured generation?

    .txt company

    We started a company to keep pushing the boundaries of structured generation. Learn more about .txt, and give our .json API a try if you need a hosted solution ✨

    Structured generation

    The first step towards reliability of systems that include large language models is to ensure that there is a well-defined interface between their output and user-defined code. Outlines provides ways to control the generation of language models to make their output more predictable.

    Multiple choices

    You can reduce the completion to a choice between multiple possibilities:

    import outlines
    
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    
    prompt = """You are a sentiment-labelling assistant.
    Is the following review positive or negative?
    
    Review: This restaurant is just awesome!
    """
    
    generator = outlines.generate.choice(model, ["Positive", "Negative"])
    answer = generator(prompt)
    

    Type constraint

    You can instruct the model to only return integers or floats:

    import outlines
    
    model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
    
    prompt = "<s>result of 9 + 9 = 18</s><s>result of 1 + 2 = "
    answer = outlines.generate.format(model, int)(prompt)
    print(answer)
    # 3
    
    prompt = "sqrt(2)="
    generator = outlines.generate.format(model, float)
    answer = generator(prompt, max_tokens=10)
    print(answer)
    # 1.41421356
    

    Efficient regex-structured generation

    Outlines also comes with fast regex-structured generation. In fact, the choice and format functions above all use regex-structured generation under the hood:

    import outlines
    
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    
    prompt = "What is the IP address of the Google DNS servers? "
    
    generator = outlines.generate.text(model)
    unstructured = generator(prompt, max_tokens=30)
    
    generator = outlines.generate.regex(
        model,
        r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)",
    )
    structured = generator(prompt, max_tokens=30)
    
    print(unstructured)
    # What is the IP address of the Google DNS servers?
    #
    # Passive DNS servers are at DNS servers that are private.
    # In other words, both IP servers are private. The database
    # does not contain Chelsea Manning
    
    print(structured)
    # What is the IP address of the Google DNS servers?
    # 2.2.6.1
    

    Unlike other libraries, regex-structured generation in Outlines is almost as fast as non-structured generation.

    Efficient JSON generation following a Pydantic model

    Outlines allows to guide the generation process so the output is guaranteed to follow a JSON schema or Pydantic model:

    from enum import Enum
    from pydantic import BaseModel, constr
    
    import outlines
    import torch
    
    
    class Weapon(str, Enum):
        sword = "sword"
        axe = "axe"
        mace = "mace"
        spear = "spear"
        bow = "bow"
        crossbow = "crossbow"
    
    
    class Armor(str, Enum):
        leather = "leather"
        chainmail = "chainmail"
        plate = "plate"
    
    
    class Character(BaseModel):
        name: constr(max_length=10)
        age: int
        armor: Armor
        weapon: Weapon
        strength: int
    
    
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    
    # Construct structured sequence generator
    generator = outlines.generate.json(model, Character)
    
    # Draw a sample
    seed = 789001
    
    character = generator("Give me a character description", seed=seed)
    
    print(repr(character))
    # Character(name='Anderson', age=28, armor=<Armor.chainmail: 'chainmail'>, weapon=<Weapon.sword: 'sword'>, strength=8)
    
    character = generator("Give me an interesting character description", rng=rng)
    
    print(repr(character))
    # Character(name='Vivian Thr', age=44, armor=<Armor.plate: 'plate'>, weapon=<Weapon.crossbow: 'crossbow'>, strength=125)
    

    The method works with union types, optional types, arrays, nested schemas, etc. Some field constraints are not supported yet, but everything else should work.

    Efficient JSON generation following a JSON Schema

    Sometimes you just want to be able to pass a JSON Schema instead of a Pydantic model. We've got you covered:

    import outlines
    
    schema = '''{
        "title": "Character",
        "type": "object",
        "properties": {
            "name": {
                "title": "Name",
                "maxLength": 10,
                "type": "string"
            },
            "age": {
                "title": "Age",
                "type": "integer"
            },
            "armor": {"$ref": "#/definitions/Armor"},
            "weapon": {"$ref": "#/definitions/Weapon"},
            "strength": {
                "title": "Strength",
                "type": "integer"
            }
        },
        "required": ["name", "age", "armor", "weapon", "strength"],
        "definitions": {
            "Armor": {
                "title": "Armor",
                "description": "An enumeration.",
                "enum": ["leather", "chainmail", "plate"],
                "type": "string"
            },
            "Weapon": {
                "title": "Weapon",
                "description": "An enumeration.",
                "enum": ["sword", "axe", "mace", "spear", "bow", "crossbow"],
                "type": "string"
            }
        }
    }'''
    
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    generator = outlines.generate.json(model, schema)
    character = generator("Give me a character description")
    

    Using context-free grammars to guide generation

    Formal grammars rule the world, and Outlines makes them rule LLMs too. You can pass any context-free grammar in the EBNF format and Outlines will generate an output that is valid to this grammar:

    import outlines
    
    arithmetic_grammar = """
        ?start: expression
    
        ?expression: term (("+" | "-") term)*
    
        ?term: factor (("*" | "/") factor)*
    
        ?factor: NUMBER
               | "-" factor
               | "(" expression ")"
    
        %import common.NUMBER
    """
    
    model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
    generator = outlines.generate.cfg(model, arithmetic_grammar)
    sequence = generator("Alice had 4 apples and Bob ate 2. Write an expression for Alice's apples:")
    
    print(sequence)
    # (8-2)
    

    This was a very simple grammar, and you can use outlines.generate.cfg to generate syntactically valid Python, SQL, and much more than this. Any kind of structured text, really. All you have to do is search for "X EBNF grammar" on the web, and take a look at the Outlines grammars module.

    Open functions

    Outlines can infer the structure of the output from the signature of a function. The result is a dictionary, and can be passed directly to the function using the usual dictionary expansion syntax **:

    import outlines
    
    
    def add(a: int, b: int):
        return a + b
    
    model = outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")
    generator = outlines.generate.json(model, add)
    result = generator("Return json with two integers named a and b respectively. a is odd and b even.")
    
    print(add(**result))
    # 3
    

    A great advantage of passing functions directly to specify the structure is that the structure of the LLM will change with the function's definition. No need to change the code at several places!

    Prompting

    Building prompts can get messy. Outlines makes it easier to write and manage prompts by encapsulating templates inside "template functions".

    These functions make it possible to neatly separate the prompt logic from the general program logic; they can be imported from other modules and libraries.

    Template functions require no superfluous abstraction, they use the Jinja2 templating engine to help build complex prompts in a concise manner:

    import outlines
    
    examples = [
        ("The food was disgusting", "Negative"),
        ("We had a fantastic night", "Positive"),
        ("Recommended", "Positive"),
        ("The waiter was rude", "Negative")
    ]
    
    @outlines.prompt
    def labelling(to_label, examples):
        """You are a sentiment-labelling assistant.
    
        {% for example in examples %}
        {{ example[0] }} // {{ example[1] }}
        {% endfor %}
        {{ to_label }} //
        """
    
    model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
    prompt = labelling("Just awesome", examples)
    answer = outlines.generate.text(model)(prompt, max_tokens=100)
    

    Join us

    Cite Outlines

    @article{willard2023efficient,
      title={Efficient Guided Generation for LLMs},
      author={Willard, Brandon T and Louf, R{\'e}mi},
      journal={arXiv preprint arXiv:2307.09702},
      year={2023}
    }
    

    what do you think about this?

  2. Assistant

    Thank you for sharing the README of the Outlines project. It appears to be a comprehensive and well-organized introduction to a library focused on robust, structured text generation using large language models. Here are some thoughts on it:

    1. Clear and Engaging Presentation: The README begins with an eye-catching logo and badges that immediately convey important information such as documentation links, contributor stats, and social media handles. This is effective in capturing the reader's interest.

    2. Installation and Getting Started: Providing a simple installation command (pip install outlines) is convenient. The prompt to visit the setup guide for first-time users is helpful for newcomers to get started smoothly.

    3. Feature Overview: Listing the features with emojis adds a friendly touch. The checklist format makes it easy to see what the library offers:

      • Support for multiple model integrations (OpenAI, Transformers, etc.).
      • Powerful prompting primitives based on Jinja templates.
      • Features like multiple choices, type constraints, dynamic stopping, and efficient regex-structured generation.
      • Capabilities for JSON generation following JSON schemas or Pydantic models.
      • Batch inference, caching, and various sampling strategies.
      • Emphasis on performance improvements and efficiency.
    4. Importance of Structured Generation: The README does a good job explaining why structured generation is beneficial, highlighting aspects such as:

      • No overhead during inference.
      • Performance enhancements over models like GPT-4 when using open-source alternatives.
      • Speed improvements and increased inference efficiency.
      • Better performance with fewer examples needed.
      • Links to blog posts and articles provide evidence and deeper insights into these claims.
    5. Company Information: Introducing the .txt company provides context about the team behind the project. It invites users to learn more and consider their hosted solutions.

    6. Detailed Examples: The examples under "Structured generation" are practical and demonstrate how to use the library effectively:

      • Multiple choices and type constraints showcase how to guide model outputs.
      • Efficient regex-structured generation and JSON generation examples illustrate advanced use cases.
      • The section on using context-free grammars is particularly interesting for generating syntactically valid code or expressions.
    7. Community Engagement: Encouraging contributions, joining Discord, and following on social media fosters a sense of community. The pointers on how to contribute and where to report issues are clear.

    8. Citation Information: Providing a citation for academic references is thoughtful for researchers who might use or reference the library in their work.

    Overall Impression:

    • The README is well-structured and informative, balancing technical details with accessibility.
    • The use of code snippets and examples helps in understanding how to implement the features.
    • Emphasizing the advantages of structured generation positions the library as a valuable tool for developers working with LLMs.

    Suggestions for Enhancement:

    • Expand the Installation Section: Including information about prerequisites, supported Python versions, or potential issues during installation could be helpful.
    • More Examples: While the examples provided are useful, linking to notebooks or a dedicated examples directory could offer more hands-on practice.
    • Highlight Use Cases: Providing specific use cases or success stories might inspire users and showcase the library's capabilities in real-world scenarios.
    • Add a Table of Contents: For longer READMEs, a table of contents can improve navigation.

    Conclusion:

    The README effectively communicates what Outlines offers and how it can benefit users interested in structured text generation with large language models. It invites users to explore the library, contribute to its development, and engage with the community. With a few minor enhancements, it could be even more user-friendly, but overall, it serves as a solid introduction to the project.