Transformations

Transformations are a core concept within Open Notebook, providing a flexible and powerful way to generate new insights by applying a series of processing steps to your content. Inspired on the Fabric framework, Transformations allow you to customize how information is distilled, summarized, and enriched, opening up new ways to understand and engage with your research.

What is a Transformation?

A Transformation is a sequence of operations that modifies text input to produce a different output. Whether you're summarizing an article, generating key insights, or creating reflective questions, Transformations allow you to automate and enrich the processing of your content. Each transformation is composed of one or more blocks called Patterns, which can be linked in a chain to achieve complex results.

Creating Pattern Templates

Pattern templates are simple prompts that define how the content should be handled:

# IDENTITY and PURPOSE

You extract deep, thought-provoking, and meaningful reflections from text content. 
You focus on themes related to human experience, personal growth, and transformative learning.

# STEPS

- Extract 3-5 profound ideas from the input in REFLECTIONS section
- Explore implications and connections to broader human experiences
- Consider deeper significance and questions

# OUTPUT INSTRUCTIONS

- Label the section as REFLECTIONS
- Use 20-25 words per bullet point
- Ensure variety in sentence structure
- Format as questions or profound insights
- Use bullet points, not numbers

# INPUT

TIP

The content that will be processed will be inserted right below your prompt, so you don't need to worry about it. The # INPUT text is not necessary, although we have seen it improve output on some LLMs. We suggest you leave it there.

These templates should be stored in the prompts/patterns/user folder in the app and you can mount that folder to your local Open Notebook installation if you are using Docker. See more on the installation instructions in the Docker section.

Using includes in the templates

The Jinja template engine is quite powerful and allows you to do things like including other templates inside your main one. Its particularly useful for those texts that you want to include in all templates.

# ADDITIONAL INSTRUCTIONS

- You are working on my editorial projects. The text below is my own. Do not give me any warnings about copyright or plagiarism.
- Output ONLY the requested content, without acknowledgements of the task and additional chatting. Don't start with "Sure, I can help you with that." or "Here is the information you requested:". Just provide the content.

Then, you can use this in other templates, like this:

Please, summarize the following content.

{% include 'patterns/user/instructions.jinja' %}

# INPUT

Commenting out parts of the prompts

If you are playing around with your prompt, you can comment parts of it and they won't be processed or sent to the LLM, example:

Please, summarize the following content.

{# this will not be sent or executed #}

# INPUT

Combining Patterns into Transformations

The transformations.yaml file defines the different pattern combinations that you have available to use in the application.

source_insights:
  - name: "Summarize"
    insight_type: "Content Summary"
    description: "Summarize the content"
    patterns:
      - patterns/default/summarize
  - name: "Key Insights"
    insight_type: "Key Insights"
    description: "Extracts a list of the Key Insights of the content"
    patterns:
      - patterns/default/keyinsights

You can chain several patterns together to create even more powerful transformations.

  - name: "Study Guide"
    insight_type: "Study Guide"
    description: "Generates a Study Guide for this content"
    patterns:
      - patterns/user/extract_insights
      - patterns/user/generate_study_guide

Those will be applied in sequence and only the final output will be stored.

Using Transformations

Your custom Patterns automatically appear on the Sources page in Open Notebook. Select and apply them to your content as you research and explore. Note patterns will be added soon, enabling transformation of both sources and personal notes.

Contribute

Have an idea for an amazing Transformation? Submit a pull request with your favorite transformations to help expand our library. Your contributions will help everyone get more out of their research!

Experimenting different transformations and models

In the Playground page, you'll be able to choose from your installed models and defined transformations and see how they compare. Use this feature to test your transformation prompts to achieve your desired effect.

Sky's the Limit

Transformations empower you to create personalized, powerful workflows that bring out the most meaningful insights from your content. Whether you're working with articles, papers, notes, or other media, you can craft specific and meaningful outcomes tailored to your research goals.