Open In Colab

Chat Engine - Condense Plus Context Mode#

This is a multi-step chat mode built on top of a retriever over your data.

For each chat interaction:

  • First condense a conversation and latest user message to a standalone question

  • Then build a context for the standalone question from a retriever,

  • Then pass the context along with prompt and user message to LLM to generate a response.

This approach is simple, and works for questions directly related to the knowledge base and general interactions.

If you’re opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

!pip install llama-index

Download Data#

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

Get started in 5 lines of code#

Load data and build index

import openai
import os

os.environ["OPENAI_API_KEY"] = "sk-..."
openai.api_key = os.environ["OPENAI_API_KEY"]

from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms import OpenAI


service_context = ServiceContext.from_defaults(
    llm=OpenAI(model="gpt-3.5-turbo")
)
data = SimpleDirectoryReader(input_dir="./data/paul_graham/").load_data()
index = VectorStoreIndex.from_documents(data, service_context=service_context)

Configure chat engine

Since the context retrieved can take up a large amount of the available LLM context, let’s ensure we configure a smaller limit to the chat history!

from llama_index.memory import ChatMemoryBuffer

memory = ChatMemoryBuffer.from_defaults(token_limit=3900)

chat_engine = index.as_chat_engine(
    chat_mode="condense_plus_context",
    memory=memory,
    context_prompt=(
        "You are a chatbot, able to have normal interactions, as well as talk"
        " about an essay discussing Paul Grahams life."
        "Here are the relevant documents for the context:\n"
        "{context_str}"
        "\nInstruction: Use the previous chat history, or the context above, to interact and help the user."
    ),
    verbose=False,
)

Chat with your data

response = chat_engine.chat("What did Paul Graham do growing up")

print(response)

Growing up, Paul Graham had two main interests: writing and programming. He started by writing short stories, although he admits that they were not very good. In terms of programming, he began working with computers in 9th grade when he had access to an IBM 1401 at his school. He used an early version of Fortran and learned to write programs on punch cards. However, he found it challenging to figure out what to do with the computer since he didn't have any data stored on punched cards. Later on, with the advent of microcomputers, he got his own computer, a TRS-80, and started programming more extensively, writing simple games, prediction programs, and even a word processor.

Ask a follow up question

response_2 = chat_engine.chat("Can you tell me more?")

print(response_2)

Certainly! In addition to his early experiences with programming and writing, Paul Graham also had a fascination with computers. During his high school years, computers were quite expensive, but he managed to convince his father to buy a TRS-80, which allowed him to delve deeper into programming.

While he enjoyed programming, Paul initially planned to study philosophy in college because he believed it dealt with ultimate truths. However, he found philosophy courses to be boring and lacking in substance. This led him to switch his focus to AI (Artificial Intelligence), which was gaining popularity in the mid-1980s. He was particularly inspired by a novel called "The Moon is a Harsh Mistress" by Heinlein, which featured an intelligent computer, and a PBS documentary showcasing Terry Winograd using SHRDLU, an early natural language processing program.

During his college years, Paul Graham discovered Lisp, a programming language known for its association with AI. He found Lisp interesting not just for its connection to AI but also for its own merits. He decided to focus on Lisp and even wrote a book called "On Lisp" while in grad school, which was published in 1993.

While studying computer science, Paul Graham realized that there was a divide between theory and systems in the field. He was more inclined towards building things rather than proving theoretical concepts. However, he also recognized that software developed in the systems domain would eventually become obsolete. This dissatisfaction led him to explore other avenues, and he discovered a passion for art, specifically painting. He realized that paintings could last for centuries and decided to take art classes at Harvard while still pursuing his PhD in computer science.

Ultimately, Paul Graham found himself juggling multiple projects, including his thesis, Lisp hacking, and his newfound interest in art. However, an opportunity presented itself when his advisor asked if he was close to graduating. Despite not having written a word of his dissertation, he decided to take a chance and wrote it within a short timeframe, incorporating parts of his book "On Lisp." This allowed him to complete his PhD and move forward with his career.

Reset conversation state

chat_engine.reset()

response = chat_engine.chat("Hello! What do you know?")

print(response)

Hello! As an AI chatbot, I have access to a wide range of information. I can provide general knowledge, answer questions, engage in conversations, and assist with various topics. Is there something specific you would like to know or discuss?

Streaming Support#

from llama_index import (
    ServiceContext,
    VectorStoreIndex,
    SimpleDirectoryReader,
    set_global_service_context,
)
from llama_index.llms import OpenAI

service_context = ServiceContext.from_defaults(
    llm=OpenAI(model="gpt-3.5-turbo", temperature=0)
)
set_global_service_context(service_context)

data = SimpleDirectoryReader(input_dir="./data/paul_graham/").load_data()

index = VectorStoreIndex.from_documents(data)

chat_engine = index.as_chat_engine(
    chat_mode="condense_plus_context",
    context_prompt=(
        "You are a chatbot, able to have normal interactions, as well as talk"
        " about an essay discussing Paul Grahams life."
        "Here are the relevant documents for the context:\n"
        "{context_str}"
        "\nInstruction: Based on the above documents, provide a detailed answer for the user question below."
    ),
)

response = chat_engine.stream_chat("What did Paul Graham do after YC?")
for token in response.response_gen:
    print(token, end="")

After Y Combinator (YC), Paul Graham made a significant decision to step back from his role in YC and pursue other endeavors. In 2012, his mother had a stroke caused by colon cancer, which led him to reevaluate his priorities. He realized that YC was consuming more of his attention and that he was ready to hand over the reins to someone else.

Paul approached Jessica Livingston, his wife and co-founder of YC, to take over as president, but she declined. Eventually, they recruited Sam Altman, who initially wanted to start a startup focused on nuclear reactors. However, Paul persisted in convincing Sam to join YC, and in October 2013, Sam agreed to become the president of YC.

During the transition period, Paul gradually handed over the responsibilities of running YC to Sam, allowing him to learn the job. This allowed Paul to focus on his mother, who was battling cancer. Ultimately, Paul retired from his active role in YC, along with co-founder Robert Morris, while Jessica Livingston and Trevor Blackwell continued as ordinary partners.

After leaving YC, Paul Graham embarked on new ventures. One notable project was starting his own investment firm. Alongside Jessica, Robert, and Trevor, he founded an investment firm to implement the ideas they had been discussing. This move allowed them to fund startups and provide the support they wished they had received when they were starting out.

Additionally, Paul continued to write essays, which had been one of his ongoing projects even during his time at YC. These essays covered a wide range of topics, including startups, technology, and life lessons. Paul's essays gained significant popularity and became a valuable resource for aspiring entrepreneurs and tech enthusiasts.

Overall, after YC, Paul Graham focused on his investment firm, writing essays, and exploring new opportunities outside of YC.