Streaming for Chat Engine - Condense Question Mode#
Load documents, build the VectorStoreIndex
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
from llama_index import VectorStoreIndex, SimpleDirectoryReader
INFO:numexpr.utils:Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
Note: NumExpr detected 12 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
INFO:numexpr.utils:NumExpr defaulting to 8 threads.
NumExpr defaulting to 8 threads.
/Users/suo/miniconda3/envs/llama/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.7) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.
warnings.warn(
Download Data
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
index = VectorStoreIndex.from_documents(documents)
Chat with your data
chat_engine = index.as_chat_engine(
chat_mode="condense_question", streaming=True
)
response_stream = chat_engine.chat("What did Paul Graham do after YC?")
INFO:llama_index.chat_engine.condense_question:Querying with: What was the next step in Paul Graham's career after his involvement with Y Combinator?
Querying with: What was the next step in Paul Graham's career after his involvement with Y Combinator?
response_stream.print_response_stream()
Paul Graham's next step in his career after his involvement with Y Combinator was to take up painting. He spent most of the rest of 2014 painting and then in March 2015 he started working on Lisp again.
Ask a follow up question
response_stream = chat_engine.chat("What about after that?")
INFO:llama_index.chat_engine.condense_question:Querying with: What did Paul Graham do after he started working on Lisp again in March 2015?
Querying with: What did Paul Graham do after he started working on Lisp again in March 2015?
response_stream.print_response_stream()
Paul Graham spent the rest of 2015 writing essays and working on the new dialect of Lisp he called Arc. He also looked for an apartment to buy and started to plan a second still life painting from the same objects.
response_stream = chat_engine.chat("Can you tell me more?")
INFO:llama_index.chat_engine.condense_question:Querying with: What did Paul Graham do after he started working on the new dialect of Lisp he called Arc in 2015?
Querying with: What did Paul Graham do after he started working on the new dialect of Lisp he called Arc in 2015?
response_stream.print_response_stream()
Paul Graham worked on the new dialect of Lisp he called Arc for four years, from March 26, 2015 to October 12, 2019. During this time, he wrote the new Lisp, called Bel, in Arc. He also wrote essays and took his children to the coast on a sunny day in 2015. In the summer of 2016, he and his family moved to England. Finally, in the fall of 2019, he finished the project.
Reset conversation state
chat_engine.reset()
response_stream = chat_engine.chat("What about after that?")
INFO:llama_index.chat_engine.condense_question:Querying with: What happens after the current situation?
Querying with: What happens after the current situation?
response_stream.print_response_stream()
After the current situation, the narrator resumes painting and experimenting with a new kind of still life. He also resumes his old life in New York, now that he is rich. He is able to take taxis and eat in restaurants, which is exciting for a while. He also starts to connect with other people who are trying to paint in New York.