Streaming output and events#
In real-world use, agents can take a long time to run. Providing feedback to the user about the progress of the agent is critical, and streaming allows you to do that.
AgentWorkflow
provides a set of pre-built events that you can use to stream output to the user. Let's take a look at how that's done.
First, we're going to introduce a new tool that takes some time to execute. In this case we'll use a web search tool called Tavily, which is available in LlamaHub.
pip install llama-index-tools-tavily-research
It requires an API key, which we're going to set in our .env
file as TAVILY_API_KEY
and retrieve using the os.getenv
method. Let's bring in our imports:
from llama_index.tools.tavily_research import TavilyToolSpec
import os
And initialize the tool:
tavily_tool = TavilyToolSpec(api_key=os.getenv("TAVILY_API_KEY"))
Now we'll create an agent using that tool and an LLM that we initialized just like we did previously.
workflow = FunctionAgent(
tools=tavily_tool.to_tool_list(),
llm=llm,
system_prompt="You're a helpful assistant that can search the web for information.",
)
In previous examples, we've used await
on the workflow.run
method to get the final response from the agent. However, if we don't await the response, we get an asynchronous iterator back that we can iterate over to get the events as they come in. This iterator will return all sorts of events. We'll start with an AgentStream
event, which contains the "delta" (the most recent change) to the output as it comes in. We'll need to import that event type:
from llama_index.core.agent.workflow import AgentStream
And now we can run the workflow and look for events of that type to output:
handler = workflow.run(user_msg="What's the weather like in San Francisco?")
async for event in handler.stream_events():
if isinstance(event, AgentStream):
print(event.delta, end="", flush=True)
If you run this yourself, you will see the output arriving in chunks as the agent runs, returning something like this:
The current weather in San Francisco is as follows:
- **Temperature**: 17.2°C (63°F)
- **Condition**: Sunny
- **Wind**: 6.3 mph (10.1 kph) from the NNW
- **Humidity**: 54%
- **Pressure**: 1021 mb (30.16 in)
- **Visibility**: 16 km (9 miles)
For more details, you can check the full report [here](https://www.weatherapi.com/).
AgentStream
is just one of many events that AgentWorkflow
emits as it runs. The others are:
AgentInput
: the full message object that begins the agent's executionAgentOutput
: the response from the agentToolCall
: which tools were called and with what argumentsToolCallResult
: the result of a tool call
You can see us filtering for more of these events in the full code of this example.
Next you'll learn about how to get a human in the loop to provide feedback to your agents.