OpenAI Agent with Tool Call Parser¶

Unfortunately, the tool calls by OpenAI are not always valid json, especially from older versions of the API. Up to and including the OpenAI API version 1106, this issue is relatively frequent if the argument is a long string (e.g. a python script), see for example here.

With the default tool call parser, the OpenAI Agent will fail to parse these tool calls and tries to fix the tool call in the next step. This needs another llm call, which is slow and expensive.

This notebook demonstrates how to define a custom tool call parser that can handle certain kinds of malformed function calls. The following steps are copied from the OpenAI Agent notebook, with the addition of a custom tool call parser.

Initial Setup¶

Let's start by importing some simple building blocks.

The main thing we need is:

1. the OpenAI API (using our own llama_index LLM class)
2. a place to keep conversation history
3. a definition for tools that our agent can use.

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

In [ ]:

%pip install llama-index-agent-openai
%pip install llama-index-llms-openai

%pip install llama-index-agent-openai %pip install llama-index-llms-openai

In [ ]:

!pip install llama-index

!pip install llama-index

In [ ]:

import json
from llama_index.core.tools import FunctionTool

import nest_asyncio

nest_asyncio.apply()

import json from llama_index.core.tools import FunctionTool import nest_asyncio nest_asyncio.apply()

Let's define some very simple calculator tools for our agent.

In [ ]:

def multiply(a: int, b: int) -> int:
    """Multiple two integers and returns the result integer"""
    return a * b


multiply_tool = FunctionTool.from_defaults(fn=multiply)

def multiply(a: int, b: int) -> int: """Multiple two integers and returns the result integer""" return a * b multiply_tool = FunctionTool.from_defaults(fn=multiply)

In [ ]:

def add(a: int, b: int) -> int:
    """Add two integers and returns the result integer"""
    return a + b


add_tool = FunctionTool.from_defaults(fn=add)

def add(a: int, b: int) -> int: """Add two integers and returns the result integer""" return a + b add_tool = FunctionTool.from_defaults(fn=add)

Definition of the Tool Call Parser¶

Sometimes, OpenAI tool calls are not valid json

When defining your own Tool Call Parser, you need to define a function that takes a OpenAIToolCall and returns a dictionary. The dictionary will be passed as **kwargs to the tool function.

The Parser should throw a ValueError if the tool call can't be parsed. This will be returned to the agent and it will try to fix the call on the next step.

In [ ]:

from typing import Dict
from llama_index.llms.openai.utils import OpenAIToolCall
import re

# The same parser is available as
# from llama_index.agent.openai import advanced_tool_call_parser


def custom_tool_call_parser(tool_call: OpenAIToolCall) -> Dict:
    r"""Parse tool calls that are not standard json.
    Also parses tool calls of the following forms:
    variable = \"\"\"Some long text\"\"\"
    variable = "Some long text"'
    variable = '''Some long text'''
    variable = 'Some long text'
    """
    arguments_str = tool_call.function.arguments
    if len(arguments_str.strip()) == 0:
        # OpenAI returns an empty string for functions containing no args
        return {}
    try:
        tool_call = json.loads(arguments_str)
        if not isinstance(tool_call, dict):
            raise ValueError("Tool call must be a dictionary")
        return tool_call
    except json.JSONDecodeError as e:
        # pattern to match variable names and content within quotes
        pattern = r'([a-zA-Z_][a-zA-Z_0-9]*)\s*=\s*["\']+(.*?)["\']+'
        match = re.search(pattern, arguments_str)

        if match:
            variable_name = match.group(1)  # This is the variable name
            content = match.group(2)  # This is the content within the quotes
            return {variable_name: content}
        raise ValueError(f"Invalid tool call: {e!s}")

from typing import Dict from llama_index.llms.openai.utils import OpenAIToolCall import re # The same parser is available as # from llama_index.agent.openai import advanced_tool_call_parser def custom_tool_call_parser(tool_call: OpenAIToolCall) -> Dict: r"""Parse tool calls that are not standard json. Also parses tool calls of the following forms: variable = \"\"\"Some long text\"\"\" variable = "Some long text"' variable = '''Some long text''' variable = 'Some long text' """ arguments_str = tool_call.function.arguments if len(arguments_str.strip()) == 0: # OpenAI returns an empty string for functions containing no args return {} try: tool_call = json.loads(arguments_str) if not isinstance(tool_call, dict): raise ValueError("Tool call must be a dictionary") return tool_call except json.JSONDecodeError as e: # pattern to match variable names and content within quotes pattern = r'([a-zA-Z_][a-zA-Z_0-9]*)\s*=\s*["\']+(.*?)["\']+' match = re.search(pattern, arguments_str) if match: variable_name = match.group(1) # This is the variable name content = match.group(2) # This is the content within the quotes return {variable_name: content} raise ValueError(f"Invalid tool call: {e!s}")

Defining the OpenAI Agent with Tool Call Parser¶

In [ ]:

from llama_index.agent.openai import OpenAIAgent
from llama_index.llms.openai import OpenAI

from llama_index.agent.openai import OpenAIAgent from llama_index.llms.openai import OpenAI

In [ ]:

llm = OpenAI(model="gpt-3.5-turbo-0613")
agent = OpenAIAgent.from_tools(
    [multiply_tool, add_tool],
    llm=llm,
    verbose=True,
    tool_call_parser=custom_tool_call_parser,
)

llm = OpenAI(model="gpt-3.5-turbo-0613") agent = OpenAIAgent.from_tools( [multiply_tool, add_tool], llm=llm, verbose=True, tool_call_parser=custom_tool_call_parser, )

Chat¶

In [ ]:

response = agent.chat("What is (121 * 3) + 42?")
print(str(response))

response = agent.chat("What is (121 * 3) + 42?") print(str(response))

Added user message to memory: What is (121 * 3) + 42?
=== Calling Function ===
Calling function: multiply with args: {
  "a": 121,
  "b": 3
}
Got output: 363
========================

=== Calling Function ===
Calling function: add with args: {
  "a": 363,
  "b": 42
}
Got output: 405
========================

(121 * 3) + 42 is equal to 405.

In [ ]:

# inspect sources
print(response.sources)

# inspect sources print(response.sources)

[ToolOutput(content='363', tool_name='multiply', raw_input={'args': (), 'kwargs': {'a': 121, 'b': 3}}, raw_output=363), ToolOutput(content='405', tool_name='add', raw_input={'args': (), 'kwargs': {'a': 363, 'b': 42}}, raw_output=405)]