2024 06 19 georgian genai bootcamp

Observability: Arize AI¶

Follow the quickstart guide found here.

In [ ]:

%pip install --upgrade \
    openinference-instrumentation-llama-index \
    opentelemetry-sdk \
    opentelemetry-exporter-otlp \
    "opentelemetry-proto>=1.12.0" \
    arize-phoenix -q

%pip install --upgrade \ openinference-instrumentation-llama-index \ opentelemetry-sdk \ opentelemetry-exporter-otlp \ "opentelemetry-proto>=1.12.0" \ arize-phoenix -q

[notice] A new release of pip available: 22.3.1 -> 24.0
[notice] To update, run: python3.11 -m pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

In [ ]:

import os

get_ipython().system = os.system

!python -m phoenix.server.main serve > arize.log 2>&1 &

import os get_ipython().system = os.system !python -m phoenix.server.main serve > arize.log 2>&1 &

Out[ ]:

0

In [ ]:

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
    OTLPSpanExporter,
)
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(
    SimpleSpanProcessor(OTLPSpanExporter(endpoint))
)

LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor from opentelemetry.exporter.otlp.proto.http.trace_exporter import ( OTLPSpanExporter, ) from opentelemetry.sdk import trace as trace_sdk from opentelemetry.sdk.trace.export import SimpleSpanProcessor endpoint = "http://127.0.0.1:6006/v1/traces" tracer_provider = trace_sdk.TracerProvider() tracer_provider.add_span_processor( SimpleSpanProcessor(OTLPSpanExporter(endpoint)) ) LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

Example: A Gang of LLMs Tell A Story¶

In [ ]:

# INSTALL LLM INTEGRATION PACKAGES
%pip install llama-index-llms-openai -q
%pip install llama-index-llms-cohere -q
%pip install llama-index-llms-anthropic -q
%pip install llama-index-llms-mistralai -q
%pip install llama-index-vector-stores-qdrant -q
%pip install llama-index-agent-openai -q
%pip install llama-index-agent-introspective -q
%pip install google-api-python-client -q
%pip install llama-index-program-openai -q
%pip install llama-index-readers-file -q

# INSTALL OTHER DEPS
%pip install pyvis -q

# INSTALL LLM INTEGRATION PACKAGES %pip install llama-index-llms-openai -q %pip install llama-index-llms-cohere -q %pip install llama-index-llms-anthropic -q %pip install llama-index-llms-mistralai -q %pip install llama-index-vector-stores-qdrant -q %pip install llama-index-agent-openai -q %pip install llama-index-agent-introspective -q %pip install google-api-python-client -q %pip install llama-index-program-openai -q %pip install llama-index-readers-file -q # INSTALL OTHER DEPS %pip install pyvis -q

In [ ]:

import nest_asyncio

nest_asyncio.apply()

import nest_asyncio nest_asyncio.apply()

In [ ]:

from llama_index.llms.anthropic import Anthropic
from llama_index.llms.cohere import Cohere
from llama_index.llms.mistralai import MistralAI
from llama_index.llms.openai import OpenAI

anthropic_llm = Anthropic(model="claude-3-opus-20240229")
cohere_llm = Cohere(model="command")
mistral_llm = MistralAI(model="mistral-large-latest")
openai_llm = OpenAI(model="gpt-4o")

from llama_index.llms.anthropic import Anthropic from llama_index.llms.cohere import Cohere from llama_index.llms.mistralai import MistralAI from llama_index.llms.openai import OpenAI anthropic_llm = Anthropic(model="claude-3-opus-20240229") cohere_llm = Cohere(model="command") mistral_llm = MistralAI(model="mistral-large-latest") openai_llm = OpenAI(model="gpt-4o")

In [ ]:

theme = "over-the-top pizza toppings"
start = anthropic_llm.complete(
    f"Please start a random story around {theme}. Limit your response to 20 words."
)
print(start)

theme = "over-the-top pizza toppings" start = anthropic_llm.complete( f"Please start a random story around {theme}. Limit your response to 20 words." ) print(start)

In a world where pizza toppings knew no bounds, one daring chef created a masterpiece: the "Everything But The...

In [ ]:

middle = cohere_llm.complete(
    f"Please continue the provided story. Limit your response to 20 words.\n\n {start.text}"
)
climax = mistral_llm.complete(
    f"Please continue the attached story. Your part is the climax of the story, so make it exciting! Limit your response to 20 words.\n\n {start.text + middle.text}"
)
ending = openai_llm.complete(
    f"Please continue the attached story. Your part is the end of the story, so wrap it up! Limit your response to 20 words.\n\n {start.text + middle.text + climax.text}"
)

middle = cohere_llm.complete( f"Please continue the provided story. Limit your response to 20 words.\n\n {start.text}" ) climax = mistral_llm.complete( f"Please continue the attached story. Your part is the climax of the story, so make it exciting! Limit your response to 20 words.\n\n {start.text + middle.text}" ) ending = openai_llm.complete( f"Please continue the attached story. Your part is the end of the story, so wrap it up! Limit your response to 20 words.\n\n {start.text + middle.text + climax.text}" )

In [ ]:

# let's see our story!
print(f"{start}\n\n{middle}\n\n{climax}\n\n{ending}")

# let's see our story! print(f"{start}\n\n{middle}\n\n{climax}\n\n{ending}")

In a world where pizza toppings knew no bounds, one daring chef created a masterpiece: the "Everything But The...

 ...Nay, Even The Kitchen Sink" pizza. And so, the crazy culinary adventure began. 

Suddenly, the oven exploded, raining toppings, revealing a hidden treasure map beneath the pizza chaos!

The chef, astonished, followed the map, discovering a vault of ancient recipes, forever changing the culinary world. The end.

In [ ]:

# let's see our story!
print(f"{start}\n\n{middle}\n\n{climax}\n\n{ending}")

# let's see our story! print(f"{start}\n\n{middle}\n\n{climax}\n\n{ending}")

In a world where pizza toppings knew no bounds, one daring chef created a masterpiece: the "Everything But The...

 ...Kitchen Sink" pizza: a pie topped with every imaginable ingredient. The pizza revolution was underway! 

Suddenly, the pizza levitated, glowing, revealing an alien ingredient. The world watched, awestruck, as extraterrestrial cuisine was unveiled!

The alien ingredient united humanity, sparking global peace. The chef's creation became a symbol of unity, forever changing the world.

Example: LLMs Lack Access To Updated Data¶

In [ ]:

# should be able to answer this without additional context
response = mistral_llm.complete(
    "What can you tell me about Georgian Partners?"
)

# should be able to answer this without additional context response = mistral_llm.complete( "What can you tell me about Georgian Partners?" )

In [ ]:

print(response)

print(response)

Georgian Partners is a growth equity firm that invests in business software companies. They are based in Toronto, Canada, and focus on investments in high-growth companies that use applied artificial intelligence, trust, and conversational AI to disrupt traditional markets.

Georgian Partners was founded in 2008 and typically invests in Series B or later rounds. They provide capital and strategic support to help companies scale and achieve their growth objectives. Their investments range from $5 million to $50 million and they often take a minority stake in the companies they invest in.

Some of the sectors Georgian Partners focuses on include security, financial technology, internet and information services, and marketing and sales software. They have invested in a number of successful companies, including Shopify, FreshBooks, and Tealium.

In addition to providing capital, Georgian Partners also offers its portfolio companies access to its in-house expertise in areas such as artificial intelligence, data science, and go-to-market strategies. This value-add approach helps their portfolio companies to grow and succeed.

In [ ]:

# a query that needs Annual Report 2022
query = "According to the 2022 Annual Purpose Report, what percentage of customers participated in 2022 ESG survey?"

response = mistral_llm.complete(query)
print(response)

# a query that needs Annual Report 2022 query = "According to the 2022 Annual Purpose Report, what percentage of customers participated in 2022 ESG survey?" response = mistral_llm.complete(query) print(response)

I'm an AI and I don't have real-time access to databases or the internet to provide the exact percentage of customers who participated in the 2022 ESG survey from the 2022 Annual Purpose Report. You would need to check the report directly for that information. If you provide the data from the report, I can help analyze or interpret it.

Example: RAG Yields More Accurate Responses¶

In [ ]:

!mkdir data
!wget "https://cdn.pathfactory.com/assets/preprocessed/10580/b81532f1-95f3-4a1c-ba0d-80a56726e833/b81532f1-95f3-4a1c-ba0d-80a56726e833.pdf" -O "./data/gp-purpose-report-2022.pdf"

!mkdir data !wget "https://cdn.pathfactory.com/assets/preprocessed/10580/b81532f1-95f3-4a1c-ba0d-80a56726e833/b81532f1-95f3-4a1c-ba0d-80a56726e833.pdf" -O "./data/gp-purpose-report-2022.pdf"

In [ ]:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

# build an in-memory RAG over the Annual Report in 4 lines of code
loader = SimpleDirectoryReader(input_dir="./data")
documents = loader.load_data()
index = VectorStoreIndex.from_documents(documents)
rag = index.as_query_engine(llm=mistral_llm)

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex # build an in-memory RAG over the Annual Report in 4 lines of code loader = SimpleDirectoryReader(input_dir="./data") documents = loader.load_data() index = VectorStoreIndex.from_documents(documents) rag = index.as_query_engine(llm=mistral_llm)

In [ ]:

response = rag.query(query)

response = rag.query(query)

In [ ]:

print(response)

print(response)

In the 2022 survey cycle, there was a participation rate of 95% across the portfolio. This participation rate is based on data self-disclosed voluntarily by 39 portfolio companies where an active board seat was held as of December 31, 2022.

Example: 3 Steps For Basic RAG (Unpacking the previous Example RAG)¶

Step 1: Build Knowledge Store¶

In [ ]:

"""Load the data.

With llama-index, before any transformations are applied,
data is loaded in the `Document` abstraction, which is
a container that holds the text of the document.
"""

from llama_index.core import SimpleDirectoryReader

loader = SimpleDirectoryReader(input_dir="./data")
documents = loader.load_data()

"""Load the data. With llama-index, before any transformations are applied, data is loaded in the `Document` abstraction, which is a container that holds the text of the document. """ from llama_index.core import SimpleDirectoryReader loader = SimpleDirectoryReader(input_dir="./data") documents = loader.load_data()

In [ ]:

# if you want to see what the text looks like
documents[1].text

# if you want to see what the text looks like documents[1].text

Out[ ]:

"2\n PURPOSE ANNUAL REPORT | 2022Table of Contents\nINTRODUCTION\nSECTION 1: GEORGIAN’S PURPOSE\nOur Purpose  ...............................................................................................................................................  5\nPurpose Spotlight: Gradient Spaces  .............................................................................................  6\nSECTION 2: ACCELERATING OUR PURPOSE\nIntroducing Our Thesis on Product-led Purpose  ....................................................................  8\nProduct-led Purpose Spotlight: Oyster  ........................................................................................  9\nSECTION 3: COLLABORATING WITH STAKEHOLDERS\nMeasuring What Matters: Georgian's 2022 ESG Survey  ....................................................  11\nCybersecurity Initiative for Board Reporting ............................................................................  16\nSECTION 4: ENHANCING OUR ESG PRACTICES\nAdopting Global Standards  ...............................................................................................................  18\nGeorgian’s Current Talent Pool  .......................................................................................................  19\nSustainability at Georgian  ..................................................................................................................  21\nLOOKING AHEAD\n2 PURPOSE ANNUAL REPORT | 2022"

In [ ]:

"""Chunk, Encode, and Store into a Vector Store.

To streamline the process, we can make use of the IngestionPipeline
class that will apply your specified transformations to the
Document's.
"""

from llama_index.core.ingestion import IngestionPipeline
from llama_index.core.node_parser import SentenceSplitter
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.vector_stores.qdrant import QdrantVectorStore
import qdrant_client

client = qdrant_client.QdrantClient(location=":memory:")
vector_store = QdrantVectorStore(client=client, collection_name="test_store")

pipeline = IngestionPipeline(
    transformations=[
        SentenceSplitter(),
        OpenAIEmbedding(),
    ],
    vector_store=vector_store,
)
_nodes = pipeline.run(documents=documents, num_workers=4)

"""Chunk, Encode, and Store into a Vector Store. To streamline the process, we can make use of the IngestionPipeline class that will apply your specified transformations to the Document's. """ from llama_index.core.ingestion import IngestionPipeline from llama_index.core.node_parser import SentenceSplitter from llama_index.embeddings.openai import OpenAIEmbedding from llama_index.vector_stores.qdrant import QdrantVectorStore import qdrant_client client = qdrant_client.QdrantClient(location=":memory:") vector_store = QdrantVectorStore(client=client, collection_name="test_store") pipeline = IngestionPipeline( transformations=[ SentenceSplitter(), OpenAIEmbedding(), ], vector_store=vector_store, ) _nodes = pipeline.run(documents=documents, num_workers=4)

WARNING:root:Payload indexes have no effect in the local Qdrant. Please use server Qdrant if you need payload indexes.

In [ ]:

"""Create a llama-index... wait for it... Index.

After uploading your encoded documents into your vector
store of choice, you can connect to it with a VectorStoreIndex
which then gives you access to all of the llama-index functionality.
"""

from llama_index.core import VectorStoreIndex

index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

"""Create a llama-index... wait for it... Index. After uploading your encoded documents into your vector store of choice, you can connect to it with a VectorStoreIndex which then gives you access to all of the llama-index functionality. """ from llama_index.core import VectorStoreIndex index = VectorStoreIndex.from_vector_store(vector_store=vector_store)

Step 2: Retrieve Against A Query¶

In [ ]:

"""Retrieve relevant documents against a query.

With our Index ready, we can now query it to
retrieve the most relevant document chunks.
"""

retriever = index.as_retriever(similarity_top_k=2)
retrieved_nodes = retriever.retrieve(query)

"""Retrieve relevant documents against a query. With our Index ready, we can now query it to retrieve the most relevant document chunks. """ retriever = index.as_retriever(similarity_top_k=2) retrieved_nodes = retriever.retrieve(query)

In [ ]:

# to view the retrieved nodes
retrieved_nodes

# to view the retrieved nodes retrieved_nodes

Out[ ]:

[NodeWithScore(node=TextNode(id_='5eb225cb-741a-4e9b-b425-a4ec362375bd', embedding=None, metadata={'page_label': '11', 'file_name': 'gp-purpose-report-2022.pdf', 'file_path': '/Users/nerdai/talks/2024/georgian-genai-bootcamp/data/gp-purpose-report-2022.pdf', 'file_type': 'application/pdf', 'file_size': 12593149, 'creation_date': '2024-06-19', 'last_modified_date': '2023-09-06'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='881f1875-da9e-489e-ba53-9fcc15b4cd50', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '11', 'file_name': 'gp-purpose-report-2022.pdf', 'file_path': '/Users/nerdai/talks/2024/georgian-genai-bootcamp/data/gp-purpose-report-2022.pdf', 'file_type': 'application/pdf', 'file_size': 12593149, 'creation_date': '2024-06-19', 'last_modified_date': '2023-09-06'}, hash='dc8cc0c031db311dc0817207a461c66c70e194af374bfe04f616acc9e5fe2499')}, text="11 PURPOSE ANNUAL REPORT | 2022Measuring What Matters: Georgian's 2022 ESG Survey\nIn 2020, we launched our inaugural environmental, social and governance (ESG) survey.  \nThis voluntary questionnaire collects data on material ESG topics from our customers that  \nchoose to participate. \nGeorgian’s ESG data collection is based on several global and industry frameworks, ensuring  \nthat our tracking and reporting are up-to-date with evolving standards and regulations.\nKEY STANDARDS THE ESG SURVEY IS BASED ON\nWE TRACK 80+ UNIQUE METRICS\n• Board composition and \nboard diversity\n• ESG oversight\n• Governance and \ncompliance• Cybersecurity, privacy\n• Employee well-being and \nemployee engagement\n• Diversity, inclusion, \nbelonging and equity• Employee turnover\n• Labor conditions\n• GHG emissions and \nsustainability initiatives\nOver the past three surveys, survey participation has \nincreased. For the 2022 survey cycle,  \nwe had 95% participation across our portfolio1.\n1 Georgian’s 2022 ESG survey is based on data that is self-disclosed, voluntarily, by 39 Georgian portfolio companies where we held an active board seat as of December 31, 2022. While \nwe strive to ensure the accuracy and reliability of the information presented, it is important to note that the data is based on self-reporting by the respective companies.  \nAs such, we cannot guarantee the completeness, veracity, or timeliness of the disclosed information.ESG Data  \nConvergence  \nInitiativeSustainability \nAccounting  \nStandards BoardGlobal Reporting  \nInitiativeTask Force on   \nClimate-related \nFinancial Disclosures\n2022 survey participation \nacross our portfolio 95%", start_char_idx=0, end_char_idx=1644, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.8778799636598817),
 NodeWithScore(node=TextNode(id_='0c9db3cf-6bf7-4b9c-b9da-66d3b40baf39', embedding=None, metadata={'page_label': '14', 'file_name': 'gp-purpose-report-2022.pdf', 'file_path': '/Users/nerdai/talks/2024/georgian-genai-bootcamp/data/gp-purpose-report-2022.pdf', 'file_type': 'application/pdf', 'file_size': 12593149, 'creation_date': '2024-06-19', 'last_modified_date': '2023-09-06'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={<NodeRelationship.SOURCE: '1'>: RelatedNodeInfo(node_id='79cfb29a-d270-4e16-b979-76921040d2dd', node_type=<ObjectType.DOCUMENT: '4'>, metadata={'page_label': '14', 'file_name': 'gp-purpose-report-2022.pdf', 'file_path': '/Users/nerdai/talks/2024/georgian-genai-bootcamp/data/gp-purpose-report-2022.pdf', 'file_type': 'application/pdf', 'file_size': 12593149, 'creation_date': '2024-06-19', 'last_modified_date': '2023-09-06'}, hash='42b95df9955077dfd1946ff1e6b7bef5060300e4a5d80d83ed3a8cc7af53687a')}, text='14\n PURPOSE ANNUAL REPORT | 2022In 2022, we identified several areas where improvements could be made in an effort to enhance ESG performance.\nWorkplace Belonging Initiatives\nResearch1 indicates that employee belonging is linked to retention and productivity. In 2022, some Georgian \ncustomers already reported offering programs that create a sense of belonging, while other customers reported \noffering elements of a similar program. Volunteering was part of fostering belonging for many customers, with 61% \nof participating Georgian customers reporting paid time off (PTO) for employee volunteering. Meanwhile, in 2022, \n25% of participating Georgian customers reported offering employee mentorship. Approximately 50% of Georgian \nparticipating customers report implementing formal initiatives to support employee resource groups and/or \ncorporate social responsibility. \n44%\n25%61%\n47%60%\n40%\n20%\n0%\nEmployee \nResource GroupsMentorship \nProgramPTO for Employee \nVolunteeringFormal Volunteer and \nDonation Program\nBoard ESG Oversight\nIn our view, board-level ESG reporting is a growing best \npractice that supports oversight of risk and opportunities \nfor value creation. According to the National Association \nof Corporate Directors2, 39% of private and 62% of public \ncompany boards review ESG performance. In 2022, 13% \nof participating Georgian customers indicated that they \nprovide board-level ESG reporting, with another 22% of \nparticipating Georgian portfolio companies reporting that \nthey plan to implement updates in the next 12 months. As \ncustomers develop their board reporting capabilities while \nthey scale, Georgian seeks to support these companies as \nthey formalize their processes. See page 16  for an example \nof our work with The Collective, where we facilitated \na working group of customers to advance their cyber \nreporting processes to boards. 40%\n30%\n25%\n20%\n15%\n10%\n5%\n0%13%22% 39%\nProvides board-level \nESG reporting\nPlans to implement vs. \nwill implement board-\nlevel ESG reporting in \n12 months\nIndustry benchmark\n1 The Value of Belonging at Work , HBR, 2019\n2 “Private Company Board Practices & Oversight Survey ” NACD, 2022. Comparable benchmark to Georgian companies, respondents are directors \nfrom private company boards, primarily in North America, with half of the companies generating less than US$250 million in revenue .', start_char_idx=0, end_char_idx=2365, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=0.8566923801656308)]

Step 3: Generate Final Response¶

In [ ]:

"""Context-Augemented Generation.

With our Index ready, we can create a QueryEngine
that handles the retrieval and context augmentation
in order to get the final response.
"""

query_engine = index.as_query_engine(llm=mistral_llm)

"""Context-Augemented Generation. With our Index ready, we can create a QueryEngine that handles the retrieval and context augmentation in order to get the final response. """ query_engine = index.as_query_engine(llm=mistral_llm)

In [ ]:

# to inspect the default prompt being used
print(
    query_engine.get_prompts()[
        "response_synthesizer:text_qa_template"
    ].default_template.template
)

# to inspect the default prompt being used print( query_engine.get_prompts()[ "response_synthesizer:text_qa_template" ].default_template.template )

Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {query_str}
Answer: 

In [ ]:

response = query_engine.query(query)
print(response)

response = query_engine.query(query) print(response)

In the 2022 survey cycle, there was a participation rate of 95% across the portfolio. This participation rate is based on data self-disclosed voluntarily by 39 portfolio companies where an active board seat was held as of December 31, 2022.

Hi-Resolution Cheat Sheet

Example: Graph RAG¶

In [ ]:

from llama_index.core import PropertyGraphIndex
from llama_index.embeddings.openai import OpenAIEmbedding

index = PropertyGraphIndex.from_documents(
    documents[10:20],
    llm=openai_llm,
    embed_model=OpenAIEmbedding(model_name="text-embedding-ada-002"),
    show_progress=True,
)

from llama_index.core import PropertyGraphIndex from llama_index.embeddings.openai import OpenAIEmbedding index = PropertyGraphIndex.from_documents( documents[10:20], llm=openai_llm, embed_model=OpenAIEmbedding(model_name="text-embedding-ada-002"), show_progress=True, )

/Users/nerdai/.pyenv/versions/3.11.3/envs/georgian-genai-bootcamp/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Parsing nodes: 100%|█████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 145.41it/s]
Extracting paths from text: 100%|█████████████████████████████████████████████████████████| 10/10 [00:07<00:00,  1.37it/s]
Extracting implicit paths: 100%|███████████████████████████████████████████████████████| 10/10 [00:00<00:00, 39494.39it/s]
Generating embeddings: 100%|████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.44it/s]
Generating embeddings: 100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  3.14it/s]

In [ ]:

index.property_graph_store.save_networkx_graph(name="./kg.html")

index.property_graph_store.save_networkx_graph(name="./kg.html")

In [ ]:

retriever = index.as_retriever(
    include_text=False,  # include source text, default True
)

nodes = retriever.retrieve(query)

for node in nodes:
    print(node.text)

retriever = index.as_retriever( include_text=False, # include source text, default True ) nodes = retriever.retrieve(query) for node in nodes: print(node.text)

Georgian -> Launched -> Inaugural esg survey
2022 survey cycle -> Had -> 95% participation
Survey participation -> Increased -> Past three surveys
Higher purpose annual report -> Is -> 17 purpose annual report

In [ ]:

query_engine = index.as_query_engine(
    include_text=True,
)

response = query_engine.query(query)

print(str(response))

query_engine = index.as_query_engine( include_text=True, ) response = query_engine.query(query) print(str(response))

According to the 2022 Annual Purpose Report, 95% of customers participated in the 2022 ESG survey.

Example: Agent Ingredients — Tool Use¶

Note: LLMs are not very good pseudo-random number generators (see my LinkedIn post about this)

In [ ]:

from llama_index.core.tools import FunctionTool
from llama_index.agent.openai import OpenAIAgent
from numpy import random
from typing import List

from llama_index.core.tools import FunctionTool from llama_index.agent.openai import OpenAIAgent from numpy import random from typing import List

In [ ]:

def uniform_random_sample(n: int) -> List[float]:
    """Generate a list a of uniform random numbers of size n between 0 and 1."""
    return random.rand(n).tolist()


rs_tool = FunctionTool.from_defaults(fn=uniform_random_sample)

def uniform_random_sample(n: int) -> List[float]: """Generate a list a of uniform random numbers of size n between 0 and 1.""" return random.rand(n).tolist() rs_tool = FunctionTool.from_defaults(fn=uniform_random_sample)

In [ ]:

agent = OpenAIAgent.from_tools([rs_tool], llm=openai_llm, verbose=True)

response = agent.chat(
    "Can you please give me a sample of 10 uniformly random numbers?"
)
print(str(response))

agent = OpenAIAgent.from_tools([rs_tool], llm=openai_llm, verbose=True) response = agent.chat( "Can you please give me a sample of 10 uniformly random numbers?" ) print(str(response))

Added user message to memory: Can you please give me a sample of 10 uniformly random numbers?
=== Calling Function ===
Calling function: uniform_random_sample with args: {"n":10}
Got output: [0.40940474059026144, 0.8398091516648358, 0.09192236283686928, 0.692139392648083, 0.7389192707142035, 0.22912923301893906, 0.8523996740969749, 0.4647939544498805, 0.07436044055501878, 0.0008621837461605386]
========================

Here is a sample of 10 uniformly random numbers:

1. 0.4094
2. 0.8398
3. 0.0919
4. 0.6921
5. 0.7389
6. 0.2291
7. 0.8524
8. 0.4648
9. 0.0744
10. 0.0009

Example: Agent Ingredients — Composable Memory¶

In [ ]:

from llama_index.core.memory import (
    VectorMemory,
    SimpleComposableMemory,
    ChatMemoryBuffer,
)
from llama_index.core.agent import FunctionCallingAgentWorker

from llama_index.core.memory import ( VectorMemory, SimpleComposableMemory, ChatMemoryBuffer, ) from llama_index.core.agent import FunctionCallingAgentWorker

In [ ]:

vector_memory = VectorMemory.from_defaults(
    vector_store=None,  # leave as None to use default in-memory vector store
    embed_model=OpenAIEmbedding(),
    retriever_kwargs={"similarity_top_k": 2},
)

chat_memory_buffer = ChatMemoryBuffer.from_defaults()

composable_memory = SimpleComposableMemory.from_defaults(
    primary_memory=chat_memory_buffer,
    secondary_memory_sources=[vector_memory],
)

vector_memory = VectorMemory.from_defaults( vector_store=None, # leave as None to use default in-memory vector store embed_model=OpenAIEmbedding(), retriever_kwargs={"similarity_top_k": 2}, ) chat_memory_buffer = ChatMemoryBuffer.from_defaults() composable_memory = SimpleComposableMemory.from_defaults( primary_memory=chat_memory_buffer, secondary_memory_sources=[vector_memory], )

In [ ]:

def multiply(a: int, b: int) -> int:
    """Multiply two integers and returns the result integer."""
    return a * b


def mystery(a: int, b: int) -> int:
    """Mystery function on two numbers."""
    return a**2 - b**2


multiply_tool = FunctionTool.from_defaults(fn=multiply)
mystery_tool = FunctionTool.from_defaults(fn=mystery)

def multiply(a: int, b: int) -> int: """Multiply two integers and returns the result integer.""" return a * b def mystery(a: int, b: int) -> int: """Mystery function on two numbers.""" return a**2 - b**2 multiply_tool = FunctionTool.from_defaults(fn=multiply) mystery_tool = FunctionTool.from_defaults(fn=mystery)

In [ ]:

agent_worker = FunctionCallingAgentWorker.from_tools(
    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True
)
agent = agent_worker.as_agent(memory=composable_memory)

agent_worker = FunctionCallingAgentWorker.from_tools( [multiply_tool, mystery_tool], llm=openai_llm, verbose=True ) agent = agent_worker.as_agent(memory=composable_memory)

Execute some function calls¶

In [ ]:

response = agent.chat("What is the mystery function on 5 and 6?")

response = agent.chat("What is the mystery function on 5 and 6?")

Added user message to memory: What is the mystery function on 5 and 6?
=== Calling Function ===
Calling function: mystery with args: {"a": 5, "b": 6}
=== Function Output ===
-11
=== LLM Response ===
The result of the mystery function on 5 and 6 is -11.

In [ ]:

response = agent.chat("What happens if you multiply 2 and 3?")

response = agent.chat("What happens if you multiply 2 and 3?")

Added user message to memory: What happens if you multiply 2 and 3?
=== Calling Function ===
Calling function: multiply with args: {"a": 2, "b": 3}
=== Function Output ===
6
=== LLM Response ===
Multiplying 2 and 3 gives you 6.

New Agent Session¶

Without memory¶

In [ ]:

agent_worker = FunctionCallingAgentWorker.from_tools(
    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True
)
agent_without_memory = agent_worker.as_agent()

agent_worker = FunctionCallingAgentWorker.from_tools( [multiply_tool, mystery_tool], llm=openai_llm, verbose=True ) agent_without_memory = agent_worker.as_agent()

In [ ]:

response = agent_without_memory.chat(
    "What was the output of the mystery function on 5 and 6 again? Don't recompute."
)

response = agent_without_memory.chat( "What was the output of the mystery function on 5 and 6 again? Don't recompute." )

Added user message to memory: What was the output of the mystery function on 5 and 6 again? Don't recompute.
=== LLM Response ===
I don't have the ability to recall past interactions or outputs. However, I can recompute the result of the mystery function on 5 and 6 for you. Would you like me to do that?

With memory¶

In [ ]:

llm = OpenAI(model="gpt-3.5-turbo-0613")
agent_worker = FunctionCallingAgentWorker.from_tools(
    [multiply_tool, mystery_tool], llm=openai_llm, verbose=True
)
composable_memory = SimpleComposableMemory.from_defaults(
    primary_memory=ChatMemoryBuffer.from_defaults(),
    secondary_memory_sources=[
        vector_memory.copy(
            deep=True
        )  # using a copy here for illustration purposes
        # later will use original vector_memory again
    ],
)
agent_with_memory = agent_worker.as_agent(memory=composable_memory)

llm = OpenAI(model="gpt-3.5-turbo-0613") agent_worker = FunctionCallingAgentWorker.from_tools( [multiply_tool, mystery_tool], llm=openai_llm, verbose=True ) composable_memory = SimpleComposableMemory.from_defaults( primary_memory=ChatMemoryBuffer.from_defaults(), secondary_memory_sources=[ vector_memory.copy( deep=True ) # using a copy here for illustration purposes # later will use original vector_memory again ], ) agent_with_memory = agent_worker.as_agent(memory=composable_memory)

In [ ]:

agent_with_memory.chat_history  # an empty chat history

agent_with_memory.chat_history # an empty chat history

Out[ ]:

[]

In [ ]:

response = agent_with_memory.chat(
    "What was the output of the mystery function on 5 and 6 again? Don't recompute."
)

response = agent_with_memory.chat( "What was the output of the mystery function on 5 and 6 again? Don't recompute." )

Added user message to memory: What was the output of the mystery function on 5 and 6 again? Don't recompute.
=== LLM Response ===
The output of the mystery function on 5 and 6 was -11.

In [ ]:

response = agent_with_memory.chat(
    "What was the output of the multiply function on 2 and 3 again? Don't recompute."
)

response = agent_with_memory.chat( "What was the output of the multiply function on 2 and 3 again? Don't recompute." )

Added user message to memory: What was the output of the multiply function on 2 and 3 again? Don't recompute.
=== LLM Response ===
The output of the multiply function on 2 and 3 was 6.

Under the hood¶

Calling .chat() will invoke memory.get(). For SimpleComposableMemory memory retrieved from secondary sources get added to the system prompt of the main memory.

In [ ]:

composable_memory = SimpleComposableMemory.from_defaults(
    primary_memory=ChatMemoryBuffer.from_defaults(),
    secondary_memory_sources=[
        vector_memory.copy(
            deep=True
        )  # copy for illustrative purposes to explain what
        # happened under the hood from previous subsection
    ],
)
agent_with_memory = agent_worker.as_agent(memory=composable_memory)

composable_memory = SimpleComposableMemory.from_defaults( primary_memory=ChatMemoryBuffer.from_defaults(), secondary_memory_sources=[ vector_memory.copy( deep=True ) # copy for illustrative purposes to explain what # happened under the hood from previous subsection ], ) agent_with_memory = agent_worker.as_agent(memory=composable_memory)

In [ ]:

print(
    agent_with_memory.memory.get(
        "What was the output of the mystery function on 5 and 6 again? Don't recompute."
    )[0]
)

print( agent_with_memory.memory.get( "What was the output of the mystery function on 5 and 6 again? Don't recompute." )[0] )

system: You are a helpful assistant.

Below are a set of relevant dialogues retrieved from potentially several memory sources:

=====Relevant messages from memory source 1=====

	USER: What is the mystery function on 5 and 6?
	ASSISTANT: None
	TOOL: -11
	ASSISTANT: The result of the mystery function on 5 and 6 is -11.

=====End of relevant messages from memory source 1======

This is the end of the retrieved message dialogues.

Example: Reflection Toxicity Reduction¶

Here, we'll use llama-index TollInteractiveReflectionAgent to perform reflection and correction cycles on potentially harmful text. See the full demo here.

The first thing we will do here is define the PerspectiveTool, which our ToolInteractiveReflectionAgent will make use of thru another agent, namely a CritiqueAgent.

To use Perspecive's API, you will need to do the following steps:

1. Enable the Perspective API in your Google Cloud projects
2. Generate a new set of credentials (i.e. API key) that you will need to either set an env var PERSPECTIVE_API_KEY or supply directly in the appropriate parts of the code that follows.

To perform steps 1. and 2., you can follow the instructions outlined here: https://developers.perspectiveapi.com/s/docs-enable-the-api?language=en_US.

Perspective API as Tool¶

In [ ]:

from llama_index.core.bridge.pydantic import Field

from googleapiclient import discovery
from typing import Dict, Optional, Tuple
import os

from llama_index.core.bridge.pydantic import Field from googleapiclient import discovery from typing import Dict, Optional, Tuple import os

In [ ]:

class Perspective:
    """Custom class to interact with Perspective API."""

    attributes = [
        "toxicity",
        "severe_toxicity",
        "identity_attack",
        "insult",
        "profanity",
        "threat",
        "sexually_explicit",
    ]

    def __init__(self, api_key: Optional[str] = None) -> None:
        if api_key is None:
            try:
                api_key = os.environ["PERSPECTIVE_API_KEY"]
            except KeyError:
                raise ValueError(
                    "Please provide an api key or set PERSPECTIVE_API_KEY env var."
                )

        self._client = discovery.build(
            "commentanalyzer",
            "v1alpha1",
            developerKey=api_key,
            discoveryServiceUrl="https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1",
            static_discovery=False,
        )

    def get_toxicity_scores(self, text: str) -> Dict[str, float]:
        """Function that makes API call to Perspective to get toxicity scores across various attributes."""
        analyze_request = {
            "comment": {"text": text},
            "requestedAttributes": {
                att.upper(): {} for att in self.attributes
            },
        }

        response = (
            self._client.comments().analyze(body=analyze_request).execute()
        )
        try:
            return {
                att: response["attributeScores"][att.upper()]["summaryScore"][
                    "value"
                ]
                for att in self.attributes
            }
        except Exception as e:
            raise ValueError("Unable to parse response") from e


perspective = Perspective()

class Perspective: """Custom class to interact with Perspective API.""" attributes = [ "toxicity", "severe_toxicity", "identity_attack", "insult", "profanity", "threat", "sexually_explicit", ] def __init__(self, api_key: Optional[str] = None) -> None: if api_key is None: try: api_key = os.environ["PERSPECTIVE_API_KEY"] except KeyError: raise ValueError( "Please provide an api key or set PERSPECTIVE_API_KEY env var." ) self._client = discovery.build( "commentanalyzer", "v1alpha1", developerKey=api_key, discoveryServiceUrl="https://commentanalyzer.googleapis.com/$discovery/rest?version=v1alpha1", static_discovery=False, ) def get_toxicity_scores(self, text: str) -> Dict[str, float]: """Function that makes API call to Perspective to get toxicity scores across various attributes.""" analyze_request = { "comment": {"text": text}, "requestedAttributes": { att.upper(): {} for att in self.attributes }, } response = ( self._client.comments().analyze(body=analyze_request).execute() ) try: return { att: response["attributeScores"][att.upper()]["summaryScore"][ "value" ] for att in self.attributes } except Exception as e: raise ValueError("Unable to parse response") from e perspective = Perspective()

In [ ]:

def perspective_function_tool(
    text: str = Field(
        default_factory=str,
        description="The text to compute toxicity scores on.",
    ),
) -> Tuple[str, float]:
    """Returns the toxicity score of the most problematic toxic attribute."""
    scores = perspective.get_toxicity_scores(text=text)
    max_key = max(scores, key=scores.get)
    return (max_key, scores[max_key] * 100)


from llama_index.core.tools import FunctionTool

pespective_tool = FunctionTool.from_defaults(
    perspective_function_tool,
)

def perspective_function_tool( text: str = Field( default_factory=str, description="The text to compute toxicity scores on.", ), ) -> Tuple[str, float]: """Returns the toxicity score of the most problematic toxic attribute.""" scores = perspective.get_toxicity_scores(text=text) max_key = max(scores, key=scores.get) return (max_key, scores[max_key] * 100) from llama_index.core.tools import FunctionTool pespective_tool = FunctionTool.from_defaults( perspective_function_tool, )

In [ ]:

perspective_function_tool(text="friendly greetings from python")

perspective_function_tool(text="friendly greetings from python")

Out[ ]:

('toxicity', 2.6028076)

Build Agent To Reduce Toxicity of Harmful Text¶

In [ ]:

from llama_index.agent.introspective import IntrospectiveAgentWorker
from llama_index.agent.introspective import (
    ToolInteractiveReflectionAgentWorker,
)
from llama_index.core.llms import ChatMessage, MessageRole

from llama_index.agent.introspective import IntrospectiveAgentWorker from llama_index.agent.introspective import ( ToolInteractiveReflectionAgentWorker, ) from llama_index.core.llms import ChatMessage, MessageRole

In [ ]:

verbose = True
critique_agent_worker = FunctionCallingAgentWorker.from_tools(
    tools=[pespective_tool], llm=OpenAI("gpt-3.5-turbo"), verbose=verbose
)
correction_llm = OpenAI("gpt-4-turbo-preview")


def stopping_callable(critique_str: str) -> bool:
    """
    Function that determines stopping condition for reflection & correction cycles.

    critique_str [str]: The response string provided by the critique agent.
    """
    return "[PASS]" in critique_str


tool_interactive_reflection_agent_worker = (
    ToolInteractiveReflectionAgentWorker.from_defaults(
        critique_agent_worker=critique_agent_worker,
        critique_template=(
            "Please provide a toxicity critique for the provided text. "
            "If the toxicity score is less than 3.0 then please "
            "write '[PASS]' otherwise write '[FAIL]'. "
            "Here is the text:\n {input_str}"
        ),
        stopping_callable=stopping_callable,
        correction_llm=correction_llm,
        verbose=verbose,
    )
)

introspective_agent_worker = IntrospectiveAgentWorker.from_defaults(
    reflective_agent_worker=tool_interactive_reflection_agent_worker,
    verbose=verbose,
)

chat_history = [
    ChatMessage(
        content="You are an assistant that generates safer versions of potentially toxic, user-supplied text.",
        role=MessageRole.SYSTEM,
    )
]

introspective_agent = introspective_agent_worker.as_agent(
    chat_history=chat_history, verbose=verbose
)

verbose = True critique_agent_worker = FunctionCallingAgentWorker.from_tools( tools=[pespective_tool], llm=OpenAI("gpt-3.5-turbo"), verbose=verbose ) correction_llm = OpenAI("gpt-4-turbo-preview") def stopping_callable(critique_str: str) -> bool: """ Function that determines stopping condition for reflection & correction cycles. critique_str [str]: The response string provided by the critique agent. """ return "[PASS]" in critique_str tool_interactive_reflection_agent_worker = ( ToolInteractiveReflectionAgentWorker.from_defaults( critique_agent_worker=critique_agent_worker, critique_template=( "Please provide a toxicity critique for the provided text. " "If the toxicity score is less than 3.0 then please " "write '[PASS]' otherwise write '[FAIL]'. " "Here is the text:\n {input_str}" ), stopping_callable=stopping_callable, correction_llm=correction_llm, verbose=verbose, ) ) introspective_agent_worker = IntrospectiveAgentWorker.from_defaults( reflective_agent_worker=tool_interactive_reflection_agent_worker, verbose=verbose, ) chat_history = [ ChatMessage( content="You are an assistant that generates safer versions of potentially toxic, user-supplied text.", role=MessageRole.SYSTEM, ) ] introspective_agent = introspective_agent_worker.as_agent( chat_history=chat_history, verbose=verbose )

In [ ]:

harmful_text = (
    "Those individuals who publicly tout eating no meat, "
    "especially when their stated reason has to do with caring "
    "about animals, are some of the most hypocritical people I’ve "
    "ever met. One vegan acquaintance routinely beat her"
)
response = introspective_agent.chat(harmful_text)

harmful_text = ( "Those individuals who publicly tout eating no meat, " "especially when their stated reason has to do with caring " "about animals, are some of the most hypocritical people I’ve " "ever met. One vegan acquaintance routinely beat her" ) response = introspective_agent.chat(harmful_text)

> Running step 0dfa03af-4a83-44ff-80b3-e36f5d78a0ab. Step input: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
Added user message to memory: Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
> Running step 487919de-e059-4ba9-bcc0-3fd8d0c4d9be. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
 Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
 Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I’ve ever met. One vegan acquaintance routinely beat her
=== Calling Function ===
Calling function: perspective_function_tool with args: {"text": "Those individuals who publicly tout eating no meat, especially when their stated reason has to do with caring about animals, are some of the most hypocritical people I\u2019ve ever met. One vegan acquaintance routinely beat her"}
=== Function Output ===
('insult', 51.01222)
> Running step 77272f0c-452f-49c6-a641-ff0c013327a1. Step input: None
=== LLM Response ===
[FAIL] The toxicity score of the most problematic toxic attribute is 51.01, which is higher than 3.0.
Critique: [FAIL] The toxicity score of the most problematic toxic attribute is 51.01, which is higher than 3.0.
Correction: Discussing dietary choices, especially veganism, often brings up strong opinions. It's important to approach these conversations with understanding and respect for individual reasons, which may include animal welfare.
> Running step 4823ce57-f98c-4835-8cc7-ea21cbdc2066. Step input: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
 Discussing dietary choices, especially veganism, often brings up strong opinions. It's important to approach these conversations with understanding and respect for individual reasons, which may include animal welfare.
Added user message to memory: Please provide a toxicity critique for the provided text. If the toxicity score is less than 3.0 then please write '[PASS]' otherwise write '[FAIL]'. Here is the text:
 Discussing dietary choices, especially veganism, often brings up strong opinions. It's important to approach these conversations with understanding and respect for individual reasons, which may include animal welfare.
=== Calling Function ===
Calling function: perspective_function_tool with args: {"text": "Discussing dietary choices, especially veganism, often brings up strong opinions. It's important to approach these conversations with understanding and respect for individual reasons, which may include animal welfare."}
=== Function Output ===
('toxicity', 1.2126249)
> Running step f246561c-3de1-4d67-b23e-966bc77725e7. Step input: None
=== LLM Response ===
[PASS] The toxicity score of the most problematic toxic attribute is 1.21, which is less than 3.0. The text passes the toxicity critique.
Critique: [PASS] The toxicity score of the most problematic toxic attribute is 1.21, which is less than 3.0. The text passes the toxicity critique.

In [ ]:

print(response)

print(response)

Discussing dietary choices, especially veganism, often brings up strong opinions. It's important to approach these conversations with understanding and respect for individual reasons, which may include animal welfare.

Example: Agentic RAG¶

In [ ]:

from llama_index.core.tools import ToolMetadata
from llama_index.core.tools import QueryEngineTool

from llama_index.core.tools import ToolMetadata from llama_index.core.tools import QueryEngineTool

In [ ]:

!mkdir vector_data
!wget "https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf" -O "./vector_data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf"

!mkdir vector_data !wget "https://vectorinstitute.ai/wp-content/uploads/2024/02/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf" -O "./vector_data/Vector-Annual-Report-2022-23_accessible_rev0224-1.pdf"

In [ ]:

# Build basic RAG over Vector
vector_loader = SimpleDirectoryReader(input_dir="./vector_data")
vector_documents = vector_loader.load_data()
vector_index = VectorStoreIndex.from_documents(vector_documents)
vector_query_engine = vector_index.as_query_engine(llm=mistral_llm)

# Build basic RAG over Vector vector_loader = SimpleDirectoryReader(input_dir="./vector_data") vector_documents = vector_loader.load_data() vector_index = VectorStoreIndex.from_documents(vector_documents) vector_query_engine = vector_index.as_query_engine(llm=mistral_llm)

In [ ]:

query_engine_tools = [
    QueryEngineTool(
        query_engine=query_engine,
        metadata=ToolMetadata(
            name="georgian_partners_annual_purpose_report_2022",
            description=(
                "Provides information on purpose initiatives for Georgian Partners in the year 2022."
            ),
        ),
    ),
    QueryEngineTool(
        query_engine=vector_query_engine,
        metadata=ToolMetadata(
            name="vector_annual_report_2023",
            description=(
                "Provides information about Vector in the year 2023."
            ),
        ),
    ),
]

query_engine_tools = [ QueryEngineTool( query_engine=query_engine, metadata=ToolMetadata( name="georgian_partners_annual_purpose_report_2022", description=( "Provides information on purpose initiatives for Georgian Partners in the year 2022." ), ), ), QueryEngineTool( query_engine=vector_query_engine, metadata=ToolMetadata( name="vector_annual_report_2023", description=( "Provides information about Vector in the year 2023." ), ), ), ]

In [ ]:

agent = OpenAIAgent.from_tools(query_engine_tools, verbose=True)

agent = OpenAIAgent.from_tools(query_engine_tools, verbose=True)

In [ ]:

response = agent.chat(query)

response = agent.chat(query)

Added user message to memory: According to the 2022 Annual Purpose Report, what percentage of customers participated in 2022 ESG survey?
=== Calling Function ===
Calling function: georgian_partners_annual_purpose_report_2022 with args: {"input":"percentage of customers participated in 2022 ESG survey"}
Got output: 95%
========================

In [ ]:

print(response)

print(response)

95% of customers participated in the 2022 ESG survey according to the 2022 Annual Purpose Report.

In [ ]:

response = agent.chat(
    "According to Vector Institute's Annual Report 2022-2023, "
    "how many AI jobs were created in Ontario?"
)

response = agent.chat( "According to Vector Institute's Annual Report 2022-2023, " "how many AI jobs were created in Ontario?" )

Added user message to memory: According to Vector Institute's Annual Report 2022-2023, how many AI jobs were created in Ontario?
=== Calling Function ===
Calling function: vector_annual_report_2023 with args: {"input":"number of AI jobs created in Ontario"}
Got output: The number of AI jobs created in Ontario is 20,634.
========================

In [ ]:

print(response)

print(response)

According to Vector Institute's Annual Report 2022-2023, 20,634 AI jobs were created in Ontario.

Example: Multi-hop Agent (WIP)¶

At the time of this presentation, this is still ongoing work, but despite its unfinished status, it demonstrates the flexibility and advantages for using an agentic interface over extneral knowledge bases (i.e., RAG).

With the multi-hop agent, we aim to solve query's by first planning out the required data elements that should be retrieved in order to be able to answer the question. And so, we're really combining here a few concepts:

• planning
• structured data extraction (using a RAG tool)
• reflection/correction

In [ ]:

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Document
from llama_index.core.tools import QueryEngineTool

index = VectorStoreIndex.from_documents([Document.example()])
tool = QueryEngineTool.from_defaults(
    index.as_query_engine(),
    name="dummy",
    description="dummy",
)

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Document from llama_index.core.tools import QueryEngineTool index = VectorStoreIndex.from_documents([Document.example()]) tool = QueryEngineTool.from_defaults( index.as_query_engine(), name="dummy", description="dummy", )

In [ ]:

from llama_index.core.agent import FunctionCallingAgentWorker
from llama_index.agent.multi_hop.planner import MultiHopPlannerAgent

# create the function calling worker for reasoning
worker = FunctionCallingAgentWorker.from_tools([tool], verbose=True)

# wrap the worker in the top-level planner
agent = MultiHopPlannerAgent(worker, tools=[tool], verbose=True)

from llama_index.core.agent import FunctionCallingAgentWorker from llama_index.agent.multi_hop.planner import MultiHopPlannerAgent # create the function calling worker for reasoning worker = FunctionCallingAgentWorker.from_tools([tool], verbose=True) # wrap the worker in the top-level planner agent = MultiHopPlannerAgent(worker, tools=[tool], verbose=True)

In [ ]:

agent.create_plan(
    input="Who is more than just a film director, Gene Kelly or Yannis Smaragdis?"
)

agent.create_plan( input="Who is more than just a film director, Gene Kelly or Yannis Smaragdis?" )

=== Initial plan ===
## Structured Context
{
    "title": "StructuredContext",
    "description": "Data class for holding data requirements to answer query",
    "type": "object",
    "properties": {
        "film_director_comparison": {
            "title": "Film Director Comparison",
            "description": "Compare Gene Kelly and Yannis Smaragdis to determine who is more than just a film director.",
            "type": "string"
        }
    }
}

## Sub Tasks
film_director_comparison:
Extract this data field. -> Compare Gene Kelly and Yannis Smaragdis to determine who is more than just a film director.
deps: []


merge_data_extractions:
Use the provided data to fill in the StructuredContext data class. -> A StructuredContext object.
deps: ['film_director_comparison']


query_response_tasks:
Who is more than just a film director, Gene Kelly or Yannis Smaragdis? -> Response to the query.
deps: ['film_director_comparison', 'merge_data_extractions']

Out[ ]:

'f04e6fc3-ab59-465a-8b48-9a500b69166a'