Open In Colab

Pinecone Vector Store#

If you’re opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

%pip install llama-index-vector-stores-pinecone
!pip install llama-index>=0.9.31 pinecone-client>=3.0.0
import logging
import sys
import os

logging.basicConfig(stream=sys.stdout, level=logging.INFO)

Creating a Pinecone Index#

from pinecone import Pinecone, ServerlessSpec
] = "<Your Pinecone API key, from>"
os.environ["OPENAI_API_KEY"] = "sk-..."

api_key = os.environ["PINECONE_API_KEY"]

pc = Pinecone(api_key=api_key)
# delete if needed
# pc.delete_index("quickstart")
# dimensions are for text-embedding-ada-002

    spec=ServerlessSpec(cloud="aws", region="us-west-2"),

# If you need to create a PodBased Pinecone index, you could alternatively do this:
# from pinecone import Pinecone, PodSpec
# pc = Pinecone(api_key='xxx')
# pc.create_index(
# 	 name='my-index',
# 	 dimension=1536,
# 	 metric='cosine',
# 	 spec=PodSpec(
# 		 environment='us-east1-gcp',
# 		 pod_type='p1.x1',
# 		 pods=1
# 	 )
# )
pinecone_index = pc.Index("quickstart")

Load documents, build the PineconeVectorStore and VectorStoreIndex#

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.pinecone import PineconeVectorStore
from IPython.display import Markdown, display

Download Data

!mkdir -p 'data/paul_graham/'
!wget '' -O 'data/paul_graham/paul_graham_essay.txt'
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/home/loganm/.wget-hsts'. HSTS will be disabled.
--2024-01-16 11:56:25--
Resolving (,,, ...
Connecting to (||:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 75042 (73K) [text/plain]
Saving to: ‘data/paul_graham/paul_graham_essay.txt’

data/paul_graham/pa 100%[===================>]  73.28K  --.-KB/s    in 0.04s   

2024-01-16 11:56:25 (1.79 MB/s) - ‘data/paul_graham/paul_graham_essay.txt’ saved [75042/75042]
# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
# initialize without metadata filter
from llama_index.core import StorageContext

if "OPENAI_API_KEY" not in os.environ:
    raise EnvironmentError(f"Environment variable OPENAI_API_KEY is not set")

vector_store = PineconeVectorStore(pinecone_index=pinecone_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
INFO:httpx:HTTP Request: POST "HTTP/1.1 200 OK"
HTTP Request: POST "HTTP/1.1 200 OK"

Query Index#

May take a minute or so for the index to be ready!

# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")
INFO:httpx:HTTP Request: POST "HTTP/1.1 200 OK"
HTTP Request: POST "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST "HTTP/1.1 200 OK"
HTTP Request: POST "HTTP/1.1 200 OK"

The author, growing up, worked on writing and programming. They wrote short stories and tried writing programs on an IBM 1401 computer. They later got a microcomputer and started programming more extensively, writing simple games and a word processor.