Tair Vector Store

In this notebook we are going to show a quick demo of using the TairVectorStore.

If you’re opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

!pip install llama-index

import os
import sys
import logging
import textwrap

import warnings

warnings.filterwarnings("ignore")

# stop huggingface warnings
os.environ["TOKENIZERS_PARALLELISM"] = "false"

# Uncomment to see debug logs
# logging.basicConfig(stream=sys.stdout, level=logging.INFO)
# logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, Document
from llama_index.vector_stores import TairVectorStore
from IPython.display import Markdown, display

Setup OpenAI

Lets first begin by adding the openai api key. This will allow us to access openai for embeddings and to use chatgpt.

import os

os.environ["OPENAI_API_KEY"] = "sk-<your key here>"

Download Data

!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'

Read in a dataset

# load documents
documents = SimpleDirectoryReader("./data/paul_graham").load_data()
print(
    "Document ID:",
    documents[0].doc_id,
    "Document Hash:",
    documents[0].doc_hash,
)

Build index from documents

Let’s build a vector index with GPTVectorStoreIndex, using TairVectorStore as its backend. Replace tair_url with the actual url of your Tair instance.

from llama_index.storage.storage_context import StorageContext

tair_url = "redis://{username}:{password}@r-bp****************.redis.rds.aliyuncs.com:{port}"

vector_store = TairVectorStore(
    tair_url=tair_url, index_name="pg_essays", overwrite=True
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = GPTVectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

Query the data

Now we can use the index as knowledge base and ask questions to it.

query_engine = index.as_query_engine()
response = query_engine.query("What did the author learn?")
print(textwrap.fill(str(response), 100))

response = query_engine.query("What was a hard moment for the author?")
print(textwrap.fill(str(response), 100))

Deleting documents

To delete a document from the index, use delete method.

document_id = documents[0].doc_id
document_id

info = vector_store.client.tvs_get_index("pg_essays")
print("Number of documents", int(info["data_count"]))

vector_store.delete(document_id)

info = vector_store.client.tvs_get_index("pg_essays")
print("Number of documents", int(info["data_count"]))

Deleting index

Delete the entire index using delete_index method.

vector_store.delete_index()

print("Check index existence:", vector_store.client._index_exists())