Open In Colab


Elasticsearch is a search database, that supports full text and vector searches.

Basic Example

In this basic example, we take the a Paul Graham essay, split it into chunks, embed it using an open-source embedding model, load it into Elasticsearch, and then query it.

If you’re opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

# !pip install llama-index elasticsearch --quiet
# !pip install sentence-transformers
# !pip install pydantic==1.10.11
# import
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.vector_stores import ElasticsearchStore
from import StorageContext
from IPython.display import Markdown, display
# set up OpenAI
import os
import getpass

os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
import openai

openai.api_key = os.environ["OPENAI_API_KEY"]

Download Data

!mkdir -p 'data/paul_graham/'
!wget '' -O 'data/paul_graham/paul_graham_essay.txt'
# define embedding function
embed_model = "local/BAAI/bge-small-en-v1.5"

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()

vector_store = ElasticsearchStore(
    index_name="paul_graham_essay", es_url="http://localhost:9200"
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, service_context=service_context

# Query Data
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")

The author worked on writing and programming outside of school. They wrote short stories and tried writing programs on an IBM 1401 computer. They also built a microcomputer kit and started programming on it, writing simple games and a word processor.