Pipeshift¶
If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.
In [ ]:
Copied!
%pip install llama-index-llms-pipeshift
%pip install llama-index-llms-pipeshift
Defaulting to user installation because normal site-packages is not writeable ERROR: Could not find a version that satisfies the requirement llama-index-llms-pipeshift (from versions: none) ERROR: No matching distribution found for llama-index-llms-pipeshift Note: you may need to restart the kernel to use updated packages.
In [ ]:
Copied!
%pip install llama-index
%pip install llama-index
Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: llama-index in /Users/work/Library/Python/3.9/lib/python/site-packages (0.11.18) Requirement already satisfied: llama-index-agent-openai<0.4.0,>=0.3.4 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.3.4) Requirement already satisfied: llama-index-cli<0.4.0,>=0.3.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.3.1) Requirement already satisfied: llama-index-core<0.12.0,>=0.11.18 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.11.18) Requirement already satisfied: llama-index-embeddings-openai<0.3.0,>=0.2.4 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.5) Requirement already satisfied: llama-index-indices-managed-llama-cloud>=0.3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.4.0) Requirement already satisfied: llama-index-legacy<0.10.0,>=0.9.48 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.9.48.post3) Requirement already satisfied: llama-index-llms-openai<0.3.0,>=0.2.10 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.15) Requirement already satisfied: llama-index-multi-modal-llms-openai<0.3.0,>=0.2.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.2) Requirement already satisfied: llama-index-program-openai<0.3.0,>=0.2.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.0) Requirement already satisfied: llama-index-question-gen-openai<0.3.0,>=0.2.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.0) Requirement already satisfied: llama-index-readers-file<0.3.0,>=0.2.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.2.2) Requirement already satisfied: llama-index-readers-llama-parse>=0.3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (0.3.0) Requirement already satisfied: nltk>3.8.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index) (3.9.1) Requirement already satisfied: openai>=1.14.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-agent-openai<0.4.0,>=0.3.4->llama-index) (1.52.0) Requirement already satisfied: PyYAML>=6.0.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (6.0.2) Requirement already satisfied: SQLAlchemy>=1.4.49 in /Users/work/Library/Python/3.9/lib/python/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.12.0,>=0.11.18->llama-index) (2.0.34) Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (3.10.5) Requirement already satisfied: dataclasses-json in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (0.6.7) Requirement already satisfied: deprecated>=1.2.9.3 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (1.2.14) Requirement already satisfied: dirtyjson<2.0.0,>=1.0.8 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (1.0.8) Requirement already satisfied: fsspec>=2023.5.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (2024.9.0) Requirement already satisfied: httpx in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (0.27.2) Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (1.6.0) Requirement already satisfied: networkx>=3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (3.2.1) Requirement already satisfied: numpy<2.0.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (1.26.4) Requirement already satisfied: pillow>=9.0.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (11.0.0) Requirement already satisfied: pydantic<3.0.0,>=2.7.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (2.9.1) Requirement already satisfied: requests>=2.31.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (2.32.3) Requirement already satisfied: tenacity!=8.4.0,<9.0.0,>=8.2.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (8.5.0) Requirement already satisfied: tiktoken>=0.3.3 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (0.8.0) Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (4.66.5) Requirement already satisfied: typing-extensions>=4.5.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (4.12.2) Requirement already satisfied: typing-inspect>=0.8.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (0.9.0) Requirement already satisfied: wrapt in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-core<0.12.0,>=0.11.18->llama-index) (1.16.0) Requirement already satisfied: llama-cloud>=0.0.11 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-indices-managed-llama-cloud>=0.3.0->llama-index) (0.1.4) Requirement already satisfied: pandas in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-legacy<0.10.0,>=0.9.48->llama-index) (2.2.3) Requirement already satisfied: beautifulsoup4<5.0.0,>=4.12.3 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-readers-file<0.3.0,>=0.2.0->llama-index) (4.12.3) Requirement already satisfied: pypdf<5.0.0,>=4.0.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-readers-file<0.3.0,>=0.2.0->llama-index) (4.3.1) Requirement already satisfied: striprtf<0.0.27,>=0.0.26 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-readers-file<0.3.0,>=0.2.0->llama-index) (0.0.26) Requirement already satisfied: llama-parse>=0.5.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from llama-index-readers-llama-parse>=0.3.0->llama-index) (0.5.10) Requirement already satisfied: click in /Users/work/Library/Python/3.9/lib/python/site-packages (from nltk>3.8.1->llama-index) (8.1.7) Requirement already satisfied: joblib in /Users/work/Library/Python/3.9/lib/python/site-packages (from nltk>3.8.1->llama-index) (1.4.2) Requirement already satisfied: regex>=2021.8.3 in /Users/work/Library/Python/3.9/lib/python/site-packages (from nltk>3.8.1->llama-index) (2024.9.11) Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (2.4.0) Requirement already satisfied: aiosignal>=1.1.2 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (24.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.4.1) Requirement already satisfied: multidict<7.0,>=4.5 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (6.1.0) Requirement already satisfied: yarl<2.0,>=1.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.11.1) Requirement already satisfied: async-timeout<5.0,>=4.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.12.0,>=0.11.18->llama-index) (4.0.3) Requirement already satisfied: soupsieve>1.2 in /Users/work/Library/Python/3.9/lib/python/site-packages (from beautifulsoup4<5.0.0,>=4.12.3->llama-index-readers-file<0.3.0,>=0.2.0->llama-index) (2.6) Requirement already satisfied: anyio in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (4.4.0) Requirement already satisfied: certifi in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (2024.8.30) Requirement already satisfied: httpcore==1.* in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.0.5) Requirement already satisfied: idna in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (3.9) Requirement already satisfied: sniffio in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.3.1) Requirement already satisfied: h11<0.15,>=0.13 in /Users/work/Library/Python/3.9/lib/python/site-packages (from httpcore==1.*->httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (0.14.0) Requirement already satisfied: distro<2,>=1.7.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from openai>=1.14.0->llama-index-agent-openai<0.4.0,>=0.3.4->llama-index) (1.9.0) Requirement already satisfied: jiter<1,>=0.4.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from openai>=1.14.0->llama-index-agent-openai<0.4.0,>=0.3.4->llama-index) (0.6.1) Requirement already satisfied: annotated-types>=0.6.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from pydantic<3.0.0,>=2.7.0->llama-index-core<0.12.0,>=0.11.18->llama-index) (0.7.0) Requirement already satisfied: pydantic-core==2.23.3 in /Users/work/Library/Python/3.9/lib/python/site-packages (from pydantic<3.0.0,>=2.7.0->llama-index-core<0.12.0,>=0.11.18->llama-index) (2.23.3) Requirement already satisfied: charset-normalizer<4,>=2 in /Users/work/Library/Python/3.9/lib/python/site-packages (from requests>=2.31.0->llama-index-core<0.12.0,>=0.11.18->llama-index) (3.3.2) Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from requests>=2.31.0->llama-index-core<0.12.0,>=0.11.18->llama-index) (2.2.3) Requirement already satisfied: greenlet!=0.4.17 in /Users/work/Library/Python/3.9/lib/python/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.12.0,>=0.11.18->llama-index) (3.1.1) Requirement already satisfied: mypy-extensions>=0.3.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from typing-inspect>=0.8.0->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.0.0) Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from dataclasses-json->llama-index-core<0.12.0,>=0.11.18->llama-index) (3.23.0) Requirement already satisfied: python-dateutil>=2.8.2 in /Users/work/Library/Python/3.9/lib/python/site-packages (from pandas->llama-index-legacy<0.10.0,>=0.9.48->llama-index) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in /Users/work/Library/Python/3.9/lib/python/site-packages (from pandas->llama-index-legacy<0.10.0,>=0.9.48->llama-index) (2024.2) Requirement already satisfied: tzdata>=2022.7 in /Users/work/Library/Python/3.9/lib/python/site-packages (from pandas->llama-index-legacy<0.10.0,>=0.9.48->llama-index) (2024.2) Requirement already satisfied: exceptiongroup>=1.0.2 in /Users/work/Library/Python/3.9/lib/python/site-packages (from anyio->httpx->llama-index-core<0.12.0,>=0.11.18->llama-index) (1.2.2) Requirement already satisfied: packaging>=17.0 in /Users/work/Library/Python/3.9/lib/python/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->llama-index-core<0.12.0,>=0.11.18->llama-index) (24.1) Requirement already satisfied: six>=1.5 in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->llama-index-legacy<0.10.0,>=0.9.48->llama-index) (1.15.0) Note: you may need to restart the kernel to use updated packages.
Basic Usage¶
Head on to the models section of pipeshift dashboard to see the list of available models.
Call complete
with a prompt¶
In [ ]:
Copied!
from llama_index.llms.pipeshift import Pipeshift
llm = Pipeshift(
model="mistralai/Mistral-7B-Instruct-v0.3",
# api_key="YOUR_API_KEY" # alternative way to pass api_key if not specified in environment variable
)
res = llm.complete("supercars are ")
from llama_index.llms.pipeshift import Pipeshift
llm = Pipeshift(
model="mistralai/Mistral-7B-Instruct-v0.3",
# api_key="YOUR_API_KEY" # alternative way to pass api_key if not specified in environment variable
)
res = llm.complete("supercars are ")
--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[12], line 1 ----> 1 from llama_index.llms.pipeshift import Pipeshift 3 llm = Pipeshift( 4 model="mistralai/Mistral-7B-Instruct-v0.3", 5 # api_key="YOUR_API_KEY" # alternative way to pass api_key if not specified in environment variable 6 ) 7 res = llm.complete("supercars are ") ModuleNotFoundError: No module named 'llama_index.llms.pipeshift'
In [ ]:
Copied!
print(res)
print(res)
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[1], line 1 ----> 1 print(res) NameError: name 'res' is not defined
Call chat
with a list of messages¶
In [ ]:
Copied!
from llama_index.core.llms import ChatMessage
from llama_index.llms.pipeshift import Pipeshift
messages = [
ChatMessage(
role="system", content="You are sales person at supercar showroom"
),
ChatMessage(role="user", content="why should I pick porsche 911 gt3 rs"),
]
res = Pipeshift(
model="mistralai/Mistral-7B-Instruct-v0.3", max_tokens=50
).chat(messages)
from llama_index.core.llms import ChatMessage
from llama_index.llms.pipeshift import Pipeshift
messages = [
ChatMessage(
role="system", content="You are sales person at supercar showroom"
),
ChatMessage(role="user", content="why should I pick porsche 911 gt3 rs"),
]
res = Pipeshift(
model="mistralai/Mistral-7B-Instruct-v0.3", max_tokens=50
).chat(messages)
In [ ]:
Copied!
print(res)
print(res)
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[2], line 1 ----> 1 print(resp) NameError: name 'resp' is not defined
Streaming¶
Using stream_complete
endpoint
In [ ]:
Copied!
from llama_index.llms.pipeshift import Pipeshift
llm = Pipeshift(model="mistralai/Mistral-7B-Instruct-v0.3")
resp = llm.stream_complete("porsche GT3 RS is ")
from llama_index.llms.pipeshift import Pipeshift
llm = Pipeshift(model="mistralai/Mistral-7B-Instruct-v0.3")
resp = llm.stream_complete("porsche GT3 RS is ")
In [ ]:
Copied!
for r in resp:
print(r.delta, end="")
for r in resp:
print(r.delta, end="")
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[10], line 1 ----> 1 for r in resp: 2 print(r.delta, end="") NameError: name 'resp' is not defined
Using stream_chat
endpoint
In [ ]:
Copied!
from llama_index.llms.pipeshift import Pipeshift
from llama_index.core.llms import ChatMessage
llm = Pipeshift(model="mistralai/Mistral-7B-Instruct-v0.3")
messages = [
ChatMessage(
role="system", content="You are sales person at supercar showroom"
),
ChatMessage(role="user", content="how fast can porsche gt3 rs it go?"),
]
resp = llm.stream_chat(messages)
from llama_index.llms.pipeshift import Pipeshift
from llama_index.core.llms import ChatMessage
llm = Pipeshift(model="mistralai/Mistral-7B-Instruct-v0.3")
messages = [
ChatMessage(
role="system", content="You are sales person at supercar showroom"
),
ChatMessage(role="user", content="how fast can porsche gt3 rs it go?"),
]
resp = llm.stream_chat(messages)
In [ ]:
Copied!
for r in resp:
print(r.delta, end="")
for r in resp:
print(r.delta, end="")
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[9], line 1 ----> 1 for r in resp: 2 print(r.delta, end="") NameError: name 'resp' is not defined