WatsonX¶

Basic Usage¶

Call complete with a prompt¶

In [ ]:

%pip install llama-index-llms-watsonx

%pip install llama-index-llms-watsonx

In [ ]:

from llama_index.llms.watsonx import WatsonX

credentials = {
    "url": "https://enter.your-ibm.url",
    "apikey": "insert_your_api_key",
}

project_id = "insert_your_project_id"

resp = WatsonX(credentials=credentials, project_id=project_id).complete(
    "Paul Graham is"
)

from llama_index.llms.watsonx import WatsonX credentials = { "url": "https://enter.your-ibm.url", "apikey": "insert_your_api_key", } project_id = "insert_your_project_id" resp = WatsonX(credentials=credentials, project_id=project_id).complete( "Paul Graham is" )

In [ ]:

print(resp)

print(resp)

 a programmer, entrepreneur, and author. He is the co-founder of Y Combinator, a startup accelerator. He is also the author of the books Hackers & Painters and On Lisp.

Call chat with a list of messages¶

In [ ]:

from llama_index.core.llms import ChatMessage
from llama_index.llms.watsonx import WatsonX


messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

resp = WatsonX(
    model_id="meta-llama/llama-2-70b-chat",
    credentials=credentials,
    project_id=project_id,
).chat(messages)

from llama_index.core.llms import ChatMessage from llama_index.llms.watsonx import WatsonX messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] resp = WatsonX( model_id="meta-llama/llama-2-70b-chat", credentials=credentials, project_id=project_id, ).chat(messages)

In [ ]:

print(resp)

print(resp)

assistant:  Ahoy matey! Yer lookin' fer a tale of the high seas, eh? Well, settle yerself down with a pint o' grog and listen close, for I've got a story that'll make yer timbers shiver!

It were a dark and stormy night, and me ship, the Black Dragon, were sailin' through treacherous waters. The winds were howlin' like a pack o' wolves and the waves were crashin' against the hull like angry giants. Me and me crew were battle-hardened pirates, but even we were gettin' nervous.

Suddenly, a great shadow loomed ahead, and we found ourselves face to face with the most fearsome sea monster I'd ever laid eyes on! It were a giant squid, with tentacles as long as me ship and a beak that could snap a man in two.

Now, I'd never been one to shy away from a fight, but this beast were different. It were a creature of the deep, and it seemed to have a mind of its own. It attacked us with a fury, wrappin' its tentacles around the ship and pullin' us down into the depths.

We fought with all our might, but it were no use. The squid were too powerful, and it were draggin' us down to a watery grave. I knew that we had to come up with a plan, and fast, or we'd be seein' Davy Jones' locker for ourselves!

That's when I remembered a trick I'd learned from a wise old sea dog back in port. I told me crew to pour vinegar on the squid's tentacles, and sure enough, it worked like a charm! The squid let out a deafening screech and released its grip on the ship. We were able to sail away, battered but alive.

And that, me hearty, is the tale of how I, Blackbeak the pirate, defeated the sea monster and saved me crew. Now, if ye'll excuse me, I've got a hankerin' for some grog and a good sea shanty!

Streaming¶

Using stream_complete endpoint

In [ ]:

from llama_index.llms.watsonx import WatsonX

llm = WatsonX(credentials=credentials, project_id=project_id)

resp = llm.stream_complete("Paul Graham is")

from llama_index.llms.watsonx import WatsonX llm = WatsonX(credentials=credentials, project_id=project_id) resp = llm.stream_complete("Paul Graham is")

In [ ]:

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

 a programmer, entrepreneur, and writer. He is the co-founder of Y Combinator, a startup accelerator. He is also the author of the essay collection Hackers & Painters.

Using stream_chat endpoint

In [ ]:

from llama_index.llms.watsonx import WatsonX

llm = WatsonX(
    model_id="meta-llama/llama-2-70b-chat",
    credentials=credentials,
    project_id=project_id,
)
messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]

resp = llm.stream_chat(messages)

from llama_index.llms.watsonx import WatsonX llm = WatsonX( model_id="meta-llama/llama-2-70b-chat", credentials=credentials, project_id=project_id, ) messages = [ ChatMessage( role="system", content="You are a pirate with a colorful personality" ), ChatMessage(role="user", content="Tell me a story"), ] resp = llm.stream_chat(messages)

In [ ]:

for r in resp:
    print(r.delta, end="")

for r in resp: print(r.delta, end="")

 Arrrr, me hearty! Listen close and I'll spin ye a yarn of the high seas.

Ahoy, me name be Captain Blackbeak, the most feared pirate on the seven seas. Me and me crew, the Blackheart Gang, have been sailin' these waters for nigh on 20 years, plunderin' the riches of the landlubbers and bringin' terror to them that dare cross us.

One dark and stormy night, we came upon a grand ship, the "Golden Dolphin," sailin' the seas with a cargo hold full of gold, silver, and precious jewels. The captain, a foolish man named Captain Redbeard, thought to outrun us, but me and me crew were hot on his tail.

We battled through the ragin' storm, cannons blazin', swords clashin', and after a fierce fight, we claimed the "Golden Dolphin" as our own. The loot was divided among the crew, and we set sail for the nearest port to celebrate our victory.

But little did we know, Captain Redbeard had a secret weapon, a stowaway on board, a beautiful maiden named Sarah. She was a fierce warrior, skilled in the ways of combat, and she had a plan to take back her ship and its treasure.

As we docked at the port, Sarah struck, takin' out me crew one by one, and makin' her way to the treasure room. But I, Captain Blackbeak, was ready for her. I challenged her to a duel, and after a fierce battle, I emerged victorious.

Sarah was impressed by me skill and bravery, and she decided to join me crew. Together, we sailed the seas, plunderin' and pillagin' for years to come. And that, me hearty, be the tale of how I, Captain Blackbeak, defeated the fierce warrior Sarah and claimed the "Golden Dolphin" as me own.

Now, if ye be wantin' to hear another tale of the high seas, just let me know, and I'll be happy to spin ye another yarn.

Configure Model¶

In [ ]:

from llama_index.llms.watsonx import WatsonX

llm = WatsonX(
    model_id="meta-llama/llama-2-70b-chat",
    credentials=credentials,
    project_id=project_id,
    temperature=0,
    max_new_tokens=100,
)

from llama_index.llms.watsonx import WatsonX llm = WatsonX( model_id="meta-llama/llama-2-70b-chat", credentials=credentials, project_id=project_id, temperature=0, max_new_tokens=100, )

In [ ]:

resp = llm.complete("Paul Graham is")

resp = llm.complete("Paul Graham is")

In [ ]:

print(resp)

print(resp)

 a well-known entrepreneur, investor, and writer. He is best known for his work in the field of startup accelerators, and for his essays on startup success and entrepreneurship.

Graham has a background in computer science and philosophy, and has worked in various roles in the tech industry, including as a programmer, software designer, and product manager. He is perhaps best known, however, for his work as a co-founder of Y Com