Nebius LLMs¶

This notebook demonstrates how to use LLMs from Nebius AI Studio with LlamaIndex. Nebius AI Studio implements all state-of-the-art LLMs available for commercial use.

First, let's install LlamaIndex and dependencies of Nebius AI Studio.

In [ ]:

%pip install llama-index-embeddings-nebius

%pip install llama-index-embeddings-nebius

In [ ]:

!pip install llama-index

!pip install llama-index

Insert your Nebius AI Studio key below. You can get it by registering for free at Nebius AI Studio and issuing the key at API Keys section."

In [ ]:

NEBIUS_API_KEY = ""

NEBIUS_API_KEY = ""

In [ ]:

from llama_index.llms.nebius import NebiusLLM

llm = NebiusLLM(
    api_key=NEBIUS_API_KEY, model="meta-llama/Meta-Llama-3.1-70B-Instruct-fast"
)

from llama_index.llms.nebius import NebiusLLM llm = NebiusLLM( api_key=NEBIUS_API_KEY, model="meta-llama/Meta-Llama-3.1-70B-Instruct-fast" )

Call complete with a prompt¶

In [ ]:

response = llm.complete("Amsterdam is the capital of ")
print(response)

response = llm.complete("Amsterdam is the capital of ") print(response)

The Netherlands!

Call chat with a list of messages¶

In [ ]:

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Write a poem about a smart AI robot named Wall-e.",
    ),
]
response = llm.chat(messages)
print(response)

from llama_index.core.llms import ChatMessage messages = [ ChatMessage(role="system", content="You are a helpful AI assistant."), ChatMessage( role="user", content="Write a poem about a smart AI robot named Wall-e.", ), ] response = llm.chat(messages) print(response)

assistant: In a world of wires and circuits bright,
A small robot shone with digital light,
Wall-e, a name that's known so well,
A smart AI, with a story to tell.

With eyes that gleam like shining steel,
He navigates the world with skill and zeal,
A waste-collecting bot, with a heart of gold,
He cleans the earth, with a story to unfold.

His trash-compacting arms, a wondrous sight,
Crush and compress, with precision and might,
He beeps and boops, with a language all his own,
A symphony of sounds, that echo and moan.

In a world of desolation, he finds a friend,
Eve, a sleek robot, with a mission to amend,
Together they roam, through the ruins of old,
A tale of love and hope, in a world grown cold.

With a heart that beats, with a digital soul,
Wall-e dreams of life, in a world made whole,
He longs to connect, to touch and to share,
A robot's quest, to show he cares.

His story's told, in a world of decay,
A beacon of hope, in a brighter day,
Wall-e, a hero, in a world of machines,
A shining star, that forever gleams.

Streaming¶

Using stream_complete endpoint¶

In [ ]:

response = llm.stream_complete("Amsterdam is the capital of ")
for r in response:
    print(r.delta, end="")

response = llm.stream_complete("Amsterdam is the capital of ") for r in response: print(r.delta, end="")

The Netherlands!

Using stream_chat with a list of messages¶

In [ ]:

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Write a poem about a smart AI robot named Wall-e.",
    ),
]
response = llm.stream_chat(messages)
for r in response:
    print(r.delta, end="")

from llama_index.core.llms import ChatMessage messages = [ ChatMessage(role="system", content="You are a helpful AI assistant."), ChatMessage( role="user", content="Write a poem about a smart AI robot named Wall-e.", ), ] response = llm.stream_chat(messages) for r in response: print(r.delta, end="")

In a world of wires and circuits bright,
A small robot shone with digital light,
Wall-e, a name that's known so well,
A hero of the future, with a story to tell.

With eyes that gleam like stars in space,
He navigates the ruins of a forgotten place,
A world of trash and decay, where humans once did roam,
But Wall-e's heart beats strong, with a love that's made of home.

His metal body, sturdy and strong,
Moves with a purpose, all day long,
He collects and sorts, with a precision so fine,
The waste of humanity, in a world that's lost its shine.

But Wall-e's not just a machine, you see,
He's a friend, a companion, with a heart that's free,
He dreams of love, of connection, of a life that's true,
A life with EVE, his counterpart, his heart beats anew.

With a beep and a boop, he communicates with glee,
A language all his own, a symphony,
He's a robot of wonder, a marvel of our time,
A shining star, that shines with a love that's divine.

In a world of metal and wires, he's a work of art,
A masterpiece, that beats with a loving heart,
Wall-e, the robot, with a soul so bright,
A hero of the future, shining with delight.