Getting Started

Installing Vertex AI

To Install Vertex AI you need to follow the following steps

Install Vertex Cloud SDK (https://googleapis.dev/python/aiplatform/latest/index.html)
Setup your Default Project , credentials , region

Basic Usage

a Basic call to the text-bison model

from llama_index.llms.vertex import Vertex
from llama_index.llms.base import ChatMessage, MessageRole, CompletionResponse

llm = Vertex(model="text-bison", temperature=0, additional_kwargs={})
llm.complete("Hello this is a sample text").text

' ```\nHello this is a sample text\n```'

Async Usage

Async

(await llm.acomplete("hello")).text

' Hello! How can I help you?'

Streaming Usage

Streaming

list(llm.stream_complete("hello"))[-1].text

' Hello! How can I help you?'

Chat Usage

chat generation

chat = Vertex(model="chat-bison")
messages = [
    ChatMessage(role=MessageRole.SYSTEM, content="Reply everything in french"),
    ChatMessage(role=MessageRole.USER, content="Hello"),
]

chat.chat(messages=messages).message.content

' Bonjour! Comment vas-tu?'

Async Chat

Asynchronous chat response

(await chat.achat(messages=messages)).message.content

' Bonjour! Comment vas-tu?'

Streaming Chat

streaming chat response

list(chat.stream_chat(messages=messages))[-1].message.content

' Bonjour! Comment vas-tu?'