OpenAILike

pydantic model llama_index.llms.openai_like.OpenAILike

OpenAILike is a thin wrapper around the OpenAI model that makes it compatible with 3rd party tools that provide an openai-compatible api.

Currently, llama_index prevents using custom models with their OpenAI class because they need to be able to infer some metadata from the model name.

NOTE: You still need to set the OPENAI_BASE_API and OPENAI_API_KEY environment variables or the api_key and api_base constructor arguments. OPENAI_API_KEY/api_key can normally be set to anything in this case, but will depend on the tool you’re using.

Show JSON schema

{
   "title": "OpenAILike",
   "description": "OpenAILike is a thin wrapper around the OpenAI model that makes it compatible with\n3rd party tools that provide an openai-compatible api.\n\nCurrently, llama_index prevents using custom models with their OpenAI class\nbecause they need to be able to infer some metadata from the model name.\n\nNOTE: You still need to set the OPENAI_BASE_API and OPENAI_API_KEY environment\nvariables or the api_key and api_base constructor arguments.\nOPENAI_API_KEY/api_key can normally be set to anything in this case,\nbut will depend on the tool you're using.",
   "type": "object",
   "properties": {
      "callback_manager": {
         "title": "Callback Manager"
      },
      "system_prompt": {
         "title": "System Prompt",
         "description": "System prompt for LLM calls.",
         "type": "string"
      },
      "messages_to_prompt": {
         "title": "Messages To Prompt"
      },
      "completion_to_prompt": {
         "title": "Completion To Prompt"
      },
      "output_parser": {
         "title": "Output Parser"
      },
      "pydantic_program_mode": {
         "default": "default",
         "allOf": [
            {
               "$ref": "#/definitions/PydanticProgramMode"
            }
         ]
      },
      "query_wrapper_prompt": {
         "title": "Query Wrapper Prompt"
      },
      "model": {
         "title": "Model",
         "description": "The OpenAI model to use.",
         "default": "gpt-3.5-turbo",
         "type": "string"
      },
      "temperature": {
         "title": "Temperature",
         "description": "The temperature to use during generation.",
         "default": 0.1,
         "gte": 0.0,
         "lte": 1.0,
         "type": "number"
      },
      "max_tokens": {
         "title": "Max Tokens",
         "description": "The maximum number of tokens to generate.",
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "additional_kwargs": {
         "title": "Additional Kwargs",
         "description": "Additional kwargs for the OpenAI API.",
         "type": "object"
      },
      "max_retries": {
         "title": "Max Retries",
         "description": "The maximum number of API retries.",
         "default": 3,
         "gte": 0,
         "type": "integer"
      },
      "timeout": {
         "title": "Timeout",
         "description": "The timeout, in seconds, for API requests.",
         "default": 60.0,
         "gte": 0,
         "type": "number"
      },
      "default_headers": {
         "title": "Default Headers",
         "description": "The default headers for API requests.",
         "type": "object",
         "additionalProperties": {
            "type": "string"
         }
      },
      "reuse_client": {
         "title": "Reuse Client",
         "description": "Reuse the OpenAI client between requests. When doing anything with large volumes of async API calls, setting this to false can improve stability.",
         "default": true,
         "type": "boolean"
      },
      "api_key": {
         "title": "Api Key",
         "description": "The OpenAI API key.",
         "type": "string"
      },
      "api_base": {
         "title": "Api Base",
         "description": "The base URL for OpenAI API.",
         "type": "string"
      },
      "api_version": {
         "title": "Api Version",
         "description": "The API version for OpenAI API.",
         "type": "string"
      },
      "context_window": {
         "title": "Context Window",
         "description": "Total number of tokens the model can be input and output for one response.",
         "default": 3900,
         "type": "integer"
      },
      "is_chat_model": {
         "title": "Is Chat Model",
         "description": "Set True if the model exposes a chat interface (i.e. can be passed a sequence of messages, rather than text), like OpenAI's /v1/chat/completions endpoint.",
         "default": false,
         "type": "boolean"
      },
      "is_function_calling_model": {
         "title": "Is Function Calling Model",
         "description": "Set True if the model supports function calling messages, similar to OpenAI's function calling API. For example, converting 'Email Anya to see if she wants to get coffee next Friday' to a function call like `send_email(to: string, body: string)`.",
         "default": false,
         "type": "boolean"
      },
      "tokenizer": {
         "title": "Tokenizer"
      },
      "class_name": {
         "title": "Class Name",
         "type": "string",
         "default": "OpenAILike"
      }
   },
   "required": [
      "api_base",
      "api_version"
   ],
   "definitions": {
      "PydanticProgramMode": {
         "title": "PydanticProgramMode",
         "description": "Pydantic program mode.",
         "enum": [
            "default",
            "openai",
            "llm",
            "guidance",
            "lm-format-enforcer"
         ],
         "type": "string"
      }
   }
}

Config

• arbitrary_types_allowed: bool = True

Fields

• context_window (int)

• is_chat_model (bool)

• is_function_calling_model (bool)

• tokenizer (Optional[Union[llama_index.llms.openai.Tokenizer, str]])

Validators

• _validate_callback_manager » callback_manager

• set_completion_to_prompt » completion_to_prompt

• set_messages_to_prompt » messages_to_prompt

field context_window: int = 3900

Total number of tokens the model can be input and output for one response.

field is_chat_model: bool = False

Set True if the model exposes a chat interface (i.e. can be passed a sequence of messages, rather than text), like OpenAI’s /v1/chat/completions endpoint.

field is_function_calling_model: bool = False

Set True if the model supports function calling messages, similar to OpenAI’s function calling API. For example, converting ‘Email Anya to see if she wants to get coffee next Friday’ to a function call like send_email(to: string, body: string).

field tokenizer: Optional[Union[Tokenizer, str]] = None

An instance of a tokenizer object that has an encode method, or the name of a tokenizer model from Hugging Face. If left as None, then this disables inference of max_tokens.

async achat(messages: Sequence[ChatMessage], **kwargs: Any) → ChatResponse

Chat with the model.

async acomplete(prompt: str, formatted: bool = False, **kwargs: Any) → CompletionResponse

Complete the prompt.

async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) → AsyncGenerator[ChatResponse, None]

Async streaming chat endpoint for LLM.

async astream_complete(prompt: str, formatted: bool = False, **kwargs: Any) → AsyncGenerator[CompletionResponse, None]

Stream complete the prompt.

chat(messages: Sequence[ChatMessage], **kwargs: Any) → ChatResponse

Chat with the model.

classmethod class_name() → str

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

complete(prompt: str, formatted: bool = False, **kwargs: Any) → CompletionResponse

Complete the prompt.

stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) → Generator[ChatResponse, None, None]

Streaming chat endpoint for LLM.

stream_complete(prompt: str, formatted: bool = False, **kwargs: Any) → Generator[CompletionResponse, None, None]

Stream complete the prompt.

property metadata: LLMMetadata

LLM metadata.