Replicate

pydantic model llama_index.llms.replicate.Replicate

Show JSON schema

{
   "title": "Replicate",
   "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `_complete`,\n    `_stream_complete`, and `metadata` methods.",
   "type": "object",
   "properties": {
      "callback_manager": {
         "title": "Callback Manager"
      },
      "system_prompt": {
         "title": "System Prompt",
         "description": "System prompt for LLM calls.",
         "type": "string"
      },
      "messages_to_prompt": {
         "title": "Messages To Prompt"
      },
      "completion_to_prompt": {
         "title": "Completion To Prompt"
      },
      "output_parser": {
         "title": "Output Parser"
      },
      "pydantic_program_mode": {
         "default": "default",
         "allOf": [
            {
               "$ref": "#/definitions/PydanticProgramMode"
            }
         ]
      },
      "query_wrapper_prompt": {
         "title": "Query Wrapper Prompt"
      },
      "model": {
         "title": "Model",
         "description": "The Replicate model to use.",
         "type": "string"
      },
      "temperature": {
         "title": "Temperature",
         "description": "The temperature to use for sampling.",
         "default": 0.75,
         "gte": 0.01,
         "lte": 1.0,
         "type": "number"
      },
      "image": {
         "title": "Image",
         "description": "The image file for multimodal model to use. (optional)",
         "default": "",
         "type": "string"
      },
      "context_window": {
         "title": "Context Window",
         "description": "The maximum number of context tokens for the model.",
         "default": 3900,
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "prompt_key": {
         "title": "Prompt Key",
         "description": "The key to use for the prompt in API calls.",
         "default": "prompt",
         "type": "string"
      },
      "additional_kwargs": {
         "title": "Additional Kwargs",
         "description": "Additional kwargs for the Replicate API.",
         "type": "object"
      },
      "is_chat_model": {
         "title": "Is Chat Model",
         "description": "Whether the model is a chat model.",
         "default": false,
         "type": "boolean"
      },
      "class_name": {
         "title": "Class Name",
         "type": "string",
         "default": "Replicate_llm"
      }
   },
   "required": [
      "model"
   ],
   "definitions": {
      "PydanticProgramMode": {
         "title": "PydanticProgramMode",
         "description": "Pydantic program mode.",
         "enum": [
            "default",
            "openai",
            "llm",
            "guidance",
            "lm-format-enforcer"
         ],
         "type": "string"
      }
   }
}

Config

• arbitrary_types_allowed: bool = True

Fields

• additional_kwargs (Dict[str, Any])

• context_window (int)

• image (str)

• is_chat_model (bool)

• model (str)

• prompt_key (str)

• temperature (float)

Validators

• _validate_callback_manager » callback_manager

• set_completion_to_prompt » completion_to_prompt

• set_messages_to_prompt » messages_to_prompt

field additional_kwargs: Dict[str, Any] [Optional]

Additional kwargs for the Replicate API.

field context_window: int = 3900

The maximum number of context tokens for the model.

Constraints

• exclusiveMinimum = 0

field image: str = ''

The image file for multimodal model to use. (optional)

field is_chat_model: bool = False

Whether the model is a chat model.

field model: str [Required]

The Replicate model to use.

field prompt_key: str = 'prompt'

The key to use for the prompt in API calls.

field temperature: float = 0.75

The temperature to use for sampling.

chat(messages: Sequence[ChatMessage], **kwargs: Any) → Any

Chat endpoint for LLM.

classmethod class_name() → str

Get the class name, used as a unique ID in serialization.

This provides a key that makes serialization robust against actual class name changes.

complete(*args: Any, **kwargs: Any) → Any

Completion endpoint for LLM.

stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) → Any

Streaming chat endpoint for LLM.

stream_complete(*args: Any, **kwargs: Any) → Any

Streaming completion endpoint for LLM.

property metadata: LLMMetadata

LLM metadata.