Callbacks#

class llama_index.callbacks.AimCallback(repo: Optional[str] = None, experiment_name: Optional[str] = None, system_tracking_interval: Optional[int] = 1, log_system_params: Optional[bool] = True, capture_terminal_logs: Optional[bool] = True, event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None, run_params: Optional[Dict[str, Any]] = None)#

AimCallback callback class.

Parameters

repo (str, optional) – Aim repository path or Repo object to which Run object is bound. If skipped, default Repo is used.
experiment_name (str, optional) – Sets Run’s experiment property. ‘default’ if not specified. Can be used later to query runs/sequences.
system_tracking_interval (int, optional) – Sets the tracking interval in seconds for system usage metrics (CPU, Memory, etc.). Set to None to disable system metrics tracking.
log_system_params (bool, optional) – Enable/Disable logging of system params such as installed packages, git info, environment variables, etc.
capture_terminal_logs (bool, optional) – Enable/Disable terminal stdout logging.
event_starts_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event starts.
event_ends_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event ends.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.
parent_id (str) – parent event id.

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

class llama_index.callbacks.CBEvent(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, time: str = '', id_: str = '')#: Generic class to store event information.

class llama_index.callbacks.CBEventType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

Callback manager event types.

CHUNKING#: Logs for the before and after of text splitting.

NODE_PARSING#: Logs for the documents and the nodes that they are parsed into.

EMBEDDING#: Logs for the number of texts embedded.

LLM#: Logs for the template and response of LLM calls.

QUERY#: Keeps track of the start and end of each query.

RETRIEVE#: Logs for the nodes retrieved for a query.

SYNTHESIZE#: Logs for the result for synthesize calls.

TREE#: Logs for the summary and level of summaries generated.

SUB_QUESTION#: Logs for a generated sub question and answer.

capitalize()#

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()#: Return a version of the string suitable for caseless comparisons.

center(width, fillchar=' ', /)#

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count(sub[, start[, end]]) → int#: Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

encode(encoding='utf-8', errors='strict')#

Encode the string using the codec registered for encoding.

encoding: The encoding in which to encode the string.
errors: The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

endswith(suffix[, start[, end]]) → bool#: Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

expandtabs(tabsize=8)#

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

format(*args, **kwargs) → str#: Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

format_map(mapping) → str#: Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{‘ and ‘}’).

index(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

isalnum()#

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isalpha()#

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isascii()#

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

isdecimal()#

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()#

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isidentifier()#

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

islower()#

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isnumeric()#

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isprintable()#

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

isspace()#

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

istitle()#

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isupper()#

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

join(iterable, /)#

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

ljust(width, fillchar=' ', /)#

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()#: Return a copy of the string converted to lowercase.

lstrip(chars=None, /)#

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

static maketrans()#

Return a translation table usable for str.translate().

If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.

partition(sep, /)#

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

removeprefix(prefix, /)#

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)#

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

replace(old, new, count=-1, /)#

Return a copy with all occurrences of substring old replaced by new.

count
Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

rfind(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)#

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rpartition(sep, /)#

Partition the string into three parts using the given separator.

This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

rsplit(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

rstrip(chars=None, /)#

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

split(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

splitlines(keepends=False)#

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

startswith(prefix[, start[, end]]) → bool#: Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

strip(chars=None, /)#

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()#: Convert uppercase characters to lowercase and lowercase characters to uppercase.

title()#

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

translate(table, /)#

Replace each character in the string using the given translation table.

table
Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.

upper()#: Return a copy of the string converted to uppercase.

zfill(width, /)#

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

class llama_index.callbacks.CallbackManager(handlers: Optional[List[BaseCallbackHandler]] = None)#

Callback manager that handles callbacks for events within LlamaIndex.

The callback manager provides a way to call handlers on event starts/ends.

Additionally, the callback manager traces the current stack of events. It does this by using a few key attributes. - trace_stack - The current stack of events that have not ended yet.

When an event ends, it’s removed from the stack. Since this is a contextvar, it is unique to each thread/task.

trace_map - A mapping of event ids to their children events.
On the start of events, the bottom of the trace stack is used as the current parent event for the trace map.
trace_id - A simple name for the current trace, usually denoting the
entrypoint (query, index_construction, insert, etc.)

Parameters: handlers (List[BaseCallbackHandler]) – list of handlers to use.

Usage:

with callback_manager.event(CBEventType.QUERY) as event:: event.on_start(payload={key, val}) … event.on_end(payload={key, val})

add_handler(handler: BaseCallbackHandler) → None#: Add a handler to the callback manager.

as_trace(trace_id: str) → Generator[None, None, None]#: Context manager tracer for lanching and shutdown of traces.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

event(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: Optional[str] = None) → Generator[EventContext, None, None]#

Context manager for lanching and shutdown of events.

Handles sending on_evnt_start and on_event_end to handlers for specified event.

Usage:

with callback_manager.event(CBEventType.QUERY, payload={key, val}) as event:: … event.on_end(payload={key, val}) # optional

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: Optional[str] = None, **kwargs: Any) → None#: Run handlers when an event ends.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: Optional[str] = None, parent_id: Optional[str] = None, **kwargs: Any) → str#: Run handlers when an event starts and return id of event.

remove_handler(handler: BaseCallbackHandler) → None#: Remove a handler from the callback manager.

set_handlers(handlers: List[BaseCallbackHandler]) → None#: Set handlers as the only handlers on the callback manager.

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

class llama_index.callbacks.EventPayload(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

capitalize()#

Return a capitalized version of the string.

More specifically, make the first character have upper case and the rest lower case.

casefold()#: Return a version of the string suitable for caseless comparisons.

center(width, fillchar=' ', /)#

Return a centered string of length width.

Padding is done using the specified fill character (default is a space).

count(sub[, start[, end]]) → int#: Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.

encode(encoding='utf-8', errors='strict')#

Encode the string using the codec registered for encoding.

encoding: The encoding in which to encode the string.
errors: The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.

endswith(suffix[, start[, end]]) → bool#: Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.

expandtabs(tabsize=8)#

Return a copy where all tab characters are expanded using spaces.

If tabsize is not given, a tab size of 8 characters is assumed.

find(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

format(*args, **kwargs) → str#: Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{‘ and ‘}’).

format_map(mapping) → str#: Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{‘ and ‘}’).

index(sub[, start[, end]]) → int#

Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

isalnum()#

Return True if the string is an alpha-numeric string, False otherwise.

A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.

isalpha()#

Return True if the string is an alphabetic string, False otherwise.

A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.

isascii()#

Return True if all characters in the string are ASCII, False otherwise.

ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.

isdecimal()#

Return True if the string is a decimal string, False otherwise.

A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.

isdigit()#

Return True if the string is a digit string, False otherwise.

A string is a digit string if all characters in the string are digits and there is at least one character in the string.

isidentifier()#

Return True if the string is a valid Python identifier, False otherwise.

Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.

islower()#

Return True if the string is a lowercase string, False otherwise.

A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.

isnumeric()#

Return True if the string is a numeric string, False otherwise.

A string is numeric if all characters in the string are numeric and there is at least one character in the string.

isprintable()#

Return True if the string is printable, False otherwise.

A string is printable if all of its characters are considered printable in repr() or if it is empty.

isspace()#

Return True if the string is a whitespace string, False otherwise.

A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.

istitle()#

Return True if the string is a title-cased string, False otherwise.

In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.

isupper()#

Return True if the string is an uppercase string, False otherwise.

A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.

join(iterable, /)#

Concatenate any number of strings.

The string whose method is called is inserted in between each given string. The result is returned as a new string.

Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’

ljust(width, fillchar=' ', /)#

Return a left-justified string of length width.

Padding is done using the specified fill character (default is a space).

lower()#: Return a copy of the string converted to lowercase.

lstrip(chars=None, /)#

Return a copy of the string with leading whitespace removed.

If chars is given and not None, remove characters in chars instead.

static maketrans()#

Return a translation table usable for str.translate().

partition(sep, /)#

Partition the string into three parts using the given separator.

This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.

If the separator is not found, returns a 3-tuple containing the original string and two empty strings.

removeprefix(prefix, /)#

Return a str with the given prefix string removed if present.

If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.

removesuffix(suffix, /)#

Return a str with the given suffix string removed if present.

If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.

replace(old, new, count=-1, /)#

Return a copy with all occurrences of substring old replaced by new.

count
Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.

If the optional argument count is given, only the first count occurrences are replaced.

rfind(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Return -1 on failure.

rindex(sub[, start[, end]]) → int#

Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.

Raises ValueError when the substring is not found.

rjust(width, fillchar=' ', /)#

Return a right-justified string of length width.

Padding is done using the specified fill character (default is a space).

rpartition(sep, /)#

Partition the string into three parts using the given separator.

If the separator is not found, returns a 3-tuple containing two empty strings and the original string.

rsplit(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Splitting starts at the end of the string and works to the front.

rstrip(chars=None, /)#

Return a copy of the string with trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

split(sep=None, maxsplit=-1)#

Return a list of the substrings in the string, using sep as the separator string.

sep
The separator used to split the string.

When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.

maxsplit
Maximum number of splits (starting from the left). -1 (the default value) means no limit.

Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.

splitlines(keepends=False)#

Return a list of the lines in the string, breaking at line boundaries.

Line breaks are not included in the resulting list unless keepends is given and true.

startswith(prefix[, start[, end]]) → bool#: Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

strip(chars=None, /)#

Return a copy of the string with leading and trailing whitespace removed.

If chars is given and not None, remove characters in chars instead.

swapcase()#: Convert uppercase characters to lowercase and lowercase characters to uppercase.

title()#

Return a version of the string where each word is titlecased.

More specifically, words start with uppercased characters and all remaining cased characters have lower case.

translate(table, /)#

Replace each character in the string using the given translation table.

table
Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.

upper()#: Return a copy of the string converted to uppercase.

zfill(width, /)#

Pad a numeric string with zeros on the left, to fill a field of the given width.

The string is never truncated.

class llama_index.callbacks.GradientAIFineTuningHandler#

Callback handler for Gradient AI fine-tuning.

This handler will collect all messages sent to the LLM, along with their responses. It will then save these messages in a .jsonl format that can be used for fine-tuning with Gradient AI’s API.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

get_finetuning_events() → Dict[str, Dict[str, Any]]#: Get finetuning events.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#: Run when an event ends.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#: Run when an event starts and return id of event.

save_finetuning_events(path: str) → None#

Save the finetuning events to a file.

This saved format can be used for fine-tuning with OpenAI’s API. The structure for each json line is as follows: {

“inputs”: “<full_prompt_str>”

},#

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

class llama_index.callbacks.LlamaDebugHandler(event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None, print_trace_on_end: bool = True)#

Callback handler that keeps track of debug info.

NOTE: this is a beta feature. The usage within our codebase, and the interface may change.

This handler simply keeps track of event starts/ends, separated by event types. You can use this callback handler to keep track of and debug events.

Parameters

event_starts_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event starts.
event_ends_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event ends.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Shutdown the current trace.

flush_event_logs() → None#: Clear all events from memory.

get_event_pairs(event_type: Optional[CBEventType] = None) → List[List[CBEvent]]#: Pair events by ID, either all events or a specific type.

get_events(event_type: Optional[CBEventType] = None) → List[CBEvent]#: Get all events for a specific event type.

get_llm_inputs_outputs() → List[List[CBEvent]]#: Get the exact LLM inputs and outputs.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#

Store event end data by event type.

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#

Store event start data by event type.

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.
parent_id (str) – parent event id.

print_trace_map() → None#: Print simple trace map to terminal for debugging of the most recent trace.

start_trace(trace_id: Optional[str] = None) → None#: Launch a trace.

class llama_index.callbacks.OpenAIFineTuningHandler#

Callback handler for OpenAI fine-tuning.

This handler will collect all messages sent to the LLM, along with their responses. It will then save these messages in a .jsonl format that can be used for fine-tuning with OpenAI’s API.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

get_finetuning_events() → Dict[str, Dict[str, Any]]#: Get finetuning events.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#: Run when an event ends.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#: Run when an event starts and return id of event.

save_finetuning_events(path: str) → None#

Save the finetuning events to a file.

This saved format can be used for fine-tuning with OpenAI’s API. The structure for each json line is as follows: {

messages: [
{ rol: “system”, content: “Text”}, { role: “user”, content: “Text” },

]

},#

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

class llama_index.callbacks.OpenInferenceCallbackHandler(callback: Optional[Callable[[List[QueryData], List[NodeData]], None]] = None)#

Callback handler for storing generation data in OpenInference format. OpenInference is an open standard for capturing and storing AI model inferences. It enables production LLMapp servers to seamlessly integrate with LLM observability solutions such as Arize and Phoenix.

For more information on the specification, see https://github.com/Arize-ai/open-inference-spec

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

flush_node_data_buffer() → List[NodeData]#

Clears the node data buffer and returns the data.

Returns: The node data.
Return type: List[NodeData]

flush_query_data_buffer() → List[QueryData]#

Clears the query data buffer and returns the data.

Returns: The query data.
Return type: List[QueryData]

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#: Run when an event ends.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#: Run when an event starts and return id of event.

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

class llama_index.callbacks.TokenCountingHandler(tokenizer: Optional[Callable[[str], List]] = None, event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None, verbose: bool = False)#

Callback handler for counting tokens in LLM and Embedding events.

Parameters

tokenizer – Tokenizer to use. Defaults to the global tokenizer (see llama_index.utils.globals_helper).
event_starts_to_ignore – List of event types to ignore at the start of a trace.
event_ends_to_ignore – List of event types to ignore at the end of a trace.

property completion_llm_token_count: int#: Get the current total LLM completion token count.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#: Count the LLM or Embedding tokens as needed.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#: Run when an event starts and return id of event.

property prompt_llm_token_count: int#: Get the current total LLM prompt token count.

reset_counts() → None#: Reset the token counts.

start_trace(trace_id: Optional[str] = None) → None#: Run when an overall trace is launched.

property total_embedding_token_count: int#: Get the current total Embedding token count.

property total_llm_token_count: int#: Get the current total LLM token count.

class llama_index.callbacks.WandbCallbackHandler(run_args: Optional[WandbRunArgs] = None, tokenizer: Optional[Callable[[str], List]] = None, event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None)#

Callback handler that logs events to wandb.

NOTE: this is a beta feature. The usage within our codebase, and the interface may change.

Use the WandbCallbackHandler to log trace events to wandb. This handler is useful for debugging and visualizing the trace events. It captures the payload of the events and logs them to wandb. The handler also tracks the start and end of events. This is particularly useful for debugging your LLM calls.

The WandbCallbackHandler can also be used to log the indices and graphs to wandb using the persist_index method. This will save the indexes as artifacts in wandb. The load_storage_context method can be used to load the indexes from wandb artifacts. This method will return a StorageContext object that can be used to build the index, using load_index_from_storage, load_indices_from_storage or load_graph_from_storage functions.

Parameters

event_starts_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event starts.
event_ends_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event ends.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None#: Run when an overall trace is exited.

finish() → None#: Finish the callback handler.

load_storage_context(artifact_url: str, index_download_dir: Optional[str] = None) → StorageContext#

Download an index from wandb and return a storage context.

Use this storage context to load the index into memory using load_index_from_storage, load_indices_from_storage or load_graph_from_storage functions.

Parameters

artifact_url (str) – url of the artifact to download. The artifact url will be of the form: entity/project/index_name:version and can be found in the W&B UI.
index_download_dir (Union[str, None]) – directory to download the index to.

log_trace_tree() → None#: Log the trace tree to wandb.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None#

Store event end data by event type.

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', parent_id: str = '', **kwargs: Any) → str#

Store event start data by event type.

Parameters

event_type (CBEventType) – event type to store.
payload (Optional[Dict[str, Any]]) – payload to store.
event_id (str) – event id to store.
parent_id (str) – parent event id.

persist_index(index: IndexType, index_name: str, persist_dir: Optional[str] = None) → None#

Upload an index to wandb as an artifact. You can learn more about W&B artifacts here: https://docs.wandb.ai/guides/artifacts.

For the ComposableGraph index, the root id is stored as artifact metadata.

Parameters

index (IndexType) – index to upload.
index_name (str) – name of the index. This will be used as the artifact name.
persist_dir (Union[str, None]) – directory to persist the index. If None, a temporary directory will be created and used.

start_trace(trace_id: Optional[str] = None) → None#: Launch a trace.

llama_index.callbacks.trace_method(trace_id: str, callback_manager_attr: str = 'callback_manager') → Callable[[Callable], Callable]#

Decorator to trace a method.

Example

@trace_method(“my_trace_id”) def my_method(self):

pass

Assumes that the self instance has a CallbackManager instance in an attribute named callback_manager. This can be overridden by passing in a callback_manager_attr keyword argument.