Advanced Prompt Techniques (Variable Mappings, Functions)¶

In this notebook we show some advanced prompt techniques. These features allow you to define more custom/expressive prompts, re-use existing ones, and also express certain operations in fewer lines of code.

We show the following features:

Partial formatting
Prompt template variable mappings
Prompt function mappings

In [ ]:

Copied!

%pip install llama-index-llms-openai
%pip install llama-index-llms-openai

Looking in indexes: https://mirrors.ustc.edu.cn/pypi/simple
Collecting llama-index-llms-openai
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/e8/45/e2849d077ad5510b56dd072c9cbfb73daa47debcac5a9582dd118f6659c5/llama_index_llms_openai-0.3.10-py3-none-any.whl (14 kB)
Collecting llama-index-core<0.13.0,>=0.12.4 (from llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/57/78/71984bb2cc8fe5c70d7986ee5034f5b10e07ecf751ada58ca1212ecc7bf7/llama_index_core-0.12.5-py3-none-any.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 7.0 MB/s eta 0:00:00a 0:00:01
Collecting openai<2.0.0,>=1.57.1 (from llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/37/e7/95437fb676381e927d4cb3f9f8dd90ed24cfd264f572db4d395037428594/openai-1.57.2-py3-none-any.whl (389 kB)
Requirement already satisfied: PyYAML>=6.0.1 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (6.0.2)
Collecting SQLAlchemy>=1.4.49 (from SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/07/15/68ef91de5b8b7f80fb2d2b3b31ed42180c6227fe0a701aed9d01d34f98ec/SQLAlchemy-2.0.36-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 7.1 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: aiohttp<4.0.0,>=3.8.6 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (3.10.5)
Collecting dataclasses-json (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/c3/be/d0d44e092656fe7a06b55e6103cbce807cdbdee17884a5367c68c9860853/dataclasses_json-0.6.7-py3-none-any.whl (28 kB)
Collecting deprecated>=1.2.9.3 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/1d/8f/c7f227eb42cfeaddce3eb0c96c60cbca37797fa7b34f8e1aeadf6c5c0983/Deprecated-1.2.15-py2.py3-none-any.whl (9.9 kB)
Collecting dirtyjson<2.0.0,>=1.0.8 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/68/69/1bcf70f81de1b4a9f21b3a62ec0c83bdff991c88d6cc2267d02408457e88/dirtyjson-1.0.8-py3-none-any.whl (25 kB)
Collecting filetype<2.0.0,>=1.2.0 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/18/79/1b8fa1bb3568781e84c9200f951c735f3f157429f44be0495da55894d620/filetype-1.2.0-py2.py3-none-any.whl (19 kB)
Requirement already satisfied: fsspec>=2023.5.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2024.9.0)
Requirement already satisfied: httpx in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (0.27.2)
Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.6.0)
Requirement already satisfied: networkx>=3.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (3.3)
Collecting nltk>3.8.1 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/4d/66/7d9e26593edda06e8cb531874633f7c2372279c3b0f46235539fe546df8b/nltk-3.9.1-py3-none-any.whl (1.5 MB)
Requirement already satisfied: numpy in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.26.4)
Requirement already satisfied: pillow>=9.0.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (10.4.0)
Requirement already satisfied: pydantic>=2.8.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2.8.2)
Requirement already satisfied: requests>=2.31.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2.32.3)
Collecting tenacity!=8.4.0,<10.0.0,>=8.2.0 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/b6/cb/b86984bed139586d01532a587464b5805f12e397594f19f931c4c2fbfa61/tenacity-9.0.0-py3-none-any.whl (28 kB)
Requirement already satisfied: tiktoken>=0.3.3 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (0.7.0)
Requirement already satisfied: tqdm<5.0.0,>=4.66.1 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (4.66.5)
Requirement already satisfied: typing-extensions>=4.5.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (4.12.2)
Collecting typing-inspect>=0.8.0 (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/65/f3/107a22063bf27bdccf2024833d3445f4eea42b2e598abfbd46f6a63b6cb0/typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)
Collecting wrapt (from llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Using cached https://mirrors.ustc.edu.cn/pypi/packages/cd/c7/b8c89bf5ca5c4e6a2d0565d149d549cdb4cffb8916d1d1b546b62fb79281/wrapt-1.17.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (82 kB)
Requirement already satisfied: anyio<5,>=3.5.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from openai<2.0.0,>=1.57.1->llama-index-llms-openai) (4.4.0)
Requirement already satisfied: distro<2,>=1.7.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from openai<2.0.0,>=1.57.1->llama-index-llms-openai) (1.9.0)
Collecting jiter<1,>=0.4.0 (from openai<2.0.0,>=1.57.1->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/4d/a0/3993cda2e267fe679b45d0bcc2cef0b4504b0aa810659cdae9737d6bace9/jiter-0.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (345 kB)
Requirement already satisfied: sniffio in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from openai<2.0.0,>=1.57.1->llama-index-llms-openai) (1.3.1)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (24.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.9.7)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.6->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (4.0.3)
Requirement already satisfied: idna>=2.8 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.57.1->llama-index-llms-openai) (3.10)
Requirement already satisfied: exceptiongroup>=1.0.2 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.57.1->llama-index-llms-openai) (1.2.2)
Requirement already satisfied: certifi in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpx->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2024.8.30)
Requirement already satisfied: httpcore==1.* in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpx->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.0.5)
Requirement already satisfied: h11<0.15,>=0.13 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpcore==1.*->httpx->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (0.14.0)
Requirement already satisfied: click in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk>3.8.1->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (8.1.7)
Requirement already satisfied: joblib in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk>3.8.1->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (1.4.2)
Requirement already satisfied: regex>=2021.8.3 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk>3.8.1->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2024.7.24)
Requirement already satisfied: annotated-types>=0.4.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic>=2.8.0->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (0.7.0)
Requirement already satisfied: pydantic-core==2.20.1 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic>=2.8.0->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2.20.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests>=2.31.0->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests>=2.31.0->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (2.2.3)
Collecting greenlet!=0.4.17 (from SQLAlchemy>=1.4.49->SQLAlchemy[asyncio]>=1.4.49->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/cf/69/79e4d63b9387b48939096e25115b8af7cd8a90397a304f92436bcb21f5b2/greenlet-3.1.1-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (599 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 599.5/599.5 kB 7.0 MB/s eta 0:00:00
Collecting mypy-extensions>=0.3.0 (from typing-inspect>=0.8.0->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/2a/e2/5d3f6ada4297caebe1a2add3b126fe800c96f56dbe5d1988a2cbe0b267aa/mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai)
  Downloading https://mirrors.ustc.edu.cn/pypi/packages/ac/a7/a78ff54e67ef92a3d12126b98eb98ab8abab3de4a8c46d240c87e514d6bb/marshmallow-3.23.1-py3-none-any.whl (49 kB)
Requirement already satisfied: packaging>=17.0 in /home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->llama-index-core<0.13.0,>=0.12.4->llama-index-llms-openai) (24.1)
WARNING: Error parsing dependencies of huggingface-hub: [Errno 2] No such file or directory: '/home/zhangyh/.pyenv/versions/3.10.14/lib/python3.10/site-packages/huggingface_hub-0.24.6.dist-info/METADATA'
Installing collected packages: filetype, dirtyjson, wrapt, tenacity, nltk, mypy-extensions, marshmallow, jiter, greenlet, typing-inspect, SQLAlchemy, deprecated, openai, dataclasses-json, llama-index-core, llama-index-llms-openai
  Attempting uninstall: openai
    Found existing installation: openai 1.39.0
    Uninstalling openai-1.39.0:
      Successfully uninstalled openai-1.39.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xinference 0.14.4 requires huggingface-hub>=0.19.4, which is not installed.
xinference 0.14.4 requires openai<1.40,>1, but you have openai 1.57.2 which is incompatible.
Successfully installed SQLAlchemy-2.0.36 dataclasses-json-0.6.7 deprecated-1.2.15 dirtyjson-1.0.8 filetype-1.2.0 greenlet-3.1.1 jiter-0.8.2 llama-index-core-0.12.5 llama-index-llms-openai-0.3.10 marshmallow-3.23.1 mypy-extensions-1.0.0 nltk-3.9.1 openai-1.57.2 tenacity-9.0.0 typing-inspect-0.9.0 wrapt-1.17.0

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

In [ ]:

Copied!

from llama_index.core import PromptTemplate
from llama_index.llms.openai import OpenAI
from llama_index.core import PromptTemplate
from llama_index.llms.openai import OpenAI

1. Partial Formatting¶

Partial formatting (partial_format) allows you to partially format a prompt, filling in some variables while leaving others to be filled in later.

This is a nice convenience function so you don't have to maintain all the required prompt variables all the way down to format, you can partially format as they come in.

This will create a copy of the prompt template.

In [ ]:

Copied!





qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Please write the answer in the style of {tone_name}
Query: {query_str}
Answer: \
"""

prompt_tmpl = PromptTemplate(qa_prompt_tmpl_str)
qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Please write the answer in the style of {tone_name}
Query: {query_str}
Answer: \
"""

prompt_tmpl = PromptTemplate(qa_prompt_tmpl_str)

In [ ]:

Copied!

partial_prompt_tmpl = prompt_tmpl.partial_format(tone_name="Shakespeare")
partial_prompt_tmpl = prompt_tmpl.partial_format(tone_name="Shakespeare")

In [ ]:

Copied!

partial_prompt_tmpl.kwargs
partial_prompt_tmpl.kwargs

Out[ ]:

{'tone_name': 'Shakespeare'}

In [ ]:

Copied!





fmt_prompt = partial_prompt_tmpl.format(
    context_str="In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters",
    query_str="How many params does llama 2 have",
)
print(fmt_prompt)
fmt_prompt = partial_prompt_tmpl.format(
    context_str="In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters",
    query_str="How many params does llama 2 have",
)
print(fmt_prompt)

Context information is below.
---------------------
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters
---------------------
Given the context information and not prior knowledge, answer the query.
Please write the answer in the style of Shakespeare
Query: How many params does llama 2 have
Answer:

2. Prompt Template Variable Mappings¶

Template var mappings allow you to specify a mapping from the "expected" prompt keys (e.g. context_str and query_str for response synthesis), with the keys actually in your template.

This allows you re-use your existing string templates without having to annoyingly change out the template variables.

In [ ]:

Copied!





# NOTE: here notice we use `my_context` and `my_query` as template variables

qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{my_context}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {my_query}
Answer: \
"""

template_var_mappings = {"context_str": "my_context", "query_str": "my_query"}

prompt_tmpl = PromptTemplate(
    qa_prompt_tmpl_str, template_var_mappings=template_var_mappings
)
# NOTE: here notice we use `my_context` and `my_query` as template variables

qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{my_context}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {my_query}
Answer: \
"""

template_var_mappings = {"context_str": "my_context", "query_str": "my_query"}

prompt_tmpl = PromptTemplate(
    qa_prompt_tmpl_str, template_var_mappings=template_var_mappings
)

In [ ]:

Copied!





fmt_prompt = prompt_tmpl.format(
    context_str="In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters",
    query_str="How many params does llama 2 have",
)
print(fmt_prompt)
fmt_prompt = prompt_tmpl.format(
    context_str="In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters",
    query_str="How many params does llama 2 have",
)
print(fmt_prompt)

Context information is below.
---------------------
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters
---------------------
Given the context information and not prior knowledge, answer the query.
Query: How many params does llama 2 have
Answer:

3. Prompt Function Mappings¶

You can also pass in functions as template variables instead of fixed values.

This allows you to dynamically inject certain values, dependent on other values, during query-time.

Here are some basic examples. We show more advanced examples (e.g. few-shot examples) in our Prompt Engineering for RAG guide.

In [ ]:

Copied!





qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {query_str}
Answer: \
"""


def format_context_fn(**kwargs):
    # format context with bullet points
    context_list = kwargs["context_str"].split("\n\n")
    fmtted_context = "\n\n".join([f"- {c}" for c in context_list])
    return fmtted_context


prompt_tmpl = PromptTemplate(
    qa_prompt_tmpl_str, function_mappings={"context_str": format_context_fn}
)
qa_prompt_tmpl_str = """\
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {query_str}
Answer: \
"""


def format_context_fn(**kwargs):
    # format context with bullet points
    context_list = kwargs["context_str"].split("\n\n")
    fmtted_context = "\n\n".join([f"- {c}" for c in context_list])
    return fmtted_context


prompt_tmpl = PromptTemplate(
    qa_prompt_tmpl_str, function_mappings={"context_str": format_context_fn}
)

In [ ]:

Copied!





context_str = """\
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.

Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases.

Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models.
"""

fmt_prompt = prompt_tmpl.format(
    context_str=context_str, query_str="How many params does llama 2 have"
)
print(fmt_prompt)
context_str = """\
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.

Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases.

Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models.
"""

fmt_prompt = prompt_tmpl.format(
    context_str=context_str, query_str="How many params does llama 2 have"
)
print(fmt_prompt)

Context information is below.
---------------------
- In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.

- Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases.

- Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models.

---------------------
Given the context information and not prior knowledge, answer the query.
Query: How many params does llama 2 have
Answer: