NVIDIA NIMs¶
NVIDIA NIMs give users easy access to NVIDIA hosted API endpoints for AI models like Mixtral 8x22B, Llama 3, Stable Diffusion, etc. These models, hosted on the https://build.nvidia.com, are optimized, tested, and hosted on the NVIDIA AI platform, making them fast and easy to evaluate, further customize, and seamlessly run at peak performance on any accelerated stack.
With NVIDIA NIMs, you can get quick results from a fully accelerated stack running on NVIDIA DGX Cloud. These models can be deployed anywhere with enterprise-grade security, stability, and support using NVIDIA AI Enterprise.
These models can be easily accessed via the
llama-index-postprocessor-nvidia-rerank
package, as shown below.
This example goes over how to use LlamaIndex to interact with the supported NVIDIA Retrieval QA Ranking Model for retrieval-augmented generation via the NVIDIARerank
class.
Reranking¶
Reranking is a critical piece of high accuracy, efficient retrieval pipelines.
Two important use cases:
- Combining results from multiple data sources
- Enhancing accuracy for single data sources
Combining results from multiple sources¶
Consider a pipeline with data from a semantic store, such as VectorStoreIndex, as well as a BM25 store.
Each store is queried independently and returns results that the individual store considers to be highly relevant. Figuring out the overall relevance of the results is where reranking comes into play.
Follow along with the Advanced - Hybrid Retriever + Re-Ranking use case, substitute the reranker with -
%pip install --upgrade --quiet llama-index-postprocessor-nvidia-rerank
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
reranker = NVIDIARerank(top_n=4)
Connecting to local NIMs¶
In addition to connecting to hosted NVIDIA NIMs, this connector can be used to connect to local microservice instances. This helps you take your applications local when necessary.
For instructions on how to setup local microservice instances, see https://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/
from llama_index.postprocessor.nvidia_rerank import NVIDIARerank
# reranker = NVIDIARerank(top_n...) from above
reranker = reranker.mode("nim", base_url="http://0.0.0.0:1976/v1")