Embeddings and Reranking for Retrieval

If you hardcoded a NIM (NVIDIA Inference Microservices, NVIDIA's hosted model API) reranker model ID (the model that re-scores your search results for relevance) into a retrieval pipeline and haven't looked at it since, there's a real chance it's already gone. This lesson's own first code sample already lived through it: the reranker it originally taught, llama-3.2-nv-rerankqa-1b-v2, was flagged for a scheduled deprecation, and checked directly against NVIDIA's own catalog, that deprecation date has now passed. The model is retired and renamed. That's not a hypothetical risk this lesson is warning you about. It's a thing that already happened to this exact lesson.

Your retrieval pipeline embeds a query, searches a vector store, and reranks the top results before handing them to a model. Three model choices go into that pipeline: an embedding model, a search index, and a reranker. Which of those three should you assume you'll need to swap out soonest? The reranker, and not as a guess: NIM's reranking layer has a documented replace-and-retire cadence you can verify on the model cards themselves.

Embeddings: turning text into a comparable vector

An embedding model converts text into a fixed-length vector, so "how do I reset my password" and "password reset steps" land close together in vector space even though they share almost no words. NIM's current generalist embedding model is nv-embed-v1: Mistral-7B-based, 4096 dimensions, up to 32k tokens of input. A code-specialized sibling, nv-embedcode-7b-v1, exists for embedding source code and technical queries rather than prose.

from openai import OpenAI

client = OpenAI(
    base_url="https://integrate.api.nvidia.com/v1",
    api_key="nvapi-your-key-here",
)

response = client.embeddings.create(
    input=["How do I reset my password?"],
    model="nvidia/nv-embed-v1",
    encoding_format="float",
)

vector = response.data[0].embedding

Same client, same base_url, same call shape as chat completion. The only thing that changed is which endpoint you're calling and what the response contains: a vector instead of text.

Reranking: the step that catches what embedding similarity misses

Embedding similarity is fast but approximate. It ranks candidates by vector distance, which is a proxy for relevance, not relevance itself. A reranking model looks at the actual query and each candidate passage together and scores them directly, which is slower per pair but far more accurate at picking the true best match out of, say, the top 20 embedding results.

import requests

headers = {"Authorization": "Bearer nvapi-your-key-here", "Accept": "application/json"}
payload = {
    "model": "nvidia/llama-nemotron-rerank-1b-v2",
    "query": {"text": "What is the GPU memory bandwidth of an H100 SXM?"},
    "passages": [
        {"text": "The Grace Hopper chip-to-chip interconnect delivers 900GB/s of bandwidth."},
        {"text": "A100 can be partitioned into seven GPU instances for dynamic workloads."},
    ],
}
response = requests.post(
    "https://ai.api.nvidia.com/v1/retrieval/nvidia/reranking",
    headers=headers,
    json=payload,
)
result = response.json()

The finding worth building your pipeline around

Here's the part that's easy to miss if you copy a reranker's model ID once and never look at it again: this lesson originally flagged llama-3.2-nv-rerankqa-1b-v2 and llama-3.2-nemoretriever-500m-rerank-v2 for a scheduled 2026-05-18 deprecation. Re-checked against NVIDIA's own model cards and API reference this session, that date has now passed, and the prediction played out exactly as flagged: both models are gone, each renamed and replaced by a v2 successor under NVIDIA's newer llama-nemotron-rerank-* naming.

That's three full generations of reranking model retired since this course's own 2024 source tutorial: the original nv-rerank-qa-mistral-4b:1, the pair that replaced it and this lesson originally taught, and now the llama-nemotron-rerank-*-v2 pair above. The code sample earlier in this lesson already uses the current name.

What this buys you

A retrieval pipeline built with the model ID isolated as a config value, checked against NIM's live catalog on a schedule, survives a reranker retirement as a config change. A pipeline that doesn't survives it as an outage, discovered in production when the old model ID stops resolving.