Skip to main content

fetch_context_window

Function fetch_context_window 

pub async fn fetch_context_window(
    provider_type: &str,
    config: &ModelProviderConfig,
) -> Option<usize>
Expand description

Attempt to fetch context window from provider’s /models endpoint. Returns None on any failure (network, parsing, missing field) — caller uses fallback.

Which families are asked, and how each authenticates, is derived from the family registry, not listed here. It used to be listed here, and that was the defect: together | groq | fireworks | deepinfra | hyperbolic | anyscale | novita | nebius was eight names maintained by hand, so a family that also serves this catalog silently fell through to None and its operators kept the unconfigured 32,000-token fallback with nothing failing to say so. A family now declares the fact beside its own spec, where the person adding the family is looking.

Eligibility is per-family and opt-in, never inferred from chat-wire compatibility: speaking the OpenAI-compatible chat protocol says nothing about whether GET {base}/models exists, what shape it returns, or whether the stored credential can be presented to it as-is.

None means unknown: the caller must leave the setting unset rather than substitute a value. An unset context window and a fabricated default are different states and must stay distinguishable downstream.