Skip to main content

Module multimodal

Module multimodal 

Structs§

ImageMarkerSummary
Byte count of the non-marker text and the number of loadable references, computed by the same scanner as parse_image_markers without building the cleaned string or copying references. This is the text-level primitive: it sees a content string, not a message, so it must never be pointed at a tool-result carrier, whose images are its declared attachments rather than marker syntax in its body. Message-level callers go through message_image_summary, which dispatches on carrier shape and falls back to this scan for ordinary text.
LocalImageCache
Per-path cache for resolved local image data URIs. Keyed by absolute path; stores (len, mtime) for freshness checks ((0, 0) sentinel = immutable upload). LRU evicts by both entry count and total bytes.
PreparedMessages
ProviderImageId
Stable identity for one normalized provider-visible image reference.

Enums§

ImageMarkerDisposition
How multimodal preparation will treat the [IMAGE:...] markers in a message at its position in the history.
MultimodalError

Constants§

IMAGE_MARKER_PREFIX
MAX_RESOLVABILITY_CANDIDATES
Count image markers in the latest user message whose references would actually resolve to an attachment: an inline data: URI that passes the shared structural check, a remote http(s) URL when remote_allowed is set (counted without any network access), or a local path that the caller’s path_allowed predicate accepts and that exists as a file. Marker syntax alone does not count, so prose that merely looks like a marker cannot fail a turn on a text-only model.
MEDIA_PLACEHOLDER
Text a degraded media marker is replaced with before the history reaches a model that cannot consume the payload. The model may echo it verbatim into a reply, so it is plain prose rather than bracket syntax: it must never parse as an outbound delivery marker ([KIND:target], which the channel parsers key on [ and :), and it contains no JSON-special characters so a marker replaced inside a native tool-result blob leaves the surrounding object valid.

Functions§

contains_image_markers
count_image_markers
count_latest_user_image_markers
count_latest_user_resolvable_image_markers
count_user_image_markers
extract_ollama_image_payload
image_marker_dispositions
One disposition per message, computed with the same predicates preparation uses (is_tool_result_carrier, current_turn_tool_result_indices, should_normalize_message_images). Dispositions do not model the config-dependent image limits (max_images, max_image_turns), which can strip or cap images preparation would otherwise dispatch.
image_marker_summary
message_image_summary
Byte count of the text a message contributes and the number of images it contributes, following preparation. A tool-result carrier counts its declared IMAGE attachments and is never body-scanned, so marker syntax quoted in a carrier body is text, and a legacy carrier (nothing declared at the fixed position) contributes no images at all. A carrier’s text_bytes is its retained body length: the envelope’s content string for a native carrier, and the carrier text with its image attachment lines lifted (non-image lines retained) for a prompt carrier. That is what a current-turn dispatch delivers; a stale carrier is priced as its bytes by the estimator instead (see ImageMarkerDisposition::Stripped). Every other message falls back to the text-level image_marker_summary scan of its content.
omit_provider_image_ids
Build a provider-only replay view with selected normalized images removed. The caller retains canonical history unchanged.
parse_image_markers
parse_user_message_image_refs
Split user-role content for image lifting: a prompt-mode tool carrier contributes only its declared image attachment lines (the body is never scanned, and non-image attachment lines stay in the text); every other user message lifts text markers exactly as before.
prepare_messages_for_provider
prepare_messages_for_provider_cached
Like prepare_messages_for_provider but reuses a LocalImageCache across calls so each unique local image file is read from disk at most once per session (or once per modification for mutable files).
provider_image_ids
Return normalized image identities in provider wire order.
sanitize_audio_markers
Strip loadable audio markers (see strip_unplayable_audio_markers) across every message in messages, logging one degradation warning when any are removed. Returns the input borrowed when no candidate marker is present (the common, allocation-free path) or an owned rebuilt vector otherwise.
sanitize_image_markers
Strip image markers that cannot be delivered inline (see strip_undeliverable_image_markers) across every message, logging one degradation warning when any are removed. Returns the input borrowed when no message needs the seam (the common, allocation-free path) or an owned rebuilt vector otherwise.
strip_media_markers
strip_media_markers_model_visible
Strip every media marker from the model-visible text of one message, for the text-only degrade path: all marker kinds, all roles.
strip_message_media
Degrade one message for a text-only route: a tool-result carrier loses exactly its declared attachments and keeps its body verbatim (legacy carriers included — a marker in tool body text is text, and rewriting it is the incident this contract exists to prevent); every other message has its media markers replaced with the placeholder as before, through the model-visible rewrite so signed envelope fields replay byte-for-byte.
warm_lazy_regexes
Force-compile this module’s lazy regexes on the caller’s thread.