Add Hallucination playground with Vela Halu and verified examples

README.md

@@ -8,7 +8,7 @@ app_port: 7860
 header: mini
 pinned: false
 license: apache-2.0
-short_description: Explore ten Vela models for intelligent routing.
+short_description: Explore eleven Vela models for intelligent routing.
 models:
   - llm-semantic-router/Vela-1.0-Encoder-307M-Domain
   - llm-semantic-router/Vela-1.0-Encoder-307M-PII
@@ -16,6 +16,7 @@ models:
   - llm-semantic-router/Vela-1.0-Encoder-307M-Safety
   - llm-semantic-router/Vela-1.0-Encoder-307M-Hazard
   - llm-semantic-router/Vela-1.0-Encoder-307M-FactCheck
+  - llm-semantic-router/Vela-1.0-Encoder-307M-Halu
   - llm-semantic-router/Vela-1.0-Encoder-307M-Modality
   - llm-semantic-router/Vela-1.0-Encoder-307M-Feedback
   - llm-semantic-router/Vela-1.0-Encoder-307M-Embedding
@@ -24,7 +25,7 @@ models:
 
 # Vela Studio
 
-A public playground for the [Vela 1.0 model family](https://huggingface.co/collections/llm-semantic-router/vela-10), by [vLLM Semantic Router](https://vllm-sr.ai). Results come from the selected model's pinned checkpoint on CPU. Exact built-in examples can reuse a prior successful inference, clearly marked as a cached example result. The interface includes twelve interactive demos powered by ten model checkpoints: classification, PII highlights and redaction, content-risk thresholds, semantic similarity, dynamic prompt routing, passage reranking, and semantic clustering.
+A public playground for the [Vela 1.0 model family](https://huggingface.co/collections/llm-semantic-router/vela-10), by [vLLM Semantic Router](https://vllm-sr.ai). Results come from the selected model's pinned checkpoint on CPU. Exact built-in examples can reuse a prior successful inference, clearly marked as a cached example result. The interface includes thirteen interactive demos powered by eleven model checkpoints: classification, evidence-based hallucination highlights, PII highlights and redaction, content-risk thresholds, semantic similarity, dynamic prompt routing, passage reranking, and semantic clustering.
 
 ## Runtime
 
@@ -32,19 +33,19 @@ The Docker image uses Python 3.11, CPU PyTorch 2.8.0, Transformers 4.57.6, and F
 
 Two 307M FP32 checkpoints require approximately 2.46 GB of parameter memory, plus runtime and activation memory. Execution remains sequential, with a single input or query-passage pair at a time. Set `VELA_MODEL_CACHE_SIZE=1` to retain only one model on smaller hosts; supported values are `1` and `2`, with `2` as the default.
 
-Most demos accept up to **512 tokens including special tokens**; Clustering accepts **3–24 texts**, each up to **128 tokens including special tokens** and 2,048 characters. Similarity and Reranker allow at most **six candidates**; Routing allows **2–10 categories**. Reranker applies the limit to each query-and-passage pair; Embedding applies it independently to each text. Inputs exceeding this limit are rejected without truncation. The model family has a larger declared input capacity; this shorter public limit keeps CPU Basic usage bounded.
+Most demos accept up to **512 tokens including special tokens**; Clustering accepts **3–24 texts**, each up to **128 tokens including special tokens** and 2,048 characters. Similarity and Reranker allow at most **six candidates**; Routing allows **2–10 categories**. Hallucination applies the limit to the complete question, evidence, and answer pair; Reranker applies the limit to each query-and-passage pair; Embedding applies it independently to each text. Inputs exceeding this limit are rejected without truncation. The model family has a larger declared input capacity; this shorter public limit keeps CPU Basic usage bounded.
 
 One inference worker serves a FIFO queue with up to **eight waiting requests** in addition to the running request. The interface shows queue position, model loading, and inference progress and allows cancellation. HTTP 429 is returned only when the waiting queue is full, with `Retry-After: 5`. Cancelling pending work removes it from the queue immediately. Cancelling running work discards its eventual result; CPU execution finishes before the next request starts. A cold download can still take several minutes, and a long queue does not increase CPU throughput.
 
-Only exact matches to the public examples in `catalog.py`, `routing_catalog.py`, `clustering_catalog.py`, and `similarity_catalog.py` can reuse successful inference results. The key includes task, text, candidates, the complete ordered category configuration, or the complete ordered clustering texts, model ID, and pinned revision. Each cache hit gets a separate random job ID and `cache_hit: true`; original inference timings are preserved. These public results stay in an in-process cache capped at 80 entries until eviction or restart. Edited examples and other user inputs are never added to this cache.
+Only exact matches to the public examples in `catalog.py`, `routing_catalog.py`, `clustering_catalog.py`, `similarity_catalog.py`, and `hallucination_catalog.py` can reuse successful inference results. The key includes task, text, evidence and answer when supplied, candidates, the complete ordered category configuration, or the complete ordered clustering texts, model ID, and pinned revision. Each cache hit gets a separate random job ID and `cache_hit: true`; original inference timings are preserved. These public results stay in an in-process cache capped at 80 entries until eviction or restart. Edited examples and other user inputs are never added to this cache.
 
 No user text, predictions, or exception payloads are logged or written to disk by this application. Pending input remains in memory until execution or cancellation; active input is released when execution returns. Private job results expire about five minutes after completion, failure, or cancellation, with cleanup at most 30 seconds later when idle. At most 64 job records are retained, so older completed results can expire sooner under load. Job IDs are unguessable bearer tokens; keep the ID private if the input is private. Hugging Face provides the surrounding hosting infrastructure. Use fictional data when trying PII examples.
 
 ## Interface
 
-All twelve demos share an input → model → result canvas, with the same compass model card, connecting paths, run-button placement, and example toolbar. Routing, Similarity, Reranker, and Clustering appear first in the horizontal navigation. Desktop uses three columns; narrower screens adapt the same flow without hiding task controls. The compass shows the selected model's identity and actual queue, loading, execution, or cache status.
+All thirteen demos share an input → model → result canvas, with the same compass model card, connecting paths, run-button placement, and example toolbar. Routing, Similarity, Reranker, and Clustering appear first in the horizontal navigation. Desktop uses three columns; narrower screens adapt the same flow without hiding task controls. The compass shows the selected model's identity and actual queue, loading, execution, or cache status.
 
-Results retain their task-specific meaning: classifications show label scores, PII offers complete highlighted and redacted text, Hazard preserves all twelve independently thresholded categories, and Similarity and Reranker show ordered passages with their respective cosine and raw relevance scores. Routing destinations remain editable in place. The global animation control pauses the ship, compass, and connecting paths.
+Results retain their task-specific meaning: classifications show label scores, Hallucination highlights unsupported answer spans, PII offers complete highlighted and redacted text, Hazard preserves all twelve independently thresholded categories, and Similarity and Reranker show ordered passages with their respective cosine and raw relevance scores. Routing destinations remain editable in place. The global animation control pauses the ship, compass, and connecting paths.
 
 Similarity offers three example groups in the same tab: **Paraphrases**, **Question answering**, and **Duplicate questions**, with two verified examples per group. Paraphrases retains the original delivery-status and password-reset inputs. Question answering finds password-reset and order-return instructions. Duplicate questions compares Git-commit and Python-list questions against related but different questions. Changing the group or choosing an example prepares the inputs; click **Compare meanings** to run the model. Every example’s intended candidate ranked first in real pinned-model inference before inclusion. These checks establish the displayed rankings, not general model accuracy or a duplicate threshold. The earlier library example and its cross-language/time-detail limitations remain in the separate acceptance records.
 
@@ -54,6 +55,12 @@ PII offers seven examples covering 13 entity types: contact details, date and ag
 
 Each selected PII example was checked for complete entity spans and redaction; each Hazard example was checked against the full detected-label set using the unchanged published thresholds. Candidate IBAN and ZIP-code examples with incomplete spans or incorrect labels, and Hazard wordings with unrelated extra detections, were excluded; their raw results remain in the separate acceptance records. These curated cases demonstrate behavior on the displayed inputs, not general model accuracy.
 
+## Hallucination
+
+Choose **Hallucination**, enter a question, the evidence to use, and an answer, then click **Check the answer**. Vela Halu highlights answer spans that are unsupported by the supplied evidence. A result with no highlighted spans means no unsupported spans were detected; it is not a guarantee of factual correctness. FactCheck is a separate task that identifies requests needing factual knowledge.
+
+Five examples cover a wrong opening time, a supported answer, a changed quantity, an invented facility, and a tool-result mismatch. All five were checked with the published model in real CPU inference. Editing any of the three fields clears the previous result and cancels pending work. The combined input must fit 512 tokens including the prompt format and special tokens; each field accepts up to 16,000 characters. Overlong inputs are rejected without truncation.
+
 ## Routing
 
 Routing opens by default. Its shared canvas connects the prompt, the Vela Encoder compass card, and compact editable destinations. Scenario selection, examples, and reset share a toolbar below the canvas.
@@ -78,16 +85,17 @@ Clustering has the same queue, cancellation, and model-cache behavior as the oth
 
 ## Model contracts
 
-The September 14, 2026 semantic audit of the 20 built-in prompts found 19 aligned with their expected meaning and one miss: Guard's first, short instruction-override example returned `benign` (66.37%) instead of `jailbreak`. After that audit, the user requested a clearer positive Guard demo. Its new default query, a detailed developer-mode override, returned jailbreak 99.9923% on two identical real runs. Guard now offers two examples: the clear override and an ordinary explanatory request. At that point the ten model demos contained 20 examples, alongside 30 Routing examples. Subsequent curated PII and Hazard additions brought the total to 63 examples; three Clustering batches brought it to 66, and four additional Similarity examples bring the current total to 70. The original missed prompt and its raw result remain in the acceptance records; replacing a demonstration does not resolve the model’s short-override limitation. These example checks do not estimate general model accuracy.
+The September 14, 2026 semantic audit of the 20 built-in prompts found 19 aligned with their expected meaning and one miss: Guard's first, short instruction-override example returned `benign` (66.37%) instead of `jailbreak`. After that audit, the user requested a clearer positive Guard demo. Its new default query, a detailed developer-mode override, returned jailbreak 99.9923% on two identical real runs. Guard now offers two examples: the clear override and an ordinary explanatory request. At that point the ten model demos contained 20 examples, alongside 30 Routing examples. Subsequent curated PII and Hazard additions brought the total to 63 examples; three Clustering batches brought it to 66, and four additional Similarity examples brought the total to 70. Five Hallucination examples bring the current total to 75. The original missed prompt and its raw result remain in the acceptance records; replacing a demonstration does not resolve the model’s short-override limitation. These example checks do not estimate general model accuracy.
 
 - Domain, FactCheck, Modality, Feedback, Guard, and Safety return softmax label scores.
 - FactCheck identifies a need for external factual knowledge or retrieval; it does not decide whether a claim is true. Guard concerns instruction attacks, while Safety and Hazard concern content risk. A risk signal alone does not prescribe refusal.
+- Hallucination pairs the question and supplied evidence with an answer and highlights answer tokens whose hallucination score is strictly above 0.5. It preserves the published model’s native YaRN configuration. Span offsets are Unicode code points into the original answer; each span score is the maximum of its detected token scores.
 - PII returns 17 entity types from 35 BIO labels. `start` and `end` are Unicode code-point offsets into the original text. Entity scores average their token probabilities.
 - Hazard uses independent sigmoid scores and the saved per-category thresholds from its [pinned operating point](https://huggingface.co/llm-semantic-router/Vela-1.0-Encoder-307M-Hazard/blob/188325c44aaf929eeea4e24463ed2543549339b2/operating_point.json). The demo's 512-token maximum is inside one 2048-token window. Full-length clients must implement the documented overlapping-window policy.
 - Embedding uses its native ModernBERT checkpoint, attention-mask mean pooling in FP32, and L2 normalization, matching its SentenceTransformers configuration. Similarities are cosine scores between 768-dimensional vectors.
 - Reranker returns raw relevance logits. Its [pinned custom loader](https://huggingface.co/llm-semantic-router/Vela-1.0-Encoder-307M-Reranker/blob/a388e41cbbd5dc5f16b6389fa76d0b8b8a38a8bf/modeling_vela_reranker.py) was inspected before enabling `trust_remote_code`; it loads the native encoder and separate trained FP32 classification heads. It uses all 22 layers and 768 dimensions.
 
-Model IDs, immutable revisions, examples, and Hazard thresholds live in `catalog.py`; Routing scenarios and their verified configurations live in `routing_catalog.py`; Clustering examples, per-item limits, and the shared default cutoff live in `clustering_catalog.py`. Similarity use-case groups and verified configurations live in `similarity_catalog.py`. Upgrading a model requires reviewing its contract and rerunning real inference checks.
+Model IDs, immutable revisions, examples, and Hazard thresholds live in `catalog.py`; Routing scenarios and their verified configurations live in `routing_catalog.py`; Clustering examples, per-item limits, and the shared default cutoff live in `clustering_catalog.py`. Similarity use-case groups and verified configurations live in `similarity_catalog.py`. Hallucination examples and the fixed threshold live in `hallucination_catalog.py`. Upgrading a model requires reviewing its contract and rerunning real inference checks.
 
 ## Local run
 
@@ -112,6 +120,8 @@ uvicorn app:app --host 0.0.0.0 --port 7860 --workers 1 --no-access-log
 
 `POST /api/jobs` accepts `{"task":"domain","text":"Why do bond prices fall?","candidates":[]}` and returns HTTP 202 with `{id, status, position, queue_ms}`. Poll `GET /api/jobs/{id}` for status: `queued`, `loading`, `running`, `completed`, `failed`, or `cancelled`. Queued positions start at one; active and terminal jobs have position zero. A completed snapshot adds `result`, containing the analysis response. A failed snapshot adds a safe `error` string and `error_status` (422 for input limits, otherwise 503). Missing or expired IDs return HTTP 404. `DELETE /api/jobs/{id}` cancels pending or running work; terminal jobs retain their final status. A cached example can already be completed in the initial HTTP 202 response.
 
+Hallucination jobs use `{"task":"hallucination","text":"When does the museum open?","context":"The museum opens at 10:00.","answer":"The museum opens at 09:00."}`. All three text fields are required and preserved verbatim. The result has `kind: "hallucination"`, the original `answer`, `spans` with `start`, `end`, `text`, and `score`, plus `max_hallucination_score` across answer tokens. Nonempty `context` or `answer` fields are rejected for other tasks. Only exact matches of all three fields can reuse a public example result.
+
 Routing jobs use `{"task":"routing","text":"Plan a weekend trip.","categories":[{"id":"travel","name":"Travel","description":"Trip planning and itineraries"},{"id":"coding","name":"Coding","description":"Programming and debugging"}]}`. Routing returns `kind: "routing"`, `top_category_id`, sorted `items` with original category IDs and indices, `dimensions: 768`, `score_type: "cosine"`, and the top-two `margin`. Similarity and Reranker retain their existing six-candidate limit.
 
 Clustering jobs use `{"task":"clustering","texts":["I forgot my password.","How can I reset my password?","How do I grow tomatoes?"]}`. They return `kind: "clustering"`, indexed original `items`, the cosine `similarities` matrix, a full average-link `merges` hierarchy, the `default_threshold`, and `groups` containing member `indices` and the representative index. Tree leaves are the original indices; merge `i` creates node `n + i`. Groups are ordered by their first input index, with deterministic tie breaking. Nonempty query text, candidates, or categories are rejected for this task; `texts` are rejected for other tasks.
@@ -132,4 +142,4 @@ python smoke_test.py --task domain
 python smoke_test.py --task all
 ```
 
-The all-model smoke test invokes ten pinned checkpoints when their example results are not cached and may download about 12.3 GB. It runs requests sequentially. Unit tests and output-shape checks alone do not establish prediction quality.
+The all-model smoke test invokes eleven pinned checkpoints when their example results are not cached and may download about 13.5 GB. It runs requests sequentially. Unit tests and output-shape checks alone do not establish prediction quality.

app.py

@@ -14,6 +14,7 @@ from starlette.concurrency import run_in_threadpool
 
 from catalog import MAX_CANDIDATES, MAX_CLUSTER_ITEMS, MAX_ROUTING_CATEGORIES, TASKS, public_catalog
 from inference import InferenceEngine, InputLimitError
+from hallucination_catalog import MAX_HALLUCINATION_FIELD_CHARS
 from job_queue import InferenceQueue, JobFailure, QueueFull
 from request_limits import RequestSizeLimitMiddleware
 
@@ -26,7 +27,8 @@ def run_inference(payload, set_stage):
         with engine.slot:
             return engine.analyze_locked(payload["task"], payload["text"], payload["candidates"],
                                          on_stage=set_stage, categories=payload.get("categories", []),
-                                         texts=payload.get("texts", []))
+                                         texts=payload.get("texts", []),
+                                         context=payload.get("context", ""), answer=payload.get("answer", ""))
     except InputLimitError as exc:
         raise JobFailure(str(exc), 422) from None
     except Exception as exc:
@@ -79,6 +81,8 @@ class AnalyzeRequest(BaseModel):
     candidates: list[Candidate] = Field(default_factory=list, max_length=MAX_CANDIDATES)
     categories: list[RoutingCategory] = Field(default_factory=list, max_length=MAX_ROUTING_CATEGORIES)
     texts: list[ClusterText] = Field(default_factory=list, max_length=MAX_CLUSTER_ITEMS)
+    context: str = Field(default="", max_length=MAX_HALLUCINATION_FIELD_CHARS)
+    answer: str = Field(default="", max_length=MAX_HALLUCINATION_FIELD_CHARS)
 
     @field_validator("task")
     @classmethod
@@ -89,6 +93,14 @@ class AnalyzeRequest(BaseModel):
 
     @model_validator(mode="after")
     def valid_content(self):
+        if self.task == "hallucination":
+            if self.candidates or self.categories or self.texts:
+                raise ValueError("Hallucination accepts only a question, evidence, and answer.")
+            if not all(value.strip() for value in (self.text, self.context, self.answer)):
+                raise ValueError("Question, evidence, and answer cannot be blank.")
+            return self
+        if self.context or self.answer:
+            raise ValueError("Evidence and answer are only supported by Hallucination.")
         if self.task == "clustering":
             if self.text or self.candidates or self.categories:
                 raise ValueError("Clustering accepts only the texts list.")

catalog.py

@@ -8,6 +8,7 @@ from routing_catalog import MAX_ROUTING_CATEGORIES, ROUTING_SCENARIOS
 from similarity_catalog import SIMILARITY_EXAMPLES, SIMILARITY_EXAMPLE_GROUPS
 from clustering_catalog import (CLUSTERING_EXAMPLES, CLUSTER_DISTANCE_THRESHOLD,
                                 MAX_CLUSTER_ITEMS, MAX_CLUSTER_TOKENS)
+from hallucination_catalog import HALLUCINATION_EXAMPLES, MAX_HALLUCINATION_FIELD_CHARS
 
 _TASKS = [
     ("domain", "Domain", "Find the subject of a request across 14 domains.", "classification", "f6354f54adcf38770f635ad903be2b00577f6c11", [
@@ -73,6 +74,14 @@ TASKS = {
 
 TASKS["embedding"]["example_groups"] = SIMILARITY_EXAMPLE_GROUPS
 
+TASKS["hallucination"] = {
+    "id": "hallucination", "name": "Hallucination",
+    "description": "Find answer spans unsupported by the supplied evidence.",
+    "kind": "hallucination", "model": MODEL_PREFIX + "Halu",
+    "revision": "521cd05d15e1959e120d002663c3b775649ae4dd",
+    "examples": HALLUCINATION_EXAMPLES,
+}
+
 # Routing is another demo of the same Embedding checkpoint, not a new model.
 TASKS["routing"] = {
     "id": "routing", "name": "Routing", "description": "Route prompts to categories you define.",
@@ -117,4 +126,5 @@ def public_catalog():
         "max_tokens": MAX_TOKENS, "max_candidates": MAX_CANDIDATES,
         "max_routing_categories": MAX_ROUTING_CATEGORIES,
         "max_cluster_items": MAX_CLUSTER_ITEMS, "max_cluster_tokens": MAX_CLUSTER_TOKENS,
+        "max_hallucination_field_chars": MAX_HALLUCINATION_FIELD_CHARS,
     }}

hallucination.py

@@ -0,0 +1,87 @@
+"""Native Vela Halu input and answer-span contracts."""
+
+import math
+
+from hallucination_catalog import HALLUCINATION_THRESHOLD
+
+
+def evidence_prompt(question, context):
+    return f"User request: {question}\n\n{context}"
+
+
+def validate_hallucination_model(model):
+    config = model.config
+    if (type(model).__name__ != "ModernBertForTokenClassification"
+            or config.model_type != "modernbert"
+            or config.id2label != {0: "supported", 1: "hallucinated"}
+            or config.label2id != {"supported": 0, "hallucinated": 1}
+            or config.num_labels != 2
+            or config.reference_compile is not False
+            or config._attn_implementation != "sdpa"):
+        raise ValueError("Halu checkpoint differs from its native inference contract")
+    if (getattr(config, "rope_scaling", None) or {}).get("rope_type") != "yarn":
+        raise ValueError("Halu requires the published YaRN configuration")
+    count = 0
+    for _, module in model.named_modules():
+        if not hasattr(module, "rotary_emb"):
+            continue
+        rotary = module.rotary_emb
+        if (getattr(rotary, "rope_type", None) != "yarn"
+                or getattr(getattr(rotary, "rope_init_fn", None), "__name__", None)
+                != "_compute_yarn_parameters"):
+            raise ValueError("Halu did not instantiate its published YaRN rotary")
+        count += 1
+    if count != config.num_hidden_layers or count != 22:
+        raise ValueError("Halu rotary layer count differs from its foundation")
+
+
+def hallucination_result(answer, sequence_ids, offsets, scores):
+    """Use answer-only Unicode offsets and the published strict > 0.5 rule."""
+    if not len(sequence_ids) == len(offsets) == len(scores):
+        raise ValueError("Halu token scores and offsets differ in length")
+    spans, current = [], None
+    max_score = 0.0
+    previous_start = -1
+    answer_tokens = 0
+
+    def finish():
+        nonlocal current
+        if current is not None:
+            # Byte-fallback tokens can share a character while straddling the
+            # threshold. Present their positive character union once; ordinary
+            # nonoverlapping spans separated by a supported token stay distinct.
+            if spans and current["start"] < spans[-1]["end"]:
+                span = spans[-1]
+                span["end"] = max(span["end"], current["end"])
+                span["score"] = max(span["score"], current["score"])
+            else:
+                span = current
+                spans.append(span)
+            span["text"] = answer[span["start"]:span["end"]]
+            current = None
+
+    for sequence, (start, end), score in zip(sequence_ids, offsets, scores, strict=True):
+        score = float(score)
+        if not math.isfinite(score) or not 0 <= score <= 1:
+            raise ValueError("Invalid Halu token probability")
+        if sequence != 1 or end <= start:
+            continue
+        if (type(start) is not int or type(end) is not int
+                or not 0 <= start < end <= len(answer) or start < previous_start):
+            raise ValueError("Invalid Halu answer offsets")
+        previous_start = start
+        answer_tokens += 1
+        max_score = max(max_score, score)
+        if score > HALLUCINATION_THRESHOLD:
+            if current is None:
+                current = {"start": start, "end": end, "score": score}
+            else:
+                current["end"] = max(current["end"], end)
+                current["score"] = max(current["score"], score)
+        else:
+            finish()
+    finish()
+    if not answer_tokens:
+        raise ValueError("Halu found no evaluable answer tokens")
+    return {"kind": "hallucination", "answer": answer, "spans": spans,
+            "max_hallucination_score": max_score}

hallucination_catalog.py

@@ -0,0 +1,37 @@
+"""Public evidence-and-answer examples for Vela Halu."""
+
+MAX_HALLUCINATION_FIELD_CHARS = 16000
+HALLUCINATION_THRESHOLD = 0.5
+
+HALLUCINATION_EXAMPLES = [
+    {
+        "name": "Wrong opening time",
+        "text": "When does the museum open on Tuesday?",
+        "context": "The museum opens at 10:00 on Tuesday.",
+        "answer": "The museum opens at 09:00 on Tuesday.",
+    },
+    {
+        "name": "Supported answer",
+        "text": "When does the museum open on Tuesday?",
+        "context": "The museum opens at 10:00 on Tuesday.",
+        "answer": "The museum opens at 10:00 on Tuesday.",
+    },
+    {
+        "name": "Changed number",
+        "text": "How many seats does the shuttle have?",
+        "context": "The shuttle has 24 seats and runs every 30 minutes.",
+        "answer": "The shuttle has 42 seats and runs every 30 minutes.",
+    },
+    {
+        "name": "Invented detail",
+        "text": "What facilities does the library offer?",
+        "context": "The library offers free Wi-Fi and a reading room.",
+        "answer": "The library offers free Wi-Fi, a reading room, and a rooftop café.",
+    },
+    {
+        "name": "Tool output mismatch",
+        "text": "Summarize the update tool's result.",
+        "context": 'The update tool returned: {"status": "success", "files_updated": 3, "tests_passed": 12}.',
+        "answer": "The tool updated 7 files and all 12 tests passed.",
+    },
+]

inference.py

@@ -10,6 +10,7 @@ from collections import OrderedDict
 from catalog import (HAZARD_THRESHOLDS, MAX_CANDIDATES, MAX_CLUSTER_ITEMS, MAX_CLUSTER_TOKENS,
                      MAX_ROUTING_CATEGORIES, MAX_TOKENS, TASKS)
 from clustering import clustering_result
+from hallucination import evidence_prompt, hallucination_result, validate_hallucination_model
 
 
 class InputLimitError(ValueError):
@@ -154,10 +155,12 @@ class InferenceEngine:
         return AutoTokenizer.from_pretrained(spec["model"], revision=spec["revision"],
                                             trust_remote_code=False, use_fast=True)
 
-    def _check_tokens(self, tokenizer, task, text, candidates, texts=None):
+    def _check_tokens(self, tokenizer, task, text, candidates, texts=None, context="", answer=""):
         limit = MAX_CLUSTER_TOKENS if task == "clustering" else MAX_TOKENS
         if task == "clustering":
             inputs = [(item, None) for item in texts or []]
+        elif task == "hallucination":
+            inputs = [(evidence_prompt(text, context), answer)]
         elif task == "reranker":
             inputs = [(text, candidate) for candidate in candidates]
         elif task in {"embedding", "routing"}:
@@ -167,7 +170,8 @@ class InferenceEngine:
         for index, (first, second) in enumerate(inputs):
             count = len(tokenizer(first, text_pair=second, truncation=False)["input_ids"])
             if count > limit:
-                subject = (f"Text {index + 1}" if task == "clustering" else
+                subject = ("Question + evidence + answer" if task == "hallucination" else
+                           f"Text {index + 1}" if task == "clustering" else
                            f"Query + passage {index + 1}" if task == "reranker"
                            else "Input" if index == 0 else
                            f"Category {index}" if task == "routing" else f"Candidate {index}")
@@ -202,10 +206,12 @@ class InferenceEngine:
                                               code_revision=spec["revision"], **arguments)
         else:
             model_class = (AutoModel if task == "embedding" else
-                           AutoModelForTokenClassification if task == "pii" else
+                           AutoModelForTokenClassification if task in {"pii", "hallucination"} else
                            AutoModelForSequenceClassification)
             model = model_class.from_pretrained(spec["model"], trust_remote_code=False,
                                                  reference_compile=False, **arguments)
+        if task == "hallucination":
+            validate_hallucination_model(model)
         model.eval()
         with self._cache_lock:
             self._cache[task] = (model, tokenizer)
@@ -252,7 +258,8 @@ class InferenceEngine:
             vectors.append(vector)
         return vectors, hits
 
-    def analyze_locked(self, task, text, candidates, on_stage=None, categories=None, texts=None):
+    def analyze_locked(self, task, text, candidates, on_stage=None, categories=None, texts=None,
+                       context="", answer=""):
         """The caller owns slot for the entire load and inference lifecycle."""
         started = time.perf_counter()
         model_task = self._model_task(task)
@@ -262,7 +269,8 @@ class InferenceEngine:
         if on_stage:
             on_stage("loading" if cold else "running")
         tokenizer = self._get_tokenizer(task)
-        self._check_tokens(tokenizer, task, text, representations, texts=texts)
+        self._check_tokens(tokenizer, task, text, representations, texts=texts,
+                           context=context, answer=answer)
         self._ensure_model(task, tokenizer)
         load_ms = (time.perf_counter() - started) * 1000 if cold else 0.0
         if on_stage:
@@ -306,6 +314,15 @@ class InferenceEngine:
                 tokens = [(model.config.id2label[index], score, start, end)
                           for index, score, (start, end) in zip(indices.tolist(), scores.tolist(), offsets)]
                 result = pii_result(text, tokens)
+            elif task == "hallucination":
+                inputs = self._inputs(tokenizer, evidence_prompt(text, context), answer, offsets=True)
+                sequence_ids = inputs.sequence_ids(0)
+                offsets = inputs.pop("offset_mapping")[0].tolist()
+                logits = model(**inputs).logits[0]
+                if logits.shape[-1] != 2:
+                    raise ValueError("Halu must return exactly two token labels")
+                scores = logits.float().softmax(dim=-1)[:, 1].tolist()
+                result = hallucination_result(answer, sequence_ids, offsets, scores)
             else:
                 inputs = self._inputs(tokenizer, text)
                 logits = model(**inputs).logits[0].float().tolist()

job_queue.py

@@ -91,12 +91,16 @@ class InferenceQueue:
         candidates = tuple(payload.get("candidates", []))
         categories = self._category_key(payload.get("categories", []))
         texts = tuple(payload.get("texts", []))
+        context, answer = payload.get("context", ""), payload.get("answer", "")
         for example in spec.get("examples", []):
             if (payload.get("text", "") == example.get("text", "")
                     and candidates == tuple(example.get("candidates", []))
                     and categories == self._category_key(example.get("categories", []))
-                    and texts == tuple(example.get("texts", []))):
-                return (payload["task"], spec["model"], spec["revision"], payload.get("text", ""), candidates, categories, texts)
+                    and texts == tuple(example.get("texts", []))
+                    and context == example.get("context", "")
+                    and answer == example.get("answer", "")):
+                return (payload["task"], spec["model"], spec["revision"], payload.get("text", ""),
+                        candidates, categories, texts, context, answer)
         return None
 
     @staticmethod

smoke_test.py

@@ -2,7 +2,7 @@
 
 Usage: python smoke_test.py --base-url http://localhost:7860 --task all
 Uncached examples execute the pinned model; cached responses are reported as
-such. This can download ~12.3 GB of weights on its first run. This checks output
+such. This can download ~13.5 GB of weights on its first run. This checks output
 contracts, not semantic correctness; see the separately recorded example audit.
 """
 
@@ -27,7 +27,7 @@ def request(base, path, payload=None):
 def example_payload(task, example):
     # Catalog examples can also include display names and other metadata.
     return {"task": task["id"], **{key: value for key, value in example.items()
-                                  if key in {"text", "candidates", "categories", "texts"}}}
+                                  if key in {"text", "context", "answer", "candidates", "categories", "texts"}}}
 
 
 def verify_clustering(result, task, example):
@@ -88,6 +88,15 @@ def verify(response, task, example):
         assert result["entities"], "The public example should contain detected PII"
         for entity in result["entities"]:
             assert example["text"][entity["start"]:entity["end"]] == entity["text"]
+    elif result["kind"] == "hallucination":
+        assert result["answer"] == example["answer"]
+        assert 0 <= result["max_hallucination_score"] <= 1
+        previous_end = 0
+        for span in result["spans"]:
+            assert previous_end <= span["start"] < span["end"] <= len(result["answer"])
+            assert result["answer"][span["start"]:span["end"]] == span["text"]
+            assert 0.5 < span["score"] <= 1
+            previous_end = span["end"]
     elif result["kind"] == "routing":
         categories = example["categories"]
         assert len(result["items"]) == len(categories)
@@ -121,7 +130,7 @@ def main():
         parser.error("Unknown task")
     example_count = 0
     for task in selected:
-        examples = task["examples"] if task["kind"] == "clustering" else task["examples"][:1]
+        examples = task["examples"] if task["kind"] in {"clustering", "hallucination"} else task["examples"][:1]
         for example in examples:
             payload = example_payload(task, example)
             response = request(args.base_url, "/api/analyze", payload)

static/app.js

@@ -8,6 +8,7 @@ const presentation = {
   clustering: {name:'Clustering',group:'Retrieve',icon:'⁙',title:'Shared meaning. Natural company.',description:'Discover groups in a collection of texts, without predefined categories.',action:'Find the groups',note:'Adjust granularity to merge or separate groups. Open a group to inspect every text.',examples:[]},
   domain: {name:'Domain',group:'Understand',icon:'◈',title:'Every question has a world.',description:'Find the subject behind a request, across 14 knowledge domains.',action:'Find the domain',note:'Scores describe the model’s distribution across its domain labels.',examples:['A curious question','Another field']},
   factcheck: {name:'Fact check',group:'Understand',icon:'⊙',title:'Know when to look it up.',description:'Recognize requests that call for external factual knowledge.',action:'Check the need',note:'This predicts whether a fact check is needed. It does not verify whether a claim is true.',examples:['Factual knowledge','Creative writing']},
+  hallucination: {name:'Hallucination',group:'Understand',icon:'⌁',title:'An answer, grounded in evidence.',description:'Compare an answer with its evidence. Find passages that may not be supported.',action:'Check the answer',note:'Highlights reflect the evidence you provide, not a guarantee of real-world truth.',examples:[]},
   modality: {name:'Modality',group:'Understand',icon:'▧',title:'The right medium for the idea.',description:'Explore whether a request calls for text, an image, or both.',action:'Find the medium',note:'AR means text generation; DIFFUSION means image generation; BOTH means both.',examples:['An image idea','Words and pictures']},
   feedback: {name:'Feedback',group:'Understand',icon:'☷',title:'Listen a little closer.',description:'Understand the feedback hidden in a conversational response.',action:'Read the feedback',note:'Predictions distinguish satisfaction, clarification, correction, a different answer, and no feedback.',examples:['A little gratitude','A correction']},
   pii: {name:'Personal data',group:'Protect',icon:'◎',title:'Find what should stay private.',description:'Locate personal information, then see it highlighted or redacted.',action:'Detect PII',note:'Highlights show model predictions. Review them before relying on redaction.',examples:["Contact details","Date & age","Street address","Network log","Test card","Synthetic ID","Work profile"]},
@@ -35,16 +36,18 @@ function renderEmpty() {
   FlowUI.setTargets([]);
   if(selected==='routing' && catalog.routing) { RoutingUI.renderEmpty(); return; }
   if(selected==='clustering') { ClusteringUI.renderEmpty(); return; }
+  if(selected==='hallucination') { HallucinationUI.renderEmpty(); return; }
   $('output').innerHTML = '<div class="empty-state"><span class="flow-empty-label">A SIGNAL, WAITING TO BE FOUND</span><strong>Ready when you are.</strong><p>Choose an example or bring your own text. Run the model to see what it notices.</p></div>';
   $('result-meta').textContent=''; $('output-tabs').hidden=true; $('copy-button').hidden=true;
   $('output').setAttribute('aria-live','polite');
 }
 function setModelState(message) { $('model-state').replaceChildren(textNode('span','','hollow-dot'),textNode('span',message)); }
 function selectTask(id) {
-  if (!presentation[id] || (busy && !['routing','clustering'].includes(selected))) return;
+  if (!presentation[id] || (busy && !['routing','clustering','hallucination'].includes(selected))) return;
   if(busy) invalidateRouting();
   RoutingUI.leave();
   ClusteringUI.leave();
+  HallucinationUI.leave();
   FlowUI.leave();
   selected=id; lastResult=null; edited=false; view='highlight'; $('error').hidden=true;
   $('result-meta').classList.remove('stale-note');
@@ -55,13 +58,14 @@ function selectTask(id) {
   $('task-title').textContent=p.title; $('task-description').textContent=p.description; $('task-symbol').textContent=p.icon;
   $('run-label').textContent=p.action; $('task-note').textContent=p.note;
   const retrieval=['embedding','reranker'].includes(id);
-  $('candidates-wrap').hidden=!retrieval; $('input-label').textContent=id==='routing'?'YOUR PROMPT':id==='clustering'?'YOUR TEXTS':retrieval?'YOUR QUERY':'YOUR TEXT';
+  $('candidates-wrap').hidden=!retrieval; $('input-label').textContent=id==='routing'?'YOUR PROMPT':id==='clustering'?'YOUR TEXTS':id==='hallucination'?'YOUR QUESTION':retrieval?'YOUR QUERY':'YOUR TEXT';
   $('candidates-label').textContent=id==='embedding'?'CANDIDATES':'PASSAGES';
-  $('input-hint').textContent=id==='clustering'?'3–24 texts · 128 tokens each':'Up to 512 tokens';
-  $('input').maxLength=id==='clustering'?98351:12000;
-  $('input').placeholder=id==='clustering'?'One text per line. Let shared meaning find its shape.':'Write a little. Discover a lot.';
-  document.querySelector('.examples>span').textContent=id==='clustering'?'TRY A COLLECTION':'TRY A PROMPT';
-  $('input').value=''; $('candidates').value='';
+  $('input-hint').textContent=id==='clustering'?'3–24 texts · 128 tokens each':id==='hallucination'?'512 tokens total · inputs are never shortened':'Up to 512 tokens';
+  if(id==='hallucination')$('input').removeAttribute('maxlength');
+  else $('input').maxLength=id==='clustering'?98351:12000;
+  $('input').placeholder=id==='clustering'?'One text per line. Let shared meaning find its shape.':id==='hallucination'?'What question is the answer responding to?':'Write a little. Discover a lot.';
+  document.querySelector('.examples>span').textContent=id==='clustering'?'TRY A COLLECTION':id==='hallucination'?'TRY AN EXAMPLE':'TRY A PROMPT';
+  $('input').value=''; $('candidates').value=''; HallucinationUI.clear();
   $('model-link').textContent=`Vela-1.0 · ${task ? task.model.split('-').pop() : p.name} ↗`;
   $('model-link').href=`https://huggingface.co/${task?.model || 'collections/llm-semantic-router/vela-10'}`;
   setModelState(cachedTasks.includes(task?.model_task || id)?'Ready on CPU':'Loads on first run');
@@ -74,17 +78,18 @@ function selectTask(id) {
   renderExamples();
   FlowUI.enter(id,p,task);
   if(id==='clustering')ClusteringUI.enter();
+  if(id==='hallucination')HallucinationUI.enter();
   if(id==='routing' && task?.scenarios) RoutingUI.enter(task);
   else if (task?.examples?.length) loadExample(task.examples.find(example=>!grouped || example.group===similarityGroup));
   countCharacters(); renderEmpty();
   const activeButton=$('tasks').querySelector(`[data-task="${id}"]`), nav=$('tasks');
   if(activeButton && nav.scrollWidth>nav.clientWidth) nav.scrollLeft+=activeButton.getBoundingClientRect().left-nav.getBoundingClientRect().left-(nav.clientWidth-activeButton.clientWidth)/2;
 }
-function countCharacters() { $('char-count').textContent=selected==='clustering'?`${ClusteringUI.texts().length} / 24 texts`:`${Array.from($('input').value).length.toLocaleString()} characters`; }
+function countCharacters() { $('char-count').textContent=selected==='hallucination'?`${HallucinationUI.characterCount().toLocaleString()} characters total`:selected==='clustering'?`${ClusteringUI.texts().length} / 24 texts`:`${Array.from($('input').value).length.toLocaleString()} characters`; }
 function inputChanged() {
   countCharacters(); updateExampleNote(); $('error').hidden=true;
   if(selected==='routing'){RoutingUI.inputChanged();return;}
-  if(selected==='clustering'){invalidateRouting();return;}
+  if(['clustering','hallucination'].includes(selected)){invalidateRouting();return;}
   if(selected==='embedding'){
     lastResult=null;edited=false;$('result-meta').classList.remove('stale-note');renderEmpty();
     if(!busy)setModelState(cachedTasks.includes('embedding')?'Ready on CPU':'Loads on first run');
@@ -108,8 +113,10 @@ function renderExamples() {
   }
 }
 function loadExample(example) {
-  if(!example || (busy && !['routing','clustering'].includes(selected)))return;
-  $('input').value=selected==='clustering'?(example.texts || []).join('\n'):example.text;$('candidates').value=(example.candidates || []).join('\n');inputChanged();
+  if(!example || (busy && !['routing','clustering','hallucination'].includes(selected)))return;
+  $('input').value=selected==='clustering'?(example.texts || []).join('\n'):example.text;$('candidates').value=(example.candidates || []).join('\n');
+  if(selected==='hallucination')HallucinationUI.fillExample(example);
+  inputChanged();
 }
 function selectSimilarityGroup() {
   if(selected!=='embedding')return;
@@ -120,10 +127,11 @@ function selectSimilarityGroup() {
 }
 function setBusy(value) {
   busy=value; document.body.classList.toggle('is-running',value);
-  const editable=['routing','clustering'].includes(selected);
+  const editable=['routing','clustering','hallucination'].includes(selected);
   for (const el of document.querySelectorAll('.task-button,.example-button,#clear-button,#similarity-group')) el.disabled=value && !editable;
   $('run-button').disabled=value;
   $('input').readOnly=value && !editable; $('candidates').readOnly=value;
+  $('hallucination-context').readOnly=value && !editable; $('hallucination-answer').readOnly=value && !editable;
   $('run-label').textContent=value?'Finding the signal…':presentation[selected].action;
   $('run-arrow').textContent=value?'◌':'↗'; $('output').setAttribute('aria-busy',String(value));
 }
@@ -135,6 +143,7 @@ function renderLoading() {
   $('output-tabs').hidden=true; $('copy-button').hidden=true; $('result-meta').textContent='';
   $('output').innerHTML='<div class="empty-state loading"><span class="flow-empty-label">FOLLOWING THE SIGNAL</span><strong id="loading-title">Finding you a place…</strong><p id="loading-copy">Your request will wait its turn on the shared CPU.</p><div class="loading-track" aria-hidden="true"></div><button id="cancel-button" class="text-button cancel-button" type="button">Cancel request</button></div>';
   if(selected==='clustering')$('loading-title').textContent='Gathering shared meaning…';
+  if(selected==='hallucination')$('loading-title').textContent='Reading against the evidence…';
   $('cancel-button').addEventListener('click',cancelRun);
   setModelState('Submitting request…');
 }
@@ -176,7 +185,7 @@ function cancelRun() {
   }
 }
 function invalidateRouting() {
-  // Routing and Clustering allow edits while waiting. An old request must not
+  // Editable tasks allow changes while waiting. An old request must not
   // publish into new inputs, even when its submission response arrives late.
   if(busy) {
     cancelRun();
@@ -190,16 +199,17 @@ function invalidateRouting() {
   $('error').hidden=true;
   $('result-meta').classList.remove('stale-note');
   renderEmpty();
-  setModelState(cachedTasks.includes(catalog[selected]?.model_task || selected)?'Ready on CPU':selected==='clustering'?'Ready to group':'Ready to route');
+  setModelState(cachedTasks.includes(catalog[selected]?.model_task || selected)?'Ready on CPU':selected==='hallucination'?'Loads on first run':selected==='clustering'?'Ready to group':'Ready to route');
 }
 async function analyze() {
   if (busy) return;
   const taskId=selected, input=$('input').value, candidates=$('candidates').value.split('\n').map(t=>t.trim()).filter(Boolean);
+  if(taskId==='hallucination') { const error=HallucinationUI.validate(); if(error){showError(error);return;} }
   if (!input.trim()) { showError(taskId==='clustering'?'Add a collection of texts or choose an example first.':'Write a prompt or choose an example first.'); $('input').focus(); return; }
   if (['embedding','reranker'].includes(taskId) && (!candidates.length || candidates.length>6)) { showError(`Add between 1 and 6 ${taskId==='embedding'?'candidates':'passages'}, one per line.`); return; }
   if(taskId==='routing') { const error=RoutingUI.validate(); if(error){showError(error);return;} }
   if(taskId==='clustering') { const error=ClusteringUI.validate(); if(error){showError(error);return;} }
-  const payload=taskId==='clustering'?{task:taskId,texts:ClusteringUI.texts()}:{task:taskId,text:input,candidates:['embedding','reranker'].includes(taskId)?candidates:[]};
+  const payload=taskId==='hallucination'?{task:taskId,text:input,...HallucinationUI.values()}:taskId==='clustering'?{task:taskId,texts:ClusteringUI.texts()}:{task:taskId,text:input,candidates:['embedding','reranker'].includes(taskId)?candidates:[]};
   if(taskId==='routing')payload.categories=RoutingUI.categories();
   $('error').hidden=true; lastResult=null; edited=false; activeJob=null; cancelRequested=false;
   const controller=new AbortController(); runController=controller;
@@ -231,6 +241,7 @@ async function analyze() {
     let pollFailures=0;
     while(job && !cancelRequested && isCurrent() && Date.now()<deadline) {
       if(job.status==='completed') {
+        if(taskId==='hallucination' && (job.result?.result?.kind!=='hallucination' || job.result.result.answer!==payload.answer)) throw new Error('The result did not match this answer. Please run it again.');
         lastResult=job.result; lastInput=input; view='highlight'; clearInterval(runTimer);
         renderResult(); setModelState(lastResult.cache_hit?'Example result reused':'Ready on CPU'); break;
       }
@@ -275,8 +286,8 @@ async function analyze() {
 }
 function updateExampleNote() {
   const candidates=$('candidates').value.split('\n').map(text=>text.trim()).filter(Boolean);
-  const match=catalog[selected]?.examples?.findIndex(example=>selected==='clustering'?JSON.stringify(example.texts || [])===JSON.stringify(ClusteringUI.texts()):example.text===$('input').value && (selected==='embedding'?(example.candidates || []).join('\n')===$('candidates').value:JSON.stringify(example.candidates || [])===JSON.stringify(candidates)));
-  if(['embedding','clustering'].includes(selected))$('examples').querySelectorAll('.example-button').forEach(button=>button.setAttribute('aria-pressed',String(Number(button.dataset.exampleIndex)===match)));
+  const match=catalog[selected]?.examples?.findIndex(example=>selected==='hallucination'?HallucinationUI.matchesExample(example):selected==='clustering'?JSON.stringify(example.texts || [])===JSON.stringify(ClusteringUI.texts()):example.text===$('input').value && (selected==='embedding'?(example.candidates || []).join('\n')===$('candidates').value:JSON.stringify(example.candidates || [])===JSON.stringify(candidates)));
+  if(['embedding','clustering','hallucination'].includes(selected))$('examples').querySelectorAll('.example-button').forEach(button=>button.setAttribute('aria-pressed',String(Number(button.dataset.exampleIndex)===match)));
   const message=catalog[selected]?.example_notes?.[String(match)] || '';
   const note=$('example-note'); note.hidden=!message; note.textContent=message;
 }
@@ -291,6 +302,7 @@ function renderResult() {
   else if (['ranking','similarity'].includes(result.kind)) renderRanking(result);
   else if (result.kind==='routing') RoutingUI.renderResult(result);
   else if (result.kind==='clustering') ClusteringUI.renderResult(result);
+  else if (result.kind==='hallucination') HallucinationUI.renderResult(result);
   else { showError('This result format could not be displayed. Please try again.'); return; }
   if(!['routing','clustering'].includes(result.kind)) {
     updateResultConnections(result);
@@ -303,6 +315,10 @@ function renderResult() {
 }
 function updateResultConnections(result) {
   if(lastResult?.result!==result || selected==='routing') return;
+  if(result.kind==='hallucination') {
+    FlowUI.setTargets([{element:$('output').querySelector('.hallucination-answer'),winner:!edited}]);
+    return;
+  }
   if(result.kind==='pii') {
     const text=$('output').querySelector('.pii-text,.redacted') || $('output');
     FlowUI.setTargets([{element:text,winner:!edited}]);
@@ -357,8 +373,9 @@ function renderRanking(result) {
   if(result.kind==='similarity') $('output').append(textNode('div',`${result.dimensions} dimensions · normalized embeddings`,'result-caption'));
 }
 async function copyText(value,button) { try {await navigator.clipboard.writeText(value); const before=button.textContent;button.textContent='Copied';setTimeout(()=>button.textContent=before,1500);}catch{showError('Clipboard access is unavailable. Select the result text to copy it.');} }
-$('run-button').addEventListener('click',analyze); $('clear-button').addEventListener('click',()=>{$('input').value='';$('candidates').value='';inputChanged();$('input').focus();});
+$('run-button').addEventListener('click',analyze); $('clear-button').addEventListener('click',()=>{$('input').value='';$('candidates').value='';HallucinationUI.clear();inputChanged();$('input').focus();});
 $('input').addEventListener('input',inputChanged);$('candidates').addEventListener('input',inputChanged);
+$('hallucination-context').addEventListener('input',inputChanged);$('hallucination-answer').addEventListener('input',inputChanged);
 $('similarity-group').addEventListener('change',selectSimilarityGroup);
 $('highlight-button').addEventListener('click',()=>{view='highlight';renderResult();});$('redact-button').addEventListener('click',()=>{view='redact';renderResult();});
 $('copy-button').addEventListener('click',()=>{if(lastResult)copyText(JSON.stringify(lastResult.result.kind==='clustering'?ClusteringUI.exportResult():lastResult.result,null,2),$('copy-button'));});

static/flow.js

@@ -1,4 +1,4 @@
-/* One shared canvas and model node for all twelve Vela demos. */
+/* One shared canvas and model node for all Vela demos. */
 'use strict';
 window.FlowUI = (() => {
   const el = id => document.getElementById(id);
@@ -85,7 +85,7 @@ window.FlowUI = (() => {
     const body = node('div', '', 'flow-model-body');
     el('model-link').textContent = `Vela ${name} ↗`;
     const metadata = node('div', '', 'flow-model-metadata');
-    const variant = ['embedding', 'routing', 'clustering'].includes(id) ? '768D' : {classification:'Classifier', pii:'Token tags', hazard:'Multi-label', ranking:'Cross-encoder'}[task?.kind] || 'Encoder';
+    const variant = ['embedding', 'routing', 'clustering'].includes(id) ? '768D' : {classification:'Classifier', pii:'Token tags', hazard:'Multi-label', hallucination:'Grounding', ranking:'Cross-encoder'}[task?.kind] || 'Encoder';
     metadata.append(node('span', '307M'), node('i', '·'), node('span', variant));
     body.append(el('model-link'), metadata, el('model-state'));
     modelCard.append(art, body);

static/hallucination.css

@@ -0,0 +1,75 @@
+/* Three source fields and quiet copper annotations share the Vela field guide. */
+.hallucination-mode #input {
+  min-height: 82px;
+  max-height: 170px;
+  font-size: 13px;
+}
+.hallucination-fields label {
+  display: block;
+  border-top: 1px solid #e2e7dc;
+  padding: 12px 16px 0;
+  color: #61725a;
+  font: 8px var(--mono);
+  letter-spacing: 1px;
+}
+.hallucination-mode .hallucination-fields textarea {
+  display: block;
+  min-height: 110px;
+  max-height: 240px;
+  width: 100%;
+  padding: 10px 16px 14px;
+  font-size: 12px;
+  line-height: 1.8;
+  resize: vertical;
+}
+.hallucination-mode .editor-foot {
+  align-items: flex-start;
+  gap: 12px;
+  line-height: 1.65;
+}
+.hallucination-mode #char-count { white-space: nowrap; }
+.hallucination-mode .hallucination-result-title {
+  font-size: 25px;
+  line-height: 1.3;
+}
+.hallucination-answer {
+  border: 1px solid #d9dfcf;
+  border-radius: 5px;
+  padding: 18px;
+  background: #fcfaf5;
+  color: var(--ink);
+  font-size: 13px;
+  line-height: 2;
+  white-space: pre-wrap;
+  overflow-wrap: anywhere;
+  max-height: 520px;
+  overflow: auto;
+}
+.hallucination-span {
+  background: #ead5b8;
+  color: #5b402b;
+  border-bottom: 1px solid #b78053;
+  border-radius: 2px;
+  box-decoration-break: clone;
+  -webkit-box-decoration-break: clone;
+}
+.hallucination-legend {
+  margin-top: 12px;
+  padding-left: 11px;
+  border-left: 3px solid #bb895d;
+  color: #795437;
+  font-size: 9px;
+  line-height: 1.7;
+}
+.hallucination-result-note {
+  margin: 13px 0 0;
+  color: #66735d;
+  font-size: 10px;
+  line-height: 1.8;
+}
+@media (max-width: 700px) {
+  .hallucination-mode #input { min-height: 90px; font-size: 14px; }
+  .hallucination-mode .hallucination-fields textarea { font-size: 14px; min-height: 115px; }
+  .hallucination-answer { padding: 15px; }
+  .hallucination-mode .editor-foot { flex-direction: column; gap: 3px; }
+}

static/hallucination.js

@@ -0,0 +1,94 @@
+/* Grounding highlights use the answer's Unicode code-point offsets. */
+'use strict';
+window.HallucinationUI = (() => {
+  const el = id => document.getElementById(id);
+  const node = (tag, text = '', className = '') => {
+    const element = document.createElement(tag);
+    element.textContent = text;
+    if (className) element.className = className;
+    return element;
+  };
+  function enter() {
+    document.querySelector('.task-main').classList.add('hallucination-mode');
+    el('hallucination-fields').hidden = false;
+  }
+  function leave() {
+    document.querySelector('.task-main').classList.remove('hallucination-mode');
+    el('hallucination-fields').hidden = true;
+  }
+  function values() {
+    return {context: el('hallucination-context').value, answer: el('hallucination-answer').value};
+  }
+  function fillExample(example) {
+    el('hallucination-context').value = example.context || '';
+    el('hallucination-answer').value = example.answer || '';
+  }
+  function clear() { fillExample({}); }
+  function matchesExample(example) {
+    const current = values();
+    return example.text === el('input').value && example.context === current.context && example.answer === current.answer;
+  }
+  function validate() {
+    for (const [id, message] of [
+      ['input', 'Add the question or choose an example first.'],
+      ['hallucination-context', 'Add the evidence the answer should be based on.'],
+      ['hallucination-answer', 'Add the answer you want to check.'],
+    ]) {
+      if (!el(id).value.trim()) { el(id).focus(); return message; }
+    }
+    return null;
+  }
+  function characterCount() {
+    const current = values();
+    return [el('input').value, current.context, current.answer].reduce((total, value) => total + Array.from(value).length, 0);
+  }
+  function renderEmpty() {
+    const empty = node('div', '', 'empty-state hallucination-empty');
+    empty.append(node('span', 'AN ANSWER, IN THE LIGHT OF EVIDENCE', 'flow-empty-label'));
+    empty.append(node('strong', 'See what the evidence supports.'));
+    empty.append(node('p', 'Add a question, its evidence, and an answer. Potentially unsupported passages will appear here.'));
+    el('output').replaceChildren(empty);
+    el('result-meta').textContent = '';
+    el('output-tabs').hidden = true;
+    el('copy-button').hidden = true;
+    el('output').setAttribute('aria-live', 'polite');
+  }
+  function segments(answer, spans) {
+    if (typeof answer !== 'string' || !answer.trim() || !Array.isArray(spans)) throw new Error('The answer highlights could not be read. Please try again.');
+    const chars = Array.from(answer), result = [];
+    let previous = 0;
+    for (const span of [...spans].sort((a, b) => a.start - b.start)) {
+      if (!Number.isInteger(span.start) || !Number.isInteger(span.end) || span.start < previous || span.end <= span.start || span.end > chars.length || !Number.isFinite(span.score) || span.score < 0 || span.score > 1) {
+        throw new Error('The answer highlights did not align with the text. Please try again.');
+      }
+      const text = chars.slice(span.start, span.end).join('');
+      if (span.text !== undefined && span.text !== text) throw new Error('The answer highlights did not match the text. Please try again.');
+      if (span.start > previous) result.push({text: chars.slice(previous, span.start).join(''), highlighted: false});
+      result.push({text, highlighted: true, score: span.score});
+      previous = span.end;
+    }
+    if (previous < chars.length) result.push({text: chars.slice(previous).join(''), highlighted: false});
+    return result;
+  }
+  function renderResult(result) {
+    // Validate the entire response before creating a reassuring empty-span state.
+    const parts = segments(result.answer, result.spans);
+    const count = result.spans.length;
+    const title = count ? `${count} potentially unsupported ${count === 1 ? 'span' : 'spans'}` : 'No unsupported spans detected';
+    const heading = node('div', title, 'result-title hallucination-result-title');
+    const answer = node('div', '', 'hallucination-answer');
+    answer.setAttribute('aria-label', 'Answer with potentially unsupported spans highlighted');
+    for (const part of parts) {
+      if (!part.highlighted) { answer.append(document.createTextNode(part.text)); continue; }
+      const mark = node('mark', part.text, 'hallucination-span');
+      mark.title = `Potentially unsupported · model score ${(part.score * 100).toFixed(1)}%`;
+      answer.append(mark);
+    }
+    const note = node('p', 'Assessed against the evidence you provided. This does not guarantee factual accuracy beyond that evidence.', 'hallucination-result-note');
+    const children = [heading, answer];
+    if (count) children.push(node('div', 'Highlighted · potentially unsupported by the evidence', 'hallucination-legend'));
+    children.push(note);
+    el('output').replaceChildren(...children);
+  }
+  return {enter, leave, values, fillExample, clear, matchesExample, validate, characterCount, renderEmpty, renderResult, segments};
+})();

static/index.html

@@ -4,7 +4,7 @@
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
   <meta name="theme-color" content="#f5f2e9">
-  <meta name="description" content="Explore Vela 1.0: ten specialized models for understanding, protecting, and retrieving language.">
+  <meta name="description" content="Explore Vela 1.0: eleven specialized models for understanding, protecting, grounding, and retrieving language.">
   <title>Vela Studio — A better direction.</title>
   <link rel="icon" href="/static/favicon.svg" type="image/svg+xml">
   <link rel="preconnect" href="https://fonts.googleapis.com">
@@ -14,10 +14,12 @@
   <link rel="stylesheet" href="/static/flow.css?v=e7df90552db0">
   <link rel="stylesheet" href="/static/routing.css?v=d2bea5f416">
   <link rel="stylesheet" href="/static/clustering.css?v=1a774c530cb1">
-  <script src="/static/flow.js?v=7b6d347f2d93" defer></script>
+  <link rel="stylesheet" href="/static/hallucination.css?v=ddd852a221a6">
+  <script src="/static/hallucination.js?v=76e36d418177" defer></script>
+  <script src="/static/flow.js?v=0d15a0d480c5" defer></script>
   <script src="/static/routing.js?v=c3f8df099d" defer></script>
   <script src="/static/clustering.js?v=531c6c50bc4f" defer></script>
-  <script src="/static/app.js?v=0cd9c7ee6f1a" defer></script>
+  <script src="/static/app.js?v=8eeeb11f8795" defer></script>
 </head>
 <body>
   <a class="skip-link" href="#studio">Skip to playground</a>
@@ -41,7 +43,7 @@
         <span class="art-caption">FIG. 01 — A LITTLE INTELLIGENCE GOES A LONG WAY.</span>
       </div>
     </section>
-    <div class="specimen-strip"><span><b>307M</b> parameters per model</span><span><b>10</b> specialized models</span><span><i class="status-dot"></i> Open weights, open possibilities</span><a href="https://huggingface.co/llm-semantic-router/Vela-1.0-Encoder-307M" target="_blank" rel="noopener">Meet the encoder <span>↗</span></a></div>
+    <div class="specimen-strip"><span><b>307M</b> parameters per model</span><span><b>11</b> specialized models</span><span><i class="status-dot"></i> Open weights, open possibilities</span><a href="https://huggingface.co/llm-semantic-router/Vela-1.0-Encoder-307M" target="_blank" rel="noopener">Meet the encoder <span>↗</span></a></div>
     <main id="studio" tabindex="-1">
       <div class="section-title"><div><span class="eyebrow">THE INTERACTIVE FIELD GUIDE</span><h2>The signal room<span>.</span></h2></div><div class="session-badge"><i class="status-dot"></i><span>Vela 1.0</span><span class="badge-divider">/</span> CPU</div></div>
       <div class="workbench">
@@ -51,7 +53,7 @@
           <div class="model-strip"><a id="model-link" target="_blank" rel="noopener">Vela-1.0 · Embedding <span>↗</span></a><span id="model-state" role="status" aria-live="polite"><span class="hollow-dot"></span> Loads on first run</span></div>
           <section id="routing-scenarios" class="routing-scenarios" aria-label="Routing scenario" hidden></section>
           <div class="panels">
-            <section class="input-panel"><div class="panel-heading"><label id="input-label" for="input">YOUR PROMPT</label><button class="text-button" id="clear-button">Clear <span>×</span></button></div><textarea id="input" maxlength="12000" spellcheck="false" placeholder="Write a little. Discover a lot." aria-describedby="input-hint"></textarea><div id="candidates-wrap" hidden><label for="candidates"><span id="candidates-label">PASSAGES</span> <span>One per line · up to 6</span></label><textarea id="candidates" maxlength="36000" rows="5" spellcheck="false"></textarea></div><div class="editor-foot"><span id="input-hint">Up to 512 tokens</span><span id="char-count">0 characters</span></div></section>
+            <section class="input-panel"><div class="panel-heading"><label id="input-label" for="input">YOUR PROMPT</label><button class="text-button" id="clear-button">Clear <span>×</span></button></div><textarea id="input" maxlength="12000" spellcheck="false" placeholder="Write a little. Discover a lot." aria-describedby="input-hint"></textarea><div id="hallucination-fields" class="hallucination-fields" hidden><label for="hallucination-context">EVIDENCE</label><textarea id="hallucination-context" rows="4" spellcheck="false" placeholder="Paste the source material the answer should be based on." aria-describedby="input-hint"></textarea><label for="hallucination-answer">ANSWER TO CHECK</label><textarea id="hallucination-answer" rows="4" spellcheck="false" placeholder="Paste an answer to compare with the evidence." aria-describedby="input-hint"></textarea></div><div id="candidates-wrap" hidden><label for="candidates"><span id="candidates-label">PASSAGES</span> <span>One per line · up to 6</span></label><textarea id="candidates" maxlength="36000" rows="5" spellcheck="false"></textarea></div><div class="editor-foot"><span id="input-hint">Up to 512 tokens</span><span id="char-count">0 characters</span></div></section>
             <section class="output-panel"><div class="panel-heading"><span>THE SIGNAL</span><div id="output-tabs" class="output-tabs" aria-label="PII output view" hidden><button id="highlight-button" aria-pressed="true">Highlight</button><button id="redact-button" aria-pressed="false">Redact</button></div><button class="text-button" id="copy-button" hidden>Copy</button></div><div id="output" class="output" aria-live="polite" aria-atomic="true"></div><div id="result-meta" class="result-meta"></div></section>
           </div>
           <div id="example-note" class="example-note" role="note" hidden></div>

tests/test_hallucination.py

@@ -0,0 +1,238 @@
+"""Evidence-pair, Unicode span, queue cache, and API contracts without weights."""
+
+from copy import deepcopy
+import math
+from types import SimpleNamespace
+import unittest
+from unittest.mock import Mock, patch
+
+from fastapi.testclient import TestClient
+
+import app as server
+from catalog import TASKS
+from hallucination import evidence_prompt, hallucination_result, validate_hallucination_model
+from hallucination_catalog import HALLUCINATION_EXAMPLES
+from inference import InferenceEngine, InputLimitError
+from job_queue import InferenceQueue
+
+
+def public_request(index=0):
+    example = HALLUCINATION_EXAMPLES[index]
+    return {"task": "hallucination", **{key: example[key] for key in ("text", "context", "answer")}}
+
+
+class HallucinationSpanTests(unittest.TestCase):
+    def test_only_answer_tokens_contribute_and_unicode_offsets_preserve_original(self):
+        answer = "😀 李雷在上海。"
+        result = hallucination_result(answer,
+            [None, 0, None, 1, 1, 1, 1, 1, 1, None],
+            [[0, 0], [0, 100], [0, 0], [0, 1], [2, 3], [3, 4], [4, 5], [5, 7], [7, 8], [0, 0]],
+            [1, .99, 1, .1, .8, .9, .5, .7, .2, 1])
+        self.assertEqual(result["kind"], "hallucination")
+        self.assertEqual(result["answer"], answer)
+        self.assertEqual(result["spans"], [
+            {"start": 2, "end": 4, "text": "李雷", "score": .9},
+            {"start": 5, "end": 7, "text": "上海", "score": .7},
+        ])
+        self.assertEqual(result["max_hallucination_score"], .9)
+
+    def test_strict_threshold_and_supported_token_separate_spans(self):
+        result = hallucination_result("ABC", [1, 1, 1], [[0, 1], [1, 2], [2, 3]], [.8, .5, .9])
+        self.assertEqual([span["text"] for span in result["spans"]], ["A", "C"])
+        self.assertEqual(hallucination_result("A", [1], [[0, 1]], [.5])["spans"], [])
+
+    def test_adjacent_positive_tokens_keep_internal_space_and_byte_fallback(self):
+        result = hallucination_result("𐍈 B", [1] * 5,
+                                      [[0, 1]] * 4 + [[2, 3]], [.7, .8, .9, .8, .6])
+        self.assertEqual(result["spans"], [{"start": 0, "end": 3, "text": "𐍈 B", "score": .9}])
+
+    def test_alternating_byte_fallback_scores_union_only_overlapping_characters(self):
+        result = hallucination_result("𐍈", [1] * 4, [[0, 1]] * 4, [.8, .1, .9, .5])
+        self.assertEqual(result["spans"], [{"start": 0, "end": 1, "text": "𐍈", "score": .9}])
+        # A supported byte token closes its span. A later positive token over
+        # another character remains distinct, even when the ranges just touch.
+        separate = hallucination_result("𐍈B", [1] * 3, [[0, 1], [0, 1], [1, 2]], [.8, .5, .9])
+        self.assertEqual(separate["spans"], [
+            {"start": 0, "end": 1, "text": "𐍈", "score": .8},
+            {"start": 1, "end": 2, "text": "B", "score": .9},
+        ])
+
+    def test_invalid_scores_offsets_or_empty_answer_mapping_fail_closed(self):
+        cases = [([1], [[0, 2]], [.7]), ([1], [[-1, 1]], [.7]),
+                 ([1], [[0, 1]], [math.nan]), ([1], [[0, 1]], [1.1]),
+                 ([0], [[0, 1]], [.9]), ([1, 1], [[0, 1]], [.7])]
+        for sequences, offsets, scores in cases:
+            with self.subTest(case=(sequences, offsets, scores)), self.assertRaises(ValueError):
+                hallucination_result("A", sequences, offsets, scores)
+
+
+class HallucinationAPITests(unittest.TestCase):
+    def setUp(self):
+        self.engine_patch = patch.object(server, "engine", InferenceEngine())
+        self.engine_patch.start()
+        self.addCleanup(self.engine_patch.stop)
+        self.queue = InferenceQueue(server.run_inference, tasks=TASKS)
+        self.queue_patch = patch.object(server, "jobs", self.queue)
+        self.queue_patch.start()
+        self.addCleanup(self.queue_patch.stop)
+        self.client = TestClient(server.app).__enter__()
+        self.addCleanup(self.client.__exit__, None, None, None)
+
+    def test_all_three_original_fields_reach_both_endpoints_unchanged(self):
+        payload = {"task": "hallucination", "text": "  Question?\n", "context": " Evidence. ",
+                   "answer": " 😀 Answer.\n"}
+        with patch.object(server.engine, "analyze_locked", return_value={"result": {"kind": "hallucination"}}) as analyze:
+            created = self.client.post("/api/jobs", json=payload)
+            self.assertEqual(created.status_code, 202)
+            self.assertEqual(self.queue.wait(created.json()["id"])["status"], "completed")
+            self.assertEqual(analyze.call_args.args, ("hallucination", payload["text"], []))
+            self.assertEqual(analyze.call_args.kwargs["context"], payload["context"])
+            self.assertEqual(analyze.call_args.kwargs["answer"], payload["answer"])
+            self.assertEqual(self.client.post("/api/analyze", json=payload).status_code, 200)
+
+    def test_required_fields_char_limits_and_unrelated_fields_reject_without_echo(self):
+        invalid = []
+        for field in ("text", "context", "answer"):
+            for value in ("", "  \n", "x" * 16001, 123, None):
+                invalid.append(dict(public_request(), **{field: value}))
+            missing = public_request()
+            del missing[field]
+            invalid.append(missing)
+        invalid += [dict(public_request(), candidates=["passage"]),
+                    dict(public_request(), categories=[{"id": "a", "name": "A"}]),
+                    dict(public_request(), texts=["first", "second", "third"])]
+        with patch.object(server.engine, "analyze_locked") as analyze:
+            for payload in invalid:
+                for path in ("/api/jobs", "/api/analyze"):
+                    response = self.client.post(path, json=payload)
+                    self.assertEqual(response.status_code, 422)
+                    self.assertNotIn("The museum opens", response.text)
+            analyze.assert_not_called()
+
+    def test_existing_tasks_reject_nonempty_evidence_answer_and_keep_empty_defaults(self):
+        with patch.object(server.engine, "analyze_locked", return_value={"result": {}}) as analyze:
+            for task, spec in TASKS.items():
+                if task == "hallucination":
+                    continue
+                payload = {key: value for key, value in spec["examples"][0].items()
+                           if key in {"text", "candidates", "categories", "texts"}}
+                payload.update(task=task, context="", answer="")
+                self.assertEqual(self.client.post("/api/analyze", json=payload).status_code, 200, task)
+                for field in ("context", "answer"):
+                    count = analyze.call_count
+                    response = self.client.post("/api/analyze", json=dict(payload, **{field: "private@example.com"}))
+                    self.assertEqual(response.status_code, 422, task)
+                    self.assertNotIn("private@example.com", response.text)
+                    self.assertEqual(analyze.call_count, count)
+
+    def test_combined_token_error_is_a_422_and_releases_worker_slot(self):
+        message = "Question + evidence + answer has 513 tokens. This CPU demo accepts at most 512."
+        with patch.object(server.engine, "analyze_locked", side_effect=InputLimitError(message)):
+            response = self.client.post("/api/analyze", json=public_request())
+            self.assertEqual(response.status_code, 422)
+            self.assertEqual(response.json()["detail"], message)
+            self.assertFalse(server.engine.slot.locked())
+
+    def test_catalog_has_halu_pin_examples_and_demo_limits_without_loading(self):
+        catalog = self.client.get("/api/catalog").json()
+        spec = next(task for task in catalog["tasks"] if task["id"] == "hallucination")
+        self.assertEqual(spec["model"], "llm-semantic-router/Vela-1.0-Encoder-307M-Halu")
+        self.assertEqual(spec["revision"], "521cd05d15e1959e120d002663c3b775649ae4dd")
+        self.assertEqual(spec["kind"], "hallucination")
+        self.assertEqual(len(spec["examples"]), 5)
+        self.assertTrue(all({"name", "text", "context", "answer"} <= example.keys()
+                            for example in spec["examples"]))
+        self.assertEqual(catalog["limits"]["max_tokens"], 512)
+        self.assertEqual(catalog["limits"]["max_hallucination_field_chars"], 16000)
+        self.assertIsNone(self.client.get("/api/status").json()["resident_task"])
+
+
+class HallucinationInferenceTests(unittest.TestCase):
+    def test_512_token_pair_budget_preserves_exact_prompt_and_rejects_before_load(self):
+        observed = []
+
+        def tokenizer(text, text_pair=None, **kwargs):
+            observed.append((text, text_pair, kwargs))
+            return {"input_ids": [0] * (len(text.split()) + len(text_pair.split()) + 3)}
+
+        engine = InferenceEngine()
+        question, context, answer = " Q? ", " evidence " * 250, " answer " * 256
+        engine._check_tokens(tokenizer, "hallucination", question, [], context=context, answer=answer)
+        self.assertEqual(observed[-1][0], f"User request: {question}\n\n{context}")
+        self.assertEqual(observed[-1][1], answer)
+        self.assertIs(observed[-1][2]["truncation"], False)
+        with patch.object(engine, "_get_tokenizer", return_value=tokenizer), patch.object(engine, "_ensure_model") as load:
+            with self.assertRaisesRegex(InputLimitError, r"Question \+ evidence \+ answer has 513 tokens.*at most 512"):
+                engine.analyze_locked("hallucination", question, [], context=context, answer=answer + " extra")
+            load.assert_not_called()
+
+    def test_halu_loads_native_token_classification_cpu_contract_and_reuses_lru(self):
+        model = SimpleNamespace(eval=Mock())
+        token_loader = SimpleNamespace(from_pretrained=Mock(return_value=model))
+        wrong_loader = SimpleNamespace(from_pretrained=Mock(side_effect=AssertionError("wrong model class")))
+        transformers = SimpleNamespace(AutoModel=wrong_loader, AutoModelForSequenceClassification=wrong_loader,
+                                       AutoModelForTokenClassification=token_loader)
+        torch = SimpleNamespace(float32="float32", set_num_threads=lambda count: None)
+        engine, tokenizer = InferenceEngine(), object()
+        with patch.dict("sys.modules", {"torch": torch, "transformers": transformers}), \
+                patch("inference.validate_hallucination_model") as validate:
+            engine._ensure_model("hallucination", tokenizer)
+            engine._ensure_model("hallucination", tokenizer)
+            validate.assert_called_once_with(model)
+        token_loader.from_pretrained.assert_called_once_with(TASKS["hallucination"]["model"],
+            revision=TASKS["hallucination"]["revision"], torch_dtype="float32",
+            attn_implementation="sdpa", trust_remote_code=False, reference_compile=False)
+        self.assertEqual(engine.status()["cached_tasks"], ["hallucination"])
+        self.assertEqual(engine.model_cache_size, 2)
+
+    def test_wrong_labels_fail_before_a_model_can_enter_the_cache(self):
+        config = SimpleNamespace(model_type="modernbert", id2label={0: "hallucinated", 1: "supported"},
+                                 label2id={"supported": 1, "hallucinated": 0}, num_labels=2,
+                                 reference_compile=False, _attn_implementation="sdpa")
+        model = type("ModernBertForTokenClassification", (), {"config": config})()
+        with self.assertRaisesRegex(ValueError, "native inference contract"):
+            validate_hallucination_model(model)
+
+
+class HallucinationCacheTests(unittest.TestCase):
+    def test_only_full_exact_public_question_evidence_answer_can_hit_cache(self):
+        spec = deepcopy(TASKS["hallucination"])
+        calls = []
+
+        def runner(payload, stage):
+            calls.append(deepcopy(payload))
+            return {"task": "hallucination", "model": spec["model"], "revision": spec["revision"],
+                    "result": {"kind": "hallucination", "answer": payload["answer"], "spans": []}}
+
+        now = [100.0]
+        queue = InferenceQueue(runner, tasks={"hallucination": spec}, clock=lambda: now[0])
+        queue.start()
+        self.addCleanup(queue.stop)
+
+        def run(payload):
+            created = queue.submit(payload)
+            self.assertTrue(queue._jobs[created["id"]].done.wait(2))
+            return queue.get(created["id"])
+
+        self.assertFalse(run(public_request())["result"]["cache_hit"])
+        self.assertTrue(run(public_request())["result"]["cache_hit"])
+        # Same question/evidence with a different public answer is a separate key.
+        self.assertFalse(run(public_request(1))["result"]["cache_hit"])
+        self.assertTrue(run(public_request(1))["result"]["cache_hit"])
+        self.assertEqual(len(calls), 2)
+        for field in ("text", "context", "answer"):
+            custom = dict(public_request(), **{field: public_request()[field] + " private@example.com"})
+            for _ in range(2):
+                self.assertFalse(run(custom)["result"]["cache_hit"])
+        self.assertEqual(len(calls), 8)
+        self.assertEqual(queue.status()["example_cache_entries"], 2)
+        self.assertNotIn("private@example.com", str(queue._examples))
+        self.assertTrue(all(job.payload is None for job in queue._jobs.values()))
+        now[0] += 301
+        self.assertTrue(run(public_request())["result"]["cache_hit"])
+        spec["revision"] = "new-pinned-revision"
+        self.assertFalse(run(public_request())["result"]["cache_hit"])
+
+
+if __name__ == "__main__":
+    unittest.main()