FP8 card

README.md

@@ -0,0 +1,39 @@
+---
+license: cc-by-nc-4.0
+base_model: openjev/openjev
+tags:
+- decision-model
+- zero-shot-classification
+- agents
+- fp8
+- vllm
+---
+
+# OpenJev, FP8 checkpoint
+
+The FP8 file of [OpenJev](https://huggingface.co/openjev/openjev): the same weights stored as FP8 (e4m3, 128×128 weight blocks, dynamic activations), about 29 GB instead of 54 GB. Everything about the model, the API and the results is in the main card; this page only says how this file was checked and how to run it.
+
+## Measured, not assumed
+
+A one-pass readout lives on small score differences, so the file was re-run on the same frozen tests as the 16-bit model, with the same helper and the same fixed calibration:
+
+| test | 16-bit model as served | this FP8 file |
+|---|---|---|
+| 10,000 text questions, 34 public sources | 84.0% (8,403) | 84.2% (8,420); +0.17 points, 95% interval −0.13 to +0.45 |
+| 2,000 desktop screenshots, next action | 88.0% (1,759) | 88.0% (1,760); +0.05 points |
+
+About 2% of individual answers change between the two files (219 of 10,000 text answers, 40 of 2,000 screenshot answers), in both directions. The rule fixed before the run was "no more than 0.5 points below the served model"; both tests pass it.
+
+## Run it
+
+```bash
+vllm serve openjev/openjev-FP8 --served-model-name qwen --port 8000 --enable-prefix-caching \
+  --max-model-len 16384 --gpu-memory-utilization 0.90 --limit-mm-per-prompt '{"image":1}' --trust-remote-code \
+  --max-num-seqs 16 --max-logprobs 64 --gdn-prefill-backend triton --quantization fp8
+```
+
+Then start the helper from the main repository (`helper/shim.py`) exactly as its card describes; the calibration settings are the same as for the 16-bit model. The conversion receipt (`FP8_CONVERSION_RECEIPT.json`) records the source checkpoint and the conversion tool.
+
+## Licence
+
+Weights: CC BY-NC 4.0 (research and non-commercial use), the same as the main repository. Helper and serving files: Apache 2.0.