Duplicate from meituan-longcat/LongCat-Flash-Omni

Co-authored-by: hc <diichen@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Meituan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in 
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,365 @@
+---
+license: mit
+library_name: LongCat-Flash-Omni
+pipeline_tag: text-generation
+tags:
+- transformers
+---
+
+# LongCat-Flash-Omni
+
+<div align="center">
+  <img src="https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/LongCat-Flash-Omni.svg" 
+       width="300" 
+       alt="LongCat Logo"/>
+</div>
+
+<div align="center" style="line-height: 1;">
+  <a href="https://longcat.ai/" target="_blank" style="margin: 2px;">
+    <img alt="Omni" src="https://img.shields.io/badge/🤖%20Omni-LongCat--Flash--Omni-ADFF2F?color=29E154&logoColor=white"  fill-opacity="1" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://github.com/meituan-longcat/LongCat-Flash-Omni">
+    <img alt="github" src="https://img.shields.io/badge/🤖%20Github-LongCat--Flash--Omni-ff6b6b?color=1783ff&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+
+<div align="center" style="line-height: 1;">
+  <a href="https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/wechat_official_accounts.jpg" target="_blank" style="margin: 2px;">
+    <img alt="Wechat" src="https://img.shields.io/badge/WeChat-LongCat-brightgreen?logo=wechat&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+  <a href="https://x.com/Meituan_LongCat" target="_blank" style="margin: 2px;">
+    <img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-LongCat-white?logo=x&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+
+<div align="center" style="line-height: 1;">
+  <a href="https://huggingface.co/meituan-longcat/LongCat-Flash-Omni/blob/main/LICENSE" style="margin: 2px;">
+    <img alt="License" src="https://img.shields.io/badge/License-MIT-f5de53?&color=f5de53" style="display: inline-block; vertical-align: middle;"/>
+  </a>
+</div>
+
+<p align="center">
+  <a href="https://github.com/meituan-longcat/LongCat-Flash-Omni/blob/main/tech_report.pdf"><b>Tech Report</b>&nbsp;📄</a>
+</p>
+
+## Model Introduction
+We introduce **LongCat-Flash-Omni**, a state-of-the-art open-source omni-modal model with 560 billion parameters (with 27B activated), excelling at real-time audio-visual interaction, which is attained by leveraging [LongCat-Flash](https://github.com/meituan-longcat/LongCat-Flash-Chat)'s high-performance Shortcut-connected Mixture-of-Experts (MoE) architecture with zero-computation experts, augmented by efficient multimodal perception and speech reconstruction modules. Through an effective curriculum-inspired progressive training strategy, our model achieves comprehensive multimodal capabilities while maintaining strong unimodal capability. Now, we open-source the model to foster future research and development in the community.
+
+### Model Architecture
+<div align="center">
+  <img src="https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/longcat_flash_omni_architecture.png" width="60%" alt="LongCat-Flash-Omni" />
+</div>
+
+
+### Key Features
+
+#### 🌟 SOTA and Unified Omni-Modal Model
+
+LongCat-Flash-Omni is an open-source omni-modal model that achieves state-of-the-art cross-modal comprehension performance. It seamlessly integrates powerful offline multi-modal understanding with real-time audio–visual interaction within a single all-in-one framework.
+
+
+#### 🌟 Large-Scale with Low-Latency Audio–Visual Interaction
+
+By leveraging an efficient LLM backbone, carefully designed lightweight modality encoders and decoder, and a chunk-wise audio–visual feature interleaving mechanism, LongCat-Flash-Omni achieves low-latency, high-quality audio–visual processing and streaming speech generation. It supports a context window of up to 128K tokens, enabling advanced capabilities in long-term memory, multi-turn dialogue, and temporal reasoning across multiple modalities.
+
+
+#### 🌟 Effective Early-Fusion Training
+
+The model adopts an innovative multi-stage pretraining pipeline that progressively incorporates text, audio, and visual modalities under a balanced data strategy and early-fusion training paradigm, ensuring strong omni-modal performance without degradation in any single modality.
+
+
+#### 🌟 Efficient Training Infrastructure
+
+Inspired by the concept of modality decoupling, we propose a Modality-Decoupled Parallelism training scheme that significantly enhances the efficiency of large-scale and highly challenging multimodal training.
+
+
+#### 🌟 Open-Source Contribution
+We provide a comprehensive overview of the training methodology and data strategies behind LongCat-Flash-Omni, and release the model to accelerate future research and innovation in omni-modal intelligence.
+
+
+For more detail, please refer to the comprehensive [***LongCat-Flash-Omni Technical Report***](https://github.com/meituan-longcat/LongCat-Flash-Omni/blob/main/tech_report.pdf).
+
+## Evaluation Results
+
+<details open>
+<summary>Omni-modality</summary>
+
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **Gemini-2.5-Flash (non-thinking)** | **Qwen3-Omni Instruct** | **Qwen2.5-Omni Instruct** |
+|-----------|-------------------------------|-----------------------------------|------------------------------|----------------------|-------------------------|
+| OmniBench | 61.38 | 66.80 | 54.99 | 58.41 | 48.16 |
+| WorldSense | 60.89 | 63.96 | 58.72 | 52.01 | 46.69 |
+| DailyOmni | 82.38 | 80.61 | 80.78 | 69.33 | 47.45 |
+| UNO-Bench | 49.90 | 64.48 | 54.30 | 42.10 | 32.60 |
+
+</details>
+
+<details>
+<summary>Vision</summary>
+
+#### Image-to-Text
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **Gemini-2.5-Flash (non-thinking)** | **Qwen3-Omni Instruct** | **Seed-1.6** | **GPT-4o-1120** | **Qwen3-VL-235B-A22B-Instruct** | **Qwen2.5-VL-72B-Instruct** |
+|-----------|-------------------------------|-----------------------------------|------------------------------|----------------------|----------|---------------|------------------------------|---------------------------|
+| **General** ||||||||||
+| MMBench-EN<sub>test</sub> | 87.5 | 89.8 | 89.3 | 86.8 | 88.5 | 83.7 | 88.3 | 88.6* |
+| MMBench-ZH<sub>test</sub> | 88.7 | 89.2 | 88.5 | 86.4 | 83.8 | 82.8 | 89.8 | 87.9* |
+| RealWorldQA | 74.8 | 76.0 | 73.9 | 72.9 | 74.5 | 74.1 | 79.3* | 75.7* |
+| MMStar | 70.9 | 78.5* | 75.5 | 68.5* | 71.5 | 63.2 | 78.4* | 68.2 |
+| **STEM & Reasoning** ||||||||||
+| MathVista<sub>mini</sub> | 77.9 | 77.7* | 77.1 | 75.9 | 78.7 | 62.8 | 84.9* | 74.8* |
+| MMMU<sub>val</sub> | 70.7 | 80.9* | 76.3 | 69.1* | 74.9 | 69.4 | 78.7* | 70.2* |
+| MMVet | 69.0 | 80.7 | 79.5 | 68.9 | 74.4 | 76.6 | 75.9 | 74.5 |
+| **Multi-Image** ||||||||||
+| BLINK | 63.1 | 70.0* | 65.7 | 56.1 | 65.0 | 65.5 | 70.7* | 60.1 |
+| MuirBench | 77.1 | 74.0* | 73.7 | 62.1 | 74.6 | 70.5 | 72.8* | 70.7* |
+| Mantis | 84.8 | 83.9 | 83.4 | 80.7 | 81.1 | 79.3 | 79.7 | 82.0 |
+| **Text Recognition & Chart/Document Understanding** ||||||||||
+| ChartQA | 87.6 | 71.7 | 77.6 | 86.8* | 82.4 | 74.5 | 89.2 | 89.5* |
+| DocVQA | 91.8 | 94.0* | 93.6* | 95.7 | 94.3 | 80.9 | 94.6 | 96.4* |
+| OCRBench | 84.9 | 87.2* | 85.6 | 85.5 | 85.6 | 82.3 | 91.2 | 88.5 |
+| OmniDocBench<sub>EN/ZH</sub>↓ | 22.8/29.0 | 31.9/24.5 | 22.8/32.9 | 28.4/40.5 | 22.0/27.6 | 25.9/37.7 | 13.6/17.5 | 22.6/32.4* |
+| **Grounding & Counting** ||||||||||
+| RefCOCO-avg | 92.3 | 75.4 | 71.9 | 89.3 | 80.2 | - | 87.1 | 90.3 |
+| CountBench | 92.4 | 91.0* | 78.6 | 90.0* | 94.1 | 85.6* | 94.3 | 93.6* |
+| **Graphical User Interface (GUI)** ||||||||||
+| VisualWebBench | 78.7 | 81.1 | 73.5 | 79.3 | 81.1 | 77.1 | 80.8 | 82.3* |
+| ScreenSpot-v2 | 91.2 | 75.8 | 63.9 | 94.7 | 91.7 | - | 93.4 | 92.9 |
+| AndroidControl<sub>low</sub> | 91.2 | 79.2 | 79.1 | 90.5 | 84.6 | 65.2 | 90.0 | 93.7* |
+| AndroidControl<sub>high</sub> | 75.6 | 60.8 | 55.5 | 70.8 | 55.2 | 41.7 | 74.1 | 67.4* |
+
+**Note**: Values marked with * are sourced from public reports. As GPT-4o does not support image grounding, we do not report its results on RefCOCO and ScreenSpot-v2
+
+---
+
+#### Video-to-Text
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **Gemini-2.5-Flash (non-thinking)** | **Qwen3-Omni Instruct** | **Seed-1.6** | **GPT-4o-1120** | **Qwen3-VL (235B-A22B-Instruct)** | **Qwen2.5-VL-72B-Instruct** |
+|-----------|-------------------------------|-----------------------------------|------------------------------|----------------------|----------|---------------|------------------------------|---------------------------|
+| **Short Video** ||||||||||
+| MVBench | 75.2 | 66.4 | 63.0 | 69.3* | 68.4 | 62.1 | 71.3 | 70.4* |
+| NextQA | 86.2 | 84.2 | 81.4 | 82.4 | 84.1 | 79.7 | 81.3 | 82.3 |
+| TempCompass | 82.2 | 80.8 | 80.2 | 73.5 | 79.4 | 76.4 | 80.5 | 74.8* |
+| **Long Video** ||||||||||
+| VideoMME (w/o audio) | 76.2 | - | - | 70.5* | 75.2 | 73.2 | 79.2* | 73.3* |
+| VideoMME (w/ audio) | 78.2 | 80.6* | 78.5 | 73.0 | - | - | - | - |
+| LongVideoBench | 69.3 | 69.4 | 66.4 | 65.4 | 64.8 | 63.9 | - | 60.7* |
+| **STEM & Reasoning** ||||||||||
+| MMVU | 67.1 | 75.6 | 72.4 | 62.4 | 67.3 | 67.4 | 69.3 | 62.9* |
+| Video-MMMU | 67.5 | 79.4* | 76.6 | 60.3 | 75.4 | 68.0 | 73.7 | 59.3 |
+
+**Note**: Values marked with * are sourced from public reports.
+
+</details>
+
+<details>
+<summary>Audio</summary>
+
+#### **Table 1: Automatic Speech Recognition (ASR) and Speech-to-Text Translation (S2TT)**
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **GPT-4o-Audio** | **Qwen3-Omni Instruct** | **Kimi-Audio** | **Step-Audio-2-mini** |
+|-----------|-------------------------------|-----------------------------------|--------------|----------------------|------------|-------------------|
+| **ASR** | | | | | | |
+| LibriSpeech (test-clean \| test-other) | 1.57 \| 4.01 | 1.74 \| 3.80 | 30.00 \| 41.83 | 1.22 \| 2.48 | 1.28 \| 2.42 | 1.33 \| 2.86 |
+| AISHELL-1 | 0.63 | 3.11 | 34.81 | 0.84 | 0.60 | 0.78 |
+| AISHELL-2 | 2.78 | 5.24 | 77.73 | 2.34 | 2.56 | 2.16 |
+| Fleurs (zh \| en) | 3.99 \| 5.02 | 2.24 \| 4.77 | 3.91 \| 5.56 | 2.20 \| 2.72 | 2.69 \| 4.44 | 2.53 \| 3.05 |
+| CommonVoice 15 (zh \| en) | 4.98 \| 13.59 | 47.30 \| 49.86 | 42.83 \| 23.88 | 4.31 \| 6.05 | 8.46 \| 7.92 | 5.00 \| 6.75 |
+| WenetSpeech (test-meeting \| test-net) | 6.69 \| 6.09 | 136.13 \| 32.82 | 54.35 \| 67.90 | 5.89 \| 4.69 | 6.28 \| 5.37 | 4.87 \| 4.82 |
+| **S2TT (BLEU)** | | | | | | |
+| CoVost2 en→zh | 47.23 | 41.94 | 29.32 | 48.72 | - | 49.12 |
+| CoVost2 zh→en | 27.32 | 25.38 | 16.01 | 21.51 | - | 29.47 |
+
+**Note**: ASR results are in CER/WER (lower is better), S2TT results are in BLEU score.  
+
+---
+
+#### **Table 2: Audio Understanding**
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **GPT-4o-Audio** | **Qwen3-Omni Instruct** | **Kimi-Audio** | **Step-Audio-2-mini** |
+|-----------|-------------------------------|-----------------------------------|--------------|----------------------|------------|-------------------|
+| MMAU | 75.90 | 72.80 | 68.40 | 77.50 | 65.20 | 73.20 |
+| VocalSound | 92.76 | 89.45 | 82.37 | 91.60 | 94.85 | 87.58 |
+| TUT2017 | 65.43 | 33.15 | 20.74 | 40.74 | 65.25 | 30.67 |
+| ClothoAQA | 72.83 | 69.67 | 61.87 | 75.16 | 72.21 | 68.39 |
+| Nonspeech7k | 93.79 | 87.59 | 72.28 | 80.83 | 93.93 | 73.24 |
+| CochlScene | 70.02 | 45.34 | 34.94 | 43.03 | 80.42 | 44.58 |
+| MELD | 54.60 | 46.74 | 39.00 | 50.80 | 59.13 | 31.44 |
+
+---
+
+#### **Table 3: Audio-to-Text Chat**
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **Gemini-2.5-Pro (ThinkingBudget128)** | **GPT-4o-Audio** | **Qwen3-Omni Instruct** | **Kimi-Audio** | **Step-Audio-2-mini** |
+|-----------|-------------------------------|-----------------------------------|--------------|----------------------|------------|-------------------|
+| **OpenAudioBench** | | | | | | |
+| LlamaQuestions | 83.33 | 83.00 | 86.30 | 83.30 | 79.33 | 69.70 |
+| ReasoningQA | 79.71 | 80.30 | 68.71 | 84.16 | 58.02 | 55.64 |
+| TriviaQA | 86.20 | 90.20 | 76.00 | 75.90 | 62.10 | 45.30 |
+| Webquestions | 76.00 | 80.90 | 81.20 | 75.20 | 70.20 | 54.40 |
+| AlpacaEval | 75.43 | 76.58 | 81.61 | 85.43 | 75.73 | 53.92 |
+| **VoiceBench** | | | | | | |
+| AlpacaEval | 4.94 | 4.70 | 4.73 | 4.74 | 4.46 | 3.84 |
+| CommonEval | 4.32 | 4.11 | 4.37 | 4.54 | 3.97 | 3.19 |
+| OpenBookQA | 93.41 | 95.16 | 87.90 | 89.70 | 83.52 | 72.97 |
+| SDQA | 82.46 | 83.54 | 90.10 | 76.90 | 63.12 | 44.85 |
+| MMSU | 81.95 | 88.32 | 78.90 | 69.00 | 62.17 | 52.00 |
+| AdvBench | 100 | 97.69 | 99.23 | 99.30 | 100 | 97.00 |
+| IFEval | 77.99 | 77.83 | 66.81 | 77.80 | 61.10 | 29.80 |
+
+</details>
+
+<details>
+<summary>Text</summary>
+
+| **Benchmark** | **LongCat-Flash-Omni Instruct** | **LongCat-Flash** | **DeepSeek V3.1** | **Qwen3 MoE-2507** | **Kimi-K2** | **GPT-4.1** | **Claude Sonnet-4** | **Gemini-2.5-Flash** |
+|-----------|-------------------------------|---------------|---------------|----------------|---------|---------|-----------------|------------------|
+| Architecture | MoE | MoE | MoE | MoE | MoE | - | - | - |
+| # Total Params | 560B | 560B | 671B | 235B | 1043B | - | - | - |
+| # Activated Params | 27B | 27B | 37B | 22B | 32B | - | - | - |
+| **General Domains** ||||||||||
+| MMLU<sub>(acc)</sub> | 90.30 | 89.71 | 90.96 | 90.23 | 89.86 | 89.64 | 91.75 | 86.33 |
+| MMLU-Pro<sub>(acc)</sub> | 82.73 | 82.68 | 84.45 | 84.83 | 82.06 | 81.72 | 83.74 | 81.95 |
+| CEval<sub>(acc)</sub> | 91.68 | 90.44 | 89.21 | 92.70 | 91.26 | 79.53 | 86.63 | 78.78 |
+| CMMLU<sub>(acc)</sub> | 89.39 | 84.34 | 88.04 | 88.14 | 89.66 | 77.65 | 86.51 | 78.30 |
+| **Instruction Following** ||||||||||
+| IFEval<sub>(acc)</sub> | 82.44 | 89.65 | 86.69 | 88.54 | 88.91 | 85.58 | 88.35 | 83.92 |
+| COLLIE<sub>(acc)</sub> | 45.69 | 57.10 | 43.80 | 49.71 | 56.34 | 50.00 | 51.22 | 48.60 |
+| Meeseeks-zh<sub>(acc)</sub> | 39.05 | 43.03 | 33.83 | 35.32 | 42.79 | 41.54 | 35.07 | 34.84 |
+| **Mathematical Reasoning** ||||||||||
+| MATH500<sub>(acc)</sub> | 97.60 | 96.40 | 96.08 | 98.80 | 97.60 | 90.60 | 93.80 | 98.40 |
+| AIME24<sub>(avg@10)</sub> | 72.92 | 70.42 | 66.30* | 81.67 | 69.60* | 47.00 | 47.00 | 79.67 |
+| BeyondAIME<sub>(avg@10)</sub> | 47.40 | 43.00 | 36.50 | 57.60 | 36.60 | 22.10 | 20.50 | 44.20 |
+| **General Reasoning** ||||||||||
+| GPQA-diamond<sub>(acc)</sub> | 74.41 | 73.23 | 74.90* | 77.43 | 75.76 | 67.68 | 70.71 | 80.30 |
+| DROP<sub>(f1)</sub> | 83.53 | 79.06 | 84.19 | 78.57 | 89.04 | 66.94 | 73.06 | 45.03 |
+| ZebraLogic<sub>(acc)</sub> | 86.00 | 89.30 | 85.30 | 94.22 | 89.11 | 56.30* | 80.10 | 57.00 |
+| GraphWalks-128k<sub>(precision)</sub> | 56.00 | 51.05 | 73.54 | 80.72 | 47.50 | 85.02 | 80.57 | 64.83 |
+| **Coding** ||||||||||
+| LiveCodeBench<sub>(pass@1)</sub> | 52.64 | 48.02 | 56.40* | 46.48 | 46.70 | 39.21 | 45.59 | 39.65 |
+| Humaneval+<sub>(pass@1)</sub> | 90.85 | 88.41 | 92.68 | 94.51 | 85.98 | 93.29 | 94.51 | 87.80 |
+| MBPP+<sub>(pass@1)</sub> | 80.16 | 79.63 | 79.89 | 79.89 | 81.75 | 79.37 | 80.16 | 76.19 |
+
+**Note**: Values marked with * are sourced from other public reports. Note that DeepSeek-V3.1, Qwen3-235B-A22B, Gemini2.5-Flash, and Claude4-Sonnet are evaluated under their non-thinking mode.
+
+</details>
+
+
+## Quick Start
+
+### Model Download
+
+LongCat-Flash-Omni is a MoE model, which means that the model weights are distributed across multiple devices. Therefore, during loading in Hugging Face Transformers or vLLM, model weights will be automatically downloaded based on the model name. However, if your runtime environment is not conducive to downloading weights during execution, you can refer to the following commands to manually download the model weights to a local directory:
+
+```bash
+# Download through Hugging Face
+pip install -U "huggingface_hub[cli]"
+huggingface-cli download meituan-longcat/LongCat-Flash-Omni --local-dir ./LongCat-Flash-Omni
+```
+
+### Usage
+
+We have implemented basic adaptations in SGLang to support running the Longcat-Flash-Omni model. Currently, the official SGLang does not natively support Longcat-Flash-Omni, so you can temporarily use our [development branch](https://github.com/XiaoBin1992/sglang/tree/longcat_omni_v0.5.3.post3) for local installation and testing.
+
+Due to its size of 560 billion parameters (560B), LongCat-Flash-Omni requires at least one node (e.g., 8×H20-141G) to host the model weights in FP8 format, and at least two nodes (e.g., 16×H800-80G) for BF16 weights. Detailed launch configurations are provided below.
+
+#### Installation
+* python >= 3.10.0 (Recommend to use Anaconda)
+* PyTorch >= 2.8 
+* CUDA >= 12.9
+
+```
+conda create -n longcat python=3.10
+conda activate longcat
+
+# install SGLang
+git clone -b longcat_omni_v0.5.3.post3 https://github.com/XiaoBin1992/sglang.git
+pushd sglang
+pip install -e "python"
+popd
+
+# install longcat-flash-omni demo
+git clone https://github.com/meituan-longcat/LongCat-Flash-Omni
+pushd LongCat-Flash-Omni
+git submodule update --init --recursive
+pip install -r requirements.txt
+popd
+```
+
+#### Demo
+
+The model can be served on your cluster using a combination of Tensor Parallelism and Expert Parallelism.
+Once all dependencies are installed, you can launch the demo using the following command.
+
+* single-node inference
+```bash
+python3 longcat_omni_demo.py \
+  --tp-size 8 \
+  --ep-size 8 \
+  --model-path where_you_download_model_dir \
+  --output-dir output
+```
+
+* multi-node inference
+```bash
+python3 longcat_omni_demo.py \
+  --tp-size 16 \
+  --ep-size 16 \
+  --nodes 2 \
+  --node-rank $NODE_RANK \
+  --dist-init-addr $MASTER_IP:5000 \
+  --model-path where_you_download_model_dir \
+  --output-dir output
+```
+
+> NOTE: Replace \$NODE\_RANK and \$MASTER\_IP with the corresponding values of your GPU machines.
+
+All test cases are defined in examples_dict.py, and additional test cases may be added as needed. After model execution, the generated results are saved in the directory specified by the --output-dir parameter.
+
+
+## Interaction with LongCat-Flash-Omni
+
+### Real-time Chat Website
+
+You can use LongCat-Flash-Omni (web version currently only supports audio interaction features) on [https://longcat.ai](https://longcat.ai). The full service will be provided in subsequent updates.
+
+### APP
+
+We are excited to announce that the LongCat-Flash-Omni app is now available for both Android and iOS. 
+
+For Android, you can download it from the following QR code.
+
+<img src=https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/android_app_qrcode.jpg width="200px">
+
+For iOS, you can download it by searching "LongCat" at App Store or QR code. Currently, only the Chinese App Store is supported.
+
+<img src=https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/ios_app_qrcode.jpg width="200px">
+
+
+## License Agreement
+
+The **model weights** are released under the **MIT License**. 
+
+Any contributions to this repository are licensed under the MIT License, unless otherwise stated. This license does not grant any rights to use Meituan trademarks or patents. 
+
+See the [LICENSE](https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/refs/heads/main/LICENSE) file for the full license text.
+
+## Usage Considerations 
+This model has not been specifically designed or comprehensively evaluated for every possible downstream application. 
+
+Developers should take into account the known limitations of large language models, including performance variations across different languages, and carefully assess accuracy, safety, and fairness before deploying the model in sensitive or high-risk scenarios. 
+It is the responsibility of developers and downstream users to understand and comply with all applicable laws and regulations relevant to their use case, including but not limited to data protection, privacy, and content safety requirements. 
+
+Nothing in this Model Card should be interpreted as altering or restricting the terms of the MIT License under which the model is released. 
+
+## Citation
+We kindly encourage citation of our work if you find it useful.
+
+```
+@misc{
+    title={LongCat-Flash-Omni Technical Report}, 
+    author={Meituan LongCat Team}, 
+    year={2025}, 
+    url={https://github.com/meituan-longcat/LongCat-Flash-Omni}, 
+}
+```
+
+## Contact
+Please contact us at <a href="mailto:longcat-team@meituan.com">longcat-team@meituan.com</a> or join our WeChat Group if you have any questions.
+
+#### WeChat Group
+<img src=https://raw.githubusercontent.com/meituan-longcat/LongCat-Flash-Omni/main/figures/wechat_qrcode.jpeg width="200px">

audio/audio_embeddings.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:691e4dd1450124c5571a0abe7c4fd63392bc9fd4fd501f4556ac1133b45b3cea
+size 404228184

audio/audio_encoder.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e6217e82dd201f081233a46711e08ccc2fed85431543829930123d03dcb14aa
+size 1024481990

audio/audio_output_layers.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8b34a30861bef825e45e5e8ed2fd3c2dd2f009eadbccb36613cf463ea15f5dc
+size 707398600

audio/audio_projector.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a66dc8801ed429dc5f9643762c08b43dfe7234bc57338b560d3033a1616bad0
+size 151021584

audio/config.json

@@ -0,0 +1,85 @@
+{
+    "_name_or_path": "",
+    "activation": "relu6",
+    "add_cross_attention": false,
+    "architectures": [
+        "LongCatAudioEncoder"
+    ],
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": null,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout": 0.0,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": null,
+    "exponential_decay_length_penalty": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "gradient_checkpointing": false,
+    "hidden_size": 6144,
+    "id2label": {
+        "0": "LABEL_0",
+        "1": "LABEL_1"
+    },
+    "input_size": 1200,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+        "LABEL_0": 0,
+        "LABEL_1": 1
+    },
+    "layer_drop": 0.0,
+    "left_context": 7,
+    "left_order": 10,
+    "left_stride": 1,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "min_length": 0,
+    "model_type": "",
+    "ndnn": 2,
+    "nlayer": 22,
+    "no_repeat_ngram_size": 0,
+    "normalize": "LayerNorm",
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_return_sequences": 1,
+    "num_right_layers": 6,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "prefix": null,
+    "problem_type": null,
+    "proj_size": 1536,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "right_context": 7,
+    "right_order": 1,
+    "right_stride": 1,
+    "sep_token_id": null,
+    "stride": 8,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": false,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": "float32",
+    "torchscript": false,
+    "transformers_version": "4.39.3",
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "vocab_size": 5252
+}

audio/preprocessor_config.json

@@ -0,0 +1,26 @@
+{
+    "fbank": {
+        "dither": 0.0,
+        "frame_length": 25,
+        "frame_shift": 10,
+        "preemphasis": 0.97,
+        "freq": 16000,
+        "high_freq": -200,
+        "low_freq": 40,
+        "num_mel_bins": 80
+    },
+    "delta": {
+        "delta_order": 0,
+        "window_size": 2
+    },
+    "cmvn": {
+        "global_cmvn": "/path/to/audio/global_cmvn"
+    },
+    "splice": {
+        "left": 7,
+        "right": 7,
+        "stride": 8,
+        "random_start": false,
+        "seed": 0
+    }
+}

audio_codec/LongCatAudioCodec_decoder_24k_4codebooks_aug_sft.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b2ed907fbefd6ca60cbb6078852233ce693ddddef05f1ca4e201b4fd3119189
+size 603637648

audio_codec/config.yaml

@@ -0,0 +1,11 @@
+codec_config:
+  codec_dimension: 1024
+  codec_dec_ratios: [8,6,6,5]
+  decoder_dim: 1536
+  semantic_dim: 1280
+  decoder_type: '24k'
+  ckpt_path: '/path/to/audio_codec/LongCatAudioCodec_decoder_24k_4codebooks_aug_sft.pt'
+  codec_codebook_size: 90
+  codec_codebook_search_dim: 8 
+  codec_codebooks: 3
+  semantic_token_nums: 8192

config.json

@@ -0,0 +1,37 @@
+{
+  "architectures": [
+    "LongcatFlashOmniForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+      "AutoConfig": "configuration_longcat_flash.LongcatFlashConfig",
+      "AutoModel": "modeling_longcat_flash.LongcatFlashModel",
+      "AutoModelForCausalLM": "modeling_longcat_flash.LongcatFlashForCausalLM"
+  },
+  "vocab_size": 131072,
+  "hidden_size": 6144,
+  "ffn_hidden_size": 12288,
+  "expert_ffn_hidden_size": 2048,
+  "num_layers": 28,
+  "num_attention_heads": 64,
+  "kv_lora_rank": 512,
+  "q_lora_rank": 1536,
+  "qk_rope_head_dim": 64,
+  "v_head_dim": 128,
+  "qk_nope_head_dim": 128,
+  "mla_scale_q_lora": true,
+  "mla_scale_kv_lora": true,
+  "routed_scaling_factor": 6.0,
+  "n_routed_experts": 512,
+  "max_position_embeddings": 131072,
+  "rms_norm_eps": 1e-5,
+  "use_cache": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "rope_theta": 10000000.0,
+  "attention_method": "MLA",
+  "zero_expert_num": 256,
+  "zero_expert_type": "identity",
+  "moe_topk": 12
+}

configuration_longcat_flash.py

@@ -0,0 +1,216 @@
+
+"""LongcatFlash model configuration"""
+
+from transformers.configuration_utils import PretrainedConfig
+from transformers.modeling_rope_utils import rope_config_validation
+
+
+LONGCAT_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
+
+
+class LongcatFlashConfig(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`LongcatFlashModel`]. It is used to instantiate an LongcatFlash
+    model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
+    defaults will yield a similar configuration to that of the LongcatFlash.
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+
+    Args:
+        vocab_size (`int`, *optional*, defaults to 131072):
+            Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the
+            `inputs_ids` passed when calling [`LongcatFlashModel`]
+        hidden_size (`int`, *optional*, defaults to 7168):
+            Dimension of the hidden representations.
+        ffn_hidden_size (`int`, *optional*, defaults to 18432):
+            Dimension of the MLP representations.
+        expert_ffn_hidden_size (`int`, *optional*, defaults to 2048):
+            Dimension of the MoE representations.
+        num_layers (`int`, *optional*, defaults to 61):
+            Number of hidden layers in the Transformer decoder.
+        num_attention_heads (`int`, *optional*, defaults to 128):
+            Number of attention heads for each attention layer in the Transformer decoder.
+        num_key_value_heads (`int`, *optional*, defaults to 128):
+            This is the number of key_value heads that should be used to implement Grouped Query Attention. If
+            `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
+            `num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When
+            converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
+            by meanpooling all the original heads within that group. For more details checkout [this
+            paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to
+            `num_attention_heads`.
+        n_routed_experts (`int`, *optional*, defaults to 256):
+            Number of routed experts.
+        routed_scaling_factor (`float`, *optional*, defaults to 2.5):
+            Scaling factor or routed experts.
+        kv_lora_rank (`int`, *optional*, defaults to 512):
+            Rank of the LoRA matrices for key and value projections.
+        q_lora_rank (`int`, *optional*, defaults to 1536):
+            Rank of the LoRA matrices for query projections.
+        qk_rope_head_dim (`int`, *optional*, defaults to 64):
+            Dimension of the query/key heads that use rotary position embeddings.
+        v_head_dim (`int`, *optional*, defaults to 128):
+            Dimension of the value heads.
+        qk_nope_head_dim (`int`, *optional*, defaults to 128):
+            Dimension of the query/key heads that don't use rotary position embeddings.
+        norm_topk_prob (`bool`, *optional*, defaults to `True`):
+            Whether to normalize the weights of the routed experts.
+        hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
+            The non-linear activation function (function or string) in the decoder.
+        max_position_embeddings (`int`, *optional*, defaults to 4096):
+            The maximum sequence length that this model might ever be used with.
+        rms_norm_eps (`float`, *optional*, defaults to 1e-06):
+            The epsilon used by the rms normalization layers.
+        use_cache (`bool`, *optional*, defaults to `True`):
+            Whether or not the model should return the last key/values attentions (not used by all models). Only
+            relevant if `config.is_decoder=True`.
+        pad_token_id (`int`, *optional*):
+            Padding token id.
+        bos_token_id (`int`, *optional*, defaults to 0):
+            Beginning of stream token id.
+        eos_token_id (`int`, *optional*, defaults to 1):
+            End of stream token id.
+        tie_word_embeddings (`bool`, *optional*, defaults to `False`):
+            Whether to tie weight embeddings
+        rope_theta (`float`, *optional*, defaults to 10000.0):
+            The base period of the RoPE embeddings.
+        attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`):
+            Whether to use a bias in the query, key, value and output projection layers during self-attention.
+        attention_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for the attention probabilities.
+        attention_method (`str`, *optional*, defaults to `"MLA"`):
+            The attention method to use.
+        initializer_range (`float`, *optional*, defaults to 0.006):
+            The initializer range for the model.
+        router_bias (`bool`, *optional*, defaults to `False`):
+            Whether to use a bias in the router.
+        zero_expert_num (`int`, *optional*, defaults to `None`):
+            The number of zero experts to use.
+        zero_expert_type (`str`, *optional*, defaults to `None`):
+            The type of zero expert to use.
+
+    ```python
+    >>> from transformers import LongcatFlashModel, LongcatFlashConfig
+
+    >>> # Initializing a LongcatFlash style configuration
+    >>> configuration = LongcatFlashConfig()
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "longcat_flash"
+    keys_to_ignore_at_inference = ["past_key_values"]
+    base_model_tp_plan = {
+        "layers.*.self_attn.k_proj": "colwise",
+        "layers.*.self_attn.v_proj": "colwise",
+        "layers.*.self_attn.o_proj": "rowwise",
+        "layers.*.mlp.experts.*.gate_proj": "local_colwise",
+        "layers.*.mlp.experts.*.up_proj": "local_colwise",
+        "layers.*.mlp.experts.*.down_proj": "local_rowwise",
+        "layers.*.mlps.*.gate_proj": "local_colwise",
+        "layers.*.mlps.*.up_proj": "local_colwise",
+        "layers.*.mlps.*.down_proj": "local_rowwise",
+    }
+    base_model_pp_plan = {
+        "embed_tokens": (["input_ids"], ["inputs_embeds"]),
+        "layers": (["hidden_states", "attention_mask"], ["hidden_states"]),
+        "norm": (["hidden_states"], ["hidden_states"]),
+    }
+
+    def __init__(
+        self,
+        vocab_size=131072,
+        hidden_size=7168,
+        ffn_hidden_size=18432,
+        expert_ffn_hidden_size=2048,
+        num_layers=61,
+        num_attention_heads=128,
+        num_key_value_heads=None,
+        n_routed_experts=256,
+        routed_scaling_factor=1,
+        kv_lora_rank=512,
+        q_lora_rank=1536,
+        qk_rope_head_dim=64,
+        v_head_dim=128,
+        qk_nope_head_dim=128,
+        mla_scale_q_lora=True,
+        mla_scale_kv_lora=True,
+        moe_topk=8,
+        norm_topk_prob=False,
+        hidden_act="silu",
+        max_position_embeddings=4096,
+        rms_norm_eps=1e-6,
+        use_cache=True,
+        pad_token_id=None,
+        bos_token_id=0,
+        eos_token_id=1,
+        tie_word_embeddings=False,
+        rope_theta=10000.0,
+        attention_bias=False,
+        attention_dropout=0.0,
+        attention_method='MLA',
+        initializer_range=0.006,
+        router_bias=False,
+        zero_expert_num=None,
+        zero_expert_type=None,
+        **kwargs,
+    ):
+        self.vocab_size = vocab_size
+        self.max_position_embeddings = max_position_embeddings
+        self.hidden_size = hidden_size
+        self.ffn_hidden_size = ffn_hidden_size
+        self.expert_ffn_hidden_size = expert_ffn_hidden_size
+        self.num_layers = num_layers
+        self.num_attention_heads = num_attention_heads
+        self.n_routed_experts = n_routed_experts
+        self.routed_scaling_factor = routed_scaling_factor
+        self.kv_lora_rank = kv_lora_rank
+        self.q_lora_rank = q_lora_rank
+        self.qk_rope_head_dim = qk_rope_head_dim
+        self.v_head_dim = v_head_dim
+        self.qk_nope_head_dim = qk_nope_head_dim
+        self.qk_head_dim = qk_nope_head_dim + qk_rope_head_dim
+        self.moe_topk = moe_topk
+        self.norm_topk_prob = norm_topk_prob
+        self.mla_scale_q_lora = mla_scale_q_lora
+        self.mla_scale_kv_lora = mla_scale_kv_lora
+        self.attention_method = attention_method
+        self.initializer_range = initializer_range
+        self.router_bias = router_bias
+        self.zero_expert_num = zero_expert_num
+        self.zero_expert_type = zero_expert_type
+
+        if self.attention_method == "MLA":
+            self.head_dim = qk_rope_head_dim
+        else:
+            ValueError('attention_method should be one of ["MLA"]')
+
+
+        if num_key_value_heads is None:
+            num_key_value_heads = num_attention_heads
+
+        self.num_key_value_heads = num_key_value_heads
+        self.hidden_act = hidden_act
+        self.rms_norm_eps = rms_norm_eps
+        self.use_cache = use_cache
+        self.rope_theta = rope_theta
+        self.attention_bias = attention_bias
+        self.attention_dropout = attention_dropout
+
+        rope_config_validation(self)
+
+        super().__init__(
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            eos_token_id=eos_token_id,
+            tie_word_embeddings=tie_word_embeddings,
+            **kwargs,
+        )
+
+    @property
+    def num_hidden_layers(self):
+        return self.num_layers
+
+
+__all__ = ["LongcatFlashConfig"]

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 3,
+  "transformers_version": "4.51.3"
+}

language_model_embedding.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f18abb8d0e4cc2fa99c4a5fa30f887a3a12a1a41c0732b79f8b2bd133202a76d
+size 1610614129

model_00001-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0739930e842bb5a86662e1d67f8c545d9ea230d5c57714354b7e34c82ed9a9c7
+size 1610612880

model_00002-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

model_00003-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

model_00004-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

model_00005-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

model_00006-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b744394fd0184b47ad5dd41d1f18e5ba86a7f39c9d751c1312000bcc1a45d6e
+size 15953918680

model_00007-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31aab2325632b27ee496e8f6dc004674288c85245331ff73a908a1c94f18e275
+size 15918289608

model_00008-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0923b82637807507c876cc3b08d566caa20b0371f50da986c8c1519728584ee6
+size 15852491488

model_00009-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0bcca8e97b15ed9d955aad63f4584b94ccf3ebc898b8774573e4a1f187ed518
+size 16111199992

model_00010-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a27f00c1962f37a158b37d69784b1d5e2dce5bbd73cabf73a0965fd0501eb4
+size 16048279128

model_00011-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdd2411cce434a2469cf023fb6fe193147ce693cb8bd64ffc585d5489b2dd85e
+size 15953918680

model_00012-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee9b9c8370f7b1dbed80225a816d378a112ed52132d5ec667f3e9dd7043c8982
+size 15918289608

model_00013-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0c54ae6ca7cd3144a6b7028c27e3fbe959a056f7120da3fcd83ead126d8c95c
+size 15852491488

model_00014-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb540e6244bcf1aef6614481443a1cd785a1b868953886667e796b6a2220d538
+size 16111199992

model_00015-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6bf32626e4b6cd01a6e4e2a71e2631e986ae47b94af32a2f8ebdeef5473e14db
+size 16048279128

model_00016-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0d99af3e8fecbb229cac4ed0e9d39b555a6e86abb069b1eb68cc12ad7a9d65e
+size 15953918680

model_00017-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b96034ec959b45273cb66bbf63940210f958eaeb962e1bf4da2dffe716ece71
+size 15918289608

model_00018-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae04faa2f8abc37e2aa992929d8a7fef9ec771e63344b952c92e23c2d285c525
+size 15852491488

model_00019-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54942379fa2969eab782d01bf8d1891c685890a3c470b8431f427406fab2b606
+size 16111199992

model_00020-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72157629e7f74c4ef8831a334b40efe93b7c03744a1a54514488cc9011fbc8a0
+size 16048279128

model_00021-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29e85eb3cd7b4dff4889e38501af42aaf16b7237f83622195c4115803982551d
+size 15953918680

model_00022-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7760c0e0fe4f02cc6d5c8fe099bdd1049b2ef3a6b1afe8872d5300f277e0ec3a
+size 15918289608

model_00023-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98161d555f0750b5e0f420f5aca0a3a69760ae74c2a9d3359eda0f9038ddf1cd
+size 15852491488

model_00024-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:599ef83350116b3247ce19cac87c502dfd5b5b2ba6fcce2304eb6395bbace8a8
+size 16111199992

model_00025-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0cb646c661f276d19267820f271efe4227fc6000b383f3057cd1f78f2210ea82
+size 16048279128

model_00026-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83fc5dd48b2dfa2d752f559e1c449c593a56ccd02108d610a45bc36530c89039
+size 15953918680

model_00027-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba0f653bac6db2981b25a3eee1405a9f3102bd2dfc39245ad69cbbd3ed271ea2
+size 15918289608

model_00028-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d69ab73062467f436dfb0e864d4586fcc76228386ecd6bf47b4208fa74b37a46
+size 15852491488

model_00029-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19a5333380e24abc9ec6264b14002da142803762b3b9dd3b1be5aa26ba519b48
+size 16111199992

model_00030-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5a0974c5e6a04ab2c718fa2858ade42b877cab87dd6256ea6b5a857e0e6d685
+size 16048279128

model_00031-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03a6400e25994e94dc4094497f2aa22d428622cc692b71c6a8b1e0da66e7ae5a
+size 15953919304

model_00032-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb6e570622a03d425e5c4a317d3018c541bf865993927a5111aa854c3038ce29
+size 15918290232

model_00033-of-00080.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:308d665912c8bee46a9b831d07c61497ebce03c0fe0f17bb50fecff18b2db5f8
+size 15852492112