yuxintian commited on
Commit
65a6d1d
·
verified ·
1 Parent(s): f431df9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -7,7 +7,7 @@ license: mit
7
  <p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp;&nbsp; | &nbsp;&nbsp;🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a>&nbsp;&nbsp; | &nbsp;&nbsp;🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a>&nbsp;&nbsp;</p>
8
 
9
  # Introduction
10
- We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.4B(Non-emb 1.14B)** activated parameters per token.
11
  It is designed to deliver strong reasoning and agentic capabilities at low inference cost, making advanced model capabilities more accessible for local and resource-constrained deployment.
12
  BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
13
 
@@ -18,12 +18,12 @@ Key highlights of the model are summarized below:
18
  + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
19
 
20
  # Model Overview
21
- Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.4B(Non-emb 1.14B) parameters activated per token.
22
 
23
  The architecture of Ling-3.0-tiny is designed to make computational efficiency serve real-world agentic performance.
24
 
25
  + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer per 4-layer block) provides more efficient long-context processing;
26
- + A sparse MoE FFN with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.4B(Non-emb 1.14B) activated parameters per token.
27
  + Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
28
 
29
  Overall, these designs deliver the inference efficiency needed to deploy lightweight models in real-world agentic workflows.
@@ -35,7 +35,7 @@ Overall, these designs deliver the inference efficiency needed to deploy lightwe
35
  We evaluated Ling-3.0-tiny across agentic tasks, coding, long-context understanding, knowledge reliability, mathematical and scientific reasoning, and instruction following.
36
  Ling-3.0-tiny achieves a score of **25** on the Artificial Analysis Intelligence Index v4.1.1 and **16** on the Artificial Analysis Agentic Index.
37
  In Artificial Analysis testing, Ling-3.0-tiny reaches an output speed of over **160 tokens/s**, with approximately **18 seconds** of end-to-end latency for a 500-token response, including reasoning time.
38
- These results highlight the model's efficiency relative to its 1.4B(Non-emb 1.14B) activated parameter footprint.
39
 
40
  The following table presents representative benchmarks for Ling-3.0-tiny:
41
 
 
7
  <p align="center">🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp;&nbsp; | &nbsp;&nbsp;🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope </a>&nbsp;&nbsp; | &nbsp;&nbsp;🐙 <a href="https://openrouter.ai/inclusionai/ling-3.0-tiny:free">OpenRouter </a>&nbsp;&nbsp;</p>
8
 
9
  # Introduction
10
+ We are introducing **Ling-3.0-tiny**, a lightweight hybrid reasoning MoE model with **7.9B** total parameters and only **1.3B** activated parameters per token.
11
  It is designed to deliver strong reasoning and agentic capabilities at low inference cost, making advanced model capabilities more accessible for local and resource-constrained deployment.
12
  BF16, FP8, and INT4 weights are provided for a wide range of hardware and deployment settings.
13
 
 
18
  + **Local and Edge Deployment:** Designed for efficient local deployment, Ling-3.0-tiny has been validated on **NVIDIA DGX Spark, Apple Silicon MacBook, and Mac mini**, enabling capable reasoning and agentic workloads without datacenter-class GPUs. With FP8, Ling-3.0-tiny reaches around **100-105 tokens/s on DGX Spark** and **86-90 tokens/s on an M4 Pro MacBook**, with approximately **8.34 GiB peak memory usage** at an 8K context length.
19
 
20
  # Model Overview
21
+ Ling-3.0-tiny inherits the hybrid linear attation architecture of Ling-3.0 series, while being specifically optimized for lightweight and accessible deployment. The model has 7.9B total parameters, with only 1.3B parameters activated per token.
22
 
23
  The architecture of Ling-3.0-tiny is designed to make computational efficiency serve real-world agentic performance.
24
 
25
  + A 3:1 KDA–MLA architecture (3 KDA layers and 1 MLA layer per 4-layer block) provides more efficient long-context processing;
26
+ + A sparse MoE FFN with 128 experts activates 8 routed experts and 1 shared expert per token, enabling broad model capabilities with only 1.3B activated parameters per token.
27
  + Native hybrid reasoning enables fast responses for routine tasks and multi-step reasoning for complex tasks within a single model.
28
 
29
  Overall, these designs deliver the inference efficiency needed to deploy lightweight models in real-world agentic workflows.
 
35
  We evaluated Ling-3.0-tiny across agentic tasks, coding, long-context understanding, knowledge reliability, mathematical and scientific reasoning, and instruction following.
36
  Ling-3.0-tiny achieves a score of **25** on the Artificial Analysis Intelligence Index v4.1.1 and **16** on the Artificial Analysis Agentic Index.
37
  In Artificial Analysis testing, Ling-3.0-tiny reaches an output speed of over **160 tokens/s**, with approximately **18 seconds** of end-to-end latency for a 500-token response, including reasoning time.
38
+ These results highlight the model's efficiency relative to its 1.3B activated parameter footprint.
39
 
40
  The following table presents representative benchmarks for Ling-3.0-tiny:
41