CCCCyx commited on
Commit
641be01
·
verified ·
1 Parent(s): 80bb676

Update final v3 quantization benchmarks

Browse files
.gitattributes CHANGED
@@ -39,3 +39,5 @@ assets/mossvl_quantization_benchmark_comparison_zh_4k.png filter=lfs diff=lfs me
39
  assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
40
  assets/architecture.png filter=lfs diff=lfs merge=lfs -text
41
  assets/logo.png filter=lfs diff=lfs merge=lfs -text
 
 
 
39
  assets/mossvl_quantization_benchmark_comparison_en_4k.png filter=lfs diff=lfs merge=lfs -text
40
  assets/architecture.png filter=lfs diff=lfs merge=lfs -text
41
  assets/logo.png filter=lfs diff=lfs merge=lfs -text
42
+ assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png filter=lfs diff=lfs merge=lfs -text
43
+ assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -53,14 +53,15 @@ The checkpoint carries its bitsandbytes configuration. Load it directly and
53
  do not add a second runtime quantization configuration. This variant does not
54
  enable HQQ KV8; `generation_config.json` uses the standard BF16 KV cache.
55
 
56
- ## Base model benchmark
57
 
58
- The following figure reports the benchmark results of the original
59
- MOSS-VL-Instruct-0708 release. It provides base-model context; it is not
60
- presented as a separate quality evaluation of this NF4 checkpoint.
 
61
 
62
  <p align="center">
63
- <img src="assets/benchmark-offline.png" alt="MOSS-VL-Instruct-0708 benchmark" width="100%"/>
64
  </p>
65
 
66
  ## Hardware requirements
 
53
  do not add a second runtime quantization configuration. This variant does not
54
  enable HQQ KV8; `generation_config.json` uses the standard BF16 KV cache.
55
 
56
+ ## Quantization benchmark
57
 
58
+ The final evaluation compares the original BF16 model with all four release
59
+ profiles on their corresponding benchmark suites. This offline NF4 checkpoint
60
+ scores 89.53 on DocVQA, 67.30 on VideoMME, 75.86 on MLVU_dev, 51.00/48.17/59.33
61
+ on the three TimeLens subsets, and 61.76 on VSIBench.
62
 
63
  <p align="center">
64
+ <img src="assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png" alt="MOSS-VL quantization benchmark comparison" width="100%"/>
65
  </p>
66
 
67
  ## Hardware requirements
README_zh.md CHANGED
@@ -51,13 +51,14 @@ tags:
51
  checkpoint 已包含 bitsandbytes 配置,加载时不要再传入第二份运行时量化配置。
52
  该版本没有启用 HQQ KV8,`generation_config.json` 使用标准 BF16 KV Cache。
53
 
54
- ## 基础模型 Benchmark
55
 
56
- 下图是原始 MOSS-VL-Instruct-0708 的 benchmark,用于说明基础模型能力;
57
- 它不作为该 NF4 checkpoint 的独立精度测评结果。
 
58
 
59
  <p align="center">
60
- <img src="assets/benchmark-offline.png" alt="MOSS-VL-Instruct-0708 benchmark" width="100%"/>
61
  </p>
62
 
63
  ## 硬件要求
 
51
  checkpoint 已包含 bitsandbytes 配置,加载时不要再传入第二份运行时量化配置。
52
  该版本没有启用 HQQ KV8,`generation_config.json` 使用标准 BF16 KV Cache。
53
 
54
+ ## 量化 Benchmark
55
 
56
+ 最终测评在各自对应的 benchmark 上对比原始 BF16 模型与四个量化发布配置。
57
+ 该离线 NF4 checkpoint 的 DocVQA、VideoMME、MLVU_dev 分别为 89.53、
58
+ 67.30、75.86;三个 TimeLens 子集为 51.00、48.17、59.33,VSIBench 为 61.76。
59
 
60
  <p align="center">
61
+ <img src="assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png" alt="MOSS-VL 量化配置 benchmark 对比" width="100%"/>
62
  </p>
63
 
64
  ## 硬件要求
assets/mossvl_quantization_benchmark_comparison_final_v3_en_4k.png ADDED

Git LFS Details

  • SHA256: d516d803e3cdb2bbae4348035f7249012597c5f92b82139a0e5d0735e3585875
  • Pointer size: 132 Bytes
  • Size of remote file: 1.71 MB
assets/mossvl_quantization_benchmark_comparison_final_v3_zh_4k.png ADDED

Git LFS Details

  • SHA256: 6ea3e7a3bfabdadbfed70b83d07f7dd084d44e366ba5123a776d3e557ddeb6e8
  • Pointer size: 132 Bytes
  • Size of remote file: 1.63 MB