122 lines
4.6 KiB
Markdown
122 lines
4.6 KiB
Markdown
# CPU Benchmark (ONNX-python)
|
|
|
|
## Configuration
|
|
### Data set:
|
|
Aishell1 [test set](https://www.openslr.org/33/) , the total audio duration is 36108.919 seconds.
|
|
|
|
### Tools
|
|
#### Install Requirements
|
|
Install ModelScope and FunASR
|
|
```shell
|
|
pip install -U modelscope funasr
|
|
# For the users in China, you could install with the command:
|
|
#pip install -U funasr -i https://mirror.sjtu.edu.cn/pypi/web/simple
|
|
```
|
|
|
|
Install requirements
|
|
```shell
|
|
git clone https://github.com/alibaba-damo-academy/FunASR.git && cd FunASR
|
|
cd funasr/runtime/python/utils
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
#### Recipe
|
|
|
|
|
|
##### test_rtf
|
|
set the model, data path and output_dir
|
|
```shell
|
|
nohup bash test_rtf.sh &> log.txt &
|
|
```
|
|
|
|
##### test_cer
|
|
set the model, data path and output_dir
|
|
```shell
|
|
nohup bash test_cer.sh &> log.txt &
|
|
```
|
|
|
|
## [Paraformer-large](https://www.modelscope.cn/models/damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary)
|
|
|
|
Number of Parameter: 220M
|
|
|
|
Storage size: 880MB
|
|
|
|
Storage size after int8-quant: 237MB
|
|
|
|
CER: 1.95%
|
|
|
|
CER after int8-quant: 1.95%
|
|
|
|
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
|
|
|
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
|
|:----------------:|:------------------:|:-------:|:------------:|
|
|
| 1 (onnx fp32) | 2806 | 0.0777 | 12.9 |
|
|
| 1 (onnx int8) | 1611 | 0.0446 | 22.4 |
|
|
| 8 (onnx fp32) | 538 | 0.0149 | 67.1 |
|
|
| 8 (onnx int8) | 210 | 0.0058 | 172.4 |
|
|
| 16 (onnx fp32) | 288 | 0.0080 | 125.2 |
|
|
| 16 (onnx int8) | 117 | 0.0032 | 309.9 |
|
|
| 32 (onnx fp32) | 167 | 0.0046 | 216.5 |
|
|
| 32 (onnx int8) | 86 | 0.0024 | 420.0 |
|
|
| 64 (onnx fp32) | 158 | 0.0044 | 228.1 |
|
|
| 64 (onnx int8) | 82 | 0.0023 | 442.8 |
|
|
| 96 (onnx fp32) | 151 | 0.0042 | 238.0 |
|
|
| 96 (onnx int8) | 80 | 0.0022 | 452.0 |
|
|
|
|
|
|
### Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz 16core-32processor with avx512_vnni
|
|
|
|
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
|
|:----------------:|:------------------:|:------:|:------------:|
|
|
| 1 (onnx fp32) | 2613 | 0.0724 | 13.8 |
|
|
| 1 (onnx int8) | 1321 | 0.0366 | 22.4 |
|
|
| 32 (onnx fp32) | 170 | 0.0047 | 212.7 |
|
|
| 32 (onnx int8) | 89 | 0.0025 | 407.0 |
|
|
| 64 (onnx fp32) | 166 | 0.0046 | 217.1 |
|
|
| 64 (onnx int8) | 87 | 0.0024 | 414.7 |
|
|
|
|
|
|
### Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni
|
|
|
|
|
|
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
|
|:----------------:|:------------------:|:------:|:------------:|
|
|
| 1 (onnx fp32) | 2959 | 0.0820 | 12.2 |
|
|
| 1 (onnx int8) | 2814 | 0.0778 | 12.8 |
|
|
| 16 (onnx fp32) | 373 | 0.0103 | 96.9 |
|
|
| 16 (onnx int8) | 331 | 0.0091 | 109.0 |
|
|
| 32 (onnx fp32) | 211 | 0.0058 | 171.4 |
|
|
| 32 (onnx int8) | 181 | 0.0050 | 200.0 |
|
|
| 64 (onnx fp32) | 153 | 0.0042 | 235.9 |
|
|
| 64 (onnx int8) | 103 | 0.0029 | 349.9 |
|
|
| 96 (onnx fp32) | 146 | 0.0041 | 247.0 |
|
|
| 96 (onnx int8) | 108 | 0.0030 | 334.1 |
|
|
|
|
## [Paraformer](https://modelscope.cn/models/damo/speech_paraformer_asr_nat-zh-cn-16k-common-vocab8358-tensorflow1/summary)
|
|
|
|
Number of Parameter: 68M
|
|
|
|
Storage size: 275MB
|
|
|
|
Storage size after int8-quant: 81MB
|
|
|
|
CER: 3.73%
|
|
|
|
CER after int8-quant: 3.78%
|
|
|
|
### Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni
|
|
|
|
| concurrent-tasks | processing time(s) | RTF | Speedup Rate |
|
|
|:----------------:|:------------------:|:------:|:------------:|
|
|
| 1 (onnx fp32) | 1173 | 0.0325 | 30.8 |
|
|
| 1 (onnx int8) | 976 | 0.0270 | 37.0 |
|
|
| 16 (onnx fp32) | 91 | 0.0025 | 395.2 |
|
|
| 16 (onnx int8) | 78 | 0.0022 | 463.0 |
|
|
| 32 (onnx fp32) | 60 | 0.0017 | 598.8 |
|
|
| 32 (onnx int8) | 40 | 0.0011 | 892.9 |
|
|
| 64 (onnx fp32) | 55 | 0.0015 | 653.6 |
|
|
| 64 (onnx int8) | 31 | 0.0009 | 1162.8 |
|
|
| 96 (onnx fp32) | 57 | 0.0016 | 632.9 |
|
|
| 96 (onnx int8) | 33 | 0.0009 | 1098.9 |
|