Altair-sh
74 lines
4.5 KiB
Markdown
74 lines
4.5 KiB
Markdown
# Compression levels
|
|
|
|
Reference for `saveUpload.zstdCompressionLevel` in
|
|
[src/config/AppConfig.ts](src/config/AppConfig.ts). If compression ever turns
|
|
out to be too slow or too memory hungry on some device, pick a different level
|
|
from the table below rather than measuring again.
|
|
|
|
The level only affects the uploader. Decompression on the parser backend costs
|
|
the same at every level.
|
|
|
|
## What was measured
|
|
|
|
A real Europa Universalis IV save, as the game wrote it:
|
|
|
|
| | |
|
|
| ------------- | ----------------------------------------------------- |
|
|
| File | `Ibevar1475_01_02save.eu4`, a zip archive |
|
|
| Size on disk | **11,502,739 bytes** (11.0 MB) |
|
|
| Entries | `gamestate` (87,188,760 bytes), `meta`, `ai` |
|
|
| Input to zstd | **87,342,592 bytes** — the tar of those three entries |
|
|
|
|
So the ratios below are against the original 11.5 MB game archive, and zstd
|
|
itself is working on the 87 MB of uncompressed save text inside it.
|
|
|
|
Each level ran through the project's own `packSave` in a separate process. The
|
|
heap columns are the WebAssembly linear memory the encoder and the decoder grow
|
|
to, read from the instance itself — that is the memory a browser tab spends,
|
|
and it is what to look at when a device runs out. The machine was a desktop;
|
|
**a phone is several times slower**, so treat the times as a lower bound.
|
|
|
|
Every level round tripped to the same 87,342,592 byte tar, so each row is
|
|
correct output rather than merely fast output.
|
|
|
|
## The table
|
|
|
|
| level | compressed | ratio | compress | decompress | encoder heap | decoder heap | window |
|
|
| -----: | ------------: | --------: | ---------: | ---------: | -----------: | -----------: | -----: |
|
|
| 1 | 6,698,281 | 1.72x | 0.54 s | 0.28 s | 8 MB | 8 MB | 1 MB |
|
|
| 2 | 6,600,451 | 1.74x | 0.54 s | 0.29 s | 8 MB | 8 MB | 1 MB |
|
|
| 3 | 6,094,116 | 1.89x | 0.57 s | 0.28 s | 8 MB | 8 MB | 2 MB |
|
|
| 4 | 6,082,362 | 1.89x | 0.61 s | 0.28 s | 8 MB | 8 MB | 2 MB |
|
|
| 5 | 5,280,090 | 2.18x | 0.92 s | 0.28 s | 8 MB | 8 MB | 2 MB |
|
|
| 6 | 4,943,366 | 2.33x | 1.13 s | 0.27 s | 8 MB | 8 MB | 2 MB |
|
|
| 7 | 4,738,515 | 2.43x | 1.27 s | 0.27 s | 9 MB | 8 MB | 2 MB |
|
|
| 8 | 4,514,853 | 2.55x | 1.52 s | 0.28 s | 9 MB | 8 MB | 2 MB |
|
|
| 9 | 4,445,368 | 2.59x | 1.51 s | 0.26 s | 16 MB | 8 MB | 4 MB |
|
|
| 10 | 4,280,567 | 2.69x | 1.74 s | 0.27 s | 26 MB | 8 MB | 4 MB |
|
|
| **11** | **4,157,684** | **2.77x** | **2.44 s** | **0.32 s** | **26 MB** | **8 MB** | 4 MB |
|
|
| 12 | 4,155,668 | 2.77x | 2.39 s | 0.27 s | 46 MB | 8 MB | 4 MB |
|
|
| 13 | 4,257,141 | 2.70x | 3.76 s | 0.27 s | 38 MB | 8 MB | 4 MB |
|
|
| 14 | 4,108,558 | 2.80x | 5.39 s | 0.28 s | 54 MB | 8 MB | 4 MB |
|
|
| 15 | 3,968,262 | 2.90x | 8.13 s | 0.33 s | 70 MB | 8 MB | 4 MB |
|
|
| 16 | 3,862,217 | 2.98x | 11.63 s | 0.26 s | 38 MB | 8 MB | 4 MB |
|
|
| 17 | 3,733,198 | 3.08x | 12.39 s | 0.26 s | 58 MB | 10 MB | 8 MB |
|
|
| 18 | 3,705,141 | 3.10x | 15.11 s | 0.26 s | 59 MB | 10 MB | 8 MB |
|
|
| 19 | 3,449,475 | 3.33x | 27.78 s | 0.28 s | 91 MB | 10 MB | 8 MB |
|
|
| 20 | 3,344,642 | 3.44x | 28.24 s | 0.27 s | 195 MB | 34 MB | 32 MB |
|
|
| 21 | 3,285,836 | 3.50x | 33.91 s | 0.30 s | 387 MB | 66 MB | 64 MB |
|
|
| 22 | 3,245,980 | 3.54x | 49.02 s | 0.30 s | 835 MB | 130 MB | 128 MB |
|
|
|
|
## Picking a level
|
|
|
|
**11 is the default**, because it is where the curve flattens. It gives good compression in reasonable time.
|
|
|
|
If there will be troubles with this level in the future, just change it in config.
|
|
|
|
## Reproducing
|
|
|
|
There is no script in the repository for this; it was a one-off measurement.
|
|
To repeat it, stream a save through `packSave` once per level in a separate
|
|
process, and read `memory.buffer.byteLength` off the encoder and decoder
|
|
instances of [@tootallnate/zstd-wasm](https://www.npmjs.com/package/@tootallnate/zstd-wasm)
|
|
for the heap figures.
|