paradox-save-parser-frontend/COMPRESSION.md

Compression levels

Reference for saveUpload.zstdCompressionLevel in src/config/AppConfig.ts. If compression ever turns out to be too slow or too memory hungry on some device, pick a different level from the table below rather than measuring again.

The level only affects the uploader. Decompression costs the same at every level, both on the parser backend and in the browser.

What was measured

A real Europa Universalis IV save, as the game wrote it:

File	Ibevar1475_01_02save.eu4, a zip archive
Size on disk	11,502,739 bytes (11.0 MB)
Entries	gamestate (87,188,760 bytes), meta, ai
Input to zstd	87,342,592 bytes — the tar of those three entries

So the ratios below are against the original 11.5 MB game archive, and zstd itself is working on the 87 MB of uncompressed save text inside it.

Each level ran through the project's own packSave in a separate process. The heap columns are the WebAssembly linear memory the encoder and the decoder grow to, read from the instance itself — that is the memory a browser tab spends, and it is what to look at when a device runs out. The machine was a desktop; a phone is several times slower, so treat the times as a lower bound.

Every level round tripped to the same 87,342,592 byte tar, so each row is correct output rather than merely fast output.

The table

level	compressed	ratio	compress	decompress	encoder heap	decoder heap
1	6,698,281	1.72x	0.54 s	0.28 s	8 MB	8 MB
2	6,600,451	1.74x	0.54 s	0.29 s	8 MB	8 MB
3	6,094,116	1.89x	0.57 s	0.28 s	8 MB	8 MB
4	6,082,362	1.89x	0.61 s	0.28 s	8 MB	8 MB
5	5,280,090	2.18x	0.92 s	0.28 s	8 MB	8 MB
6	4,943,366	2.33x	1.13 s	0.27 s	8 MB	8 MB
7	4,738,515	2.43x	1.27 s	0.27 s	9 MB	8 MB
8	4,514,853	2.55x	1.52 s	0.28 s	9 MB	8 MB
9	4,445,368	2.59x	1.51 s	0.26 s	16 MB	8 MB
10	4,280,567	2.69x	1.74 s	0.27 s	26 MB	8 MB
11	4,157,684	2.77x	2.44 s	0.32 s	26 MB	8 MB
12	4,155,668	2.77x	2.39 s	0.27 s	46 MB	8 MB
13	4,257,141	2.70x	3.76 s	0.27 s	38 MB	8 MB
14	4,108,558	2.80x	5.39 s	0.28 s	54 MB	8 MB
15	3,968,262	2.90x	8.13 s	0.33 s	70 MB	8 MB
16	3,862,217	2.98x	11.63 s	0.26 s	38 MB	8 MB
17	3,733,198	3.08x	12.39 s	0.26 s	58 MB	10 MB
18	3,705,141	3.10x	15.11 s	0.26 s	59 MB	10 MB
19	3,449,475	3.33x	27.78 s	0.28 s	91 MB	10 MB
20	3,344,642	3.44x	28.24 s	0.27 s	195 MB	34 MB
21	3,285,836	3.50x	33.91 s	0.30 s	387 MB	66 MB
22	3,245,980	3.54x	49.02 s	0.30 s	835 MB	130 MB

Picking a level

10 is the default, because it is where the curve flattens. It gives good compression in reasonable time: 11 is only 3% smaller but takes 40% longer.

If there will be troubles with this level in the future, just change it in config.

Reproducing

There is no script in the repository for this; it was a one-off measurement. To repeat it, stream a save through packSave once per level in a separate process, and read memory.buffer.byteLength off the encoder and decoder instances of @tootallnate/zstd-wasm for the heap figures.