added benchmarks

ParadoxSaveParser.Benchmarks/README.md

@@ -0,0 +1,111 @@
+# ParadoxSaveParser.Benchmarks
+
+Compares the query engine of this repo (`SearchExpression` + `SaveParserEU4`) against
+regex engines and `jq` on one realistic task:
+
+> extract the `technology` block (adm/dip/mil level) of every country from an EU4 save
+
+| engine | query |
+|---|---|
+| this repo | `countries.*.technology` |
+| jq | `.countries \| map_values(.technology)` |
+| .NET `Regex` / PCRE.NET | see `Extractors.FlatPattern` / `Extractors.PathAwarePattern` |
+| `Utf8JsonReader` / `JsonDocument` | hand written navigation |
+
+## Corpus
+
+* `gamestate` — a real 109 MB EU4 1.37 save (text format, 6.0M lines, 2803 country blocks,
+  1870 of which have a `technology` block).
+* `gamestate.min.json` — 92 MB compact JSON twin, produced from the save by this project's
+  own parser. It exists because jq and the `System.Text.Json` engines cannot read the
+  Paradox format at all.
+
+The save is **not** in the repository. Point `PSP_BENCH_DATA` at the directory holding it,
+or drop it in `ParadoxSaveParser.CLI/bin/Debug/net10.0/` (the default location).
+
+## Usage
+
+```sh
+dotnet build ParadoxSaveParser.Benchmarks -c Release
+cd ParadoxSaveParser.Benchmarks/bin/Release/net10.0
+
+./ParadoxSaveParser.Benchmarks.exe prepare   # build the JSON twin (once)
+./ParadoxSaveParser.Benchmarks.exe verify    # one run of every engine + correctness check
+./ParadoxSaveParser.Benchmarks.exe --filter '*'   # full BenchmarkDotNet run
+```
+
+Every engine returns the checksum `countries * 1_000_000 + sum_of_tech_levels`, so `verify`
+shows immediately when an engine finds something different from the others.
+
+## Fairness notes
+
+* All in-process engines read from memory; file I/O is excluded. jq is an external process,
+  so its numbers include process start (measured separately by the `jq startup only`
+  benchmark) and reading the JSON from the OS page cache.
+* The regex engines work on the save text, which is 109 MB; jq works on 92 MB of JSON.
+* jq is the only engine here that is a general query language: it can do arithmetic,
+  filtering and reshaping, which neither the regexes nor `SearchExpression` can.
+
+## Results
+
+AMD Ryzen 7 5700X, 16 logical cores, Windows 10 21H2, .NET 10.0.12, jq 1.8.2,
+109 MB save / 92 MB JSON twin, BenchmarkDotNet `RunStrategy.Monitoring`, 5 iterations.
+
+| engine | mean | vs this project | allocated | correct? |
+|---|---:|---:|---:|---|
+| .NET Regex flat, source generated | 14.8 ms | 0.02x | 1.2 MB | path-blind |
+| .NET Regex flat, compiled | 17.0 ms | 0.03x | 1.2 MB | path-blind |
+| .NET Regex flat, interpreted | 19.3 ms | 0.03x | 1.2 MB | path-blind |
+| PCRE.NET flat, JIT compiled | 97.4 ms | 0.15x | 0.9 MB | path-blind |
+| .NET Regex balanced block + flat | 112.3 ms | 0.17x | 224 MB | yes |
+| .NET Regex path aware, compiled | 116.2 ms | 0.18x | 1.4 MB | 1869/1870 |
+| PCRE.NET path aware, JIT compiled | 122.9 ms | 0.19x | 0.9 MB | 1869/1870 |
+| Utf8JsonReader over JSON twin | 155.7 ms | 0.24x | 0 B | yes |
+| JsonDocument over JSON twin | 321.4 ms | 0.50x | 0 B (native) | yes |
+| .NET Regex path aware, NonBacktracking | 444.5 ms | 0.69x | 32 MB | 1869/1870 |
+| **SearchExpression (this project)** | **645.1 ms** | **1.00x** | **1.6 MB** | yes |
+| Full parse, then select | 1831 ms | 2.84x | 1153 MB | yes |
+| jq `.countries \| map_values(.technology)` | 4340 ms | 6.73x | n/a | yes |
+| jq `empty` (parse the file, emit nothing) | 4227 ms | 6.55x | n/a | n/a |
+| jq `--stream` | 17490 ms | 27.1x | n/a | n/a |
+| jq process startup only | 4.2 ms | 0.01x | n/a | n/a |
+
+### Reading the table
+
+* **jq is ~6.7x slower than this parser** and 97% of that time is JSON parsing, not the
+  query: `jq empty` on the same file costs 4227 ms of the 4340 ms. Process startup is
+  negligible (4 ms). jq's `--stream` mode, often recommended for large inputs, is 4x
+  *slower* still. jq also needs the data converted to JSON first, which this parser has to
+  do anyway — so end to end jq is strictly more expensive here.
+* **The regexes are 5-40x faster, but they are not doing the same job.** A regex never
+  parses the structure; it scans bytes for a literal and validates a short window around it.
+  `Regex` with a literal prefix (`technology={`) is vectorized, so 109 MB is scanned at
+  several GB/s. That speed is real and the cost is real too: nothing verifies that the hit
+  is under `countries`, at the right depth, or belongs to the tag matched before it.
+* **Path-aware regexes lose both the speed and the correctness.** Forcing the tag into the
+  pattern costs 7x (17 ms -> 116 ms) and still returns 1869 of 1870 countries: the tag class
+  `[A-Z0-9]{3}` silently drops EU4's `---` pseudo-country. Pairing survives here only by
+  luck of the file layout — all 933 tech-less country blocks happen to be grouped at the end
+  of the save, so the lazy gap never runs across one. A save that interleaves them would
+  make the regex report a technology block under the wrong country tag with no error.
+* **`RegexOptions.NonBacktracking`** guarantees linear time but is 26x slower than the
+  compiled backtracking engine on this pattern and allocates 32 MB.
+* **PCRE.NET** (the most used non-BCL regex engine in .NET, ~460k downloads) is 6x slower
+  than `System.Text.RegularExpressions` on the flat pattern, because .NET's vectorized
+  literal prefix search beats PCRE2's JIT here. On the path-aware pattern the two are equal.
+  There is no reason to leave the BCL engine for this workload.
+* **The query is what makes this parser fast, not the parsing.** Same parser, same file:
+  645 ms with `countries.*.technology`, 1831 ms and 1.1 GB allocated without a query. The
+  search expression prunes ~65% of the work and 99.9% of the allocations.
+* **Against a JSON reader on equivalent data**, the parser is 4x slower than `Utf8JsonReader`
+  and 2x slower than `JsonDocument`. Those numbers exclude the pdx -> JSON conversion
+  (~4.7 s), so they are a ceiling for the format, not a usable alternative.
+
+### Where this parser's time goes
+
+645 ms for 109 MB is ~170 MB/s, or ~13 cycles per input byte. The lexer reads the save one
+byte at a time through `Stream.ReadByte()` (`SaveParserEU4.LexTextSave`), which is a virtual
+call plus bounds check per byte, and appends char by char into a `StringBuilder`. Reading
+into a `byte[]` buffer and scanning it with `ReadOnlySpan<byte>.IndexOfAny` would be the
+first thing to try — the regex numbers above show what the same hardware does when it scans
+a span instead of a stream.