Toolkit LabsTruncated JSON from a stream — 21 parsers
A streaming reply that stopped mid-object is the one failure a dialect parser cannot help with: the text is not loose JSON, it is unfinished JSON, and a parser has to decide what the missing part was. This page is one column of the leaderboard pulled out and read properly — 11 Python and 10 JavaScript parsers on the same 25 truncated cases, one run each, nothing tuned afterwards.
Cases: the truncated category of MALFORMED-300, 25 labelled outputs cut off mid-string, mid-number, mid-array and mid-object. Python measured 2026-08-19T12:31:18Z on Python 3.12.3; JavaScript 2026-08-19T12:33:03Z on node v25.8.2. Corpus sha256 ded36c275f9ff481a9e264f9e900ccbb16da9cf430ff6fe8ffc23b9c02acb88a.
Python
| parser | recovered exactly | refused | returned a wrong value | same library, all 300 |
|---|---|---|---|---|
json-repair 0.63.2 | 23 / 25 | 0 | 2 | 265 / 300 |
jsonshim (Toolkit Labs) | 23 / 25 | 0 | 2 | 282 / 300 |
partial-json-parser 0.2.1.1.post7 | 13 / 25 | 12 | 0 | 47 / 300 |
commentjson 0.9.0 | 0 / 25 | 25 | 0 | 58 / 300 |
demjson3 3.0.5 | 0 / 25 | 25 | 0 | 110 / 300 |
dirtyjson 1.0.8 | 0 / 25 | 25 | 0 | 123 / 300 |
hjson 3.1.0 | 0 / 25 | 25 | 0 | 85 / 300 |
json.loads (stdlib control) | 0 / 25 | 25 | 0 | 25 / 300 |
json5 0.15.0 | 0 / 25 | 25 | 0 | 118 / 300 |
pyjson5 2.0.1 | 0 / 25 | 25 | 0 | 118 / 300 |
simplejson 4.1.1 | 0 / 25 | 25 | 0 | 29 / 300 |
JavaScript / TypeScript
| parser | recovered exactly | refused | returned a wrong value | same library, all 300 |
|---|---|---|---|---|
jsonc-parser 3.3.1 | 20 / 25 | 4 | 1 | 157 / 300 |
best-effort-json-parser 1.5.1 | 14 / 25 | 0 | 11 | 160 / 300 |
partial-json 0.1.7 | 13 / 25 | 12 | 0 | 74 / 300 |
untruncate-json 0.0.1 | 13 / 25 | 12 | 0 | 47 / 300 |
jsonrepair 3.15.0 | 12 / 25 | 1 | 12 | 205 / 300 |
dirty-json 0.9.2 | 4 / 25 | 6 | 15 | 110 / 300 |
JSON.parse (stdlib control) v25.8.2 | 0 / 25 | 25 | 0 | 25 / 300 |
hjson 3.2.2 | 0 / 25 | 25 | 0 | 95 / 300 |
json-loose 1.2.4 | 0 / 25 | 25 | 0 | 80 / 300 |
json5 2.2.3 | 0 / 25 | 25 | 0 | 118 / 300 |
Three outcomes per case, and they add to 25 for every row: the parser returned the expected value, or it refused, or it returned something else. The last column is the one worth reading twice.
A zero here is not a broken library
8 of the 11 Python parsers and 4 of the 10 JavaScript ones recover none of the 25 cases, and that is the design working as intended. json5, pyjson5, hjson, commentjson and their neighbours are dialect parsers: they read a looser grammar than strict JSON, so comments, trailing commas and unquoted keys are inside their job, and a document nobody closed is not. There is nothing to be lenient about until something finishes the text. Every one of them refuses instead of guessing, which on this category is the safe answer.
The column that costs you
Recovering 25 of 25 is not what separates these libraries; refusing cleanly versus answering wrongly is. Across both languages a parser returned a value that was not the expected one, without refusing, 43 times — 4 in Python and 39 in JavaScript. The widest gap is dirty-json, which returns a wrong value on 15 of 25 truncated cases. A refusal is a branch in your code. A wrong value is a row in your database that looks exactly like data, and a truncated reply is precisely the case where the missing half is the half you needed.
The best Python results on this category are a tie — json-repair and jsonshim (Toolkit Labs), each recovering 23 of 25 with 0 refusals between them. In JavaScript the best is jsonc-parser at 20 of 25, refusing 4 and returning a wrong value on 1.
The tool published here does not win this page on recovery alone and is not presented as if it did — on the 275 recoverable cases of the whole corpus json-repair recovers more than jsonshim does. Read the full grid before choosing anything on one category.
Check it yourself
Every number above is computed from the two published run files and the per-case rows the harnesses wrote; the derived counts on this page are published as one small file, so nothing here rests on this page’s word.
truncated.json — the per-parser counts behind both tables · leaderboard.json · leaderboard_js.json · run_leaderboard.py · run_js.mjs
curl -O https://toolkitlabs.org/malformed300/sample30.jsonl
curl -O https://toolkitlabs.org/malformed300/score.py
python3 score.py --corpus sample30.jsonl --parser yourmodule:recoverThe free sample carries 2–3 cases from each of the 12 categories, truncation among them, and is CC0 forever. One broken reply rather than a suite? The in-browser tool runs two of these libraries and the strict parser on your own text, in the page, uploading nothing.
What this page does not tell you
- 25 cases is 25 cases. It is enough to separate a parser that closes documents from one that refuses them, and not enough to rank two libraries that sit one case apart.
- One run per library. These parsers are deterministic, but this measurement does not prove that.
- The category holds equal numbers of each cut point by construction; real traffic does not.
- “Wrong value” means the returned value is not the expected one. A library free to guess the missing half will sometimes guess it right, and that is counted as recovered here even though your pipeline could not have known.
The full corpus
The 30 free cases and the scorer are CC0 forever, bought or not. The remaining 270 cases — all 25 truncated ones among them — with the label rationale for each are the paid product, and they are what let you run this on your own parser at full resolution.
Single developer — €29 · Team / CI licence — €99
Instant download after payment; digital goods, delivered immediately. Payments processed by Stripe. Prices include VAT where applicable.