Nim JSON parsing is 30 times slower than Node.JS

@alexeypetrushin I've investigated this and fixed the performance problem, see now `jsonutils.jsonTo` doesn't have any overhead compared to `json.to` (the PR actually makes a 20x speed improvement compared to before PR)

Nim JSON parsing is 30 times slower than Node.JS

> your snippet uses jsonutils, so how can you compare apples to apples with a > version that doesn't have it? I used 1.4.2 without jsonutils, and updated recently to 1.4.6 and also updated json code to use the new `jsonutils` in all my json related code. Here's the [diff](https://gist.github.co

Nim JSON parsing is 30 times slower than Node.JS

@alexeypetrushin can you explain? > 1.4.2 without jsonutils -> 1.4.6 with jsonutils (became slower) your snippet uses jsonutils, so how can you compare apples to apples with a version that doesn't have it? please post a version that works with some prior version and is slower in devel than that

Nim JSON parsing is 30 times slower than Node.JS

There is a bunch of issues related to JSON performance: #12833 #3809 #12152 and nim-lang/RFCs#188 Perhaps it's worth linking them in and showing a big warning on the top of the page.

Nim JSON parsing is 30 times slower than Node.JS

Have you tried [sam](https://github.com/OpenSystemsLab/sam.nim)?

Nim JSON parsing is 30 times slower than Node.JS

> other problematic areas with direct JSON deserializers imo, ...and I almost forgot about inheritance. It simply cant be supported. From the [marshal](https://nim-lang.github.io/Nim/marshal.html) docs:

Nim JSON parsing is 30 times slower than Node.JS

> benchmarks should use -d:danger, not -d:release, or at least show both I disagree about `-d:danger`. Safety is one of Nim core features. Benchmarking with key feature disabled would be wrong, as it would be benchmark for something totally artificiall that never going to be used in practice. >

Nim JSON parsing is 30 times slower than Node.JS

You can always run Nim in Node ;P

Nim JSON parsing is 30 times slower than Node.JS

Besides speed, which is very important and should be accounted for, other problematic areas with direct JSON deserializers imo, are writing custom deserializers for compound objects (if you really need them, it won't work correctly due to unspecified keys order), in other words [RFCs/247](https

Nim JSON parsing is 30 times slower than Node.JS

Wow @Ward, you are a genius. Pre-allocating the string is much much faster! Your `newStringOfCap(64)` led me on this path, where turns out it is faster to scan the string figure out the exact size then populate the string using a second pass. The memory is warm any ways! When I populate the stri

Nim JSON parsing is 30 times slower than Node.JS

Quick side note: deno isn't node although they both use v8.

Nim JSON parsing is 30 times slower than Node.JS

I got a better result even than `--gc:none` by adding following code and compiling with `nim r -d:danger --opt:speed --gc:arc`. proc parseHook*(s: string, i: var int, v: var string) {.inline.} = v = newStringOfCap(64) jsony.parseHook(s, i, v) timeIt "jsony":

Nim JSON parsing is 30 times slower than Node.JS

> It seems like JSON in Nim became very slow, after recent update I noticed > that my data processing scripts became slower * slower compared to what nim version? jsonutils didn't exist in 1.2 and the snippet is faster in devel 0de3d4292f328f94c7a94af7e3e61e58ff43a252 than in 1.4.0 * ben

Nim JSON parsing is 30 times slower than Node.JS

the code gets 10x faster if you use `to` from `json` rather than `json_to` from `jsonutils`. on my machine that's like 0.08s vs 0.03s

Nim JSON parsing is 30 times slower than Node.JS

Eh , not everything is a string though, There can be unmanaged `ptr UnceckedArray[byte]`, you are interfacing with, Sockets, reading/writing directly from/to a file, etc.

Nim JSON parsing is 30 times slower than Node.JS

> That is why I made jsony Yes, thanks. I knew about it, but it was more or less tolerable so far, so I was continue to use std/json till now. > Nim's standard json parsers first creates intermediate json nodes then turns > them into nim types, causing each object and string to be double alloca

Nim JSON parsing is 30 times slower than Node.JS

Personally I would like to see, `streams2` that are just concepts (static polymorphism). I have no idea how dynamic polymorphism is useful in streams. Would that alleviate your performance concerns and security issues (overwriting proc pointers)?

Nim JSON parsing is 30 times slower than Node.JS

For me a better stream is just a string with an index. Very clear what is going on.

Nim JSON parsing is 30 times slower than Node.JS

For backwards compatibility then. It could be a possibility for Nim v2.0 though.

Nim JSON parsing is 30 times slower than Node.JS

Why not merge jsony into stdlib and replace the existing json lib?

Nim JSON parsing is 30 times slower than Node.JS

Because many people rely on current json library. This would create two json libs in std library. I think it would cause confusion. The ideas from jsony could be taken to std lib though.

Nim JSON parsing is 30 times slower than Node.JS

there was a PR that improved parsing speed Whats it's status?

Nim JSON parsing is 30 times slower than Node.JS

Thanks Ward! Its pretty cool you chose my library as first thing. I have committed some changes to the float parser but I still can't beat typescript, but I did gain about 3.5ms of speed. name ... min time avg timestd dv runs parsing ..

Nim JSON parsing is 30 times slower than Node.JS

You can use instead of std/json. Modified code: import benchy, jsony import times, os, std/json, std/jsonutils type OptionContract* = ref object id*: string right*:string expiration*:

Nim JSON parsing is 30 times slower than Node.JS

heh... I am slow...

Nim JSON parsing is 30 times slower than Node.JS

Nim's std lib parsing is kind of slow. That is why I made jsony ( ) import times, os, std/json, jsony type OptionContract* = ref object id*: string right*:string expiration*: string s

Nim JSON parsing is 30 times slower than Node.JS

It seems like JSON in Nim is really slow. It's a bit unfair to compare with Node.JS, as in Node JSON parsed by heavily optimised C code. Yet, I think the numbers should be more or less comparable, not orders of magnitude slower. The scripts below are 30 times slower with `-d:release` and 160 ti