On Tuesday, 28 February 2017 at 20:27:25 UTC, Alexey H wrote:
Hello, guys!
I'm working on a project that involves parsing of huge JSON
datasets in real-time.
Just an example of what i'm dealing with is here:
https://gist.githubusercontent.com/gdmka/125014058bb7d7f01b867fac56300a61/raw/f0c6b5be5fb01b16dd83f07c577b72f76f72c855/data.json
Can't think of any tools other that D or Go to solve this
problem.
My experience of solving the problem with Go has led me to
Stackoverflow and the community out there seemed too reluctant
to help so i assumed that the language cannot handle such a set
of operations on complex datastructures.
My experience with D was like a charm. Where i have had ~100
lines of Go code i did the same with 12 in with D. But,
nevertheless, i did some profiling (unfortunately on OS X) and
saw much heavier CPU usage with D than Go. Probably because the
Go solution was unpacking all the data strictly to struct.
So, my real question is: can i actually, by any change, get the
description of an underlying struct that the call to parseJSON
generates?
The goers have this thing https://mholt.github.io/json-to-go/
to generate structs from JSON automatically.
Since D easily parses JSON by type inference, i assume it
builds a JSONValue struct which holds all the fields and the
data.
If it is possible, then i can build a similar JSON to D tool
just for the sake of saving people's time and patience.
If you really care about performance, have a look this:
http://forum.dlang.org/post/20151014090114.60780ad6@marco-toshiba
std.json is not tuned for performance, so don't expect good
results from it.