> Sent: Wednesday, April 15, 2015 at 1:28 PM
> From: "Andre Somers" <an...@familiesomers.nl>
> To: interest@qt-project.org
> Subject: Re: [Interest] json file handling
>
> On 15-4-2015 17:25, Thiago Macieira wrote:
> > On Wednesday 15 April 2015 16:36:15 André Somers wrote:
> >>> The dilemma I'm in is, how can I tell the json writer to "append" to a
> >>> current json file without loading the whole file back in memory (eg
> >>> read json from file->edit in Qt->write back to file). Because, after a
> >>> while, I suppose the file will get quite large.
> >> I don't think JSON is particulary suitable for this kind of application
> >> (and neither is XML).
> > CBOR might be, since it does support "undefined length" maps and arrays.
> I'm not familiar with that format, but I'll look it up. I guess the
> choice for JSON or XML is made due to convenience though, as classes to
> read and write these are readily available. I doubt the same can be said
> of CBOR.
> > As for XML, the format may not be suitable but QXmlStreamWriter is since it
> > keeps the state of which tags it needs to close and allows you to simply
> > stream data into it. On the receiver side, QXmlStreamReader is capable of
> > receiving incomplete data and making the best it can with what has been
> > completed.
> I'd expect a log-file like output to be well defined at any moment in
> time, or as close as possible to that. So, while QXmlStreamWriter is
> suitable, it does not produce a valid XML file until you finalize the
> write. That may not be what you want (even though QXmlStreamReader could
> be used to parse the incomplete file*).
>
> I think a simpler aproach may be a better one in such cases: simply add
> new values to the end of a file when the become available. No fancy
> markup languages to wrap them. That will result in a smaller file
> (especially compared to XML), faster writing and no problems reading the
> data back in. Simple CSV will do if you want a text-based file, but a
> binary file could also function really well if that is not needed. Just
> append to the file when you have new data available. Such approaches
> have worked with log files for decades :-)
>
> André
>
> *) Though I seem to remember there are actually some problems with that
> in reality.
Back in the day I developed a generic serializer around the operations of push,
item(key) value(value), and pop. I think it was called RecursiveMap and I think
Thiago may have given it to me. It was based off QVariantMap, except there was
a value() member.
Anyway, when converting structured data, you can imagine everything as those
four functions.
push(): add an item (or key) to a list
pop(): end the list
item(key): create an item with key
value(value): set the value of the last item.
{"this": "is", "some": {"1": "JSON"} } =
push().item('this').value('is').item('some').push().item('1').value('JSON').pop().pop()
You can then write trivial translaters from any format to any other format.
XmlStreamReader/Writer is handy for XML. JSON is trivial as well, either from a
object perspective or text stream perspective.
Textually:
push() = '{'
item(key) = '"key":'
value(value) '"value",' (or 'value,'), if value is int, or real, etc.
pop() = '}'
Objectually:
push() = obj={} // set currentObj to new obj, [].push(obj) onto objectStack
item(key) = // set currentObj[key] to valueRef, currentObj[key]=null
value(value) // valueRef=value
pop() // pop from objectStack
Don't forget though with JSON, there is only ever one root element, which is a
dictionary. You can't have your root JSON be and array [], or a bunch of items
"1,2,3" or {},{},{}, they all must be in a {"root: _items_{}_or_[]__ }
Which means you will always need a closing '}'
_______________________________________________
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
