Re: [Development] FW: QtCS19 Serialization session

2019-11-28 Thread Arnaud Clère
-Original Message-
From: Thiago Macieira 
>
> Would it make sense if this API were not in QtCore?
> Then we could give it a try and see how many people think it's useful, what 
> its pitfalls are, etc.

Yes, it is possible and it perfectly makes sense. It does not have to
wait for Qt6 too.

It only implies to duplicate a CBOR stream writer but the new one
would have the opportunity to distinct
QByteArray containing binary data from those explicitly containing
utf8 data for better performance.
The existing QCborStreamReader can be used from the outside as
performance is not so critical for reading.

At one point, my longer term goal of providing a structured logging
facility compatible with QDebug could
bring back the subject to move a few things back to QtCore but maybe not.

How do I proceed? I guess I need to ask for a specific repo as described here:
https://wiki.qt.io/Requesting_New_Repositories

Name: QtDataTransforms ?

Description:
QtDataTransforms long-term goal is to make it easy to move data
to/from specific Qt modules based on a
common JSON-like data model made of sequences, records, and basic data
types. It defines a default view of
most Qt types following their equivalent QML definitions. It is
designed to support all of JSON and CBOR formats,
as well as in-memory data structures like QJsonValue, QCborValue or
QVariantList/Map. It aims to support more
elaborate data models such as CSV, XML, and QAbstractItemModel using
optional metadata to allow, for instance,
reading a CSV file to a TableView in a single line of code, writing a
TreeView in XML format, etc.

Responsible persons: me and/or ? Who would be reviewing it?
Desired repository name: qtdatatransforms ?
Namespace: QtDataTransforms ? Is it possible to keep a few things in
Qt namespace ?

Thanks,
Arnaud
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] FW: QtCS19 Serialization session

2019-11-27 Thread Thiago Macieira
On Tuesday, 26 November 2019 23:13:46 PST Arnaud Clère wrote:
> -Original Message-
> From: Thiago Macieira 
> 
> > > value.meta(qmColumns,"key,value").bind(map.iterator())
> > 
> > I really, REALLY do not like this line.
> 
> I was going to explain all meta(...) were never required, but well,
> you have pretty well summarized your arguments so I will just stop
> wasting our time.

Would it make sense if this API were not in QtCore? Then we could give it a 
try and see how many people think it's useful, what its pitfalls are, etc.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] FW: QtCS19 Serialization session

2019-11-26 Thread Arnaud Clère
-Original Message-
From: Thiago Macieira 

> > value.meta(qmColumns,"key,value").bind(map.iterator())
>
> I really, REALLY do not like this line.

I was going to explain all meta(...) were never required, but well,
you have pretty well summarized your arguments so I will just stop
wasting our time.

Thanks,
Arnaud
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] FW: QtCS19 Serialization session

2019-11-26 Thread Thiago Macieira
On Tuesday, 26 November 2019 06:50:28 PST Arnaud Clère wrote:
> value.meta(qmColumns,"key,value").bind(map.iterator())

I really, REALLY do not like this line.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] FW: QtCS19 Serialization session

2019-11-26 Thread Arnaud Clère
Now, this is the discussion I was hoping for at QtCS :-)

> -Original Message-
> From: Thiago Macieira 
>
> > - it is frequent to have to save/load our types and then send/receive
> > them to some server/device using, say QSettings/QJsonDocument for the
> > first and JSON/CBOR for the latter
>
> How many do so interchangeably. I'm not asking about whether we should
> support different serialisations. I am asking how many types would benefit
> from a single, standardised mechanism for multiple backends?

The details of the serialization are usually far less important than
being able to
comply with the environment requirements :
- my company has developed a comprehensive XML toolset : whatever XML will do
(possibly with XSLT in the middle)
- my device need to exchange data : I want CBOR format but I am free to define
whatever payload suits my use case
- my desktop application has to load/save user data : whatever
QSettings flavor will do

> For example, a Rectangle in CBOR might be stored as (x1,y1,x2,y2) whereas the
> QSettings format might have been XxY+W+H, which means a generic solution
> will not work.

Truly, this kind of questions can become a standardization nightmare.
But as with all
standardization efforts, the value is not so much in the final
solution chosen than in providing
users with a common, well, "standard". Your example shows Qt evolved
in different directions wrt
serialization. Let us improve this in the future!

"@QPoint(10,20,100,150)" is a necessary facility to handle legacy but
a more explicit and principled
API and data format would be better.

Apart from a few basic types (int...) that should use the format
mandated by the targeted data format
or another standardized textual representation when there is none
(say, QUuid::toRfc4122 for JSON),
I argue that all data types should be "record/object/struct" with
meaningful property names.

Indeed, the QML types already look like a very nice and useful
de-facto standard for exchanging data
on top of JSON/CBOR :

https://doc.qt.io/qt-5/qmltypes.html
https://doc.qt.io/qt-5/qml-rect.html

According to this approach, no need to enter wasteful bikeshedding :

QRect::zap(...)  {
  return value.record("rect")
.item("x",x1)
.item("x",y1)
.item("width", x2-x1)
.item("height",y2-y1); // TODO use setHeight() if (value->mode()==Read)
}

> > There are not much alternatives
> > in how to serialize QList and the ones in CBOR are covered
> > (definite/indefinite). QPoint::zap or, say, QColor::zap may not fit
> > all use cases but the API allows to easily define another serialization.
>
> I do not agree. Please back up that statement with data: how many types would 
> benefit from a standardised serialisation?

What kind of data? I think all of the QML types above would benefit
from a standard way
to serialize to/from other data formats. Are you asking which ones
would work in read/write
using the fluent interface only?

> And why are you including item models in this, what do they have to do with 
> serialisation?

The proposed API handles serialization as simply iterating over
structured data with
some Reader/Writer. The proof-of-concept demonstrates having a
standard API for this
would make many use-cases become a simple one-liner like:

// Fill the table view with the CSV file content
QCsvReader(&csvFile).read(QItemModelWriter<>(&tableview).value());

That is the kind of simplicity, or absence thereof, that makes people
move away from C++
to Python (cf list comprehensions).

> > hey, I used to understand assembly and now it looks strange to me!
>
> Ok, that's actually a good point: assembly is very different from Qt-style
> C++, but it's not usually mixed with C++. What you're proposing is a C++very,
> very different API style that matches nothing in Qt and is meant to be used 
> alongside Qt-style C++.
>
> There is no example of fluent-like API in Qt. The closest we'll see of it in
> C++ is the ranges API with operator|, which are meant to imitate a shell
> pipeline anyway.

C++ has evolved from inline assembly to ranges. I do not think Qt
should stop in the middle.
QDataStream streaming interface is a fluent interface hidden behind a
cryptic operator.
Yet, most people find it convenient and some are even happy to not have
to understand the trick of passing along the QDataStream&.

> As I said, I don't like it and I fear a very different API could be confusing.
> I understand what it's meant to do. I am saying I don't like that it does all 
> it does.

Ok, but then how can we make it easier to read some CBOR into C++ types?
Build a QCborValue in memory, and then iterate over it to copy the
data to our own C++ types?
That will always be possible but my API would require much less work,
be safer and far more efficient.

> It's also advising people to write inefficient code. For example, de-
> serialising a record requires repeated iterations and is either O(n log n) for
> sorted records (like JSON) or O(n²) for unsorted ones like