Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Tuesday 13 October 2015 22:46:36 Marc Mutz wrote:
> I would propose to package the two into a class, called 
> - you guessed it - QStringView.

I now started work on QStringView (and later QByteArrayView), here:
   https://codereview.qt-project.org/#/q/topic:qstringview,n,z

Input welcome.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Paul Olav Tvete]

On Tuesday 20. October 2015 23.37.27 Thiago Macieira wrote:
> Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu:
> > As I remember it, the change from QSubString to QStringRef was a simple
> > API
> > naming decision, i.e. independent of the implementation details. 
> 
> Then it must be a coincidence that it changed internals more or less at the 
> same time.

Both were committed at the same time (in change 
138d1ab051ed4934b303fa7a43a27dc4004e3ff9 for those who have access to the old 
Qt history).  

Date:   Mon Jan 15 14:59:53 2007 +0100
Commit message: "Removed QSubString again, and introduced a lower-level class 
QStringRef"

QSubString had a QString::Data *d member, and the initial QStringRef had a 
const QString *m_string, just like today.

I have only skimmed the thread, so I'm probably misunderstanding something, 
but I don't see how a QString pointer is that much safer than a Data pointer, 
or a QChar pointer. 

A safe class would use the implicit sharing by keeping a QString member, but I 
don't know how that would impact performance.

- Paul
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Knoll Lars]

On 22/10/15 10:56, "Paul Olav Tvete"  wrote:

>On Tuesday 20. October 2015 23.37.27 Thiago Macieira wrote:
>> Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu:
>> > As I remember it, the change from QSubString to QStringRef was a
>>simple
>> > API
>> > naming decision, i.e. independent of the implementation details.
>> 
>> Then it must be a coincidence that it changed internals more or less at
>>the 
>> same time.
>
>Both were committed at the same time (in change
>138d1ab051ed4934b303fa7a43a27dc4004e3ff9 for those who have access to the
>old 
>Qt history).  
>
>Date:   Mon Jan 15 14:59:53 2007 +0100
>Commit message: "Removed QSubString again, and introduced a lower-level
>class 
>QStringRef"
>
>QSubString had a QString::Data *d member, and the initial QStringRef had
>a 
>const QString *m_string, just like today.
>
>I have only skimmed the thread, so I'm probably misunderstanding
>something, 
>but I don't see how a QString pointer is that much safer than a Data
>pointer, 
>or a QChar pointer.

Unless you participate in the ref counting it’s only slightly safer. It’s
safe against modifications of the string that do not shrink it’s length.
> 
>
>A safe class would use the implicit sharing by keeping a QString member,
>but I 
>don't know how that would impact performance.

Not a good idea IMO, as you’ll need to increase and decrease the refcount
of often trivial operations. In addition, I think a QChar pointer + length
is a better idea, as it also allows wrapping data that’s not contained in
a QString.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

Em qua 21 out 2015, às 06:45:53, Knoll Lars escreveu:
> >That doesn't work. I've tried to change QStringRef and it broke in a lot
> >of  places.
> 
> Do you remember what broke?

Not really. When I tracked down the failures, I did find out that it was 
related to the original QString being reallocated, so the QStringRef had a 
dangling reference. But I don't remember more details.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu:
> As I remember it, the change from QSubString to QStringRef was a simple API
> naming decision, i.e. independent of the implementation details. 

Then it must be a coincidence that it changed internals more or less at the 
same time.
 
> The reason a String pointer is in there was due to an attempt to make this
> class a bit safer. As long as we guaranteed the lifetime of the underlying
> String, accessing the string ref should be ok. In hindsight, I do think
> this was a mistake, as this safety doesn't give you a whole lot. In
> practice, you couldn't modify the underlying Qstring anyway, so we could
> have simply used a QChar pointer and a length. That certainly would have
> been more flexible and would have helped us today.
 
> So I'd agree that one could use a QStringView in the XML parser today.
> QStringRef is doing exactly the same, but has the disadvantage of requiring
> a pointer to a QString. This works in the parser, as it keeps the whole
> string in memory for as long as parsing is ongoing, and you are supposed to
> consume the returned stringref. But I would not recommend using
> QStringRef/QStringView as a return value in almost all other cases.

That doesn't work. I've tried to change QStringRef and it broke in a lot of 
places.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Knoll Lars]

On 21/10/15 08:37, 
"development-bounces+lars.knoll=theqtcompany@qt-project.org on behalf of 
Thiago Macieira" 
 wrote:

>Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu:
>> As I remember it, the change from QSubString to QStringRef was a simple API
>> naming decision, i.e. independent of the implementation details. 
>
>Then it must be a coincidence that it changed internals more or less at the 
>same time.

Possible. I have to admit that I don't recall all the details anymore. I can 
try to look up the old commit logs in case we think this is important.

> 
>> The reason a String pointer is in there was due to an attempt to make this
>> class a bit safer. As long as we guaranteed the lifetime of the underlying
>> String, accessing the string ref should be ok. In hindsight, I do think
>> this was a mistake, as this safety doesn't give you a whole lot. In
>> practice, you couldn't modify the underlying Qstring anyway, so we could
>> have simply used a QChar pointer and a length. That certainly would have
>> been more flexible and would have helped us today.
> 
>> So I'd agree that one could use a QStringView in the XML parser today.
>> QStringRef is doing exactly the same, but has the disadvantage of requiring
>> a pointer to a QString. This works in the parser, as it keeps the whole
>> string in memory for as long as parsing is ongoing, and you are supposed to
>> consume the returned stringref. But I would not recommend using
>> QStringRef/QStringView as a return value in almost all other cases.
>
>That doesn't work. I've tried to change QStringRef and it broke in a lot of 
>places.

Do you remember what broke?

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Knoll Lars]

On 20/10/15 18:11, 
"development-bounces+lars.knoll=theqtcompany@qt-project.org on behalf of 
Thiago Macieira" 
 wrote:



>On Tuesday 20 October 2015 08:49:17 André Somers wrote:
>> Op 19-10-2015 om 17:54 schreef Thiago Macieira:
>> > First of all, QStringRef keeps a pointer to the original QString, so it
>> > isn't as fragile as QStringView. In fact, during the development of the
>> > XML stream classes, there was a QSubString class that did more or less
>> > what QStringView would do. It got replaced for a reason.
>> 
>> Would you mind expanding on that a bit? I would think that the XML
>> stream classes would be a natural example to use a string view, but it
>> seems that such a thing was tried and dragons were uncovered... Perhaps
>> it would be good to share a war story here so we won't run into those
>> same dragons again?
>
>Wish that I could. I wasn't around that time. I know this happened because I 
>used to pull the daily snapshots and commit to an svk repository (yes, svk!) 
>to see the differences. That's how I spotted the QSubString class being added, 
>then removed, then QStringRef showed up.
>
>Is there anyone still around who worked with Matthias on this back in 
>2006/2007?

I was there doing the work with Matthias. 

As I remember it, the change from QSubString to QStringRef was a simple API 
naming decision, i.e. independent of the implementation details. 

The reason a String pointer is in there was due to an attempt to make this 
class a bit safer. As long as we guaranteed the lifetime of the underlying 
String, accessing the string ref should be ok. In hindsight, I do think this 
was a mistake, as this safety doesn't give you a whole lot. In practice, you 
couldn't modify the underlying Qstring anyway, so we could have simply used a 
QChar pointer and a length. That certainly would have been more flexible and 
would have helped us today.

So I'd agree that one could use a QStringView in the XML parser today. 
QStringRef is doing exactly the same, but has the disadvantage of requiring a 
pointer to a QString. This works in the parser, as it keeps the whole string in 
memory for as long as parsing is ongoing, and you are supposed to consume the 
returned stringref. But I would not recommend using QStringRef/QStringView as a 
return value in almost all other cases.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[André Somers]

Op 19-10-2015 om 17:54 schreef Thiago Macieira:
> First of all, QStringRef keeps a pointer to the original QString, so it isn't
> as fragile as QStringView. In fact, during the development of the XML stream
> classes, there was a QSubString class that did more or less what QStringView
> would do. It got replaced for a reason.
Would you mind expanding on that a bit? I would think that the XML 
stream classes would be a natural example to use a string view, but it 
seems that such a thing was tried and dragons were uncovered... Perhaps 
it would be good to share a war story here so we won't run into those 
same dragons again?

André


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 20, 2015 08:43:26 Thiago Macieira  wrote:

> On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote:
>> So if deep copies loose their ineffciency myth, what reason remains to not
>> use  QSV in all functions taking QString?
>
> Complicating the API.
>
> You cannot possibly prove that doing something O(n) is as efficient as doing 
> something O(1), therefore there's no way that deep copies would be a myth.

We both know that depends on the size. The minimal size of what the CPU gets 
from the memory is a cache line. The size of a cache line today is typically 64 
bytes. So as a minimum you always push that much data around. So if you use a 
atomic pointer on more than one cpu this cache lines has to be copied all the 
time. If you change something everything which depends on it has to be wait or 
recompute. So if  you accidentally have data on the some cache line it is  
shared too. This is not very intuitive. Your application can be much slower 
because you accidentally share data. 

Okay that is simplified but we get the picture. And the worst part is that 
memory is so much slower in relation to cache access as it was for the 
difference between memory and hard disks in the paste.  Today the difference 
between caches and memories resemble more that I learned about databases 20 
years ago. And there you learned to be careful about the complexity notations 
which doesn't reason about different memory speeds. 

COW with strings can lead to more coping than simply coping the string because 
most strings are small. Sharing writeable cache lines between different cpu is 
asking for trouble but I  think you know Amdahl's law 
https://en.wikipedia.org/wiki/Amdahl%27s_law. 

So why we don't measure it? Lets compile some applications with different 
implementation. Not simple ones. Maybe qtcreator would be a good idea. 

> COW may have comparable performance to deep copies if you consider the whole 
> picture and code written properly. Our code was written for COW, so I doubt 
> that you would get the same performance by suddenly making deep copies.
>
> To summarise:
>
> 1) I'm ok with adding QStringView
> 2) I'm ok with using QStringView in applications, no restraints
> 3) I'm ok with exploring the use in API, in parameters
> 4) I'm ok with exploring it in return values, in very limited conditions that 
> are being studied very well, ahead of time
>
> Exploring does not mean you can go and start adding overloads everywhere.

I think that's a very good approach. 

> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel Open Source Technology Center
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
Sent from cellphone 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Tuesday 20 October 2015 01:08:31 Thiago Macieira wrote:
> So the summary of QStringView is:
>  a) never used as return type
>  b) used only as a parameter if the function processes the data and never 
>  stores it and will never, ever store it (lazy processing)
>  c) for existing API, needs to be overloaded with QString
> 
> I'm not buying the cost/benefit ratio.

That's because you castrate the uses too much :)

There are a ton of inlines types that would benefit, because they cannot be 
changed anyway.

If you look at uic or qdoc, you'll see QStringView use-cases pop up in every 
corner.

But mainly our impedance mismatch is that you focus on return values and I 
focus on arguments. I agree on the trickyness of the QSV result types, but 
there are always exceptions, some of which you and I mentioned, so I wouldn't 
go as far as saying (a). That would ban QXmlStreamReader, e.g.

n the function argument side of things I see no real reason to have anything 
but QStringView arguments, plus strategic QString&& ones. The only reason to 
so much restrict the use of QSV as arguments type is the wish to avoid QString 
deep copies, the fear of which you said you don't share.

So if deep copies loose their ineffciency myth, what reason remains to not use 
QSV in all functions taking QString?

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

I see. But then, if I have QStringView, doesn't that eliminate most of the 
reasons for needing the other string containing classes? If I want the 
efficiency of QStringView, won't QString underneath always be the right choice?

In Thiago's example, if I have a QStringView API on my class, would I ever want 
to switch the QString to a QByteArray?

Does using QStringView with QString work as well or better than WByteArray et 
al?

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Monday, October 19, 2015 11:26 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

On Monday 19 October 2015 21:56:54 Smith Martin wrote:
> >This API here simply cannot exist because the returned value would be a
> >dangling reference.
>
> I don't understand. Implicit for using the QStringView data() function is
> knowing that once the QByteArray is set, it is never changed. Then the
> data() function is ok, isn't it?

A QStringView points to QChar* (or ushort*) while a QByteArray only provides
char*. The fromLatin1() (say) call thus necessary will create a temporary
QString, which will bind to the returned QStringView, but will already be
destroyed (taking the data QStringView references with it) when control
returns to the caller.

This particular mistake is easy to prevent statically, though:

   QStringView(QString &&) = delete;

Thanks,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

That's QByteArray at the end, of course.


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Smith Martin <martin.sm...@theqtcompany.com>
Sent: Tuesday, October 20, 2015 9:13 AM
To: Marc Mutz; development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

I see. But then, if I have QStringView, doesn't that eliminate most of the 
reasons for needing the other string containing classes? If I want the 
efficiency of QStringView, won't QString underneath always be the right choice?

In Thiago's example, if I have a QStringView API on my class, would I ever want 
to switch the QString to a QByteArray?

Does using QStringView with QString work as well or better than WByteArray et 
al?

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Monday, October 19, 2015 11:26 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

On Monday 19 October 2015 21:56:54 Smith Martin wrote:
> >This API here simply cannot exist because the returned value would be a
> >dangling reference.
>
> I don't understand. Implicit for using the QStringView data() function is
> knowing that once the QByteArray is set, it is never changed. Then the
> data() function is ok, isn't it?

A QStringView points to QChar* (or ushort*) while a QByteArray only provides
char*. The fromLatin1() (say) call thus necessary will create a temporary
QString, which will bind to the returned QStringView, but will already be
destroyed (taking the data QStringView references with it) when control
returns to the caller.

This particular mistake is easy to prevent statically, though:

   QStringView(QString &&) = delete;

Thanks,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

I guess we need QStringView for QString and QByteArrayView for QByteArray, etc.

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Smith Martin <martin.sm...@theqtcompany.com>
Sent: Tuesday, October 20, 2015 9:13 AM
To: Marc Mutz; development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

I see. But then, if I have QStringView, doesn't that eliminate most of the 
reasons for needing the other string containing classes? If I want the 
efficiency of QStringView, won't QString underneath always be the right choice?

In Thiago's example, if I have a QStringView API on my class, would I ever want 
to switch the QString to a QByteArray?

Does using QStringView with QString work as well or better than WByteArray et 
al?

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Monday, October 19, 2015 11:26 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

On Monday 19 October 2015 21:56:54 Smith Martin wrote:
> >This API here simply cannot exist because the returned value would be a
> >dangling reference.
>
> I don't understand. Implicit for using the QStringView data() function is
> knowing that once the QByteArray is set, it is never changed. Then the
> data() function is ok, isn't it?

A QStringView points to QChar* (or ushort*) while a QByteArray only provides
char*. The fromLatin1() (say) call thus necessary will create a temporary
QString, which will bind to the returned QStringView, but will already be
destroyed (taking the data QStringView references with it) when control
returns to the caller.

This particular mistake is easy to prevent statically, though:

   QStringView(QString &&) = delete;

Thanks,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Иван Комиссаров]

Btw, isn't the QStringView is the same as Range
{ Container::Iterator begin; Container::Iterator end; } ?
Why we introduce only QStringView/QByteArrayView? Maybe we should think
about adding range API to all containers?

Alexandrescus presentation about ranges:
http://www.slideshare.net/rawwell/iteratorsmustgo

2015-10-20 11:56 GMT+03:00 Smith Martin <martin.sm...@theqtcompany.com>:

> I guess we need QStringView for QString and QByteArrayView for QByteArray,
> etc.
>
> martin
>
> 
> From: development-bounces+martin.smith=theqtcompany@qt-project.org
> <development-bounces+martin.smith=theqtcompany@qt-project.org> on
> behalf of Smith Martin <martin.sm...@theqtcompany.com>
> Sent: Tuesday, October 20, 2015 9:13 AM
> To: Marc Mutz; development@qt-project.org
> Subject: Re: [Development] RFC: Proposal for a semi-radical change in
>  Qt  APIs taking strings
>
> I see. But then, if I have QStringView, doesn't that eliminate most of the
> reasons for needing the other string containing classes? If I want the
> efficiency of QStringView, won't QString underneath always be the right
> choice?
>
> In Thiago's example, if I have a QStringView API on my class, would I ever
> want to switch the QString to a QByteArray?
>
> Does using QStringView with QString work as well or better than WByteArray
> et al?
>
> martin
>
> 
> From: development-bounces+martin.smith=theqtcompany@qt-project.org
> <development-bounces+martin.smith=theqtcompany@qt-project.org> on
> behalf of Marc Mutz <marc.m...@kdab.com>
> Sent: Monday, October 19, 2015 11:26 PM
> To: development@qt-project.org
> Subject: Re: [Development] RFC: Proposal for a semi-radical change in
>  Qt  APIs taking strings
>
> On Monday 19 October 2015 21:56:54 Smith Martin wrote:
> > >This API here simply cannot exist because the returned value would be a
> > >dangling reference.
> >
> > I don't understand. Implicit for using the QStringView data() function is
> > knowing that once the QByteArray is set, it is never changed. Then the
> > data() function is ok, isn't it?
>
> A QStringView points to QChar* (or ushort*) while a QByteArray only
> provides
> char*. The fromLatin1() (say) call thus necessary will create a temporary
> QString, which will bind to the returned QStringView, but will already be
> destroyed (taking the data QStringView references with it) when control
> returns to the caller.
>
> This particular mistake is easy to prevent statically, though:
>
>QStringView(QString &&) = delete;
>
> Thanks,
> Marc
>
> --
> Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
> KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
> Tel: +49-30-521325470
> KDAB - The Qt Experts
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
>
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 20, 2015 10:10:36 Knoll Lars  wrote:

> On 20/10/15 09:59, "Bubke Marco"  wrote:
>
>>On October 20, 2015 08:43:26 Thiago Macieira 
>>wrote:
>>
>>> On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote:
 So if deep copies loose their ineffciency myth, what reason remains to
not
 use  QSV in all functions taking QString?
>>>
>>> Complicating the API.
>>>
>>> You cannot possibly prove that doing something O(n) is as efficient as
>>>doing 
>>> something O(1), therefore there's no way that deep copies would be a
>>>myth.
>>
>>We both know that depends on the size. The minimal size of what the CPU
>>gets from the memory is a cache line. The size of a cache line today is
>>typically 64 bytes. So as a minimum you always push that much data
>>around. So if you use a atomic pointer on more than one cpu this cache
>>lines has to be copied all the time. If you change something everything
>>which depends on it has to be wait or recompute. So if  you accidentally
>>have data on the some cache line it is  shared too. This is not very
>>intuitive. Your application can be much slower because you accidentally
>>share data. 
>>
>>Okay that is simplified but we get the picture. And the worst part is
>>that memory is so much slower in relation to cache access as it was for
>>the difference between memory and hard disks in the paste.  Today the
>>difference between caches and memories resemble more that I learned about
>>databases 20 years ago. And there you learned to be careful about the
>>complexity notations which doesn't reason about different memory speeds.
>>
>>COW with strings can lead to more coping than simply coping the string
>>because most strings are small. Sharing writeable cache lines between
>>different cpu is asking for trouble but I  think you know Amdahl's law
>>https://en.wikipedia.org/wiki/Amdahl%27s_law.
>>
>>So why we don't measure it? Lets compile some applications with different
>>implementation. Not simple ones. Maybe qtcreator would be a good idea.
>
> Experimenting with that is probably a good idea, but we certainly can’t
> change away from COW any time soon. Maybe not ever because of
> compatibility concerns.

We should be careful not to get the new motif too. ;-) I think we have to find 
a middle ground again and again. It's  a question of how much pain our actions 
are producing and how much our inactions.  ;-) 

>> 
>>
>>> COW may have comparable performance to deep copies if you consider the
>>>whole 
>>> picture and code written properly. Our code was written for COW, so I
>>>doubt 
>>> that you would get the same performance by suddenly making deep copies.
>>>
>>> To summarise:
>>>
>>> 1) I'm ok with adding QStringView
>>> 2) I'm ok with using QStringView in applications, no restraints
>>> 3) I'm ok with exploring the use in API, in parameters
>>> 4) I'm ok with exploring it in return values, in very limited
>>>conditions that 
>>> are being studied very well, ahead of time
>>>
>>> Exploring does not mean you can go and start adding overloads
>>>everywhere.
>>
>>I think that's a very good approach.
>
> +1.
>
> Cheers,
> Lars
Sent from cellphone 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Tuesday 20 October 2015 11:08:51 Иван Комиссаров wrote:
> Btw, isn't the QStringView is the same as Range
> { Container::Iterator begin; Container::Iterator end; } ?

No. A string_view (or QStringView) is targeted at string operations. A range 
is just a pair of iterators.

> Why we introduce only QStringView/QByteArrayView? Maybe we should think
> about adding range API to all containers?

We want to use the std functionalities in this area (ranges and array_view), 
because the API differences are small, and this is not a core skill of Qt as a 
library. Unicode string handling is. Our containers are all much weaker than 
the std ones, from QVector over QPair to QSharedPointer, and that should give 
you the hint that we don't have the manpower (nor should be have the desire) 
to compete with the std on these matters.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Tuesday 20 October 2015 09:13:18 Smith Martin wrote:
> I see. But then, if I have QStringView, doesn't that eliminate most of the
> reasons for needing the other string containing classes? If I want the
> efficiency of QStringView, won't QString underneath always be the right
> choice?

No. Two examples:

QStringLiteral - currently, it returns a full QString instance, with 
associated need to call the out-of-line QString dtor. Q6String will solve many 
of the issues, as Thiago has already said. But with QStringView, we could just 
pass u"foo"  (a const char16_t[4]) _now_, and be done with it. The compiler 
and linker will share those string literals for us and even perform common 
suffix optimisations ("bar" and "foobar" overlap in the executable - that's 
allowed for string literals (and empty base classes) but not any other 
objects). No QString involved.

If you build a local temporary string, it makes sense to use 
QVarLengthArray to do so. Again no QString involved.

> In Thiago's example, if I have a QStringView API on my class, would I ever
> want to switch the QString to a QByteArray?

Yes. See, say, QUrl, which needs to juggle both the encoded and the decoded 
form of a URL.

Also, if you have an XML file, in most cases it will not be encoded in UTF-16 
on disk, so what you're working off of is a QByteArray (yes, I realize that 
QXmlStreamReader is working on QChar* (QStringRef, QString, 
QXmlStreamStringRef, ...), so take emails instead.

> Does using QStringView with QString work as well or better than WByteArray
> et al?

QByteArray is to QString what QLatin1String is (a bit) to QStringView, but 
we've been discussing (and agreeing on) the need for a QByteArrayView, too, 
because not all QByteArrays contain latin-1 strings.

But those who wish to experiment with a string view in Qt can use 
QLatin1String more. Now.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[André Somers]

Op 20-10-2015 om 11:08 schreef Иван Комиссаров:
> Btw, isn't the QStringView is the same as Range 
> { Container::Iterator begin; Container::Iterator end; } ?
> Why we introduce only QStringView/QByteArrayView? Maybe we should 
> think about adding range API to all containers?
>
> Alexandrescus presentation about ranges: 
> http://www.slideshare.net/rawwell/iteratorsmustgo

More recent: https://www.youtube.com/watch?v=mFUXNMfaciE (Eric Niebler 
on CppCon)

But I think it is a bit early to start doing that, seeing that the whole 
idea is still in flux in the STL. I'm sure we don't want to end up with 
something incompatible (again), right? Also, a QStringView is more than 
"just" a range, as it supports all kinds of string-related API similar 
to QString, or so I will assume based on the similar proposal for 
std::string_view. (as presented in 
https://www.youtube.com/watch?v=H9gAaNRoon4)

André

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[André Somers]

Op 19-10-2015 om 22:31 schreef Bubke Marco:
> For example the highlighting can show that the parameter is an in or in/out 
> parameter so you don't need to use pointers anymore for in/out parameters 
> etc. But it is getting off topic.
https://bugreports.qt.io/browse/QTCREATORBUG-14468

André

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote:
> So if deep copies loose their ineffciency myth, what reason remains to not
> use  QSV in all functions taking QString?

Complicating the API.

You cannot possibly prove that doing something O(n) is as efficient as doing 
something O(1), therefore there's no way that deep copies would be a myth.

COW may have comparable performance to deep copies if you consider the whole 
picture and code written properly. Our code was written for COW, so I doubt 
that you would get the same performance by suddenly making deep copies.

To summarise:

1) I'm ok with adding QStringView
2) I'm ok with using QStringView in applications, no restraints
3) I'm ok with exploring the use in API, in parameters
4) I'm ok with exploring it in return values, in very limited conditions that 
are being studied very well, ahead of time

Exploring does not mean you can go and start adding overloads everywhere.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Julien Blanc]

Le lundi 19 octobre 2015 à 16:08 -0700, Thiago Macieira a écrit : 

> 
> So the summary of QStringView is:
>  a) never used as return type
>  b) used only as a parameter if the function processes the data and never 
>  stores it and will never, ever store it (lazy processing)
>  c) for existing API, needs to be overloaded with QString
> 
> I'm not buying the cost/benefit ratio.


I don’t get b) statement.

If you need to store it, as a QString, and you can afford to expose that
to the user, then i think it’s better to provide a QString&& overload.
This :
- makes clear that you will store the value « as-is ».
- should be more efficient than copying the string
- still allows copying the string if the user needs to keep its own copy
of the data.
- cohabites well with a QStringView api

I do not see the benefit of a QString api vs a QStringView and a
QString&& api, which cover more use cases.

Regards,

Julien Blanc
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Knoll Lars]

On 20/10/15 09:59, "Bubke Marco"  wrote:

>On October 20, 2015 08:43:26 Thiago Macieira 
>wrote:
>
>> On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote:
>>> So if deep copies loose their ineffciency myth, what reason remains to
>>>not
>>> use  QSV in all functions taking QString?
>>
>> Complicating the API.
>>
>> You cannot possibly prove that doing something O(n) is as efficient as
>>doing 
>> something O(1), therefore there's no way that deep copies would be a
>>myth.
>
>We both know that depends on the size. The minimal size of what the CPU
>gets from the memory is a cache line. The size of a cache line today is
>typically 64 bytes. So as a minimum you always push that much data
>around. So if you use a atomic pointer on more than one cpu this cache
>lines has to be copied all the time. If you change something everything
>which depends on it has to be wait or recompute. So if  you accidentally
>have data on the some cache line it is  shared too. This is not very
>intuitive. Your application can be much slower because you accidentally
>share data. 
>
>Okay that is simplified but we get the picture. And the worst part is
>that memory is so much slower in relation to cache access as it was for
>the difference between memory and hard disks in the paste.  Today the
>difference between caches and memories resemble more that I learned about
>databases 20 years ago. And there you learned to be careful about the
>complexity notations which doesn't reason about different memory speeds.
>
>COW with strings can lead to more coping than simply coping the string
>because most strings are small. Sharing writeable cache lines between
>different cpu is asking for trouble but I  think you know Amdahl's law
>https://en.wikipedia.org/wiki/Amdahl%27s_law.
>
>So why we don't measure it? Lets compile some applications with different
>implementation. Not simple ones. Maybe qtcreator would be a good idea.

Experimenting with that is probably a good idea, but we certainly can’t
change away from COW any time soon. Maybe not ever because of
compatibility concerns.
> 
>
>> COW may have comparable performance to deep copies if you consider the
>>whole 
>> picture and code written properly. Our code was written for COW, so I
>>doubt 
>> that you would get the same performance by suddenly making deep copies.
>>
>> To summarise:
>>
>> 1) I'm ok with adding QStringView
>> 2) I'm ok with using QStringView in applications, no restraints
>> 3) I'm ok with exploring the use in API, in parameters
>> 4) I'm ok with exploring it in return values, in very limited
>>conditions that 
>> are being studied very well, ahead of time
>>
>> Exploring does not mean you can go and start adding overloads
>>everywhere.
>
>I think that's a very good approach.

+1.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Tuesday 20 October 2015 10:06:28 Julien Blanc wrote:
> Le lundi 19 octobre 2015 à 16:08 -0700, Thiago Macieira a écrit :
> > So the summary of QStringView is:
> >  a) never used as return type
> >  b) used only as a parameter if the function processes the data and never
> >  stores it and will never, ever store it (lazy processing)
> >  
> >  c) for existing API, needs to be overloaded with QString
> > 
> > I'm not buying the cost/benefit ratio.
> 
> I don’t get b) statement.
>
> If you need to store it, as a QString, and you can afford to expose that
> to the user, then i think it’s better to provide a QString&& overload.

QString&& won't match a regular lvalue.

QString s = "foo";
function(s);

[and even if it did, it would be wrong]

But this would compile:

function(QString(s))

because it now creates a copy before passing to the function. That is, you 
create a copy that you're giving ownership of.

I don't think I want this kind of API. It's too confusing.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Tuesday 20 October 2015 08:49:17 André Somers wrote:
> Op 19-10-2015 om 17:54 schreef Thiago Macieira:
> > First of all, QStringRef keeps a pointer to the original QString, so it
> > isn't as fragile as QStringView. In fact, during the development of the
> > XML stream classes, there was a QSubString class that did more or less
> > what QStringView would do. It got replaced for a reason.
> 
> Would you mind expanding on that a bit? I would think that the XML
> stream classes would be a natural example to use a string view, but it
> seems that such a thing was tried and dragons were uncovered... Perhaps
> it would be good to share a war story here so we won't run into those
> same dragons again?

Wish that I could. I wasn't around that time. I know this happened because I 
used to pull the daily snapshots and commit to an svk repository (yes, svk!) 
to see the differences. That's how I spotted the QSubString class being added, 
then removed, then QStringRef showed up.

Is there anyone still around who worked with Matthias on this back in 
2006/2007?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote:
> On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> > That the proposal is that every single function currently taking a 
> > QString should instead be changed to take a QStringView instead, unless 
> > a few exceptional cases (for instance the function is used to store the 
> > string somehow, plus other cases under discussion). Similar case for 
> > return values. If that doesn't look like a radical change to you...
> 
> Return values must always be QString, never QStringView. Returning a view
> is  like returning a reference and goes against Qt's library code policy.

That's like saying that functions musn't return a QStringRef. But see 
QXmlStreamReader, which uses that to (apparently) good effect.

I also mentioned the case where a plugin returns some static data as a QString 
coming from a QStringLiteral. You yourself mentioned that that's going boom 
when the plugin is unloaded. If the plugin instead returned a QStringView, 
then callers wishing to store the result would have toString() it, making a 
deep copy and insulating themselves from the plugin being unloaded, while 
users wishing to just do some mix-and-matching with the string could avoid the 
copy by keeping the returned data in a QStringView.

People should stop being so afraid of a string deep copy. It's a no-op for 
SSO'ed strings and according to Herb Sutter's benchmark on average doesn't 
matter for all other strings. Here's where safety and performance could 
actually go hand-in-hand for once.

Safety? Yes, the above QStringLiteral problem and also, as you know (and it's 
even documented) that Qt doesn't implement CoW correctly:

  QString s1 = "foo";
  QString::iterator it = s1.begin();
  QString s2 = s1;
  *it = 'b';
  // oops: s2 == "boo", too!

I would like to see proof that ref-counted strings are actually worth it, even 
in the unsafe Qt implementation, considering that the C++ world slowly moves 
away from them.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>Return values must always be QString, never QStringView. Returning a view is
>like returning a reference and goes against Qt's library code policy.

But an assumption is the user manages the ownership of the underlying string, 
so it seems different. I'm rewriting the C++ parsing in qdoc. It calls a 
tokenizer that has a token()function that returns an enum value and a lexeme() 
function that returns a QString. It builds the QString for lexeme() each time, 
when it would be better to return a QStringView, since I know the text won't 
change. For the names that have to be saved, the QStringView can be passed to 
the tree node constructor, and only one QString gets built.

martin

From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Thiago Macieira <thiago.macie...@intel.com>
Sent: Sunday, October 18, 2015 10:55 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt    
APIs taking strings

On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> That the proposal is that every single function currently taking a
> QString should instead be changed to take a QStringView instead, unless
> a few exceptional cases (for instance the function is used to store the
> string somehow, plus other cases under discussion). Similar case for
> return values. If that doesn't look like a radical change to you...

Return values must always be QString, never QStringView. Returning a view is
like returning a reference and goes against Qt's library code policy.

--
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Monday 19 October 2015 06:10:41 Smith Martin wrote:
> >Return values must always be QString, never QStringView. Returning a view
> >is like returning a reference and goes against Qt's library code policy.
>
> But an assumption is the user manages the ownership of the underlying
> string, so it seems different. 

Are you talking about a class that operates on user-provided data?

In that restricted scenario, it might make sense. The point I'm trying to make 
is that returning a non-owning copy means that something else must own the 
data. That's what goes against the library code policy.

Example:

QUrl url;
QStringView query() const;

It's not possible to write the query() function.

> I'm rewriting the C++ parsing in qdoc. It
> calls a tokenizer that has a token()function that returns an enum value and
> a lexeme() function that returns a QString. It builds the QString for
> lexeme() each time, when it would be better to return a QStringView, since
> I know the text won't change.

Good, but qdoc is not a library so it's not subject to the library code 
policy.

> For the names that have to be saved, the
> QStringView can be passed to the tree node constructor, and only one
> QString gets built.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>The point I'm trying to make 
>is that returning a non-owning copy means that something else must own the 
>data. That's what goes against the library code policy.

The library code policy is an one that has always made sense in the 
pre-string_view world. But if string_view is being added to C++, don't we have 
to add it to Qt? And if we add it to Qt, don't we have to support it in all the 
Qt classes where it could improve performance? 

Won't internet-of-things developers want a library that fully supports 
string_view?

martin



From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Thiago Macieira <thiago.macie...@intel.com>
Sent: Monday, October 19, 2015 8:21 AM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

On Monday 19 October 2015 06:10:41 Smith Martin wrote:
> >Return values must always be QString, never QStringView. Returning a view
> >is like returning a reference and goes against Qt's library code policy.
>
> But an assumption is the user manages the ownership of the underlying
> string, so it seems different.

Are you talking about a class that operates on user-provided data?

In that restricted scenario, it might make sense. The point I'm trying to make
is that returning a non-owning copy means that something else must own the
data. That's what goes against the library code policy.

Example:

QUrl url;
QStringView query() const;

It's not possible to write the query() function.

> I'm rewriting the C++ parsing in qdoc. It
> calls a tokenizer that has a token()function that returns an enum value and
> a lexeme() function that returns a QString. It builds the QString for
> lexeme() each time, when it would be better to return a QStringView, since
> I know the text won't change.

Good, but qdoc is not a library so it's not subject to the library code
policy.

> For the names that have to be saved, the
> QStringView can be passed to the tree node constructor, and only one
> QString gets built.

--
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>That's like saying that functions musn't return a QStringRef. But see
>QXmlStreamReader, which uses that to (apparently) good effect.

That's a good point, but the other points you raise are just arguing by finding 
fault with Qt. I think fully supporting QStringView has to be justified by 
showing a way to include it in the library so that it can be used safely.

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Monday, October 19, 2015 11:23 AM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt    
APIs taking strings

On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote:
> On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> > That the proposal is that every single function currently taking a
> > QString should instead be changed to take a QStringView instead, unless
> > a few exceptional cases (for instance the function is used to store the
> > string somehow, plus other cases under discussion). Similar case for
> > return values. If that doesn't look like a radical change to you...
>
> Return values must always be QString, never QStringView. Returning a view
> is  like returning a reference and goes against Qt's library code policy.

That's like saying that functions musn't return a QStringRef. But see
QXmlStreamReader, which uses that to (apparently) good effect.

I also mentioned the case where a plugin returns some static data as a QString
coming from a QStringLiteral. You yourself mentioned that that's going boom
when the plugin is unloaded. If the plugin instead returned a QStringView,
then callers wishing to store the result would have toString() it, making a
deep copy and insulating themselves from the plugin being unloaded, while
users wishing to just do some mix-and-matching with the string could avoid the
copy by keeping the returned data in a QStringView.

People should stop being so afraid of a string deep copy. It's a no-op for
SSO'ed strings and according to Herb Sutter's benchmark on average doesn't
matter for all other strings. Here's where safety and performance could
actually go hand-in-hand for once.

Safety? Yes, the above QStringLiteral problem and also, as you know (and it's
even documented) that Qt doesn't implement CoW correctly:

  QString s1 = "foo";
  QString::iterator it = s1.begin();
  QString s2 = s1;
  *it = 'b';
  // oops: s2 == "boo", too!

I would like to see proof that ref-counted strings are actually worth it, even
in the unsafe Qt implementation, considering that the C++ world slowly moves
away from them.

Thanks,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Matthew Woehlke]

On 2015-10-16 19:52, Kurt Pattyn wrote:
> On 17 Oct 2015, at 01:18, Marc Mutz wrote:
>> I find it intolerable that todays software takes 1000x the memory,
>>  1000x the CPU capacity and doesn't get a given jobs done in less
>> wallclock time than software 20 years ago.
>
> These are 'numbers'. Is it like that? Really 1000x the memory, 1000x the CPU 
> capacity?

Er... yes?

An 80286 of yesteryear ran as fast as 25 MHz¹. A modern, 6-core 4 GHz
CPU has a "theoretical" 24 GHz of "computing power". That's almost
exactly 1000x (more, considering the ISA improvements). The same machine
could address up to 16 MiB of RAM. A lot of machines around my office
have 16 GiB of RAM (my home desktop has more; OTOH, many home machines
have less). That's pedantically a factor of 1024x; again, close enough.

Basically, yeah... computers some 10-20 years ago had similar specs as
modern computers - CPU speed in the single to low double digits, memory
in similar ranges - except the numbers were in one 1000/1024-order of
magnitude less units (MHz vs. GHz, MiB vs. GiB). The increase in disk
space is, if anything, even more extreme.

There remains, of course, *some* exaggeration there... modern computers
are, after all, tackling quantities of data that computers 20 years ago
couldn't dream of touching (consider any process run on large clusters,
for example). Even so, it's sobering to think about, especially as a lot
of user software is "prettier" but in many ways not actually *better*,
at least not in proportion to the increase in resource requirements.

(¹ https://en.wikipedia.org/wiki/Intel_80286)

-- 
Matthew

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Monday 19 October 2015 11:23:38 Marc Mutz wrote:
> On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote:
> > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> > > That the proposal is that every single function currently taking a
> > > QString should instead be changed to take a QStringView instead, unless
> > > a few exceptional cases (for instance the function is used to store the
> > > string somehow, plus other cases under discussion). Similar case for
> > > return values. If that doesn't look like a radical change to you...
> > 
> > Return values must always be QString, never QStringView. Returning a view
> > is  like returning a reference and goes against Qt's library code policy.
> 
> That's like saying that functions musn't return a QStringRef. But see
> QXmlStreamReader, which uses that to (apparently) good effect.

No, it isn't.

First of all, QStringRef keeps a pointer to the original QString, so it isn't 
as fragile as QStringView. In fact, during the development of the XML stream 
classes, there was a QSubString class that did more or less what QStringView 
would do. It got replaced for a reason.

Second, this falls under the exception of parsing a large block of data and 
getting information about chunks inside.

And besides, it does make code using those classes uglier.

> I also mentioned the case where a plugin returns some static data as a
> QString coming from a QStringLiteral. You yourself mentioned that that's
> going boom when the plugin is unloaded. If the plugin instead returned a
> QStringView, then callers wishing to store the result would have toString()
> it, making a deep copy and insulating themselves from the plugin being
> unloaded, while users wishing to just do some mix-and-matching with the
> string could avoid the copy by keeping the returned data in a QStringView.

I agree on the advantage.

>From what I can see, a Qt6 QStringView is a QString with a null d pointer, 
except that it triggers deep copy everywhere whre it's assigned to a regular 
QString. I'm fine with having the class.

I'm not ok with changing the library code policy.

> People should stop being so afraid of a string deep copy. It's a no-op for
> SSO'ed strings and according to Herb Sutter's benchmark on average doesn't
> matter for all other strings. Here's where safety and performance could
> actually go hand-in-hand for once.

That's not what I am afraid of.

Again, please try writing this method:

class Foo
{
QUrl url;
public:
QStringView host() const;
};


-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Monday 19 October 2015 07:14:55 Smith Martin wrote:
> >The point I'm trying to make
> >is that returning a non-owning copy means that something else must own the
> >data. That's what goes against the library code policy.
> 
> The library code policy is an one that has always made sense in the
> pre-string_view world. But if string_view is being added to C++, don't we
> have to add it to Qt? And if we add it to Qt, don't we have to support it
> in all the Qt classes where it could improve performance?

I don't think that string_view makes a difference. The point is still the same: 
returning non-owning copies of data implies something else owns it. We haven't 
done that so far and I don't think string_view should change the policy.

> 
> Won't internet-of-things developers want a library that fully supports
> string_view?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Monday 19 October 2015 21:56:54 Smith Martin wrote:
> >This API here simply cannot exist because the returned value would be a 
> >dangling reference.
> 
> I don't understand. Implicit for using the QStringView data() function is
> knowing that once the QByteArray is set, it is never changed. Then the
> data() function is ok, isn't it?

A QStringView points to QChar* (or ushort*) while a QByteArray only provides 
char*. The fromLatin1() (say) call thus necessary will create a temporary 
QString, which will bind to the returned QStringView, but will already be 
destroyed (taking the data QStringView references with it) when control 
returns to the caller.

This particular mistake is easy to prevent statically, though:

   QStringView(QString &&) = delete;

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 19, 2015 21:38:51 Thiago Macieira  wrote:

> On Monday 19 October 2015 18:38:52 Smith Martin wrote:
>> >Again, please try writing this method:
>> Doesn't that example just mean there are some classes that can't have a
>> QStringView API? A class should have a QStringView API if it can be used
>> safely.
>
> Right. I'm simply saying that "if it can be used safely" is very, very 
> restricted.
>
> Suppose you have in v1:
>
> class Foo
> {
>   QString m_data;
> public:
>   QStringView data() const;
> };
>
> Which looks fine and works without dangling references. Then in v2, you need 
> to 
> do this:
>
> class Foo
> {
>   QByteArray m_data;
> public:
>   QStringView data() const;
> };

How can you make a QStringView from a byte array? I know we abuse byte array as 
utf8 class but if you convert it you have no ownership of the created QString. 
Do you do a reinterpret cast? ;-) 

I see QStringView as const QString & but much more explicit and portable. So 
maybe you want to show that people can return a string view of a temporary? We 
can add a error in the clang code model so it should be a non issue. 

I think we should develop and use better tools which warn you about mistakes 
like that. That makes C++ much easier. ;-) 

Today I spoke with Ossi about the integration in gerrit too. So if you have 
enough firewalls we can be much more liberal in what we can use. 

For example the highlighting can show that the parameter is an in or in/out 
parameter so you don't need to use pointers anymore for in/out parameters etc. 
But it is getting off topic. 

> This API here simply cannot exist because the returned value would be a 
> dangling reference.
>
> Therefore, QStringView returns must be treated like references: only in 
> exceptional cases.
>
> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel Open Source Technology Center
>
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
Sent from cellphone 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Monday 19 October 2015 18:38:52 Smith Martin wrote:
> >Again, please try writing this method:
> Doesn't that example just mean there are some classes that can't have a
> QStringView API? A class should have a QStringView API if it can be used
> safely.

Right. I'm simply saying that "if it can be used safely" is very, very 
restricted.

Suppose you have in v1:

class Foo
{
QString m_data;
public:
QStringView data() const;
};

Which looks fine and works without dangling references. Then in v2, you need to 
do this:

class Foo
{
QByteArray m_data;
public:
QStringView data() const;
};

This API here simply cannot exist because the returned value would be a 
dangling reference.

Therefore, QStringView returns must be treated like references: only in 
exceptional cases.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>Again, please try writing this method:

Doesn't that example just mean there are some classes that can't have a 
QStringView API? A class should have a QStringView API if it can be used safely.

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Thiago Macieira <thiago.macie...@intel.com>
Sent: Monday, October 19, 2015 5:54 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt    
APIs taking strings

On Monday 19 October 2015 11:23:38 Marc Mutz wrote:
> On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote:
> > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> > > That the proposal is that every single function currently taking a
> > > QString should instead be changed to take a QStringView instead, unless
> > > a few exceptional cases (for instance the function is used to store the
> > > string somehow, plus other cases under discussion). Similar case for
> > > return values. If that doesn't look like a radical change to you...
> >
> > Return values must always be QString, never QStringView. Returning a view
> > is  like returning a reference and goes against Qt's library code policy.
>
> That's like saying that functions musn't return a QStringRef. But see
> QXmlStreamReader, which uses that to (apparently) good effect.

No, it isn't.

First of all, QStringRef keeps a pointer to the original QString, so it isn't
as fragile as QStringView. In fact, during the development of the XML stream
classes, there was a QSubString class that did more or less what QStringView
would do. It got replaced for a reason.

Second, this falls under the exception of parsing a large block of data and
getting information about chunks inside.

And besides, it does make code using those classes uglier.

> I also mentioned the case where a plugin returns some static data as a
> QString coming from a QStringLiteral. You yourself mentioned that that's
> going boom when the plugin is unloaded. If the plugin instead returned a
> QStringView, then callers wishing to store the result would have toString()
> it, making a deep copy and insulating themselves from the plugin being
> unloaded, while users wishing to just do some mix-and-matching with the
> string could avoid the copy by keeping the returned data in a QStringView.

I agree on the advantage.

>From what I can see, a Qt6 QStringView is a QString with a null d pointer,
except that it triggers deep copy everywhere whre it's assigned to a regular
QString. I'm fine with having the class.

I'm not ok with changing the library code policy.

> People should stop being so afraid of a string deep copy. It's a no-op for
> SSO'ed strings and according to Herb Sutter's benchmark on average doesn't
> matter for all other strings. Here's where safety and performance could
> actually go hand-in-hand for once.

That's not what I am afraid of.

Again, please try writing this method:

class Foo
{
QUrl url;
public:
QStringView host() const;
};


--
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>This API here simply cannot exist because the returned value would be a 
>dangling reference.

I don't understand. Implicit for using the QStringView data() function is 
knowing that once the QByteArray is set, it is never changed. Then the data() 
function is ok, isn't it?

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Thiago Macieira <thiago.macie...@intel.com>
Sent: Monday, October 19, 2015 9:38 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in   Qt  
APIs taking strings

On Monday 19 October 2015 18:38:52 Smith Martin wrote:
> >Again, please try writing this method:
> Doesn't that example just mean there are some classes that can't have a
> QStringView API? A class should have a QStringView API if it can be used
> safely.

Right. I'm simply saying that "if it can be used safely" is very, very
restricted.

Suppose you have in v1:

class Foo
{
QString m_data;
public:
QStringView data() const;
};

Which looks fine and works without dangling references. Then in v2, you need to
do this:

class Foo
{
QByteArray m_data;
public:
QStringView data() const;
};

This API here simply cannot exist because the returned value would be a
dangling reference.

Therefore, QStringView returns must be treated like references: only in
exceptional cases.

--
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Olivier Goffart]

On Monday 19. October 2015 23:26:45 Marc Mutz wrote:
> This particular mistake is easy to prevent statically, though:
> 
>QStringView(QString &&) = delete;

No.

We want this to work:

 if (isValidIdentifier(myVariant.toString()))

with isValidIdentifier taking a QStringView and toString returning a 
temporary.

-- 
Olivier 

Woboq - Qt services and support - http://woboq.com - http://code.woboq.org

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Tuesday 20 October 2015 00:51:58 Olivier Goffart wrote:
> On Monday 19. October 2015 23:26:45 Marc Mutz wrote:
> > This particular mistake is easy to prevent statically, though:
> > 
> >QStringView(QString &&) = delete;
> 
> No.
> 
> We want this to work:
> 
>  if (isValidIdentifier(myVariant.toString()))
> 
> with isValidIdentifier taking a QStringView and toString returning a 
> temporary.

That can be mitigated by overloading isValidIdentifier with a QString &&. That 
is:

bool isValidIdentifier(QString &);
bool isValidIdentifier(const QStringView );

The first one can delegate to the second, as inside it the id parameter is 
lvalue, not rvalue. That is:

bool isValidIdentifier(QString &) 
{ return isValidIdentifier(QStringView(id)); }

Note that this overload is not necessary for existing APIs where a const 
QString& overload is already present, as that will bind to the rvalue just 
fine.

So is there any case where we'd want a QString&& overload instead of a const 
QString &?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Monday 19 October 2015 20:31:53 Bubke Marco wrote:
> On October 19, 2015 21:38:51 Thiago Macieira  
wrote:
> > Right. I'm simply saying that "if it can be used safely" is very, very
> > restricted.
> > 
> > Suppose you have in v1:
> > 
> > class Foo
> > {
> > QString m_data;
> > public:
> > QStringView data() const;
> > };
> > 
> > Which looks fine and works without dangling references. Then in v2, you
> > need to do this:
> > 
> > class Foo
> > {
> > QByteArray m_data;
> > public:
> > QStringView data() const;
> > };
> 
> How can you make a QStringView from a byte array? 

That's exactly the point. It was a tricky question and this is exactly the 
gotcha. You cannot. Therefore, the API must not be permitted, save for 
exceptional cases where you know the internal backing will NEVER EVER change 
(until the next source compatibility change, at the soonest).

So the summary of QStringView is:
 a) never used as return type
 b) used only as a parameter if the function processes the data and never 
 stores it and will never, ever store it (lazy processing)
 c) for existing API, needs to be overloaded with QString

I'm not buying the cost/benefit ratio.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>If that doesn't look like a radical change to you...

But that's way down the line. 

We can add QStringView and overloads that take QStringView without removing the 
overloads that take QString. 

Then we can deprecate the overloads that take QString.


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Giuseppe D'Angelo <giuseppe.dang...@kdab.com>
Sent: Sunday, October 18, 2015 9:27 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs 
taking strings

Il 18/10/2015 20:30, Smith Martin ha scritto:
> After watching the video on string_view, it seems clear we have to offer a 
> QStringView, so I don't see what is "semi-radical" about your proposal.

That the proposal is that every single function currently taking a
QString should instead be changed to take a QStringView instead, unless
a few exceptional cases (for instance the function is used to store the
string somehow, plus other cases under discussion). Similar case for
return values. If that doesn't look like a radical change to you...

Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Software Engineer
KDAB (UK) Ltd., a KDAB Group company | Tel: UK +44-1625-809908
KDAB - The Qt Experts

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote:
> That the proposal is that every single function currently taking a 
> QString should instead be changed to take a QStringView instead, unless 
> a few exceptional cases (for instance the function is used to store the 
> string somehow, plus other cases under discussion). Similar case for 
> return values. If that doesn't look like a radical change to you...

Return values must always be QString, never QStringView. Returning a view is 
like returning a reference and goes against Qt's library code policy.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

After watching the video on string_view, it seems clear we have to offer a 
QStringView, so I don't see what is "semi-radical" about your proposal.

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Tuesday, October 13, 2015 10:46 PM
To: development@qt-project.org
Subject: [Development] RFC: Proposal for a semi-radical change in Qt APIs   
taking strings

Hi,

After looking quite a bit into the current state of string handling in Qt for
my QtWS talk last week, I have become frustrated by the state of string
handling in Qt.

We have such powerful tools for string handling (QStringRef, QStringBuilder),
but all APIs outside QString and its immediate surroundings only deal in
QString. The correct way would be to overload every function taking QString
with QLatin1String and QStringRef versions, and then, for some other rare
cases, const QChar *, int size. Let alone std::basic_string.

I would therefore like to propose to abandon QString for new API (and over
time phase it out of existing API), and only provide (const QChar*, size_t) as
the most general form. I would propose to package the two into a class, called
- you guessed it - QStringView.

=FAQ=

Q: Why not just use QStringRef?

A: QStringRef is tied to QString. E.g. you can't create a QStringRef from a
pair of QChar*, int. It also is kind of stuck in historic mistakes making it
undesireable as a cheap-to-pass parameter type.

Q: What mistakes do you refer to?

A: The fact that it has copy ctor and assignment operator, so it's not a
trivally-copyable type and thus cannot efficiently passed by-value. It may also
be too large for pass-by-value due to the rather useless QString pointer
(should have been QStringData*, if any). Neither can be fixed before Qt 6.

Q: Why size_t?

A: The intent of QStringView (and std::experimental::string_view) is to act as
an interface between modules written with different compilers and different
flags. A std::string will never be compatible between compilers or even just
different flags, but a simple struct {char*, size_t} will always be, by way of
it's C compatibility.

So the goal is not just to accept QString, QStringRef, and (QChar*,int) (and
QVarLengthArray!) as input to QStringView, but also
std::basic_string and std::vector.

Q: What about the plans to make QString UTF-8-backed?

A: QStringView-using code will need to be ported just as QString-using code
will.

Q: What future do you have in mind for QStringRef?

A: None in particular, though I have found a need for an owning QStringRef in
some places. But I expect Qt 6' QString to be able to provide a restricted
view on shared data, such that it would subsume QStringRef completely.

Q: What about QLatin1String?

A: Once QString is backed by UTF-8, latin-1 ceases to be a special charset. We
might want something like QUsAsciiString, but it would just be a UTF-8 string,
so it could be packed into QStringView.

Q: What about QByteArray, QVector?

A: I'm unsure about QByteArrayView. It might not pull its weight compared to
std::(experimental::)string_view, but I also note that we're currently missing
a QByteArrayRef, so a QBAView might make sense while we wait for the std one
to become available to us.

I'm actively opposed to a QArrayView, because I don't think it provides us
with anything std::(experimental::)array_view doesn't already.

Q: What about a rope?

A: A rope is a more complex string that can provide complex views on existing
data as well as store rules for generating stretches of data (as opposed to
the data itself).

A rope is a very complex data structure and would not work as a universal
interface type. It would be cool if Qt had a rope, but that is outside the
scope of my proposal.

Q: What do you mean when you say "abandon QString"?

A: I mean that functions should not take QStrings as arguments, but
QStringViews. Then users can transparently pass QString, QStringRef and any of
a number of other "string" types without overloading the function on each of
them.

I do not mean to abandon QString, the class. Only QString, the interface type.

Q: What API should QStringView have?

A: Since it's mainly an interface type, it should have implicit conversions
from all kinds of "string" types, but explicit conversion _to_ those string
types. It should carry all the API from QString that can be implemented on
just a (QChar*, size_t) (e.g. trimmed(), left(), mid(), section(), split(),
but not append(), replace() (except maybe the (QChar,QChar) overload.
Corresponding QString/Ref API could (eventually) just forward to the
QStringView one.

Thanks, now fire away,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a 

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Giuseppe D'Angelo]

Il 18/10/2015 20:30, Smith Martin ha scritto:

After watching the video on string_view, it seems clear we have to offer a QStringView, 
so I don't see what is "semi-radical" about your proposal.


That the proposal is that every single function currently taking a 
QString should instead be changed to take a QStringView instead, unless 
a few exceptional cases (for instance the function is used to store the 
string somehow, plus other cases under discussion). Similar case for 
return values. If that doesn't look like a radical change to you...


Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Software Engineer
KDAB (UK) Ltd., a KDAB Group company | Tel: UK +44-1625-809908
KDAB - The Qt Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Saturday 17 October 2015 22:51:28 Bubke Marco wrote:
> But before we add new concepts we should agree in which direction we want
> to go. I have the feeling that we build an airplan but some want a jumbo
> jet, some a jet fighter and some, lets describe it so,  they want to fly
> to the moon because earth will maybe burning down and build something
> completely different which doesn't looks like an airplane anymore which we
> built over many years. I don't think we will be succed before we find a
> common context. So it would be good to describe what is our story. 

I believe I did that multiple times now.

What's _your_ story? And so what's _your_ alternative?

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Saturday 17 October 2015 22:42:35 Konstantin Ritt wrote:
> > The whole problem of QString::fromRawData is that the method you called
> > may store the QString and thus keep referencing the string you had, even
> > past the
> > point where your string changed.
> > 
> > In fact, that happens with QStringLiteral too. If you use QStringLiteral
> > in a
> > plugin and the plugin gets unloaded, the application may crash.
> 
> Indeed, it is the same issue. QStringView doesn't solve it either.
> So the best thing we can do, IMO, is to give the user a tool and describe
> how to use it properly to get some extra performance, on his own risk.

Since QStringView doesn't (ever) own, it does solve the problem. If a plugin 
returns a static-storage duration string through QStringView istead of 
QString, a user that wishes to store the QString needs to assign it to a 
QString, which performs a deep copy. Everyone else can work with the static-
storage string data directly (until the plugin gets unloaded, of course, but 
that would be visible in the client code).

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>I have not discovered anything. I merely use the existing tools and found that 
>APIs outside the core Qt string classes largely ignore them, and probably 
>rightfully so, since there's a combinatorical explosion when you combine 
>QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc.

When you use tools as you say you did, to find some knowledge we didn't know 
before, that's what discover means. But now you are changing the meaning again. 
When you started this discussion, I understood you to mean that QString, et al, 
are inefficient in the sense that there is a better way, and the QStringView is 
that better way. But now...

>QStringView is not more efficient than any of those. It is an abstraction over
>them, so that you can use, say, QStringRef in more places.

I understood your last reply to me to mean that we should be using QStringRef 
in more places anyway, whether we add QStringView to the mix or not, and that 
one thing that is clearly missing from Qt documentation is a "How and when to 
use" discussion for all of QString, QLatin1String, QString, QStringRef, QChar, 
const char*, and QByteArray, including which combinations of these classes to 
use and which combinations to avoid.

>Then documenters should ask. There's a class QStringRef. If you/they don't
>understand what the difference is to QString and why it exists, just ask.

Both classes are documented, so we have no reason to think we don't understand 
them. You can argue that the doc team should be using Qt in anger as we go, and 
we should be discovering ourselves how Qt classes should be used to achieve 
optimal efficiency, but we don't have much time for that, given all the 
documentation that has to be written, so we depend on the engineers who design 
and/or use Qt classes to tell us when we need to add this higher level how-to 
knowledge. 

>I try to shake thing up here and
>there, because I'm a C++ fanboy and want Qt to stay relevant.

We all want that. That's why we have begun to improve the docs, at your 
recommendation, with regard to using QVector over QList. This 
QString/QStringView discussion is a similar situation, but with each reply from 
you, I become more confused about what problem you are trying to solve with 
QStringView. You listed most of the Qt string handling classes above. I think 
you are stating or implying that it is a mistake for a user to arbitrarily mix 
them in an application. But this is easy to do. qdoc uses QString everywhere. 
It is after all a string manipulation program.  We have recently improved qdoc 
performance by quite a lot, both by changing its strategy and by improving its 
use of string classes. I am sure there is more to be done, because qdoc mixes 
all the Qt string classes. That's easy to do because in a routine that is 
exclusively QString based, you find you need to call a function that returns a 
QByteArray or that only accepts a parameter that is const char*. 

Is QStringView meant to avoid this by adding APIs to Qt classes that use 
QStringView? Is the goal to include a QStringView API everywhere and then 
always pass around QStringViews instead of combinations of the string handling 
classes?

martin


From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Saturday, October 17, 2015 10:13 PM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt
APIs taking strings

On Saturday 17 October 2015 15:51:35 Smith Martin wrote:
> >Please understand my POV: I am of the firm belief that Qt has no business
> >creating inefficiencies for its users by sloppy implementation. Anywhere.
>
> I think you are overreacting here, way too much. You have discovered a more
> efficient way to handle strings, but that doesn't mean Trolltech created
> inefficiency by a sloppy implementation of QString. QString has been used
> productively by many users for a long time. The fact that a moere
> efficient implementation exists doesn't mean the current implementation is
> inefficient.

I have not discovered anything. I merely use the existing tools and found that
APIs outside the core Qt string classes largely ignore them, and probably
rightfully so, since there's a combinatorical explosion when you combine
QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc.

QStringView is not more efficient than any of those. It is an abstraction over
them, so that you can use, say, QStringRef in more places.

> >And please don't forget that QString fanboys can continue to massacre the
> >heap if they so wish. After all, we've had QStringRef for a long time
> >now, but most people still write str.mid(n).toInt() instead of
> >str.midRef().toInt().

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

Hi Martin 

After watching https://youtu.be/H9gAaNRoon4 I got a much better picture of 
string view. I really recommend to watch it. My understanding is that 
qstringview would be a qt implementation of string view and the conversation 
cost would be zero. 

I find that very promising because it sould be very easy to use every string 
manipulation which is written for utf16 string view or qstringview with no cost 
and quite conveniently. So it really improves interoperability. :-) 

There are other advantages of string view.  For example instead to a pointer of 
characters and size in a function signature it could be better statically 
analyzed. 

I really think we should embrace it more openly. 

Sent from cellphone 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Saturday 17 October 2015 19:42:27 Bubke Marco wrote:
> > Last time I profiled Qt Creator startup and parsing of projects, the two
> > most  expensive calls in QtCore were qHash and the SHA1 calculator. The
> > former I've already fixed. The latter I was hoping that some colleagues
> > would fix by creating an optimised library[1] we'd use but they refused
> > to add SHA1 support to it.
> 
> Thiago, we will be using clang as the new code model and in my experiences
> it is quite slow so we have to hide the latency. This is not only clang but
> same parsing of the results too where we heavily use a QByteArray wrapper.
> If I have time I will try out QValArray but it is not so urgent. I think
> QtCreator is an example that string operations matter because every latency
> as you type is quite unpleasant. 

Roberto's parser didn't use QByteArray either and performance with Creator was 
acceptable. If you replace that with the Clang parser and that's slower, we 
conclude that the fault lies with Clang.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Saturday 17 October 2015 08:06:33 Marc Mutz wrote:
> > The word 'fanboys' disturbs me (I know you don't mean it that way) because
> > there are no 'hard' numbers on how 'bad' the current situation really is.
> > It would really be helpful to have an idea how 'real-world' applications
> > suffer from the current implementation of QString. That would give a very
> > good context to decide how Qt can help to leverage these problems.
> 
> If you want real-world numbers, follow the optimisations Milian did in moc. 
> IIRC, he documented the speedup.They are not (all) string-related, but I
> also don't claim that QStringView will single-handedly make your apps go
> faster. I wrote about QList already. There's more where these topics came
> from, but essentially they all boil down to: "minimise allocations".

On the other hand, note how real world numbers show that QtGui and QtWidgets 
consume a lot more CPU time than the combined time spent in malloc and QString 
operations.

Last time I profiled Qt Creator startup and parsing of projects, the two most 
expensive calls in QtCore were qHash and the SHA1 calculator. The former I've 
already fixed. The latter I was hoping that some colleagues would fix by 
creating an optimised library[1] we'd use but they refused to add SHA1 support 
to it.

[1] https://github.com/01org/tinycrypt
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

Hi Marc

They thread mostly concentrated on performance but what about statical 
analysis? string_view and array_view are specifically designed for that. Maybe 
you mention it already but how is QStringView working for it. 

Sent from cellphone 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

>Please understand my POV: I am of the firm belief that Qt has no business
>creating inefficiencies for its users by sloppy implementation. Anywhere.

I think you are overreacting here, way too much. You have discovered a more 
efficient way to handle strings, but that doesn't mean Trolltech created 
inefficiency by a sloppy implementation of QString. QString has been used 
productively by many users for a long time. The fact that a moere efficient 
implementation exists doesn't mean the current implementation is inefficient. 

>And please don't forget that QString fanboys can continue to massacre the heap
>if they so wish. After all, we've had QStringRef for a long time now, but most
>people still write str.mid(n).toInt() instead of str.midRef().toInt().

But that is because we don't explain in the documentation the most efficient 
ways to use these classes together. And the reason it isn't in the 
documentation is we who write the documentation don't know what you know.

>I find it intolerable that todays software
>takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs
>done in less wallclock time than software 20 years ago.

But the fact is, it usually IS tolerable, because the applications you are 
talking about mostly work in human time.

Your efforts to improve Qt are most appreciated, but you make it sound like we 
should publicly name and shame all the engineers who ever worked on QString but 
who failed to find the holy grail of the most efficient implementation.

I had no idea my heap was in such pain.

martin



From: development-bounces+martin.smith=theqtcompany@qt-project.org 
<development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of 
Marc Mutz <marc.m...@kdab.com>
Sent: Saturday, October 17, 2015 1:18 AM
To: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt
APIs taking strings

Kurt,

The mail you're replying to has nothing to do with QStringView. It was a
tangent about std::string_view.

If you need a real-world example of where QStringView would be handy: Some
3rd-party code returns you a char16_t *path, and you want to perform a
QDir::cd(). Currently, you need to create a QString (which allocates). Had
QDir::cd() taken a QStringView, you wouldn't have that allocation.

Another example I already mentioned:

> E.g. local QStrings could be replaced by QVarLengthArray,
>
> e.g. in code like this:
> QString joinstr = QLatin1String("\"\n");
> joinstr += indent;
> joinstr += indent;
> joinstr += QLatin1Char('"');
>
> QString rc(QLatin1Char('"'));
> rc += result.join(joinstr);
> rc += QLatin1Char('"');
>
> Even when replaced with QStringBuilder:
> QString joinstr = QLatin1String("\"\n") + indent + indent
>
>+ QLatin1Char('"');
>
> QString rc = QLatin1Char('"') + result.join(joinstr) +
> QLatin1Char('"');
>
> But the main benefit would be that we only ever need one overload for a
> QString-taking function, not N, N > 3.

Final with QStringView (and a minor modification to QStringBuilder):

  QVarLengthArray joinstr // doesn't alloc (for most 'indent's)
  = QLatin1String("\"\n") + indent + indent + QLatin1Char('"');

  // in this line, QStringList::join() doesn't know about QVarLengthArray.
  // but it takes a QStringView and therefore transparently accepts QVLA
  QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"');

Please understand my POV: I am of the firm belief that Qt has no business
creating inefficiencies for its users by sloppy implementation. Anywhere. We
cannot know whether one user is calling QString::multiArg() in a tight loop.
It might be dominating her runtime. 99.9% of users probably couldn't care less
about QString::multiArg() performance. But we need to care about the 0.1%,
because that number is much closer to 100% if you integrate it over all users'
critical paths (iow: every Qt user will have *some* part of Qt that she wished
was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at some
point in the past!).

The inefficiencies I (and Sergio and Milian and, I'm happy to see, Friedemann,
and others) care about are not ones you will find in a profiler in those 99.9%
of cases, because they are not used in a critical path. But even for the 99.9%
of users, if we make all of Qt 10% faster on average, their apps will run
close to 10% faster, too. Automatically.

So yes, I care about performance. I find it intolerable that todays software
takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs
done in less wallclock time than software 20 years ago. And people don't use
C++ because it's pretty. You cannot compete with Java on pro

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Saturday 17 October 2015 15:51:35 Smith Martin wrote:
> >Please understand my POV: I am of the firm belief that Qt has no business
> >creating inefficiencies for its users by sloppy implementation. Anywhere.
> 
> I think you are overreacting here, way too much. You have discovered a more
> efficient way to handle strings, but that doesn't mean Trolltech created
> inefficiency by a sloppy implementation of QString. QString has been used
> productively by many users for a long time. The fact that a moere
> efficient implementation exists doesn't mean the current implementation is
> inefficient.

I have not discovered anything. I merely use the existing tools and found that 
APIs outside the core Qt string classes largely ignore them, and probably 
rightfully so, since there's a combinatorical explosion when you combine 
QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc.

QStringView is not more efficient than any of those. It is an abstraction over 
them, so that you can use, say, QStringRef in more places.

> >And please don't forget that QString fanboys can continue to massacre the
> >heap if they so wish. After all, we've had QStringRef for a long time
> >now, but most people still write str.mid(n).toInt() instead of
> >str.midRef().toInt().
> 
> But that is because we don't explain in the documentation the most
> efficient ways to use these classes together. And the reason it isn't in
> the documentation is we who write the documentation don't know what you
> know.

Then documenters should ask. There's a class QStringRef. If you/they don't 
understand what the difference is to QString and why it exists, just ask.

> >I find it intolerable that todays software
> >takes 1000x the memory, 1000x the CPU capacity and doesn't get a given
> >jobs done in less wallclock time than software 20 years ago.
> 
> But the fact is, it usually IS tolerable, because the applications you are
> talking about mostly work in human time.

Again, if it _were_ tolerable, we would all be using Java by now.

> Your efforts to improve Qt are most appreciated, but you make it sound like
> we should publicly name and shame all the engineers who ever worked on
> QString but who failed to find the holy grail of the most efficient
> implementation.

Erm, no, that's not what I'm saying. I'm said we shouldn't use sloppy 
implementation because people here claimed it was ok until proven otherwise. I 
merely pointed out (with many words) that that is not how C++ libraries are 
supposed to work. You can profile an application and you can profile a 
function. 
You cannot profile a library.

What TT created is magnificient. It greatly eased my learning of C++. But it 
needs to evolve to stay relevant, because hardware doesn't stay the same.

In the 2000s C++ was seen as a dying language by many. It is understandable 
that TT picked concepts from Java, the then-favourite language. But now C++ 
developers have gained back their self-esteem, because the industry has 
realised that they absolutely _need_ the efficiency of C++. Java doesn't cut 
it. 
We no longer look up to Java and C#.

When Qt was created, and for a long time afterwards, CoW was a good 
optimisation. When multithreading came along, it ceased to be. That is now 
universally understood, except in Qt where things are sometimes done because 
they have been done that way since Qt 1 or 2. I try to shake thing up here and 
there, because I'm a C++ fanboy and want Qt to stay relevant.

> I had no idea my heap was in such pain.

Play around with heaptrack. And watch Milian's talks at QtWS once they become 
available.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 17, 2015 19:22:55 Thiago Macieira  wrote:

> On Saturday 17 October 2015 08:06:33 Marc Mutz wrote:
>> > The word 'fanboys' disturbs me (I know you don't mean it that way) because
>> > there are no 'hard' numbers on how 'bad' the current situation really is.
>> > It would really be helpful to have an idea how 'real-world' applications
>> > suffer from the current implementation of QString. That would give a very
>> > good context to decide how Qt can help to leverage these problems.
>> 
>> If you want real-world numbers, follow the optimisations Milian did in moc. 
>> IIRC, he documented the speedup.They are not (all) string-related, but I
>> also don't claim that QStringView will single-handedly make your apps go
>> faster. I wrote about QList already. There's more where these topics came
>> from, but essentially they all boil down to: "minimise allocations".
>
> On the other hand, note how real world numbers show that QtGui and QtWidgets 
> consume a lot more CPU time than the combined time spent in malloc and 
> QString 
> operations.
>
> Last time I profiled Qt Creator startup and parsing of projects, the two most 
> expensive calls in QtCore were qHash and the SHA1 calculator. The former I've 
> already fixed. The latter I was hoping that some colleagues would fix by 
> creating an optimised library[1] we'd use but they refused to add SHA1 
> support 
> to it.

Thiago, we will be using clang as the new code model and in my experiences it 
is quite slow so we have to hide the latency. This is not only clang but same 
parsing of the results too where we heavily use a QByteArray wrapper. If I have 
time I will try out QValArray but it is not so urgent. I think QtCreator is an 
example that string operations matter because every latency as you type is 
quite unpleasant. 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Konstantin Ritt]

2015-10-17 8:58 GMT+04:00 Thiago Macieira :

> On Saturday 17 October 2015 07:14:58 Konstantin Ritt wrote:
> > 2015-10-17 6:23 GMT+04:00 Thiago Macieira :
> > > On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote:
> > > > - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5
> } */
> > >
> > > -
> > >
> > > > explicitly means "no owning, deep-copy when necessary"
> > >
> > > And when is it necessary?
> >
> > In a given example, left() derives the original data, even when it isn't
> > shared, whilst leftRef() produces a non-shared QString, just like
> proposed
> > QStringView.
>
> You missed the point.
>
> The whole problem of QString::fromRawData is that the method you called may
> store the QString and thus keep referencing the string you had, even past
> the
> point where your string changed.
>
> In fact, that happens with QStringLiteral too. If you use QStringLiteral
> in a
> plugin and the plugin gets unloaded, the application may crash.
>

Indeed, it is the same issue. QStringView doesn't solve it either.
So the best thing we can do, IMO, is to give the user a tool and describe
how to use it properly to get some extra performance, on his own risk.


> The whole point here was "deep-copy when necessary": when is it necessary?
> If
> we can't solve the two problems above, we are going to make the problem
> worse.


We could probably introduce a QString::ensureSharable() for the advanced
user's goals
and maybe call it in the copy constructor and the assignment operator to
avoid storing a data we do not own. The latter would potentially kill all
the extra performance we got in a first place, though.

In any way, QString::fromRawData() currently has the issue you've described
above, so I have some doubts we'll make the problem even worse.


Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 17, 2015 21:05:29 Marc Mutz  wrote:

> On Saturday 17 October 2015 15:51:35 Smith Martin wrote:
>> >Please understand my POV: I am of the firm belief that Qt has no business
>> >creating inefficiencies for its users by sloppy implementation. Anywhere.
>> 
>> I think you are overreacting here, way too much. You have discovered a more
>> efficient way to handle strings, but that doesn't mean Trolltech created
>> inefficiency by a sloppy implementation of QString. QString has been used
>> productively by many users for a long time. The fact that a moere
>> efficient implementation exists doesn't mean the current implementation is
>> inefficient.
>
> I have not discovered anything. I merely use the existing tools and found 
> that 
> APIs outside the core Qt string classes largely ignore them, and probably 
> rightfully so, since there's a combinatorical explosion when you combine 
> QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc.
>
> QStringView is not more efficient than any of those. It is an abstraction 
> over 
> them, so that you can use, say, QStringRef in more places.
>
>> >And please don't forget that QString fanboys can continue to massacre the
>> >heap if they so wish. After all, we've had QStringRef for a long time
>> >now, but most people still write str.mid(n).toInt() instead of
>> >str.midRef().toInt().
>> 
>> But that is because we don't explain in the documentation the most
>> efficient ways to use these classes together. And the reason it isn't in
>> the documentation is we who write the documentation don't know what you
>> know.
>
> Then documenters should ask. There's a class QStringRef. If you/they don't 
> understand what the difference is to QString and why it exists, just ask.
>
>> >I find it intolerable that todays software
>> >takes 1000x the memory, 1000x the CPU capacity and doesn't get a given
>> >jobs done in less wallclock time than software 20 years ago.
>> 
>> But the fact is, it usually IS tolerable, because the applications you are
>> talking about mostly work in human time.
>
> Again, if it _were_ tolerable, we would all be using Java by now.
>
>> Your efforts to improve Qt are most appreciated, but you make it sound like
>> we should publicly name and shame all the engineers who ever worked on
>> QString but who failed to find the holy grail of the most efficient
>> implementation.
>
> Erm, no, that's not what I'm saying. I'm said we shouldn't use sloppy 
> implementation because people here claimed it was ok until proven otherwise. 
> I 
> merely pointed out (with many words) that that is not how C++ libraries are 
> supposed to work. You can profile an application and you can profile a 
> function. 
> You cannot profile a library.
>
> What TT created is magnificient. It greatly eased my learning of C++. But it 
> needs to evolve to stay relevant, because hardware doesn't stay the same.
>
> In the 2000s C++ was seen as a dying language by many. It is understandable 
> that TT picked concepts from Java, the then-favourite language. But now C++ 
> developers have gained back their self-esteem, because the industry has 
> realised that they absolutely _need_ the efficiency of C++. Java doesn't cut 
> it. 
> We no longer look up to Java and C#.
>
> When Qt was created, and for a long time afterwards, CoW was a good 
> optimisation. When multithreading came along, it ceased to be. That is now 
> universally understood, except in Qt where things are sometimes done because 
> they have been done that way since Qt 1 or 2. I try to shake thing up here 
> and 
> there, because I'm a C++ fanboy and want Qt to stay relevant.

Yes but you cannot change everything in one day. You are referencing to 
something like http://www.gotw.ca/publications/optimizations.htm? I still 
believe sharing mutable data between threads is not a good idea. In most cases 
you start to sterilize your threads. Working on the same cache line is not an 
good idea. I think coping or slicing data between different threads is much 
smarter. In that sense functional programming is much easier to program as you 
write your own loop. So I am not sure that our current CoW story is the answer. 
Not because it is inefficient but because it can lead to inefficient systems 
because it makes sharing so easy.

But before we add new concepts we should agree in which direction we want to 
go. I have the feeling that we build an airplan but some want a jumbo jet, some 
a jet fighter and some, lets describe it so,  they want to fly to the moon 
because earth will maybe burning down and build something completely different 
which doesn't looks like an airplane anymore which we built over many years. I 
don't think we will be succed before we find a common context. So it would be 
good to describe what is our story. 

>> I had no idea my heap was in such pain.
>
> Play around with heaptrack. And watch Milian's talks at QtWS once they become 
> available.
>
> Thanks,
> Marc

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 17, 2015 15:52:08 Smith Martin  wrote:

>>Please understand my POV: I am of the firm belief that Qt has no business
>>creating inefficiencies for its users by sloppy implementation. Anywhere.
>
> I think you are overreacting here, way too much. You have discovered a more 
> efficient way to handle strings, but that doesn't mean Trolltech created 
> inefficiency by a sloppy implementation of QString. QString has been used 
> productively by many users for a long time. The fact that a moere efficient 
> implementation exists doesn't mean the current implementation is inefficient. 

QString makes heap allocation and has atomic pointers. I think it was designed 
for convenience and  not for maximum speed. That's okay because Qt is mostly a 
GUI and not text processing library.

>>I find it intolerable that todays software
>>takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs
>>done in less wallclock time than software 20 years ago.
>
> But the fact is, it usually IS tolerable, because the applications you are 
> talking about mostly work in human time.
>
> Your efforts to improve Qt are most appreciated, but you make it sound like 
> we should publicly name and shame all the engineers who ever worked on 
> QString but who failed to find the holy grail of the most efficient 
> implementation.

Like I wrote they were very understandable design decisions. Anyway I think a 
professional developer knows that he should has distance to his code so that he 
get criticism about it not personal. Understanding criticism and argumentation 
as a process to write code is, I think, the key to get a good developer. So why 
they should be feel ashamed if something better comes up. This is called 
progress und it happens all the time.  I think I would be feel ashamed if I 
would fight other code because of personal reason. Sometimes I feel that way 
but I hopefully not stick to it. ;-) 

After reading some proposals about string view it has to my understanding this 
properties:
1. Makes string usage between different implementation more efficient 
2. Improve static analysis - makes ownership clear
3. Gives the different string implementation a common ground

You can use easily use a Qt function without creating a QString which can be a 
big advantage if performance matters.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Friday 16 October 2015 19:03:18 Marc Mutz wrote:
> The problem is that I feel we can't wait for string_view because we already 
> drown in QString equivalents. Plus, we'd get the usual replies if anyone
> suggested to use std::string_view as such a fundamental Qt type. Naturally,
> QStringView would trivially convert to and from std::u16string_view with no
> runtime overhead (unless compiler writers ruin the concept of string_view
> by implementing it as (T*, size_t) in one compiler and (T*, T*) in
> another)

The conversion from one to the other is one instruction.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Friday 16 October 2015 19:31:50 Thiago Macieira wrote:
> The conversion from one to the other is one instruction.

Which is one instruction too much for tail-call optimisation, e.g.

But the point was about std::string_view binary compatibility. As basically a 
C struct with no ownership semantics, it would be a perfect vehicle to pass 
strings around different programming languages and compilers, but only if the 
ABI (ie. the data members) were specified.

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Kurt Pattyn]

Marc,

It is clear that your main concern is performance (needless conversions) and 
convenience (being able to work efficiently with 3rd party libraries).
Regarding performance, I think it would be good if we could come up with some 
numbers. How 'bad' is the current implementation compared to an 'ideal' 
situation? And then define some acceptable target keeping into account 
convenience. Otherwise I am afraid that discussions can keep going forever over 
1 instruction more or less.
Regarding convenience, and that is what Qt is all about, I think you made a 
tremendous proposal that can really bring Qt a big step forward. The idea is 
very sound and clear. Of course, there are technical challenges, but that is 
what we are supposed to solve.

I also agree that we should aim for simplicity. QString and QStringView is 
simple to understand, iff we can get rid of all other string related classes. 
The ideal case would be just to have one QString class, although I doubt that 
is achievable.

I think it would be good to come up with some real-world use-cases that show 
how a Qt user and a Qt application could benefit from your proposal. 
Personally, I never experienced a performance problem with QString; but then 
the applications I write are not very string oriented. I am a QString 
aficionado :-)

After all, what I really like about Qt is convenience, simplicity and 
portability. I think this should be the main focus of Qt, and that also implies 
smooth collaboration with the outside world.

Cheers,

Kurt

> On 16 Oct 2015, at 21:47, Marc Mutz  wrote:
> 
>> On Friday 16 October 2015 19:31:50 Thiago Macieira wrote:
>> The conversion from one to the other is one instruction.
> 
> Which is one instruction too much for tail-call optimisation, e.g.
> 
> But the point was about std::string_view binary compatibility. As basically a 
> C struct with no ownership semantics, it would be a perfect vehicle to pass 
> strings around different programming languages and compilers, but only if the 
> ABI (ie. the data members) were specified.
> 
> -- 
> Marc Mutz  | Senior Software Engineer
> KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
> Tel: +49-30-521325470
> KDAB - The Qt Experts
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Smith Martin]

Then after we add QStringView, we will have QString for containing a string and 
QStringView for passing its contents out of Qt?

martin


From: m...@kdab.com <m...@kdab.com> on behalf of Marc Mutz <marc.m...@kdab.com>
Sent: Friday, October 16, 2015 10:07 AM
To: Smith Martin
Cc: development@qt-project.org
Subject: Re: [Development] RFC: Proposal for a semi-radical change in  Qt   
APIs taking strings

On Friday 16 October 2015 07:59:40 Smith Martin wrote:
> Can you refocus the discussion for everyone? For me at least?
>
> QString is my favorite class of all time. I use it every day in every
> program; it always works, and I never make a mistake.

Then you can continue to be blissfully ignorant of reality as you are :) If
you are a QString fanboy and don't care about 3rd-party code and/or efficiency,
then you can just continue as before.

> 1. What is the problem with QString that QStringView will solve?

TL;DR: Inefficient conversion from/to QString, leading to inefficiencies within
pure Qt-code and in particular when interfacing with 3rd-party code, BiC
issues,

> 2. Why can't QString be fixed?

Because it is a container, not an interface type.

An interface type needs to be BC across most of compilers and languages (incl.
C and - potentially Java).

A container needs to handle (as in own) memory. An interface type (quite
obviously, if it's to work with C, too) cannot.

> 3. What is a string view?

A pimped { QChar *begin, QChar *end; }

Thanks,
Marc

--
Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Ziller Eike]

> On Oct 16, 2015, at 9:10 AM, Poenitz Andre  
> wrote:
> 
> 
> Marc Mutz wrote:
>>> I think too we should embrace the standard library more and don't replicate
>>> their features.
>> 
>> So you think that QStringView is too experimental and _at the same time_
>> replicating the standard. Sounds paradoxal to me.
> 
> It's not paradoxical at all. It would be a new implementation with the usual 
> need for maturing even if it implements a well-known standard concepts.
> 
> And there is indeed room for actual experiments, e.g. when it comes 
> structure layout. Pointer+int offset is not the only possible choice. 
> 
> You cannot do that in Qt proper, due to BC promises.

And there would also be some room in Qt Creator to experiment with and gather 
experience on how all this fits with corresponding API changes, which I also do 
not see how that could be done in Qt without lots of #ifdefs.

> Compile-time opt-in
> is a way to exempt it from BC, but then your main reason to have it in Qt 
> and not in Creator, namely 'more widespread use', is void, as barely anyone 
> will opt in. You'll get more public exposure (with less risk!) if it's used by
> default in Creator code for a while before the then-matured implementation
> goes to the library.
> 
> Andre'
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

-- 
Eike Ziller, Senior Software Engineer - The Qt Company GmbH
 
The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 
144331 B

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 16, 2015 08:53:33 Marc Mutz  wrote:

> On Friday 16 October 2015 01:32:26 Bubke Marco wrote:
>> On October 16, 2015 00:20:22 Marc Mutz  wrote:
>> > Guys, this thread is for QStringView. Could we keep it on-topic, please?
>> > There are more than enough bits floating around to create your own
>> > threads (with a tip of the hat to Kai).
>> 
>> Good argument but actually I think before we introduce something new to our
>> string handling we should test it out. Why not add it Qt creator like
>> André proposed and see how it works.
>
> I have already answered why I think this is a bad idea.

Yes, I read it. That why I proposed the experimental playground. It is not so 
much about QStringView but about the following changes. What is so bad about to 
play around features before we see all the implications. My experiences whisper 
to me that if I like an idea too much I should be careful because I love to 
ignore unpleasant side effects. 

>> I think we don't want to end with something like our model view system or
>> QtControls. Lets test it before we make changes.
>> 
>> I think too we should embrace the standard library more and don't replicate
>> their features.
>
> So you think that QStringView is too experimental and _at the same time_ 
> replicating the standard. Sounds paradoxal to me.

Is it already in the standard? The standard has an experimental stage too. And 
I  like this concept very much. 

>> A better process to add features would be helpful too.  First they should
>> be experimental so we can change them easily. Second we should be better
>> at removing features. If we do not remove things we will getting slower
>> and slower to add new interesting stuff. It is hard to find the balance
>> but if you are too conservative you will getting slowly less used. We
>> tried to be very innovative with Qml and we learned much about it.
>> 
>> So the question is how can we maximize the usefulness of Qt with our man
>> power. Is replicating the standard library really helping?
>
> My stance on replicating std functionality and the NIH syndrome should by now 
> be above suspicion, even for the casual reader.

It was not about you or me. It was about us an organization. How we can work 
better together. How can we make a difference, how can we help other 
developers. 

> I have already argued why I think QStringView is needed, but QArrayView is 
> not.

Sorry if you got the impression it was about you. I should be more careful with 
my arguments. I try to disconnect arguments from the self. I thinking the 
argument should stand for himself in his context and not so much by the person 
who speaks it. 

To make it clear,  I find QStringView promising but still we should explore the 
concept with all their implications more. Actually I think we sould add more of 
this kind of changes in an experimental stage like #include 
. So we are forced to evaluate for every release 
how it works out and if it looks well we move it to the normal include path. So 
experimental would be an area in every repository so that everybody could try 
it out. 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Friday 16 October 2015 01:32:26 Bubke Marco wrote:
> On October 16, 2015 00:20:22 Marc Mutz  wrote:
> > Guys, this thread is for QStringView. Could we keep it on-topic, please?
> > There are more than enough bits floating around to create your own
> > threads (with a tip of the hat to Kai).
> 
> Good argument but actually I think before we introduce something new to our
> string handling we should test it out. Why not add it Qt creator like
> André proposed and see how it works.

I have already answered why I think this is a bad idea.

> I think we don't want to end with something like our model view system or
> QtControls. Lets test it before we make changes.
> 
> I think too we should embrace the standard library more and don't replicate
> their features.

So you think that QStringView is too experimental and _at the same time_ 
replicating the standard. Sounds paradoxal to me.

> A better process to add features would be helpful too.  First they should
> be experimental so we can change them easily. Second we should be better
> at removing features. If we do not remove things we will getting slower
> and slower to add new interesting stuff. It is hard to find the balance
> but if you are too conservative you will getting slowly less used. We
> tried to be very innovative with Qml and we learned much about it.
> 
> So the question is how can we maximize the usefulness of Qt with our man
> power. Is replicating the standard library really helping?

My stance on replicating std functionality and the NIH syndrome should by now 
be above suspicion, even for the casual reader.

I have already argued why I think QStringView is needed, but QArrayView is 
not.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Poenitz Andre]

Marc Mutz wrote:
> > I think too we should embrace the standard library more and don't replicate
> > their features.
> 
> So you think that QStringView is too experimental and _at the same time_
> replicating the standard. Sounds paradoxal to me.

It's not paradoxical at all. It would be a new implementation with the usual 
need for maturing even if it implements a well-known standard concepts.

And there is indeed room for actual experiments, e.g. when it comes 
structure layout. Pointer+int offset is not the only possible choice. 

You cannot do that in Qt proper, due to BC promises. Compile-time opt-in
is a way to exempt it from BC, but then your main reason to have it in Qt 
and not in Creator, namely 'more widespread use', is void, as barely anyone 
will opt in. You'll get more public exposure (with less risk!) if it's used by
default in Creator code for a while before the then-matured implementation
goes to the library.

Andre'
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Konstantin Ritt]

>
> > You're misrepresenting the argument. QString doesn't support other
> encodings
> > because UTF-16 is the best for the task at hand and we have too much
> legacy to
> > support. Because of that, QCollator only supports UTF-16.
>
> I buy the legacy argument but I don't buy that utf 16 is the best
> encoding. I think because of legacy we cannot change QString but I think we
> should still support UTF 8 better.
>

So what? This doesn't change the fact UTF-16 has advantages over UTF-8 and
thus QString's backing storage is UTF-16 - and won't be changed to anything
else.


Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Иван Комиссаров]

Back to the topic.
Marc, what usecases of QStringView are not covered with QString { ushort*
data; int size; QStringData *refcount; }
Not sure i understand that "interface" terminology. Would it be possible to
pass QStringView to other (non-qt) APIs?
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Friday 16 October 2015 09:53:56 Smith Martin wrote:
> Then after we add QStringView, we will have QString for containing a string
> and QStringView for passing its contents out of Qt?

Yes, except for "out of Qt". We already have - within Qt - lots of equivalent 
ways to provide a QChar-based string - and QString is only one (QStringRef and 
QVarLengthArray are others, as is QChar[]).

As I said before, this is very similar to Q(Persistent)ModelIndex - QMI is the 
interface type, and it's transparent to the compiler and thus efficient to work 
with. QPMI is the storage type, intransparent to the compiler and very slow to 
work with.

HTH,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Friday 16 October 2015 13:28:57 Иван Комиссаров wrote:
> Back to the topic.
> Marc, what usecases of QStringView are not covered with QString { ushort*
> data; int size; QStringData *refcount; }
> Not sure i understand that "interface" terminology. Would it be possible to
> pass QStringView to other (non-qt) APIs?

See my mail to Martin today.

While not the first reason to create a QStringView, yes, a (ushort*, ushort*) 
can allow zero-copy string transfer between at least C, C++ and Java, possibly 
also JavaScript. So it may have a direct benefit for QML, though I don't know 
that code at all.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Koehne Kai]

I guess it's not by incidence that there's also a std::string_view coming, see 
e.g.

https://www.youtube.com/watch?v=H9gAaNRoon4

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3762.html

So, is this really the same, except non-templated and for QString (with 
conversion from/to Qt types)? Or do you have other differences in mind?

Regards

Kai

> -Original Message-
> From: development-bounces+kai.koehne=theqtcompany@qt-project.org
> [mailto:development-bounces+kai.koehne=theqtcompany@qt-project.org]
> On Behalf Of Marc Mutz
> Sent: Tuesday, October 13, 2015 10:47 PM
> To: development@qt-project.org
> Subject: [Development] RFC: Proposal for a semi-radical change in Qt APIs
> taking strings
> 
> Hi,
> 
> After looking quite a bit into the current state of string handling in Qt for 
> my
> QtWS talk last week, I have become frustrated by the state of string handling 
> in
> Qt.
> 
> We have such powerful tools for string handling (QStringRef, QStringBuilder),
> but all APIs outside QString and its immediate surroundings only deal in 
> QString.
> The correct way would be to overload every function taking QString with
> QLatin1String and QStringRef versions, and then, for some other rare cases,
> const QChar *, int size. Let alone std::basic_string.
> 
> I would therefore like to propose to abandon QString for new API (and over 
> time
> phase it out of existing API), and only provide (const QChar*, size_t) as the 
> most
> general form. I would propose to package the two into a class, called
> - you guessed it - QStringView.
> 
> =FAQ=
> 
> Q: Why not just use QStringRef?
> 
> A: QStringRef is tied to QString. E.g. you can't create a QStringRef from a 
> pair of
> QChar*, int. It also is kind of stuck in historic mistakes making it 
> undesireable as
> a cheap-to-pass parameter type.
> 
> Q: What mistakes do you refer to?
> 
> A: The fact that it has copy ctor and assignment operator, so it's not a 
> trivally-
> copyable type and thus cannot efficiently passed by-value. It may also be too
> large for pass-by-value due to the rather useless QString pointer (should have
> been QStringData*, if any). Neither can be fixed before Qt 6.
> 
> Q: Why size_t?
> 
> A: The intent of QStringView (and std::experimental::string_view) is to act 
> as an
> interface between modules written with different compilers and different 
> flags.
> A std::string will never be compatible between compilers or even just 
> different
> flags, but a simple struct {char*, size_t} will always be, by way of it's C
> compatibility.
> 
> So the goal is not just to accept QString, QStringRef, and (QChar*,int) (and
> QVarLengthArray!) as input to QStringView, but also
> std::basic_string and std::vector.
> 
> Q: What about the plans to make QString UTF-8-backed?
> 
> A: QStringView-using code will need to be ported just as QString-using code 
> will.
> 
> Q: What future do you have in mind for QStringRef?
> 
> A: None in particular, though I have found a need for an owning QStringRef in
> some places. But I expect Qt 6' QString to be able to provide a restricted 
> view on
> shared data, such that it would subsume QStringRef completely.
> 
> Q: What about QLatin1String?
> 
> A: Once QString is backed by UTF-8, latin-1 ceases to be a special charset. We
> might want something like QUsAsciiString, but it would just be a UTF-8 
> string, so
> it could be packed into QStringView.
> 
> Q: What about QByteArray, QVector?
> 
> A: I'm unsure about QByteArrayView. It might not pull its weight compared to
> std::(experimental::)string_view, but I also note that we're currently 
> missing a
> QByteArrayRef, so a QBAView might make sense while we wait for the std one
> to become available to us.
> 
> I'm actively opposed to a QArrayView, because I don't think it provides us 
> with
> anything std::(experimental::)array_view doesn't already.
> 
> Q: What about a rope?
> 
> A: A rope is a more complex string that can provide complex views on existing
> data as well as store rules for generating stretches of data (as opposed to 
> the
> data itself).
> 
> A rope is a very complex data structure and would not work as a universal
> interface type. It would be cool if Qt had a rope, but that is outside the 
> scope of
> my proposal.
> 
> Q: What do you mean when you say "abandon QString"?
> 
> A: I mean that functions should not take QStrings as arguments, but
> QStringViews. Then users can transparently pass QString, QStringRef and any of
> a number of other "string" types without overloading the function on each of
> them.
> 
> I do not mean to abandon QString, the cl

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Friday 16 October 2015 17:03:41 Koehne Kai wrote:
> I guess it's not by incidence that there's also a std::string_view coming,
> see e.g.
> 
> https://www.youtube.com/watch?v=H9gAaNRoon4
> 
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3762.html
> 
> So, is this really the same, except non-templated and for QString (with
> conversion from/to Qt types)? Or do you have other differences in mind?

Nothing qualitatively different. Just std::basic_string_view with a 
subset of QString API, yes.

The problem is that I feel we can't wait for string_view because we already 
drown in QString equivalents. Plus, we'd get the usual replies if anyone 
suggested to use std::string_view as such a fundamental Qt type. Naturally, 
QStringView would trivially convert to and from std::u16string_view with no 
runtime overhead (unless compiler writers ruin the concept of string_view by 
implementing it as (T*, size_t) in one compiler and (T*, T*) in another)

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Saturday 17 October 2015 01:18:06 Marc Mutz wrote:
> If you need a real-world example of where QStringView would be handy: Some 
> 3rd-party code returns you a char16_t *path, and you want to perform a 
> QDir::cd(). Currently, you need to create a QString (which allocates). Had 
> QDir::cd() taken a QStringView, you wouldn't have that allocation.

Bad example. You can use QString::fromRawData. While that today allocates the 
d pointer, for Qt 6 it wouldn't (null d pointer) and there would be no code 
change for the user.

There is, however, the danger of fromRawData: you need to be sure the called 
function won't try to store the QString. With a QStringView or the unsharable 
QString that Brano proposed in the other email, it would be clear that a deep 
copy needs to happen.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Branislav Katreniak]

Thiago described Qt6 QString as { QArrayData *d; ushort* b; qsize size }.
That is pretty close to QStringView, just extra *d makes it slightly bigger.

This is how I understand this class:
QString will have (typical) case when b points into d. QString own one
reference in d in this case.
QString will have special case when b points to text section and d does not
own anything.
QString will have special case for SSO.

Can QString have special case for not owned, non persistent b pointer
(non-owning / view mode)?

Non-owning QString can be created with special static QString method. It
stays non-owning only while being passed through const &. Code that cares
about keeping QString in view mode, must stick to const QString & all the
time. Copy assignment on non-owning QString results in owning QString - it
triggers deep copy into newly created shared data.

It solves the queued connection problem that existed with QStringView
class. I suppose that queued connection internally performs QString copy.

I have no idea about performance implications for code paths that do not
care about this use case. But having just one string class would be very
convenient.

Kind regards
 Brano
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Friday 16 October 2015 22:08:59 Branislav Katreniak wrote:
> Thiago described Qt6 QString as { QArrayData *d; ushort* b; qsize size }.
> That is pretty close to QStringView, just extra *d makes it slightly bigger.
> 
> This is how I understand this class:
> QString will have (typical) case when b points into d. QString own one
> reference in d in this case.
> QString will have special case when b points to text section and d does not
> own anything.
> QString will have special case for SSO.
> 
> Can QString have special case for not owned, non persistent b pointer
> (non-owning / view mode)?

How is that different from your second case? If it doesn't own, it doesn't need 
a d pointer.

> Non-owning QString can be created with special static QString method. It
> stays non-owning only while being passed through const &. Code that cares
> about keeping QString in view mode, must stick to const QString & all the
> time. Copy assignment on non-owning QString results in owning QString - it
> triggers deep copy into newly created shared data.

Ok, I understand the difference, but why do you want this behaviour? BTW, this 
is the unsharable strings that we're getting rid of.

This would make sense to view a string of ephemeral data. But if the data is 
ephemeral, why can't we use the refcounter? In other words, if someone owns 
this data, why isn't that QString?

> It solves the queued connection problem that existed with QStringView
> class. I suppose that queued connection internally performs QString copy.

True, but I think that having QString share refcount of substrings solves the 
problem that QStringView was proposed for in the first place.

> I have no idea about performance implications for code paths that do not
> care about this use case. But having just one string class would be very
> convenient.

That is the problem. That's the reason we're trying to get rid of unsharable 
containers: performance impact on the code paths that don't need it, which are 
the majority of cases.

There are two possibilities for the unsharable flag:

1) it's inside the d pointer, which means the d pointer must be non-null. That 
means even the non-owning QString needs to allocate memory for the d pointer 
block.

2) it's outside the d pointer, which means that
 a) it doesn't need to allocate memory
 b) we need to extend QString to be:
  { QArrayData *d; ushort *b; qsize size; int flags; }
  (24 bytes if qsize is 32-bit on 64-bit platforms; 32 if it is 64-bit; always 
   16 bytes on 32-bit platforms)

Either way, the existence of the flag means that every QString copy-assignment 
needs to check this flag, instead of simply incrementing the refcount if d is 
not null and being done with it. That means code bloat for the deep copy 
showing up everywhere.

I really don't want the unsharable flag.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

Kurt,

The mail you're replying to has nothing to do with QStringView. It was a 
tangent about std::string_view.

If you need a real-world example of where QStringView would be handy: Some 
3rd-party code returns you a char16_t *path, and you want to perform a 
QDir::cd(). Currently, you need to create a QString (which allocates). Had 
QDir::cd() taken a QStringView, you wouldn't have that allocation.

Another example I already mentioned:

> E.g. local QStrings could be replaced by QVarLengthArray,
> 
> e.g. in code like this:
> QString joinstr = QLatin1String("\"\n");
> joinstr += indent;
> joinstr += indent;
> joinstr += QLatin1Char('"');
> 
> QString rc(QLatin1Char('"'));
> rc += result.join(joinstr);
> rc += QLatin1Char('"');
> 
> Even when replaced with QStringBuilder:
> QString joinstr = QLatin1String("\"\n") + indent + indent
> 
>+ QLatin1Char('"');
> 
> QString rc = QLatin1Char('"') + result.join(joinstr) +
> QLatin1Char('"');
> 
> But the main benefit would be that we only ever need one overload for a
> QString-taking function, not N, N > 3.

Final with QStringView (and a minor modification to QStringBuilder):

  QVarLengthArray joinstr // doesn't alloc (for most 'indent's)
  = QLatin1String("\"\n") + indent + indent + QLatin1Char('"');

  // in this line, QStringList::join() doesn't know about QVarLengthArray.
  // but it takes a QStringView and therefore transparently accepts QVLA
  QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"');

Please understand my POV: I am of the firm belief that Qt has no business 
creating inefficiencies for its users by sloppy implementation. Anywhere. We 
cannot know whether one user is calling QString::multiArg() in a tight loop. 
It might be dominating her runtime. 99.9% of users probably couldn't care less 
about QString::multiArg() performance. But we need to care about the 0.1%, 
because that number is much closer to 100% if you integrate it over all users' 
critical paths (iow: every Qt user will have *some* part of Qt that she wished 
was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at some 
point in the past!).

The inefficiencies I (and Sergio and Milian and, I'm happy to see, Friedemann, 
and others) care about are not ones you will find in a profiler in those 99.9% 
of cases, because they are not used in a critical path. But even for the 99.9% 
of users, if we make all of Qt 10% faster on average, their apps will run 
close to 10% faster, too. Automatically.

So yes, I care about performance. I find it intolerable that todays software 
takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs 
done in less wallclock time than software 20 years ago. And people don't use 
C++ because it's pretty. You cannot compete with Java on programmer efficiency. 
Its standard library is orders of magnitude larger than C++'s. People choose 
C++ (and accept the associated loss in programmer efficiency), because they 
require the bare metal efficiency of C++. And if Qt gets in the way, they will 
look for alternatives and find them in places that make Qt look like paradise.

That said, QStringView is as much about

- slim interface type vs. thick, fat, storage type
- interop with 3rd-party and native code

than it is about efficiency.

Bottomline: I don't need a fancy anylysis to tell me that less allocations = 
faster programs = more happy Qt users.

And please don't forget that QString fanboys can continue to massacre the heap 
if they so wish. After all, we've had QStringRef for a long time now, but most 
people still write str.mid(n).toInt() instead of str.midRef().toInt(). 
QStringView won't change that: A function taking QStringView accepts a 
QString, too (either by implicit conversion, or where it makes sence, by 
QString overload).

Thanks,
Marc

On Friday 16 October 2015 21:42:23 Kurt Pattyn wrote:
> Marc,
> 
> It is clear that your main concern is performance (needless conversions)
> and convenience (being able to work efficiently with 3rd party libraries).
> Regarding performance, I think it would be good if we could come up with
> some numbers. How 'bad' is the current implementation compared to an
> 'ideal' situation? And then define some acceptable target keeping into
> account convenience. Otherwise I am afraid that discussions can keep going
> forever over 1 instruction more or less. Regarding convenience, and that
> is what Qt is all about, I think you made a tremendous proposal that can
> really bring Qt a big step forward. The idea is very sound and clear. Of
> course, there are technical challenges, but that is what we are supposed
> to solve.
> 
> I also agree that we should aim for simplicity. QString and QStringView is
> simple to understand, iff we can get rid of all other string related
> classes. The ideal case would be just to have one QString class, although
> I doubt that is achievable.
> 
> I think it 

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Konstantin Ritt]

If QString in Qt6 could be { QArrayData *d; ushort *b; int s }, then it
supersedes QStringView in all aspects:
- QString(u"hello world", 5) /* { d=.., b=.., s=5 } */ - still has a
superpower of COW, etc
- QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */ -
explicitly means "no owning, deep-copy when necessary"

then we could get rid of QStringRef (or typedef it to QString) and do
something like:

private:
QString(QArrayData *dd, ushort *unicode, int size) { if (dd) { dd->ref(); }
d = dd; b = unicode; s = size; } /* not that trivial, of course, just to
describe the idea */
public:
QString QString::left(int n) const { return QString(d, b, n); }
QString QString::leftRef(int n) const { return QString(nullptr, b, n); }


> That said, QStringView is as much about
>
> - slim interface type vs. thick, fat, storage type
> - interop with 3rd-party and native code

So as QString now... i.e.

DWORD dwBufferSize = 0;
if (::GetUserProfileDirectory(token, NULL, ) == 0 &&
dwBufferSize != 0) {
QVarLengthArray userDirectory(dwBufferSize);
if (::GetUserProfileDirectory(token, userDirectory.data(),
) == 0) {
QString dirName = QString::fromRawData(userDirectory.constData(),
dwBufferSize); // still unsafe, but it is clearly documented for years and
nothing has really changed for its users anyways
return QDir().exists(
dirName.rightRef(dirName.lastIndexOf(QLatin1Char('\\';
}
}


Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Kurt Pattyn]

> On 17 Oct 2015, at 01:18, Marc Mutz  wrote:
> 
> Kurt,
> 
> The mail you're replying to has nothing to do with QStringView. It was a 
> tangent about std::string_view.
Yes, but this thread is diverging on a lot of different things than the 
original, genuine intent you have. So, please forgive me on replying on a 
'side-branch' :-)
> 
> If you need a real-world example of where QStringView would be handy: Some 
> 3rd-party code returns you a char16_t *path, and you want to perform a 
> QDir::cd(). Currently, you need to create a QString (which allocates). Had 
> QDir::cd() taken a QStringView, you wouldn't have that allocation.
> 
> Another example I already mentioned:
> 
>> E.g. local QStrings could be replaced by QVarLengthArray,
>> 
>> e.g. in code like this:
>>QString joinstr = QLatin1String("\"\n");
>>joinstr += indent;
>>joinstr += indent;
>>joinstr += QLatin1Char('"');
>> 
>>QString rc(QLatin1Char('"'));
>>rc += result.join(joinstr);
>>rc += QLatin1Char('"');
>> 
>> Even when replaced with QStringBuilder:
>>QString joinstr = QLatin1String("\"\n") + indent + indent
>> 
>>   + QLatin1Char('"');
>> 
>>QString rc = QLatin1Char('"') + result.join(joinstr) +
>>QLatin1Char('"');
>> 
>> But the main benefit would be that we only ever need one overload for a
>> QString-taking function, not N, N > 3.
> 
> Final with QStringView (and a minor modification to QStringBuilder):
> 
>  QVarLengthArray joinstr // doesn't alloc (for most 'indent's)
>  = QLatin1String("\"\n") + indent + indent + QLatin1Char('"');
> 
>  // in this line, QStringList::join() doesn't know about QVarLengthArray.
>  // but it takes a QStringView and therefore transparently accepts QVLA
>  QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"');
> 
> Please understand my POV: I am of the firm belief that Qt has no business 
> creating inefficiencies for its users by sloppy implementation. Anywhere. We 
> cannot know whether one user is calling QString::multiArg() in a tight loop. 
> It might be dominating her runtime. 99.9% of users probably couldn't care 
> less 
> about QString::multiArg() performance. But we need to care about the 0.1%, 
> because that number is much closer to 100% if you integrate it over all 
> users' 
> critical paths (iow: every Qt user will have *some* part of Qt that she 
> wished 
> was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at 
> some 
> point in the past!).
Don't misunderstand me, I completely understand your POV. You are right: Qt IS 
about convenience and performance. Only, if I see the 'heated' discussions, it 
would be convenient if some baseline could be defined.
> 
> The inefficiencies I (and Sergio and Milian and, I'm happy to see, 
> Friedemann, 
> and others) care about are not ones you will find in a profiler in those 
> 99.9% 
> of cases, because they are not used in a critical path. But even for the 
> 99.9% 
> of users, if we make all of Qt 10% faster on average, their apps will run 
> close to 10% faster, too. Automatically.
> 
> So yes, I care about performance. I find it intolerable that todays software 
> takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs 
> done in less wallclock time than software 20 years ago.
These are 'numbers'. Is it like that? Really 1000x the memory, 1000x the CPU 
capacity?
I tend to say 'yes you are right', but then we still need some concrete 
measurements.
> And people don't use 
> C++ because it's pretty. You cannot compete with Java on programmer 
> efficiency. 
> Its standard library is orders of magnitude larger than C++'s. People choose 
> C++ (and accept the associated loss in programmer efficiency), because they 
> require the bare metal efficiency of C++. And if Qt gets in the way, they 
> will 
> look for alternatives and find them in places that make Qt look like paradise.
But Qt IS like paradise :-)
> 
> That said, QStringView is as much about
> 
> - slim interface type vs. thick, fat, storage type
> - interop with 3rd-party and native code
> 
> than it is about efficiency.
> 
> Bottomline: I don't need a fancy anylysis to tell me that less allocations = 
> faster programs = more happy Qt users.
But to what extent? If I run my application on a workstation, the effect is 
negligible, as my application has more to do than handling strings. If I have 
an embedded application, every piece of performance gain could benefit, but my 
experience tells me (for MY use cases) that the graphics stack is more a 
bottleneck than string handling.
> 
> And please don't forget that QString fanboys can continue to massacre the 
> heap 
> if they so wish. After all, we've had QStringRef for a long time now, but 
> most 
> people still write str.mid(n).toInt() instead of str.midRef().toInt(). 
> QStringView won't change that: A function taking QStringView accepts a 
> QString, too (either by implicit conversion, or where 

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote:
> - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */ -
> explicitly means "no owning, deep-copy when necessary"

And when is it necessary?
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Saturday 17 October 2015 01:52:27 Kurt Pattyn wrote:
> > Bottomline: I don't need a fancy anylysis to tell me that less
> > allocations =  faster programs = more happy Qt users.
> 
> But to what extent? If I run my application on a workstation, the effect is
> negligible, as my application has more to do than handling strings. If I
> have an embedded application, every piece of performance gain could
> benefit, but my experience tells me (for MY use cases) that the graphics
> stack is more a bottleneck than string handling.

I object to the notion that a desktop would not benefit from more efficient 
string handling. Hardware doesn't get faster the way it used to. If the speed 
of C++ wasn't needed, your workstation would only run Java apps (and Sun 
tried, but failed).

> >
> > And please don't forget that QString fanboys can continue to massacre the
> > heap  if they so wish. After all, we've had QStringRef for a long time
> > now, but most people still write str.mid(n).toInt() instead of
> > str.midRef().toInt(). QStringView won't change that: A function taking
> > QStringView accepts a QString, too (either by implicit conversion, or
> > where it makes sence, by QString overload).
> 
> The word 'fanboys' disturbs me (I know you don't mean it that way) because
> there are no 'hard' numbers on how 'bad' the current situation really is.
> It would really be helpful to have an idea how 'real-world' applications
> suffer from the current implementation of QString. That would give a very
> good context to decide how Qt can help to leverage these problems.

If you want real-world numbers, follow the optimisations Milian did in moc. 
IIRC, he documented the speedup.They are not (all) string-related, but I also 
don't claim that QStringView will single-handedly make your apps go faster. I 
wrote about QList already. There's more where these topics came from, but 
essentially they all boil down to: "minimise allocations".

> That said, I like the idea of QStringView.

Thanks,
Marc

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Saturday 17 October 2015 07:14:58 Konstantin Ritt wrote:
> 2015-10-17 6:23 GMT+04:00 Thiago Macieira :
> > On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote:
> > > - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */
> > 
> > -
> > 
> > > explicitly means "no owning, deep-copy when necessary"
> > 
> > And when is it necessary?
> 
> In a given example, left() derives the original data, even when it isn't
> shared, whilst leftRef() produces a non-shared QString, just like proposed
> QStringView.

You missed the point.

The whole problem of QString::fromRawData is that the method you called may 
store the QString and thus keep referencing the string you had, even past the 
point where your string changed.

In fact, that happens with QStringLiteral too. If you use QStringLiteral in a 
plugin and the plugin gets unloaded, the application may crash.

The whole point here was "deep-copy when necessary": when is it necessary? If 
we can't solve the two problems above, we are going to make the problem worse.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Konstantin Ritt]

2015-10-15 11:00 GMT+03:00 Bubke Marco :

> On October 15, 2015 08:45:30 Knoll Lars 
> wrote:
>
> > On 14/10/15 23:51, "Bubke Marco"  wrote:
> >
> >>On October 14, 2015 23:10:26 Thiago Macieira 
> >>wrote:
> >>> Qt does not have to provide a comparator that operates on something
> >>>other than
> >>> its native string type.
> >>
> >>Isn't Qt a framework to help developers? Sorry your argumentation is
> >>sounds not very empirical.
> >
> > Of course our aim should be to help developers. But there will always be
> > some use cases which we will not cover. The question is whether this is
> > one of them or not.
>
> Most file and network content is in utf 8, databases too. It has simply a
> size and performance advantage for most cases. You have not so many cases
> where you have pure Chinese signs in an text. Mostly it is an mixture. In
> Linux,  which is very important in embedded, utf 8 dominates ťhe APIs. Ask
> your self if we don't want support that. We could start simply and expand
> slowly. If the standard library would support utf 8 collations on all
> platforms very well we could skip it but today you have to do your own
> solutions again and again.
>

For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel
free to compare their complexity against UTF-32).
And why "pure Chinese signs" again? Did you ever look into the Unicode's
Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all
spoken languages, without any performance hits (in compare to UTF-8), and
all we have to pay is an extra byte per every Base Latin character (in
compare to UTF-8, again).

[1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt

Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Ziller Eike]

> On Oct 14, 2015, at 5:14 PM, Matthew Woehlke  wrote:
> 
> On 2015-10-14 10:30, André Somers wrote:
>> Op 14-10-2015 om 15:59 schreef Matthew Woehlke:
>>> STL should change. In Qt and Python, you can use negative indices to
>>> refer to a distance (length) relative to the end (length) of the string.
>>> In STL you can't do that, which is a significant limitation by
>>> comparison. Please don't drop this useful functionality!
>> 
>> I'm not so sure anymore. Do you really think that for instance passing 
>> in a negative _from_ in QString::indexOf to search from the back of the 
>> string is intuitive API? I don't.
> 
> Huh? Of course it is.
> 
>  s.indexOf('c', 5); // find 'c', forward, starting at offset 5
>  s.indexOf('c', -5); // find 'c', forward, starting at offset N-5

So from where does 
s.indexOf(‘c’, i-2)
search?
This is similar to integer overflow, and I think utilizing that in an API leads 
to less readable and potentially unexpectedly behaving code.

Anyhow this seems to be only vaguely related to the things that are discussed 
in this thread.

Br, Eike

> A negative offset -K is exactly the same as N + 1 - K (N = length of
> string). It just saves having to write that out yourself. It *doesn't*
> change the behavior of the function. (I think you are confusing
> tail-relative offset with reverse operation, which is totally different
> and orthogonal.)
> 
> Even STL supports this, partially, for -1; string::npos is generally
> equivalent to -1 in Qt.
> 
> Bah. Okay, apparently Qt actually *doesn't* support tail-relative, but
> just treats n<0 like string::npos. That could be improved for Qt 6
> though, but only if n is signed.
> 
> -- 
> Matthew
> 
> ___
> Development mailing list
> Development@qt-project.org
> http://lists.qt-project.org/mailman/listinfo/development

-- 
Eike Ziller, Senior Software Engineer - The Qt Company GmbH
 
The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja
Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 
144331 B

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Thursday 15 October 2015 06:05:34 Thiago Macieira wrote:
> On Thursday 15 October 2015 02:22:50 Marc Mutz wrote:
> > On Thursday 15 October 2015 00:27:14 Thiago Macieira wrote:
> > > Way too much code would break if we did that because we allow people
> > > access
> > > to  the data pointer in QString and to iterate directly
> > > (std::{,w,u16}string don't allow that, which makes parsing them
> > > actually a lot more cumbersome).
> >
> > 
> >
> > Just chiming in to say: It does:
> > http://en.cppreference.com/w/cpp/string/basic_string/data
> 
> Ah, right. The mutable pointer is the one missing...

char *data = &*str.begin();

There might not be explicit API for it, but it's not forbidden to use that 
idiom (if !str.empty(), of course).

-- 
Marc Mutz  | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Knoll Lars]

On 14/10/15 23:51, "Bubke Marco"  wrote:

>On October 14, 2015 23:10:26 Thiago Macieira 
>wrote:
>
>> On Wednesday 14 October 2015 20:52:12 Bubke Marco wrote:
>>> On October 14, 2015 22:13:11 Thiago Macieira
>>>
>> wrote:
>>> And I don't want an utf 8 baked
>>> QString. For my use cases implicit sharing is overkill.  Move semantics
>>> would be enough. I want localAwareCompare(const char *s1, const char
>>>*s2).
>>
>> Do it on your own. You just said that ICU has the function you want, so
>>use 
>> it.
>
>So Qt is always shipping with ICU?

No, we wanted to do this at some point, but it turns out that it’s not
possible to rely on it on all platforms.
> 
>
>> Qt does not have to provide a comparator that operates on something
>>other than 
>> its native string type.
>
>Isn't Qt a framework to help developers? Sorry your argumentation is
>sounds not very empirical.

Of course our aim should be to help developers. But there will always be
some use cases which we will not cover. The question is whether this is
one of them or not.
> 
>
>>
>>> Maybe windows and mac os will bring support to the standard library so
>>>we
>>> don't need it but in the mean time it would be very helpful.
>>> 
>>> A utf 8 based QTextDocument would be maybe nice too.
>>
>> What for? It needs to keep a lot of extra structures, so the cost of
>> conversion and extra memory is minimal. And besides, QTextDocument
>>really 
>> needs a seekable string, not UTF-8.
>
>Is UTF 16 seekable? You still have surrogates and you can merge merge
>code points.

For the most parts. When it comes to positioning cursors inside the text,
you’ll always need to take care of complex text layouting, diacritics and
(in the case of utf16) surrogates. Still, a lot of the seeking is probably
easier with utf16 than with utf8.
> 
>
>Lets describe an example. I send the QTextDocument content to an library
>which expect utf8 content and gives me back positions. This gets
>interesting if you use non
>ASCII signs. Actually the new clang code model works that way.

We also have the opposite case, where we need to send utf16 to a 3rd party
or system library, and get back positions. Unfortunately, not all APIs
take the same encoding.
> 
>>
>> Even if we provide UTF-8 support classes, those will not propagate to
>>the GUI. 
>> Forget it.
>
>What about compressing UTF 16 like python is doing it for UTF 32. If you
>are only using ascii you set a flag and you can remove all that useless
>zeros. It would be have implications for data() but maybe we should not
>provide access to the internal representation. If you use UTF 32 as a
>base you don't need anymore surrogates.

That’s back to a mixed representation in QString. I personally think that
combines the worst of both worlds.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 15, 2015 08:45:30 Knoll Lars  wrote:

> On 14/10/15 23:51, "Bubke Marco"  wrote:
>
>>On October 14, 2015 23:10:26 Thiago Macieira 
>>wrote:
>>
>>> On Wednesday 14 October 2015 20:52:12 Bubke Marco wrote:
 On October 14, 2015 22:13:11 Thiago Macieira

>>> wrote:
 And I don't want an utf 8 baked
 QString. For my use cases implicit sharing is overkill.  Move semantics
 would be enough. I want localAwareCompare(const char *s1, const char
*s2).
>>>
>>> Do it on your own. You just said that ICU has the function you want, so
>>>use 
>>> it.
>>
>>So Qt is always shipping with ICU?
>
> No, we wanted to do this at some point, but it turns out that it’s not
> possible to rely on it on all platforms.

Is there an other utf 8 backend on Windows? I heard that Microsoft is using it 
now on some plattforms too. 

>> 
>>
>>> Qt does not have to provide a comparator that operates on something
>>>other than 
>>> its native string type.
>>
>>Isn't Qt a framework to help developers? Sorry your argumentation is
>>sounds not very empirical.
>
> Of course our aim should be to help developers. But there will always be
> some use cases which we will not cover. The question is whether this is
> one of them or not.

Most file and network content is in utf 8, databases too. It has simply a size 
and performance advantage for most cases. You have not so many cases where you 
have pure Chinese signs in an text. Mostly it is an mixture. In Linux,  which 
is very important in embedded, utf 8 dominates ťhe APIs. Ask your self if we 
don't want support that. We could start simply and expand slowly. If the 
standard library would support utf 8 collations on all platforms very well we 
could skip it but today you have to do your own solutions again and again. 

>> 
>>
>>>
 Maybe windows and mac os will bring support to the standard library so
we
 don't need it but in the mean time it would be very helpful.
 
 A utf 8 based QTextDocument would be maybe nice too.
>>>
>>> What for? It needs to keep a lot of extra structures, so the cost of
>>> conversion and extra memory is minimal. And besides, QTextDocument
>>>really 
>>> needs a seekable string, not UTF-8.
>>
>>Is UTF 16 seekable? You still have surrogates and you can merge merge
>>code points.
>
> For the most parts. When it comes to positioning cursors inside the text,
> you’ll always need to take care of complex text layouting, diacritics and
> (in the case of utf16) surrogates. Still, a lot of the seeking is probably
> easier with utf16 than with utf8.

Actually I would like to see measurements. Does anybody have seen anything 
about it. What is Chrome and Mozilla doing?

>> 
>>
>>Lets describe an example. I send the QTextDocument content to an library
>>which expect utf8 content and gives me back positions. This gets
>>interesting if you use non
>>ASCII signs. Actually the new clang code model works that way.
>
> We also have the opposite case, where we need to send utf16 to a 3rd party
> or system library, and get back positions. Unfortunately, not all APIs
> take the same encoding.

Yes, but in my experience C APIs are using UTF 8 or all UTF standards. There 
some older which use wchar but that is different anyway. It may be different 
for Java etc. but I don't think that people who use Qt use that much java APIs. 
 ;-) 

>> 
>>>
>>> Even if we provide UTF-8 support classes, those will not propagate to
>>>the GUI. 
>>> Forget it.
>>
>>What about compressing UTF 16 like python is doing it for UTF 32. If you
>>are only using ascii you set a flag and you can remove all that useless
>>zeros. It would be have implications for data() but maybe we should not
>>provide access to the internal representation. If you use UTF 32 as a
>>base you don't need anymore surrogates.
>
> That’s back to a mixed representation in QString. I personally think that
> combines the worst of both worlds.
>
I am not so sure eighter but I  like to have an open discussion about it with 
examples of other use cases. How other programs like mozilla,  chrome handling 
it, what does other languages support. Especially Internet centric solutions 
handle it would be interesting. 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Matthew Woehlke]

On 2015-10-15 02:38, Ziller Eike wrote:
> So from where does 's.indexOf(‘c’, i-2)' search?
>
> This is similar to integer overflow, and I think utilizing that in an
> API leads to less readable and potentially unexpectedly behaving
> code.

It depends on the value of i, of course. And you're not going to get any
better results with an unsigned offset.

> Anyhow this seems to be only vaguely related to the things that are discussed 
> in this thread.

There was a question whether to use signed or unsigned as the size type,
that somewhere turned into Qt's use of signed for string sizes/offsets
being inconsistent with STL/CSL. I suppose it has wandered off a bit on
a tangent, though.

-- 
Matthew

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 15, 2015 14:53:29 Konstantin Ritt  wrote:

> 2015-10-15 11:00 GMT+03:00 Bubke Marco 
> >:
> On October 15, 2015 08:45:30 Knoll Lars 
> > wrote:
>
>> On 14/10/15 23:51, "Bubke Marco" 
>> > wrote:
>>
>>>On October 14, 2015 23:10:26 Thiago Macieira 
>>>>
>>>wrote:
 Qt does not have to provide a comparator that operates on something
other than
 its native string type.
>>>
>>>Isn't Qt a framework to help developers? Sorry your argumentation is
>>>sounds not very empirical.
>>
>> Of course our aim should be to help developers. But there will always be
>> some use cases which we will not cover. The question is whether this is
>> one of them or not.
>
> Most file and network content is in utf 8, databases too. It has simply a 
> size and performance advantage for most cases. You have not so many cases 
> where you have pure Chinese signs in an text. Mostly it is an mixture. In 
> Linux,  which is very important in embedded, utf 8 dominates ?he APIs. Ask 
> your self if we don't want support that. We could start simply and expand 
> slowly. If the standard library would support utf 8 collations on all 
> platforms very well we could skip it but today you have to do your own 
> solutions again and again.
>
> For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel 
> free to compare their complexity against UTF-32).

Do you have mesured it? Please no  theoretical discussions. If you have larger 
texts you have annotations in ascii like XML.

> And why "pure Chinese signs" again? Did you ever look into the Unicode's 
> Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all 
> spoken languages, without any performance hits (in compare to UTF-8), and all 
> we have to pay is an extra byte per every Base Latin character (in compare to 
> UTF-8, again).

Okay,  again. Most text today is embedded in meta information and this 
information are very often ascii. Why do you think utf8 is used so widely?
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

On Thursday 15 October 2015 14:52:46 Konstantin Ritt wrote:
> For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel
> free to compare their complexity against UTF-32).
> And why "pure Chinese signs" again? Did you ever look into the Unicode's
> Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all
> spoken languages, without any performance hits (in compare to UTF-8), and
> all we have to pay is an extra byte per every Base Latin character (in
> compare to UTF-8, again).
> 
> [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt

What everyone is referring to here is this:



  
  Google
  
html { background: #fff; margin: 0 1em; }
body { font: .8125em/1.5 arial, sans-serif; text-align: center; }
h1 { font-size: 1.5em; font-weight: normal; margin: 1em 0 0; }
p#footer { color: #767676; font-size: .77em; }
p#footer a { background: url(//www.google.cn/intl/zh-
CN_cn/images/cn_icp.gif) top right no-repeat; padding: 5px 20px 5px 0; }
ul { margin: 2em; padding: 0; }
li { display: inline; padding: 0 2em; }
div { -moz-border-radius: 20px; -webkit-border-radius: 20px; border: 1px 
solid #ccc; border-radius: 20px; margin: 2em auto 1em; max-width: 650px; min-
width: 544px; }
div:hover, div:hover * { cursor: pointer; }
div:hover { border-color: #999; }
div p { margin: .5em 0 1.5em; }
img { border: 0; }
  
  
http://www.google.com.hk/webhp?hl=zh-CNsourceid=cnhp;>
  

http://www.google.com.hk/webhp?hl=zh-
CNsourceid=cnhp">google.com.hk
请收藏我们的网址
  
  
http://translate.google.cn/?sourceid=cnhp;>翻译
  
  2011 - http://www.miibeian.gov.cn/;>ICP证合字
B2-20070004号
  
var gcn=gcn||
{};gcn.IS_IMAGES=(/images\.google\.cn/.exec(window.location)||
window.location.hash=='#images'||
window.location.hash=='images');gcn.HOMEPAGE_DEST='http://www.google.com.hk/webhp?hl=zh-
CN&sourceid=cnhp';gcn.IMAGES_DEST='http://images.google.com.hk/imghp?'+'hl=zh-
CN&sourceid=cnhp';gcn.DEST_URL=gcn.IS_IMAGES?gcn.IMAGES_DEST:gcn.HOMEPAGE_DEST;gcn.READABLE_HOMEPAGE_URL='google.com.hk';gcn.READABLE_IMAGES_URL='images.google.com.hk';gcn.redirectIfLocationHasQueryParams=function()
{if(window.location.search&&/google\.cn/.exec(window.location)&&!/webhp/.exec(window.location))
{window.location=String(window.location).replace('google.cn','google.com.hk')}}
();gcn.replaceHrefsWithImagesUrl=function(){if(gcn.IS_IMAGES){var 
a=document.getElementsByTagName('a');for(var i=0,len=a.length;i | Senior Software Engineer
KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company
Tel: +49-30-521325470
KDAB - The Qt Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Matthew Woehlke]

On 2015-10-14 18:27, Thiago Macieira wrote:
> On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote:
>> Is UTF 16 seekable? You still have surrogates and you can merge merge code
>> points.
> 
> Seekable enough. It's much easier to deal with than UTF-8. A surrogate pair, 
> as its name says, appears *only* in pairs, so you always know if you're on 
> the 
> first or on the second. Moreover, all living languages are encoded in the 
> Basic 
> Multilingual Plane, so no surrogate pairs are required for any of them. 
> Handling of surrogate pairs can be moved to non-critical codepaths.

Oh? 

-- 
Matthew

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 15, 2015 15:56:35 Matthew Woehlke  wrote:

> On 2015-10-15 02:38, Ziller Eike wrote:
>> So from where does 's.indexOf(‘c’, i-2)' search?
>>
>> This is similar to integer overflow, and I think utilizing that in an
>> API leads to less readable and potentially unexpectedly behaving
>> code.
>
> It depends on the value of i, of course. And you're not going to get any
> better results with an unsigned offset.

You could get an assert or warning because you are out of range. 

>> Anyhow this seems to be only vaguely related to the things that are 
>> discussed in this thread.
>
> There was a question whether to use signed or unsigned as the size type,
> that somewhere turned into Qt's use of signed for string sizes/offsets
> being inconsistent with STL/CSL. I suppose it has wandered off a bit on
> a tangent, though.
>
 I would prefer we would be more in line with the standard library. All the 
casts to silence warnings are not very beautiful. 
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[André Somers]

Op 15-10-2015 om 14:52 schreef Konstantin Ritt:
>
>
> For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 
> (feel free to compare their complexity against UTF-32).
> And why "pure Chinese signs" again? Did you ever look into the 
> Unicode's Scripts.txt [1], for example? It clearly shows UTF-16 covers 
> [almost] all spoken languages, without any performance hits (in 
> compare to UTF-8), and all we have to pay is an extra byte per every 
> Base Latin character (in compare to UTF-8, again).
>
> [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt
>
"All we have to pay"? Isn't that quite a significant cost, if your every 
other byte in your data is going to be null? Doesn't that impact cache 
lines? Doesn't that impact how many characters you can stuff into a 
string with SSO (as planned for Qt6?) and thus when you need to start 
allocating? I would certainly think that there is an impact on things 
like XML parsing speed. But true enough, that is without measuring that.

I also think it should be the default for saving QStrings to files. Many 
years ago, I was writing with an application on a very constrained 
mobile device dealing with maps. Converting the map format to use UTF8 
instead of UTF16 for its contained strings (which were many) caused a 
very significant reduction of file size (impacting the map area the 
users could take with them on the road) and loading speed (impacting UX 
very positively).

André


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Bubke Marco]

On October 16, 2015 00:20:22 Marc Mutz  wrote:

> Guys, this thread is for QStringView. Could we keep it on-topic, please? 
> There 
> are more than enough bits floating around to create your own threads (with a 
> tip of the hat to Kai).
>

Good argument but actually I think before we introduce something new to our 
string handling we should test it out. Why not add it Qt creator like André 
proposed and see how it works. 

I think we don't want to end with something like our model view system or 
QtControls. Lets test it before we make changes. 

I think too we should embrace the standard library more and don't replicate 
their features. 

A better process to add features would be helpful too.  First they should be 
experimental so we can change them easily. Second we should be better at 
removing features. If we do not remove things we will getting slower and slower 
to add new interesting stuff. It is hard to find the balance but if you are too 
conservative you will getting slowly less used. We tried to be very innovative 
with Qml and we learned much about it. 

So the question is how can we maximize the usefulness of Qt with our man power. 
Is replicating the standard library really helping? 

>
> On Thursday 15 October 2015 23:02:09 Bubke Marco wrote:
>> On October 15, 2015 00:27:45 Thiago Macieira  
> wrote:
>> > On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote:
>> >> On October 14, 2015 23:10:26 Thiago Macieira 
>> > 
>> > wrote:
>> >> > Do it on your own. You just said that ICU has the function you want,
>> >> > so use
>> >> > it.
>> >> 
>> >> So Qt is always shipping with ICU?
>> > 
>> > It can be disabled on Windows. On OS X there's no point since it's part
>> > of the system. On Linux, if you disable it, you're going to have some
>> > other features reduced, so don't disable it.
>> > 
>> >> > Qt does not have to provide a comparator that operates on something
>> >> > other than its native string type.
>> >> 
>> >> Isn't Qt a framework to help developers? Sorry your argumentation is
>> >> sounds not very empirical.
>> > 
>> > Yes, it is. But Qt's goal is not to support every single use-case and
>> > corner- case out there. Qt should make 90% easy and 9% possible. That
>> > means there's a 1% of the realm of possibilities that Qt does not
>> > address. If your use-case calls into this group, use the fact that Qt is
>> > native code and just call other libraries.
>> 
>> Actually I think Qt is not main developing library people use. It is there
>> to make the boring stuff easy, to hide the different interfaces between
>> different platforms. That is why many people use Qt,  they want to have a
>> GUI but don't want to invest to much time in it. The interesting stuff 
>> which is differentiating you from others is mostly home grown in
>> connection with much more specialized libraries. And this libraries are
>> much more important to the users. So we should support them, their
>> interfaces and not force our interfaces on them. How many users use the
>> standard library too,  especially the new features, why don't we support
>> them not much better. Why do we have to reinvent the wheel again and
>> again. I know binary compatibility is important for you but is it really
>> that important outside of the special linux distribution cocoon. Is it
>> important under Windows,  is it important under Mac, is it important under
>> embedded Linux? I think the advantages are smaller than the drawbacks.
>> 
>> > That's one of the two main advantages of native code. There's no sandbox
>> > to escape from.
>> > 
>> > Qt already supports doing locale-aware comparison. We even have a class
>> > for it, so it can be done efficiently: QCollator and it supports our
>> > native string type (QString).
>> 
>> Do you like to live on a native island?
>> 
>> > Providing extra support for a character encoding that is not what QString
>> > uses falls in that 1%. Just use ICU.
>> 
>> You arguments sounds very tautological. Because it is unimportant we don't
>> have it a string class for it. It is unimportant because QString is not
>> supporting it.
>> 
>> I know you love plationian argumentation but it would be much more
>> effective if you would try to get in the context of other and understand
>> their arguments in their context.  Showing in your own context that their
>> arguments "makes no sense" is not very useful.
>> 
>> >> Lets describe an example. I send the QTextDocument content to an library
>> >> which expect utf8 content and gives me back positions. This gets
>> >> interesting if you use non ASCII signs. Actually the new clang code
>> >> model works that way.
>> > 
>> > That example shows how UTF-16 is better. See above on seekability of
>> > UTF-16 vs UTF-8.
>> > 
>> > The solution for this is to fix the library to accept UTF-16. When we
>> > were doing Qt 5.0, we needed PCRE to support UTF-16. Their developers
>> > were very 

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Marc Mutz]

Guys, this thread is for QStringView. Could we keep it on-topic, please? There 
are more than enough bits floating around to create your own threads (with a 
tip of the hat to Kai).

Thanks,
Marc

On Thursday 15 October 2015 23:02:09 Bubke Marco wrote:
> On October 15, 2015 00:27:45 Thiago Macieira  
wrote:
> > On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote:
> >> On October 14, 2015 23:10:26 Thiago Macieira 
> > 
> > wrote:
> >> > Do it on your own. You just said that ICU has the function you want,
> >> > so use
> >> > it.
> >> 
> >> So Qt is always shipping with ICU?
> > 
> > It can be disabled on Windows. On OS X there's no point since it's part
> > of the system. On Linux, if you disable it, you're going to have some
> > other features reduced, so don't disable it.
> > 
> >> > Qt does not have to provide a comparator that operates on something
> >> > other than its native string type.
> >> 
> >> Isn't Qt a framework to help developers? Sorry your argumentation is
> >> sounds not very empirical.
> > 
> > Yes, it is. But Qt's goal is not to support every single use-case and
> > corner- case out there. Qt should make 90% easy and 9% possible. That
> > means there's a 1% of the realm of possibilities that Qt does not
> > address. If your use-case calls into this group, use the fact that Qt is
> > native code and just call other libraries.
> 
> Actually I think Qt is not main developing library people use. It is there
> to make the boring stuff easy, to hide the different interfaces between
> different platforms. That is why many people use Qt,  they want to have a
> GUI but don't want to invest to much time in it. The interesting stuff 
> which is differentiating you from others is mostly home grown in
> connection with much more specialized libraries. And this libraries are
> much more important to the users. So we should support them, their
> interfaces and not force our interfaces on them. How many users use the
> standard library too,  especially the new features, why don't we support
> them not much better. Why do we have to reinvent the wheel again and
> again. I know binary compatibility is important for you but is it really
> that important outside of the special linux distribution cocoon. Is it
> important under Windows,  is it important under Mac, is it important under
> embedded Linux? I think the advantages are smaller than the drawbacks.
> 
> > That's one of the two main advantages of native code. There's no sandbox
> > to escape from.
> > 
> > Qt already supports doing locale-aware comparison. We even have a class
> > for it, so it can be done efficiently: QCollator and it supports our
> > native string type (QString).
> 
> Do you like to live on a native island?
> 
> > Providing extra support for a character encoding that is not what QString
> > uses falls in that 1%. Just use ICU.
> 
> You arguments sounds very tautological. Because it is unimportant we don't
> have it a string class for it. It is unimportant because QString is not
> supporting it.
> 
> I know you love plationian argumentation but it would be much more
> effective if you would try to get in the context of other and understand
> their arguments in their context.  Showing in your own context that their
> arguments "makes no sense" is not very useful.
> 
> >> Lets describe an example. I send the QTextDocument content to an library
> >> which expect utf8 content and gives me back positions. This gets
> >> interesting if you use non ASCII signs. Actually the new clang code
> >> model works that way.
> > 
> > That example shows how UTF-16 is better. See above on seekability of
> > UTF-16 vs UTF-8.
> > 
> > The solution for this is to fix the library to accept UTF-16. When we
> > were doing Qt 5.0, we needed PCRE to support UTF-16. Their developers
> > were very welcoming and wrote the version that supports UTF-16, so Qt
> > does not need to reallocate.
> 
> You have ever heard of Pippi Longstocking: "Widdiwiddiwitt, we make the
> world like we wish it should be. "  or how it is translated to english.
> You really think that you can force other larger projects to use utf16
> instead of utf8 if it has disadvantages for them.
> 
> And the PCRE is now supporting both at  runtime? Especially for large text
> it would be very helpful if you don't need to convert them to QString
> before you use  regular expressions on it.
> 
> >> > Even if we provide UTF-8 support classes, those will not propagate to
> >> > the GUI. Forget it.
> >> 
> >> What about compressing UTF 16 like python is doing it for UTF 32. If you
> >> are only using ascii you set a flag and you can remove all that useless
> >> zeros. It would be have implications for data() but maybe we should not
> >> provide access to the internal representation. If you use UTF 32 as a
> >> base you don't need anymore surrogates.
> > 
> > That's what Lars called a "hybrid solution" and vetoed. I second that.
> > 
> > Way too 

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Thursday 15 October 2015 21:02:09 Bubke Marco wrote:
> Actually I think Qt is not main developing library people use. It is there
> to make the boring stuff easy, to hide the different interfaces between
> different platforms. That is why many people use Qt,  they want to have a
> GUI but don't want to invest to much time in it. The interesting stuff 
> which is differentiating you from others is mostly home grown in connection
> with much more specialized libraries. And this libraries are much more
> important to the users. So we should support them, their interfaces and not
> force our interfaces on them. 

That's a slippery slope. If we have to support every library's interface, 
we'll have a horrible mish-mash of an API that tries to support everything and 
doesn't support anything really well.

I'd rather we supported the "One Qt Way" really well. Supporting other ways is 
possible, as long as we don't cause too much maintenance trouble.

> How many users use the standard library too, 
> especially the new features, why don't we support them not much better. Why
> do we have to reinvent the wheel again and again. 

I won't bother repeating the arguments of why we can't use some of the 
standard library features. And see the discussion on std::chrono as a 
replacement for QTime(Span).

> I know binary
> compatibility is important for you but is it really that important outside
> of the special linux distribution cocoon. 

Yes. Lots of users don't recompile the world when Qt issues a new release, but 
still try to upgrade.

And besides, if we have to maintain binary compatibility for 99% of the API 
anyway because of Linux distributions, then we might as well go the extra mile 
and keep it for everyone.

> Is it important under Windows, 
> is it important under Mac, is it important under embedded Linux? I think
> the advantages are smaller than the drawbacks.

I disagree.

Not to mention that "embedded Linux" these days is not different from "regular 
Linux". Even devices with 32 MB of RAM have package managers and install 
software from a central repository.

> > That's one of the two main advantages of native code. There's no sandbox
> > to
> > escape from.
> > 
> > Qt already supports doing locale-aware comparison. We even have a class
> > for
> > it, so it can be done efficiently: QCollator and it supports our native
> > string type (QString).
> 
> Do you like to live on a native island?

Yes. I love writing native code. That's one of the biggest powers of Qt 
compared to any other cross-platform solution out there. So, yeah, native 
island is good.

Did you mean to ask if I like constraining myself to only Qt-style APIs? The 
answer is also yes. I hate the Standard Library API because it's confusing (to 
me, obviously not so much to the people who wrote it), limited in convenience 
forcing me to write more code than focusing on getting stuff done. I often feel 
that the Standard Library tries to achieve 100% support of some tiny feature, 
overengineering it and not focusing on making developers' lives easier. The 
example is std::chrono.

> > Providing extra support for a character encoding that is not what QString
> > uses falls in that 1%. Just use ICU.
> 
> You arguments sounds very tautological. Because it is unimportant we don't
> have it a string class for it. It is unimportant because QString is not
> supporting it.

You're misrepresenting the argument. QString doesn't support other encodings 
because UTF-16 is the best for the task at hand and we have too much legacy to 
support. Because of that, QCollator only supports UTF-16.

> I know you love plationian argumentation but it would be much more effective
> if you would try to get in the context of other and understand their
> arguments in their context.  Showing in your own context that their
> arguments "makes no sense" is not very useful.

You're dangerously close to attacking me instead of attacking my arguments.

> > That example shows how UTF-16 is better. See above on seekability of
> > UTF-16 vs UTF-8.
> > 
> > The solution for this is to fix the library to accept UTF-16. When we were
> > doing Qt 5.0, we needed PCRE to support UTF-16. Their developers were very
> > welcoming and wrote the version that supports UTF-16, so Qt does not need
> > to reallocate.
> 
> You have ever heard of Pippi Longstocking: "Widdiwiddiwitt, we make the
> world like we wish it should be. "  or how it is translated to english. You
> really think that you can force other larger projects to use utf16 instead
> of utf8 if it has disadvantages for them.

And why should we support UTF-8 instead of UTF-16 if it has disadvantages for 
us?

We'll have to agree to disagree with those projects. We've chosen UTF-16 and 
we're aligned with a lot of other important API.

> And the PCRE is now supporting both at  runtime? Especially for large text
> it would be very helpful if you don't need to convert them to QString
> before you use  regular expressions on it.

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Konstantin Ritt]

2015-10-15 17:52 GMT+03:00 André Somers :

> Op 15-10-2015 om 14:52 schreef Konstantin Ritt:
> >
> >
> > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16
> > (feel free to compare their complexity against UTF-32).
> > And why "pure Chinese signs" again? Did you ever look into the
> > Unicode's Scripts.txt [1], for example? It clearly shows UTF-16 covers
> > [almost] all spoken languages, without any performance hits (in
> > compare to UTF-8), and all we have to pay is an extra byte per every
> > Base Latin character (in compare to UTF-8, again).
> >
> > [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt
> >
> "All we have to pay"? Isn't that quite a significant cost, if your every
> other byte in your data is going to be null?


Only for US-ASCII / Latin-1.

Regards,
Konstantin
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[André Somers]

Op 14-10-2015 om 15:59 schreef Matthew Woehlke:
>
>>> Yes, signed please. We can discuss whether it should be 64bit for Qt 6.
>> The current std API uses size_t. Do you (= both of you) expect that ever to
>> change? If it doesn't, Qt will forever be the odd one out, until we finally
>> drop QVector etc for std::vector etc and then porting will be a horror 
>> because
>> of MSVC's annoying warnings.
> STL should change. In Qt and Python, you can use negative indices to
> refer to a distance (length) relative to the end (length) of the string.
> In STL you can't do that, which is a significant limitation by
> comparison. Please don't drop this useful functionality!
I'm not so sure anymore. Do you really think that for instance passing 
in a negative _from_ in QString::indexOf to search from the back of the 
string is intuitive API? I don't. I would rather have a specific 
indexOfBackwards or something like that. Or one could just use the 
iterator API with a standard algorithm I guess.

André

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Matthew Woehlke]

On 2015-10-14 10:30, André Somers wrote:
> Op 14-10-2015 om 15:59 schreef Matthew Woehlke:
>> STL should change. In Qt and Python, you can use negative indices to
>> refer to a distance (length) relative to the end (length) of the string.
>> In STL you can't do that, which is a significant limitation by
>> comparison. Please don't drop this useful functionality!
>
> I'm not so sure anymore. Do you really think that for instance passing 
> in a negative _from_ in QString::indexOf to search from the back of the 
> string is intuitive API? I don't.

Huh? Of course it is.

  s.indexOf('c', 5); // find 'c', forward, starting at offset 5
  s.indexOf('c', -5); // find 'c', forward, starting at offset N-5

A negative offset -K is exactly the same as N + 1 - K (N = length of
string). It just saves having to write that out yourself. It *doesn't*
change the behavior of the function. (I think you are confusing
tail-relative offset with reverse operation, which is totally different
and orthogonal.)

Even STL supports this, partially, for -1; string::npos is generally
equivalent to -1 in Qt.

Bah. Okay, apparently Qt actually *doesn't* support tail-relative, but
just treats n<0 like string::npos. That could be improved for Qt 6
though, but only if n is signed.

-- 
Matthew

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Wednesday 14 October 2015 09:59:28 Matthew Woehlke wrote:
> On 2015-10-14 06:16, Marc Mutz wrote:
> > First, afaiu from what Thiago mentions in reviews, Q6String will have SSO
> > (small-string-optimisation) which makes many short strings expensive to
> > copy (if you think that copying 24 bytes is slower than upping an atomic
> > int through an indirection) or cheap to copy (if you think the opposite).
> > In any case, small strings will be very cheap to create (no allocation),
> > so for many strings there will be not much difference between passing a
> > QStringView or passing a QString.
> 
> Atomic operations are expensive (I think I heard once 'on the order of
> 100 instruction cycles', but that's highly apocryphal), mainly I would
> guess due to the need to maintain cache coherency. A small copy might
> happen entirely in local hot cache. 24 bytes is a whole three registers
> on a modern 64-bit machine. That's probably not going to be very slow.

This discussion is a red herring. It's not a choice between copying and 
atomically incrementing a reference counter. It's a choice between copying and 
between copying plus incrementing the reference counter.

QString s2 = s1;

needs to copy those bytes that are sizeof(QString) *anyway*, regardless of 
whether in addition to that it will increment the refcount.

> >> Yes, signed please. We can discuss whether it should be 64bit for Qt 6.
> > 
> > The current std API uses size_t. Do you (= both of you) expect that ever
> > to
> > change? If it doesn't, Qt will forever be the odd one out, until we
> > finally
> > drop QVector etc for std::vector etc and then porting will be a horror
> > because of MSVC's annoying warnings.
> 
> STL should change. In Qt and Python, you can use negative indices to
> refer to a distance (length) relative to the end (length) of the string.
> In STL you can't do that, which is a significant limitation by
> comparison. Please don't drop this useful functionality!

And see discussions in the std-discussions and std-proposals mailing list. I 
repeat what I said: the current *committee* stance is that you should use 
signed for everything, except when you need well-defined overflow behaviour.

The only problem, which I raised there and did not get resolved, is which type 
to use.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Wednesday 14 October 2015 08:51:11 Thiago Macieira wrote:
> The separation of the string itself from the size and the d pointer allows
> the  compiler, if it wants to, to share strings. In fact, disassembly of
> 
> f(QStringLiteral("foo"), QStringLiteral("foo")) 
> 
> produces one copy of u"foo" only.

Let me expand on this.

Current Qt5 QStringLiteral("foo") produces a data block of size
sizeof(QArrayData) + sizeof(u"foo") = 24 + 8 = 32 bytes

and the fact that QStringLiterals don't share will cause the innocent-looking 
above code require 64 bytes of read-only data.

My current code expands to 8 bytes of read-only data, at the expense of a 
little more code.

Current code:

leaq_ZZZ1fvENKUlvE0_clEvE15qstring_literal(%rip), %rax
movq%rax, 16(%rsp)
leaq_ZZZ1fvENKUlvE_clEvE15qstring_literal(%rip), %rax
movq%rax, (%rsp)
; followed by the call:
movq%rsp, %rdi
leaq16(%rsp), %rsi
call_Z1fRK7QStringS1_@PLT

My code:

leaq.LC0(%rip), %rcx; u"foo"
movq_ZN10QArrayData18shared_static_dataE@GOTPCREL(%rip), %rax
movq%rcx, 40(%rsp)
movl$3, 48(%rsp)
movq%rcx, 8(%rsp)
movq%rax, 32(%rsp)
movq%rax, (%rsp)
movl$3, 16(%rsp)
; the call itself is unchanged:
movq%rsp, %rdi
leaq32(%rsp), %rsi
call_Z1fRK7QStringS1_@PLT

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development

Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings

[Thiago Macieira]

On Wednesday 14 October 2015 12:16:47 Marc Mutz wrote:
> > >But as a condition to be even considered, it needs to be only for the
> > >methods 
> > >that do not hold a copy of the string. That is, methods that immediately
> > >consume the string and no longer need to reference its contents.
> 
> Thiago, I think it would help the discussion if you quickly summarised your 
> planned changes to QString in Qt 6.
> 
> AFAIK, the size and offset will move into the object, so I expected that 
> Q6String would subsume QStringRef, because each QString could provide a 
> separate view on the shared underlying data. I also was led to believe that 
> Q6String would use SSO, which, given its inceased sizeof(), would make a
> lot of sense, imo.

Indeed, that's the biggest gain. QString will contain a QStringPrivate, which 
is

struct QStringPrivate
{
QArrayData *d;
ushort *b;
qsize size; // let's bikeshed what qsize is later
};

My current code initialises a QStringLiteral like so:

#  define QStringLiteral(str) \
([]() -> QString { \
QStringPrivate holder = {  \
QArrayData::sharedStatic(), \
reinterpret_cast(const_cast(QT_UNICODE_LITERAL(str))), \
sizeof(QT_UNICODE_LITERAL(str))/2 - 1 }; \
return QString(holder); \
}()) \

The separation of the string itself from the size and the d pointer allows the 
compiler, if it wants to, to share strings. In fact, disassembly of 

f(QStringLiteral("foo"), QStringLiteral("foo")) 

produces one copy of u"foo" only.

Like you said, QString can become its own QStringView/QStringRef/QSubString. 
QString::left/mid/right can simply copy the d pointer, increment the refcount, 
then adjust b and size. This solves the issue I had with your proposal: 
passing a QStringView to a method that decides to copy it, so it wouldn't 
participate in reference counting. The drawback with this is the pathological 
case where a short substring is holding a large data block hostage.

My next objective, not yet achieved due to lack of time, is to make that 
QArrayData::sharedStatic() actually be a null pointer. That is, for anything 
that we didn't allocate memory for, the d pointer should be null. That implies 
a much faster loading of constant QStringLiterals and much faster handling of 
the decrement case. The biggest pain point in the code above in my current 
version is what happens after the call to f(): the compiler generates 2x bit 
testing of d->flags and calls to QArrayData::deallocate(), which are dead code 
and will never be run.

After that, implement SSO, which should hold 11 UTF-16 characters, including 
the null terminator. If we benchmark and find that we could use more, we can 
simply artificially increase sizeof(QString) to 32, which may have some extra 
benefits of its own, including the fact that the 24-byte short QString will be 
at odds with the null d pointer -- the if (d) check instead becomes
if (quintptr(d) & ~quintptr(1))

[also note how the order of the members in QStringPrivate needs to change for 
big-endian architectures]

[and note everything I say about QString also applies to QByteArray and 
QVector]

> And then I thought, QString would be converted to hold UTF-8. I saw 
> wip/qstring-utf8 fly by on gerrit, but ok, that hasn't received any updates 
> since 2012.

That was when we converted the QString methods taking const char* from Latin1 
to UTF-8. The backing store has never changed.

My version of QString stores an extra flag that indicates whether the string is 
US-ASCII, in which case we can run the unchecked to-Latin1 algorithm in both 
toLatin1 and toUtf8. Another idea I had but haven't investigated is to cache 
that result, which requires the returned QByteArray to share the d pointer 
with the QString.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development