Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 13 October 2015 22:46:36 Marc Mutz wrote: > I would propose to package the two into a class, called > - you guessed it - QStringView. I now started work on QStringView (and later QByteArrayView), here: https://codereview.qt-project.org/#/q/topic:qstringview,n,z Input welcome. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20. October 2015 23.37.27 Thiago Macieira wrote: > Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu: > > As I remember it, the change from QSubString to QStringRef was a simple > > API > > naming decision, i.e. independent of the implementation details. > > Then it must be a coincidence that it changed internals more or less at the > same time. Both were committed at the same time (in change 138d1ab051ed4934b303fa7a43a27dc4004e3ff9 for those who have access to the old Qt history). Date: Mon Jan 15 14:59:53 2007 +0100 Commit message: "Removed QSubString again, and introduced a lower-level class QStringRef" QSubString had a QString::Data *d member, and the initial QStringRef had a const QString *m_string, just like today. I have only skimmed the thread, so I'm probably misunderstanding something, but I don't see how a QString pointer is that much safer than a Data pointer, or a QChar pointer. A safe class would use the implicit sharing by keeping a QString member, but I don't know how that would impact performance. - Paul ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 22/10/15 10:56, "Paul Olav Tvete"wrote: >On Tuesday 20. October 2015 23.37.27 Thiago Macieira wrote: >> Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu: >> > As I remember it, the change from QSubString to QStringRef was a >>simple >> > API >> > naming decision, i.e. independent of the implementation details. >> >> Then it must be a coincidence that it changed internals more or less at >>the >> same time. > >Both were committed at the same time (in change >138d1ab051ed4934b303fa7a43a27dc4004e3ff9 for those who have access to the >old >Qt history). > >Date: Mon Jan 15 14:59:53 2007 +0100 >Commit message: "Removed QSubString again, and introduced a lower-level >class >QStringRef" > >QSubString had a QString::Data *d member, and the initial QStringRef had >a >const QString *m_string, just like today. > >I have only skimmed the thread, so I'm probably misunderstanding >something, >but I don't see how a QString pointer is that much safer than a Data >pointer, >or a QChar pointer. Unless you participate in the ref counting it’s only slightly safer. It’s safe against modifications of the string that do not shrink it’s length. > > >A safe class would use the implicit sharing by keeping a QString member, >but I >don't know how that would impact performance. Not a good idea IMO, as you’ll need to increase and decrease the refcount of often trivial operations. In addition, I think a QChar pointer + length is a better idea, as it also allows wrapping data that’s not contained in a QString. Cheers, Lars ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Em qua 21 out 2015, às 06:45:53, Knoll Lars escreveu: > >That doesn't work. I've tried to change QStringRef and it broke in a lot > >of places. > > Do you remember what broke? Not really. When I tracked down the failures, I did find out that it was related to the original QString being reallocated, so the QStringRef had a dangling reference. But I don't remember more details. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu: > As I remember it, the change from QSubString to QStringRef was a simple API > naming decision, i.e. independent of the implementation details. Then it must be a coincidence that it changed internals more or less at the same time. > The reason a String pointer is in there was due to an attempt to make this > class a bit safer. As long as we guaranteed the lifetime of the underlying > String, accessing the string ref should be ok. In hindsight, I do think > this was a mistake, as this safety doesn't give you a whole lot. In > practice, you couldn't modify the underlying Qstring anyway, so we could > have simply used a QChar pointer and a length. That certainly would have > been more flexible and would have helped us today. > So I'd agree that one could use a QStringView in the XML parser today. > QStringRef is doing exactly the same, but has the disadvantage of requiring > a pointer to a QString. This works in the parser, as it keeps the whole > string in memory for as long as parsing is ongoing, and you are supposed to > consume the returned stringref. But I would not recommend using > QStringRef/QStringView as a return value in almost all other cases. That doesn't work. I've tried to change QStringRef and it broke in a lot of places. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 21/10/15 08:37, "development-bounces+lars.knoll=theqtcompany@qt-project.org on behalf of Thiago Macieira"wrote: >Em qua 21 out 2015, às 06:29:30, Knoll Lars escreveu: >> As I remember it, the change from QSubString to QStringRef was a simple API >> naming decision, i.e. independent of the implementation details. > >Then it must be a coincidence that it changed internals more or less at the >same time. Possible. I have to admit that I don't recall all the details anymore. I can try to look up the old commit logs in case we think this is important. > >> The reason a String pointer is in there was due to an attempt to make this >> class a bit safer. As long as we guaranteed the lifetime of the underlying >> String, accessing the string ref should be ok. In hindsight, I do think >> this was a mistake, as this safety doesn't give you a whole lot. In >> practice, you couldn't modify the underlying Qstring anyway, so we could >> have simply used a QChar pointer and a length. That certainly would have >> been more flexible and would have helped us today. > >> So I'd agree that one could use a QStringView in the XML parser today. >> QStringRef is doing exactly the same, but has the disadvantage of requiring >> a pointer to a QString. This works in the parser, as it keeps the whole >> string in memory for as long as parsing is ongoing, and you are supposed to >> consume the returned stringref. But I would not recommend using >> QStringRef/QStringView as a return value in almost all other cases. > >That doesn't work. I've tried to change QStringRef and it broke in a lot of >places. Do you remember what broke? Cheers, Lars ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 20/10/15 18:11, "development-bounces+lars.knoll=theqtcompany@qt-project.org on behalf of Thiago Macieira"wrote: >On Tuesday 20 October 2015 08:49:17 André Somers wrote: >> Op 19-10-2015 om 17:54 schreef Thiago Macieira: >> > First of all, QStringRef keeps a pointer to the original QString, so it >> > isn't as fragile as QStringView. In fact, during the development of the >> > XML stream classes, there was a QSubString class that did more or less >> > what QStringView would do. It got replaced for a reason. >> >> Would you mind expanding on that a bit? I would think that the XML >> stream classes would be a natural example to use a string view, but it >> seems that such a thing was tried and dragons were uncovered... Perhaps >> it would be good to share a war story here so we won't run into those >> same dragons again? > >Wish that I could. I wasn't around that time. I know this happened because I >used to pull the daily snapshots and commit to an svk repository (yes, svk!) >to see the differences. That's how I spotted the QSubString class being added, >then removed, then QStringRef showed up. > >Is there anyone still around who worked with Matthias on this back in >2006/2007? I was there doing the work with Matthias. As I remember it, the change from QSubString to QStringRef was a simple API naming decision, i.e. independent of the implementation details. The reason a String pointer is in there was due to an attempt to make this class a bit safer. As long as we guaranteed the lifetime of the underlying String, accessing the string ref should be ok. In hindsight, I do think this was a mistake, as this safety doesn't give you a whole lot. In practice, you couldn't modify the underlying Qstring anyway, so we could have simply used a QChar pointer and a length. That certainly would have been more flexible and would have helped us today. So I'd agree that one could use a QStringView in the XML parser today. QStringRef is doing exactly the same, but has the disadvantage of requiring a pointer to a QString. This works in the parser, as it keeps the whole string in memory for as long as parsing is ongoing, and you are supposed to consume the returned stringref. But I would not recommend using QStringRef/QStringView as a return value in almost all other cases. Cheers, Lars ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Op 19-10-2015 om 17:54 schreef Thiago Macieira: > First of all, QStringRef keeps a pointer to the original QString, so it isn't > as fragile as QStringView. In fact, during the development of the XML stream > classes, there was a QSubString class that did more or less what QStringView > would do. It got replaced for a reason. Would you mind expanding on that a bit? I would think that the XML stream classes would be a natural example to use a string view, but it seems that such a thing was tried and dragons were uncovered... Perhaps it would be good to share a war story here so we won't run into those same dragons again? André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 20, 2015 08:43:26 Thiago Macieirawrote: > On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote: >> So if deep copies loose their ineffciency myth, what reason remains to not >> use QSV in all functions taking QString? > > Complicating the API. > > You cannot possibly prove that doing something O(n) is as efficient as doing > something O(1), therefore there's no way that deep copies would be a myth. We both know that depends on the size. The minimal size of what the CPU gets from the memory is a cache line. The size of a cache line today is typically 64 bytes. So as a minimum you always push that much data around. So if you use a atomic pointer on more than one cpu this cache lines has to be copied all the time. If you change something everything which depends on it has to be wait or recompute. So if you accidentally have data on the some cache line it is shared too. This is not very intuitive. Your application can be much slower because you accidentally share data. Okay that is simplified but we get the picture. And the worst part is that memory is so much slower in relation to cache access as it was for the difference between memory and hard disks in the paste. Today the difference between caches and memories resemble more that I learned about databases 20 years ago. And there you learned to be careful about the complexity notations which doesn't reason about different memory speeds. COW with strings can lead to more coping than simply coping the string because most strings are small. Sharing writeable cache lines between different cpu is asking for trouble but I think you know Amdahl's law https://en.wikipedia.org/wiki/Amdahl%27s_law. So why we don't measure it? Lets compile some applications with different implementation. Not simple ones. Maybe qtcreator would be a good idea. > COW may have comparable performance to deep copies if you consider the whole > picture and code written properly. Our code was written for COW, so I doubt > that you would get the same performance by suddenly making deep copies. > > To summarise: > > 1) I'm ok with adding QStringView > 2) I'm ok with using QStringView in applications, no restraints > 3) I'm ok with exploring the use in API, in parameters > 4) I'm ok with exploring it in return values, in very limited conditions that > are being studied very well, ahead of time > > Exploring does not mean you can go and start adding overloads everywhere. I think that's a very good approach. > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development Sent from cellphone ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 01:08:31 Thiago Macieira wrote: > So the summary of QStringView is: > a) never used as return type > b) used only as a parameter if the function processes the data and never > stores it and will never, ever store it (lazy processing) > c) for existing API, needs to be overloaded with QString > > I'm not buying the cost/benefit ratio. That's because you castrate the uses too much :) There are a ton of inlines types that would benefit, because they cannot be changed anyway. If you look at uic or qdoc, you'll see QStringView use-cases pop up in every corner. But mainly our impedance mismatch is that you focus on return values and I focus on arguments. I agree on the trickyness of the QSV result types, but there are always exceptions, some of which you and I mentioned, so I wouldn't go as far as saying (a). That would ban QXmlStreamReader, e.g. n the function argument side of things I see no real reason to have anything but QStringView arguments, plus strategic QString&& ones. The only reason to so much restrict the use of QSV as arguments type is the wish to avoid QString deep copies, the fear of which you said you don't share. So if deep copies loose their ineffciency myth, what reason remains to not use QSV in all functions taking QString? Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
I see. But then, if I have QStringView, doesn't that eliminate most of the reasons for needing the other string containing classes? If I want the efficiency of QStringView, won't QString underneath always be the right choice? In Thiago's example, if I have a QStringView API on my class, would I ever want to switch the QString to a QByteArray? Does using QStringView with QString work as well or better than WByteArray et al? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Monday, October 19, 2015 11:26 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 21:56:54 Smith Martin wrote: > >This API here simply cannot exist because the returned value would be a > >dangling reference. > > I don't understand. Implicit for using the QStringView data() function is > knowing that once the QByteArray is set, it is never changed. Then the > data() function is ok, isn't it? A QStringView points to QChar* (or ushort*) while a QByteArray only provides char*. The fromLatin1() (say) call thus necessary will create a temporary QString, which will bind to the returned QStringView, but will already be destroyed (taking the data QStringView references with it) when control returns to the caller. This particular mistake is easy to prevent statically, though: QStringView(QString &&) = delete; Thanks, Marc -- Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
That's QByteArray at the end, of course. From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Smith Martin <martin.sm...@theqtcompany.com> Sent: Tuesday, October 20, 2015 9:13 AM To: Marc Mutz; development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings I see. But then, if I have QStringView, doesn't that eliminate most of the reasons for needing the other string containing classes? If I want the efficiency of QStringView, won't QString underneath always be the right choice? In Thiago's example, if I have a QStringView API on my class, would I ever want to switch the QString to a QByteArray? Does using QStringView with QString work as well or better than WByteArray et al? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Monday, October 19, 2015 11:26 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 21:56:54 Smith Martin wrote: > >This API here simply cannot exist because the returned value would be a > >dangling reference. > > I don't understand. Implicit for using the QStringView data() function is > knowing that once the QByteArray is set, it is never changed. Then the > data() function is ok, isn't it? A QStringView points to QChar* (or ushort*) while a QByteArray only provides char*. The fromLatin1() (say) call thus necessary will create a temporary QString, which will bind to the returned QStringView, but will already be destroyed (taking the data QStringView references with it) when control returns to the caller. This particular mistake is easy to prevent statically, though: QStringView(QString &&) = delete; Thanks, Marc -- Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
I guess we need QStringView for QString and QByteArrayView for QByteArray, etc. martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Smith Martin <martin.sm...@theqtcompany.com> Sent: Tuesday, October 20, 2015 9:13 AM To: Marc Mutz; development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings I see. But then, if I have QStringView, doesn't that eliminate most of the reasons for needing the other string containing classes? If I want the efficiency of QStringView, won't QString underneath always be the right choice? In Thiago's example, if I have a QStringView API on my class, would I ever want to switch the QString to a QByteArray? Does using QStringView with QString work as well or better than WByteArray et al? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Monday, October 19, 2015 11:26 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 21:56:54 Smith Martin wrote: > >This API here simply cannot exist because the returned value would be a > >dangling reference. > > I don't understand. Implicit for using the QStringView data() function is > knowing that once the QByteArray is set, it is never changed. Then the > data() function is ok, isn't it? A QStringView points to QChar* (or ushort*) while a QByteArray only provides char*. The fromLatin1() (say) call thus necessary will create a temporary QString, which will bind to the returned QStringView, but will already be destroyed (taking the data QStringView references with it) when control returns to the caller. This particular mistake is easy to prevent statically, though: QStringView(QString &&) = delete; Thanks, Marc -- Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Btw, isn't the QStringView is the same as Range { Container::Iterator begin; Container::Iterator end; } ? Why we introduce only QStringView/QByteArrayView? Maybe we should think about adding range API to all containers? Alexandrescus presentation about ranges: http://www.slideshare.net/rawwell/iteratorsmustgo 2015-10-20 11:56 GMT+03:00 Smith Martin <martin.sm...@theqtcompany.com>: > I guess we need QStringView for QString and QByteArrayView for QByteArray, > etc. > > martin > > > From: development-bounces+martin.smith=theqtcompany@qt-project.org > <development-bounces+martin.smith=theqtcompany@qt-project.org> on > behalf of Smith Martin <martin.sm...@theqtcompany.com> > Sent: Tuesday, October 20, 2015 9:13 AM > To: Marc Mutz; development@qt-project.org > Subject: Re: [Development] RFC: Proposal for a semi-radical change in > Qt APIs taking strings > > I see. But then, if I have QStringView, doesn't that eliminate most of the > reasons for needing the other string containing classes? If I want the > efficiency of QStringView, won't QString underneath always be the right > choice? > > In Thiago's example, if I have a QStringView API on my class, would I ever > want to switch the QString to a QByteArray? > > Does using QStringView with QString work as well or better than WByteArray > et al? > > martin > > > From: development-bounces+martin.smith=theqtcompany@qt-project.org > <development-bounces+martin.smith=theqtcompany@qt-project.org> on > behalf of Marc Mutz <marc.m...@kdab.com> > Sent: Monday, October 19, 2015 11:26 PM > To: development@qt-project.org > Subject: Re: [Development] RFC: Proposal for a semi-radical change in > Qt APIs taking strings > > On Monday 19 October 2015 21:56:54 Smith Martin wrote: > > >This API here simply cannot exist because the returned value would be a > > >dangling reference. > > > > I don't understand. Implicit for using the QStringView data() function is > > knowing that once the QByteArray is set, it is never changed. Then the > > data() function is ok, isn't it? > > A QStringView points to QChar* (or ushort*) while a QByteArray only > provides > char*. The fromLatin1() (say) call thus necessary will create a temporary > QString, which will bind to the returned QStringView, but will already be > destroyed (taking the data QStringView references with it) when control > returns to the caller. > > This particular mistake is easy to prevent statically, though: > >QStringView(QString &&) = delete; > > Thanks, > Marc > > -- > Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer > KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company > Tel: +49-30-521325470 > KDAB - The Qt Experts > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development > ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 20, 2015 10:10:36 Knoll Larswrote: > On 20/10/15 09:59, "Bubke Marco" wrote: > >>On October 20, 2015 08:43:26 Thiago Macieira >>wrote: >> >>> On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote: So if deep copies loose their ineffciency myth, what reason remains to not use QSV in all functions taking QString? >>> >>> Complicating the API. >>> >>> You cannot possibly prove that doing something O(n) is as efficient as >>>doing >>> something O(1), therefore there's no way that deep copies would be a >>>myth. >> >>We both know that depends on the size. The minimal size of what the CPU >>gets from the memory is a cache line. The size of a cache line today is >>typically 64 bytes. So as a minimum you always push that much data >>around. So if you use a atomic pointer on more than one cpu this cache >>lines has to be copied all the time. If you change something everything >>which depends on it has to be wait or recompute. So if you accidentally >>have data on the some cache line it is shared too. This is not very >>intuitive. Your application can be much slower because you accidentally >>share data. >> >>Okay that is simplified but we get the picture. And the worst part is >>that memory is so much slower in relation to cache access as it was for >>the difference between memory and hard disks in the paste. Today the >>difference between caches and memories resemble more that I learned about >>databases 20 years ago. And there you learned to be careful about the >>complexity notations which doesn't reason about different memory speeds. >> >>COW with strings can lead to more coping than simply coping the string >>because most strings are small. Sharing writeable cache lines between >>different cpu is asking for trouble but I think you know Amdahl's law >>https://en.wikipedia.org/wiki/Amdahl%27s_law. >> >>So why we don't measure it? Lets compile some applications with different >>implementation. Not simple ones. Maybe qtcreator would be a good idea. > > Experimenting with that is probably a good idea, but we certainly can’t > change away from COW any time soon. Maybe not ever because of > compatibility concerns. We should be careful not to get the new motif too. ;-) I think we have to find a middle ground again and again. It's a question of how much pain our actions are producing and how much our inactions. ;-) >> >> >>> COW may have comparable performance to deep copies if you consider the >>>whole >>> picture and code written properly. Our code was written for COW, so I >>>doubt >>> that you would get the same performance by suddenly making deep copies. >>> >>> To summarise: >>> >>> 1) I'm ok with adding QStringView >>> 2) I'm ok with using QStringView in applications, no restraints >>> 3) I'm ok with exploring the use in API, in parameters >>> 4) I'm ok with exploring it in return values, in very limited >>>conditions that >>> are being studied very well, ahead of time >>> >>> Exploring does not mean you can go and start adding overloads >>>everywhere. >> >>I think that's a very good approach. > > +1. > > Cheers, > Lars Sent from cellphone ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 11:08:51 Иван Комиссаров wrote: > Btw, isn't the QStringView is the same as Range > { Container::Iterator begin; Container::Iterator end; } ? No. A string_view (or QStringView) is targeted at string operations. A range is just a pair of iterators. > Why we introduce only QStringView/QByteArrayView? Maybe we should think > about adding range API to all containers? We want to use the std functionalities in this area (ranges and array_view), because the API differences are small, and this is not a core skill of Qt as a library. Unicode string handling is. Our containers are all much weaker than the std ones, from QVector over QPair to QSharedPointer, and that should give you the hint that we don't have the manpower (nor should be have the desire) to compete with the std on these matters. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 09:13:18 Smith Martin wrote: > I see. But then, if I have QStringView, doesn't that eliminate most of the > reasons for needing the other string containing classes? If I want the > efficiency of QStringView, won't QString underneath always be the right > choice? No. Two examples: QStringLiteral - currently, it returns a full QString instance, with associated need to call the out-of-line QString dtor. Q6String will solve many of the issues, as Thiago has already said. But with QStringView, we could just pass u"foo" (a const char16_t[4]) _now_, and be done with it. The compiler and linker will share those string literals for us and even perform common suffix optimisations ("bar" and "foobar" overlap in the executable - that's allowed for string literals (and empty base classes) but not any other objects). No QString involved. If you build a local temporary string, it makes sense to use QVarLengthArray to do so. Again no QString involved. > In Thiago's example, if I have a QStringView API on my class, would I ever > want to switch the QString to a QByteArray? Yes. See, say, QUrl, which needs to juggle both the encoded and the decoded form of a URL. Also, if you have an XML file, in most cases it will not be encoded in UTF-16 on disk, so what you're working off of is a QByteArray (yes, I realize that QXmlStreamReader is working on QChar* (QStringRef, QString, QXmlStreamStringRef, ...), so take emails instead. > Does using QStringView with QString work as well or better than WByteArray > et al? QByteArray is to QString what QLatin1String is (a bit) to QStringView, but we've been discussing (and agreeing on) the need for a QByteArrayView, too, because not all QByteArrays contain latin-1 strings. But those who wish to experiment with a string view in Qt can use QLatin1String more. Now. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Op 20-10-2015 om 11:08 schreef Иван Комиссаров: > Btw, isn't the QStringView is the same as Range > { Container::Iterator begin; Container::Iterator end; } ? > Why we introduce only QStringView/QByteArrayView? Maybe we should > think about adding range API to all containers? > > Alexandrescus presentation about ranges: > http://www.slideshare.net/rawwell/iteratorsmustgo More recent: https://www.youtube.com/watch?v=mFUXNMfaciE (Eric Niebler on CppCon) But I think it is a bit early to start doing that, seeing that the whole idea is still in flux in the STL. I'm sure we don't want to end up with something incompatible (again), right? Also, a QStringView is more than "just" a range, as it supports all kinds of string-related API similar to QString, or so I will assume based on the similar proposal for std::string_view. (as presented in https://www.youtube.com/watch?v=H9gAaNRoon4) André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Op 19-10-2015 om 22:31 schreef Bubke Marco: > For example the highlighting can show that the parameter is an in or in/out > parameter so you don't need to use pointers anymore for in/out parameters > etc. But it is getting off topic. https://bugreports.qt.io/browse/QTCREATORBUG-14468 André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote: > So if deep copies loose their ineffciency myth, what reason remains to not > use QSV in all functions taking QString? Complicating the API. You cannot possibly prove that doing something O(n) is as efficient as doing something O(1), therefore there's no way that deep copies would be a myth. COW may have comparable performance to deep copies if you consider the whole picture and code written properly. Our code was written for COW, so I doubt that you would get the same performance by suddenly making deep copies. To summarise: 1) I'm ok with adding QStringView 2) I'm ok with using QStringView in applications, no restraints 3) I'm ok with exploring the use in API, in parameters 4) I'm ok with exploring it in return values, in very limited conditions that are being studied very well, ahead of time Exploring does not mean you can go and start adding overloads everywhere. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Le lundi 19 octobre 2015 à 16:08 -0700, Thiago Macieira a écrit : > > So the summary of QStringView is: > a) never used as return type > b) used only as a parameter if the function processes the data and never > stores it and will never, ever store it (lazy processing) > c) for existing API, needs to be overloaded with QString > > I'm not buying the cost/benefit ratio. I don’t get b) statement. If you need to store it, as a QString, and you can afford to expose that to the user, then i think it’s better to provide a QString&& overload. This : - makes clear that you will store the value « as-is ». - should be more efficient than copying the string - still allows copying the string if the user needs to keep its own copy of the data. - cohabites well with a QStringView api I do not see the benefit of a QString api vs a QStringView and a QString&& api, which cover more use cases. Regards, Julien Blanc ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 20/10/15 09:59, "Bubke Marco"wrote: >On October 20, 2015 08:43:26 Thiago Macieira >wrote: > >> On Tuesday 20 October 2015 09:44:23 Marc Mutz wrote: >>> So if deep copies loose their ineffciency myth, what reason remains to >>>not >>> use QSV in all functions taking QString? >> >> Complicating the API. >> >> You cannot possibly prove that doing something O(n) is as efficient as >>doing >> something O(1), therefore there's no way that deep copies would be a >>myth. > >We both know that depends on the size. The minimal size of what the CPU >gets from the memory is a cache line. The size of a cache line today is >typically 64 bytes. So as a minimum you always push that much data >around. So if you use a atomic pointer on more than one cpu this cache >lines has to be copied all the time. If you change something everything >which depends on it has to be wait or recompute. So if you accidentally >have data on the some cache line it is shared too. This is not very >intuitive. Your application can be much slower because you accidentally >share data. > >Okay that is simplified but we get the picture. And the worst part is >that memory is so much slower in relation to cache access as it was for >the difference between memory and hard disks in the paste. Today the >difference between caches and memories resemble more that I learned about >databases 20 years ago. And there you learned to be careful about the >complexity notations which doesn't reason about different memory speeds. > >COW with strings can lead to more coping than simply coping the string >because most strings are small. Sharing writeable cache lines between >different cpu is asking for trouble but I think you know Amdahl's law >https://en.wikipedia.org/wiki/Amdahl%27s_law. > >So why we don't measure it? Lets compile some applications with different >implementation. Not simple ones. Maybe qtcreator would be a good idea. Experimenting with that is probably a good idea, but we certainly can’t change away from COW any time soon. Maybe not ever because of compatibility concerns. > > >> COW may have comparable performance to deep copies if you consider the >>whole >> picture and code written properly. Our code was written for COW, so I >>doubt >> that you would get the same performance by suddenly making deep copies. >> >> To summarise: >> >> 1) I'm ok with adding QStringView >> 2) I'm ok with using QStringView in applications, no restraints >> 3) I'm ok with exploring the use in API, in parameters >> 4) I'm ok with exploring it in return values, in very limited >>conditions that >> are being studied very well, ahead of time >> >> Exploring does not mean you can go and start adding overloads >>everywhere. > >I think that's a very good approach. +1. Cheers, Lars ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 10:06:28 Julien Blanc wrote: > Le lundi 19 octobre 2015 à 16:08 -0700, Thiago Macieira a écrit : > > So the summary of QStringView is: > > a) never used as return type > > b) used only as a parameter if the function processes the data and never > > stores it and will never, ever store it (lazy processing) > > > > c) for existing API, needs to be overloaded with QString > > > > I'm not buying the cost/benefit ratio. > > I don’t get b) statement. > > If you need to store it, as a QString, and you can afford to expose that > to the user, then i think it’s better to provide a QString&& overload. QString&& won't match a regular lvalue. QString s = "foo"; function(s); [and even if it did, it would be wrong] But this would compile: function(QString(s)) because it now creates a copy before passing to the function. That is, you create a copy that you're giving ownership of. I don't think I want this kind of API. It's too confusing. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 08:49:17 André Somers wrote: > Op 19-10-2015 om 17:54 schreef Thiago Macieira: > > First of all, QStringRef keeps a pointer to the original QString, so it > > isn't as fragile as QStringView. In fact, during the development of the > > XML stream classes, there was a QSubString class that did more or less > > what QStringView would do. It got replaced for a reason. > > Would you mind expanding on that a bit? I would think that the XML > stream classes would be a natural example to use a string view, but it > seems that such a thing was tried and dragons were uncovered... Perhaps > it would be good to share a war story here so we won't run into those > same dragons again? Wish that I could. I wasn't around that time. I know this happened because I used to pull the daily snapshots and commit to an svk repository (yes, svk!) to see the differences. That's how I spotted the QSubString class being added, then removed, then QStringRef showed up. Is there anyone still around who worked with Matthias on this back in 2006/2007? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote: > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > > That the proposal is that every single function currently taking a > > QString should instead be changed to take a QStringView instead, unless > > a few exceptional cases (for instance the function is used to store the > > string somehow, plus other cases under discussion). Similar case for > > return values. If that doesn't look like a radical change to you... > > Return values must always be QString, never QStringView. Returning a view > is like returning a reference and goes against Qt's library code policy. That's like saying that functions musn't return a QStringRef. But see QXmlStreamReader, which uses that to (apparently) good effect. I also mentioned the case where a plugin returns some static data as a QString coming from a QStringLiteral. You yourself mentioned that that's going boom when the plugin is unloaded. If the plugin instead returned a QStringView, then callers wishing to store the result would have toString() it, making a deep copy and insulating themselves from the plugin being unloaded, while users wishing to just do some mix-and-matching with the string could avoid the copy by keeping the returned data in a QStringView. People should stop being so afraid of a string deep copy. It's a no-op for SSO'ed strings and according to Herb Sutter's benchmark on average doesn't matter for all other strings. Here's where safety and performance could actually go hand-in-hand for once. Safety? Yes, the above QStringLiteral problem and also, as you know (and it's even documented) that Qt doesn't implement CoW correctly: QString s1 = "foo"; QString::iterator it = s1.begin(); QString s2 = s1; *it = 'b'; // oops: s2 == "boo", too! I would like to see proof that ref-counted strings are actually worth it, even in the unsafe Qt implementation, considering that the C++ world slowly moves away from them. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>Return values must always be QString, never QStringView. Returning a view is >like returning a reference and goes against Qt's library code policy. But an assumption is the user manages the ownership of the underlying string, so it seems different. I'm rewriting the C++ parsing in qdoc. It calls a tokenizer that has a token()function that returns an enum value and a lexeme() function that returns a QString. It builds the QString for lexeme() each time, when it would be better to return a QStringView, since I know the text won't change. For the names that have to be saved, the QStringView can be passed to the tree node constructor, and only one QString gets built. martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Thiago Macieira <thiago.macie...@intel.com> Sent: Sunday, October 18, 2015 10:55 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > That the proposal is that every single function currently taking a > QString should instead be changed to take a QStringView instead, unless > a few exceptional cases (for instance the function is used to store the > string somehow, plus other cases under discussion). Similar case for > return values. If that doesn't look like a radical change to you... Return values must always be QString, never QStringView. Returning a view is like returning a reference and goes against Qt's library code policy. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 06:10:41 Smith Martin wrote: > >Return values must always be QString, never QStringView. Returning a view > >is like returning a reference and goes against Qt's library code policy. > > But an assumption is the user manages the ownership of the underlying > string, so it seems different. Are you talking about a class that operates on user-provided data? In that restricted scenario, it might make sense. The point I'm trying to make is that returning a non-owning copy means that something else must own the data. That's what goes against the library code policy. Example: QUrl url; QStringView query() const; It's not possible to write the query() function. > I'm rewriting the C++ parsing in qdoc. It > calls a tokenizer that has a token()function that returns an enum value and > a lexeme() function that returns a QString. It builds the QString for > lexeme() each time, when it would be better to return a QStringView, since > I know the text won't change. Good, but qdoc is not a library so it's not subject to the library code policy. > For the names that have to be saved, the > QStringView can be passed to the tree node constructor, and only one > QString gets built. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>The point I'm trying to make >is that returning a non-owning copy means that something else must own the >data. That's what goes against the library code policy. The library code policy is an one that has always made sense in the pre-string_view world. But if string_view is being added to C++, don't we have to add it to Qt? And if we add it to Qt, don't we have to support it in all the Qt classes where it could improve performance? Won't internet-of-things developers want a library that fully supports string_view? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Thiago Macieira <thiago.macie...@intel.com> Sent: Monday, October 19, 2015 8:21 AM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 06:10:41 Smith Martin wrote: > >Return values must always be QString, never QStringView. Returning a view > >is like returning a reference and goes against Qt's library code policy. > > But an assumption is the user manages the ownership of the underlying > string, so it seems different. Are you talking about a class that operates on user-provided data? In that restricted scenario, it might make sense. The point I'm trying to make is that returning a non-owning copy means that something else must own the data. That's what goes against the library code policy. Example: QUrl url; QStringView query() const; It's not possible to write the query() function. > I'm rewriting the C++ parsing in qdoc. It > calls a tokenizer that has a token()function that returns an enum value and > a lexeme() function that returns a QString. It builds the QString for > lexeme() each time, when it would be better to return a QStringView, since > I know the text won't change. Good, but qdoc is not a library so it's not subject to the library code policy. > For the names that have to be saved, the > QStringView can be passed to the tree node constructor, and only one > QString gets built. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>That's like saying that functions musn't return a QStringRef. But see >QXmlStreamReader, which uses that to (apparently) good effect. That's a good point, but the other points you raise are just arguing by finding fault with Qt. I think fully supporting QStringView has to be justified by showing a way to include it in the library so that it can be used safely. martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Monday, October 19, 2015 11:23 AM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote: > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > > That the proposal is that every single function currently taking a > > QString should instead be changed to take a QStringView instead, unless > > a few exceptional cases (for instance the function is used to store the > > string somehow, plus other cases under discussion). Similar case for > > return values. If that doesn't look like a radical change to you... > > Return values must always be QString, never QStringView. Returning a view > is like returning a reference and goes against Qt's library code policy. That's like saying that functions musn't return a QStringRef. But see QXmlStreamReader, which uses that to (apparently) good effect. I also mentioned the case where a plugin returns some static data as a QString coming from a QStringLiteral. You yourself mentioned that that's going boom when the plugin is unloaded. If the plugin instead returned a QStringView, then callers wishing to store the result would have toString() it, making a deep copy and insulating themselves from the plugin being unloaded, while users wishing to just do some mix-and-matching with the string could avoid the copy by keeping the returned data in a QStringView. People should stop being so afraid of a string deep copy. It's a no-op for SSO'ed strings and according to Herb Sutter's benchmark on average doesn't matter for all other strings. Here's where safety and performance could actually go hand-in-hand for once. Safety? Yes, the above QStringLiteral problem and also, as you know (and it's even documented) that Qt doesn't implement CoW correctly: QString s1 = "foo"; QString::iterator it = s1.begin(); QString s2 = s1; *it = 'b'; // oops: s2 == "boo", too! I would like to see proof that ref-counted strings are actually worth it, even in the unsafe Qt implementation, considering that the C++ world slowly moves away from them. Thanks, Marc -- Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 2015-10-16 19:52, Kurt Pattyn wrote: > On 17 Oct 2015, at 01:18, Marc Mutz wrote: >> I find it intolerable that todays software takes 1000x the memory, >> 1000x the CPU capacity and doesn't get a given jobs done in less >> wallclock time than software 20 years ago. > > These are 'numbers'. Is it like that? Really 1000x the memory, 1000x the CPU > capacity? Er... yes? An 80286 of yesteryear ran as fast as 25 MHz¹. A modern, 6-core 4 GHz CPU has a "theoretical" 24 GHz of "computing power". That's almost exactly 1000x (more, considering the ISA improvements). The same machine could address up to 16 MiB of RAM. A lot of machines around my office have 16 GiB of RAM (my home desktop has more; OTOH, many home machines have less). That's pedantically a factor of 1024x; again, close enough. Basically, yeah... computers some 10-20 years ago had similar specs as modern computers - CPU speed in the single to low double digits, memory in similar ranges - except the numbers were in one 1000/1024-order of magnitude less units (MHz vs. GHz, MiB vs. GiB). The increase in disk space is, if anything, even more extreme. There remains, of course, *some* exaggeration there... modern computers are, after all, tackling quantities of data that computers 20 years ago couldn't dream of touching (consider any process run on large clusters, for example). Even so, it's sobering to think about, especially as a lot of user software is "prettier" but in many ways not actually *better*, at least not in proportion to the increase in resource requirements. (¹ https://en.wikipedia.org/wiki/Intel_80286) -- Matthew ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 11:23:38 Marc Mutz wrote: > On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote: > > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > > > That the proposal is that every single function currently taking a > > > QString should instead be changed to take a QStringView instead, unless > > > a few exceptional cases (for instance the function is used to store the > > > string somehow, plus other cases under discussion). Similar case for > > > return values. If that doesn't look like a radical change to you... > > > > Return values must always be QString, never QStringView. Returning a view > > is like returning a reference and goes against Qt's library code policy. > > That's like saying that functions musn't return a QStringRef. But see > QXmlStreamReader, which uses that to (apparently) good effect. No, it isn't. First of all, QStringRef keeps a pointer to the original QString, so it isn't as fragile as QStringView. In fact, during the development of the XML stream classes, there was a QSubString class that did more or less what QStringView would do. It got replaced for a reason. Second, this falls under the exception of parsing a large block of data and getting information about chunks inside. And besides, it does make code using those classes uglier. > I also mentioned the case where a plugin returns some static data as a > QString coming from a QStringLiteral. You yourself mentioned that that's > going boom when the plugin is unloaded. If the plugin instead returned a > QStringView, then callers wishing to store the result would have toString() > it, making a deep copy and insulating themselves from the plugin being > unloaded, while users wishing to just do some mix-and-matching with the > string could avoid the copy by keeping the returned data in a QStringView. I agree on the advantage. >From what I can see, a Qt6 QStringView is a QString with a null d pointer, except that it triggers deep copy everywhere whre it's assigned to a regular QString. I'm fine with having the class. I'm not ok with changing the library code policy. > People should stop being so afraid of a string deep copy. It's a no-op for > SSO'ed strings and according to Herb Sutter's benchmark on average doesn't > matter for all other strings. Here's where safety and performance could > actually go hand-in-hand for once. That's not what I am afraid of. Again, please try writing this method: class Foo { QUrl url; public: QStringView host() const; }; -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 07:14:55 Smith Martin wrote: > >The point I'm trying to make > >is that returning a non-owning copy means that something else must own the > >data. That's what goes against the library code policy. > > The library code policy is an one that has always made sense in the > pre-string_view world. But if string_view is being added to C++, don't we > have to add it to Qt? And if we add it to Qt, don't we have to support it > in all the Qt classes where it could improve performance? I don't think that string_view makes a difference. The point is still the same: returning non-owning copies of data implies something else owns it. We haven't done that so far and I don't think string_view should change the policy. > > Won't internet-of-things developers want a library that fully supports > string_view? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 21:56:54 Smith Martin wrote: > >This API here simply cannot exist because the returned value would be a > >dangling reference. > > I don't understand. Implicit for using the QStringView data() function is > knowing that once the QByteArray is set, it is never changed. Then the > data() function is ok, isn't it? A QStringView points to QChar* (or ushort*) while a QByteArray only provides char*. The fromLatin1() (say) call thus necessary will create a temporary QString, which will bind to the returned QStringView, but will already be destroyed (taking the data QStringView references with it) when control returns to the caller. This particular mistake is easy to prevent statically, though: QStringView(QString &&) = delete; Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 19, 2015 21:38:51 Thiago Macieirawrote: > On Monday 19 October 2015 18:38:52 Smith Martin wrote: >> >Again, please try writing this method: >> Doesn't that example just mean there are some classes that can't have a >> QStringView API? A class should have a QStringView API if it can be used >> safely. > > Right. I'm simply saying that "if it can be used safely" is very, very > restricted. > > Suppose you have in v1: > > class Foo > { > QString m_data; > public: > QStringView data() const; > }; > > Which looks fine and works without dangling references. Then in v2, you need > to > do this: > > class Foo > { > QByteArray m_data; > public: > QStringView data() const; > }; How can you make a QStringView from a byte array? I know we abuse byte array as utf8 class but if you convert it you have no ownership of the created QString. Do you do a reinterpret cast? ;-) I see QStringView as const QString & but much more explicit and portable. So maybe you want to show that people can return a string view of a temporary? We can add a error in the clang code model so it should be a non issue. I think we should develop and use better tools which warn you about mistakes like that. That makes C++ much easier. ;-) Today I spoke with Ossi about the integration in gerrit too. So if you have enough firewalls we can be much more liberal in what we can use. For example the highlighting can show that the parameter is an in or in/out parameter so you don't need to use pointers anymore for in/out parameters etc. But it is getting off topic. > This API here simply cannot exist because the returned value would be a > dangling reference. > > Therefore, QStringView returns must be treated like references: only in > exceptional cases. > > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel Open Source Technology Center > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development Sent from cellphone ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 18:38:52 Smith Martin wrote: > >Again, please try writing this method: > Doesn't that example just mean there are some classes that can't have a > QStringView API? A class should have a QStringView API if it can be used > safely. Right. I'm simply saying that "if it can be used safely" is very, very restricted. Suppose you have in v1: class Foo { QString m_data; public: QStringView data() const; }; Which looks fine and works without dangling references. Then in v2, you need to do this: class Foo { QByteArray m_data; public: QStringView data() const; }; This API here simply cannot exist because the returned value would be a dangling reference. Therefore, QStringView returns must be treated like references: only in exceptional cases. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>Again, please try writing this method: Doesn't that example just mean there are some classes that can't have a QStringView API? A class should have a QStringView API if it can be used safely. martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Thiago Macieira <thiago.macie...@intel.com> Sent: Monday, October 19, 2015 5:54 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 11:23:38 Marc Mutz wrote: > On Sunday 18 October 2015 22:55:23 Thiago Macieira wrote: > > On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > > > That the proposal is that every single function currently taking a > > > QString should instead be changed to take a QStringView instead, unless > > > a few exceptional cases (for instance the function is used to store the > > > string somehow, plus other cases under discussion). Similar case for > > > return values. If that doesn't look like a radical change to you... > > > > Return values must always be QString, never QStringView. Returning a view > > is like returning a reference and goes against Qt's library code policy. > > That's like saying that functions musn't return a QStringRef. But see > QXmlStreamReader, which uses that to (apparently) good effect. No, it isn't. First of all, QStringRef keeps a pointer to the original QString, so it isn't as fragile as QStringView. In fact, during the development of the XML stream classes, there was a QSubString class that did more or less what QStringView would do. It got replaced for a reason. Second, this falls under the exception of parsing a large block of data and getting information about chunks inside. And besides, it does make code using those classes uglier. > I also mentioned the case where a plugin returns some static data as a > QString coming from a QStringLiteral. You yourself mentioned that that's > going boom when the plugin is unloaded. If the plugin instead returned a > QStringView, then callers wishing to store the result would have toString() > it, making a deep copy and insulating themselves from the plugin being > unloaded, while users wishing to just do some mix-and-matching with the > string could avoid the copy by keeping the returned data in a QStringView. I agree on the advantage. >From what I can see, a Qt6 QStringView is a QString with a null d pointer, except that it triggers deep copy everywhere whre it's assigned to a regular QString. I'm fine with having the class. I'm not ok with changing the library code policy. > People should stop being so afraid of a string deep copy. It's a no-op for > SSO'ed strings and according to Herb Sutter's benchmark on average doesn't > matter for all other strings. Here's where safety and performance could > actually go hand-in-hand for once. That's not what I am afraid of. Again, please try writing this method: class Foo { QUrl url; public: QStringView host() const; }; -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>This API here simply cannot exist because the returned value would be a >dangling reference. I don't understand. Implicit for using the QStringView data() function is knowing that once the QByteArray is set, it is never changed. Then the data() function is ok, isn't it? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Thiago Macieira <thiago.macie...@intel.com> Sent: Monday, October 19, 2015 9:38 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Monday 19 October 2015 18:38:52 Smith Martin wrote: > >Again, please try writing this method: > Doesn't that example just mean there are some classes that can't have a > QStringView API? A class should have a QStringView API if it can be used > safely. Right. I'm simply saying that "if it can be used safely" is very, very restricted. Suppose you have in v1: class Foo { QString m_data; public: QStringView data() const; }; Which looks fine and works without dangling references. Then in v2, you need to do this: class Foo { QByteArray m_data; public: QStringView data() const; }; This API here simply cannot exist because the returned value would be a dangling reference. Therefore, QStringView returns must be treated like references: only in exceptional cases. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19. October 2015 23:26:45 Marc Mutz wrote: > This particular mistake is easy to prevent statically, though: > >QStringView(QString &&) = delete; No. We want this to work: if (isValidIdentifier(myVariant.toString())) with isValidIdentifier taking a QStringView and toString returning a temporary. -- Olivier Woboq - Qt services and support - http://woboq.com - http://code.woboq.org ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Tuesday 20 October 2015 00:51:58 Olivier Goffart wrote: > On Monday 19. October 2015 23:26:45 Marc Mutz wrote: > > This particular mistake is easy to prevent statically, though: > > > >QStringView(QString &&) = delete; > > No. > > We want this to work: > > if (isValidIdentifier(myVariant.toString())) > > with isValidIdentifier taking a QStringView and toString returning a > temporary. That can be mitigated by overloading isValidIdentifier with a QString &&. That is: bool isValidIdentifier(QString &); bool isValidIdentifier(const QStringView ); The first one can delegate to the second, as inside it the id parameter is lvalue, not rvalue. That is: bool isValidIdentifier(QString &) { return isValidIdentifier(QStringView(id)); } Note that this overload is not necessary for existing APIs where a const QString& overload is already present, as that will bind to the rvalue just fine. So is there any case where we'd want a QString&& overload instead of a const QString &? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Monday 19 October 2015 20:31:53 Bubke Marco wrote: > On October 19, 2015 21:38:51 Thiago Macieirawrote: > > Right. I'm simply saying that "if it can be used safely" is very, very > > restricted. > > > > Suppose you have in v1: > > > > class Foo > > { > > QString m_data; > > public: > > QStringView data() const; > > }; > > > > Which looks fine and works without dangling references. Then in v2, you > > need to do this: > > > > class Foo > > { > > QByteArray m_data; > > public: > > QStringView data() const; > > }; > > How can you make a QStringView from a byte array? That's exactly the point. It was a tricky question and this is exactly the gotcha. You cannot. Therefore, the API must not be permitted, save for exceptional cases where you know the internal backing will NEVER EVER change (until the next source compatibility change, at the soonest). So the summary of QStringView is: a) never used as return type b) used only as a parameter if the function processes the data and never stores it and will never, ever store it (lazy processing) c) for existing API, needs to be overloaded with QString I'm not buying the cost/benefit ratio. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>If that doesn't look like a radical change to you... But that's way down the line. We can add QStringView and overloads that take QStringView without removing the overloads that take QString. Then we can deprecate the overloads that take QString. From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Giuseppe D'Angelo <giuseppe.dang...@kdab.com> Sent: Sunday, October 18, 2015 9:27 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings Il 18/10/2015 20:30, Smith Martin ha scritto: > After watching the video on string_view, it seems clear we have to offer a > QStringView, so I don't see what is "semi-radical" about your proposal. That the proposal is that every single function currently taking a QString should instead be changed to take a QStringView instead, unless a few exceptional cases (for instance the function is used to store the string somehow, plus other cases under discussion). Similar case for return values. If that doesn't look like a radical change to you... Cheers, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Software Engineer KDAB (UK) Ltd., a KDAB Group company | Tel: UK +44-1625-809908 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Sunday 18 October 2015 21:27:31 Giuseppe D'Angelo wrote: > That the proposal is that every single function currently taking a > QString should instead be changed to take a QStringView instead, unless > a few exceptional cases (for instance the function is used to store the > string somehow, plus other cases under discussion). Similar case for > return values. If that doesn't look like a radical change to you... Return values must always be QString, never QStringView. Returning a view is like returning a reference and goes against Qt's library code policy. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
After watching the video on string_view, it seems clear we have to offer a QStringView, so I don't see what is "semi-radical" about your proposal. martin From: development-bounces+martin.smith=theqtcompany@qt-project.orgon behalf of Marc Mutz Sent: Tuesday, October 13, 2015 10:46 PM To: development@qt-project.org Subject: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings Hi, After looking quite a bit into the current state of string handling in Qt for my QtWS talk last week, I have become frustrated by the state of string handling in Qt. We have such powerful tools for string handling (QStringRef, QStringBuilder), but all APIs outside QString and its immediate surroundings only deal in QString. The correct way would be to overload every function taking QString with QLatin1String and QStringRef versions, and then, for some other rare cases, const QChar *, int size. Let alone std::basic_string. I would therefore like to propose to abandon QString for new API (and over time phase it out of existing API), and only provide (const QChar*, size_t) as the most general form. I would propose to package the two into a class, called - you guessed it - QStringView. =FAQ= Q: Why not just use QStringRef? A: QStringRef is tied to QString. E.g. you can't create a QStringRef from a pair of QChar*, int. It also is kind of stuck in historic mistakes making it undesireable as a cheap-to-pass parameter type. Q: What mistakes do you refer to? A: The fact that it has copy ctor and assignment operator, so it's not a trivally-copyable type and thus cannot efficiently passed by-value. It may also be too large for pass-by-value due to the rather useless QString pointer (should have been QStringData*, if any). Neither can be fixed before Qt 6. Q: Why size_t? A: The intent of QStringView (and std::experimental::string_view) is to act as an interface between modules written with different compilers and different flags. A std::string will never be compatible between compilers or even just different flags, but a simple struct {char*, size_t} will always be, by way of it's C compatibility. So the goal is not just to accept QString, QStringRef, and (QChar*,int) (and QVarLengthArray!) as input to QStringView, but also std::basic_string and std::vector. Q: What about the plans to make QString UTF-8-backed? A: QStringView-using code will need to be ported just as QString-using code will. Q: What future do you have in mind for QStringRef? A: None in particular, though I have found a need for an owning QStringRef in some places. But I expect Qt 6' QString to be able to provide a restricted view on shared data, such that it would subsume QStringRef completely. Q: What about QLatin1String? A: Once QString is backed by UTF-8, latin-1 ceases to be a special charset. We might want something like QUsAsciiString, but it would just be a UTF-8 string, so it could be packed into QStringView. Q: What about QByteArray, QVector? A: I'm unsure about QByteArrayView. It might not pull its weight compared to std::(experimental::)string_view, but I also note that we're currently missing a QByteArrayRef, so a QBAView might make sense while we wait for the std one to become available to us. I'm actively opposed to a QArrayView, because I don't think it provides us with anything std::(experimental::)array_view doesn't already. Q: What about a rope? A: A rope is a more complex string that can provide complex views on existing data as well as store rules for generating stretches of data (as opposed to the data itself). A rope is a very complex data structure and would not work as a universal interface type. It would be cool if Qt had a rope, but that is outside the scope of my proposal. Q: What do you mean when you say "abandon QString"? A: I mean that functions should not take QStrings as arguments, but QStringViews. Then users can transparently pass QString, QStringRef and any of a number of other "string" types without overloading the function on each of them. I do not mean to abandon QString, the class. Only QString, the interface type. Q: What API should QStringView have? A: Since it's mainly an interface type, it should have implicit conversions from all kinds of "string" types, but explicit conversion _to_ those string types. It should carry all the API from QString that can be implemented on just a (QChar*, size_t) (e.g. trimmed(), left(), mid(), section(), split(), but not append(), replace() (except maybe the (QChar,QChar) overload. Corresponding QString/Ref API could (eventually) just forward to the QStringView one. Thanks, now fire away, Marc -- Marc Mutz | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Il 18/10/2015 20:30, Smith Martin ha scritto: After watching the video on string_view, it seems clear we have to offer a QStringView, so I don't see what is "semi-radical" about your proposal. That the proposal is that every single function currently taking a QString should instead be changed to take a QStringView instead, unless a few exceptional cases (for instance the function is used to store the string somehow, plus other cases under discussion). Similar case for return values. If that doesn't look like a radical change to you... Cheers, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Software Engineer KDAB (UK) Ltd., a KDAB Group company | Tel: UK +44-1625-809908 KDAB - The Qt Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 22:51:28 Bubke Marco wrote: > But before we add new concepts we should agree in which direction we want > to go. I have the feeling that we build an airplan but some want a jumbo > jet, some a jet fighter and some, lets describe it so, they want to fly > to the moon because earth will maybe burning down and build something > completely different which doesn't looks like an airplane anymore which we > built over many years. I don't think we will be succed before we find a > common context. So it would be good to describe what is our story. I believe I did that multiple times now. What's _your_ story? And so what's _your_ alternative? Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 22:42:35 Konstantin Ritt wrote: > > The whole problem of QString::fromRawData is that the method you called > > may store the QString and thus keep referencing the string you had, even > > past the > > point where your string changed. > > > > In fact, that happens with QStringLiteral too. If you use QStringLiteral > > in a > > plugin and the plugin gets unloaded, the application may crash. > > Indeed, it is the same issue. QStringView doesn't solve it either. > So the best thing we can do, IMO, is to give the user a tool and describe > how to use it properly to get some extra performance, on his own risk. Since QStringView doesn't (ever) own, it does solve the problem. If a plugin returns a static-storage duration string through QStringView istead of QString, a user that wishes to store the QString needs to assign it to a QString, which performs a deep copy. Everyone else can work with the static- storage string data directly (until the plugin gets unloaded, of course, but that would be visible in the client code). -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>I have not discovered anything. I merely use the existing tools and found that >APIs outside the core Qt string classes largely ignore them, and probably >rightfully so, since there's a combinatorical explosion when you combine >QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc. When you use tools as you say you did, to find some knowledge we didn't know before, that's what discover means. But now you are changing the meaning again. When you started this discussion, I understood you to mean that QString, et al, are inefficient in the sense that there is a better way, and the QStringView is that better way. But now... >QStringView is not more efficient than any of those. It is an abstraction over >them, so that you can use, say, QStringRef in more places. I understood your last reply to me to mean that we should be using QStringRef in more places anyway, whether we add QStringView to the mix or not, and that one thing that is clearly missing from Qt documentation is a "How and when to use" discussion for all of QString, QLatin1String, QString, QStringRef, QChar, const char*, and QByteArray, including which combinations of these classes to use and which combinations to avoid. >Then documenters should ask. There's a class QStringRef. If you/they don't >understand what the difference is to QString and why it exists, just ask. Both classes are documented, so we have no reason to think we don't understand them. You can argue that the doc team should be using Qt in anger as we go, and we should be discovering ourselves how Qt classes should be used to achieve optimal efficiency, but we don't have much time for that, given all the documentation that has to be written, so we depend on the engineers who design and/or use Qt classes to tell us when we need to add this higher level how-to knowledge. >I try to shake thing up here and >there, because I'm a C++ fanboy and want Qt to stay relevant. We all want that. That's why we have begun to improve the docs, at your recommendation, with regard to using QVector over QList. This QString/QStringView discussion is a similar situation, but with each reply from you, I become more confused about what problem you are trying to solve with QStringView. You listed most of the Qt string handling classes above. I think you are stating or implying that it is a mistake for a user to arbitrarily mix them in an application. But this is easy to do. qdoc uses QString everywhere. It is after all a string manipulation program. We have recently improved qdoc performance by quite a lot, both by changing its strategy and by improving its use of string classes. I am sure there is more to be done, because qdoc mixes all the Qt string classes. That's easy to do because in a routine that is exclusively QString based, you find you need to call a function that returns a QByteArray or that only accepts a parameter that is const char*. Is QStringView meant to avoid this by adding APIs to Qt classes that use QStringView? Is the goal to include a QStringView API everywhere and then always pass around QStringViews instead of combinations of the string handling classes? martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Saturday, October 17, 2015 10:13 PM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Saturday 17 October 2015 15:51:35 Smith Martin wrote: > >Please understand my POV: I am of the firm belief that Qt has no business > >creating inefficiencies for its users by sloppy implementation. Anywhere. > > I think you are overreacting here, way too much. You have discovered a more > efficient way to handle strings, but that doesn't mean Trolltech created > inefficiency by a sloppy implementation of QString. QString has been used > productively by many users for a long time. The fact that a moere > efficient implementation exists doesn't mean the current implementation is > inefficient. I have not discovered anything. I merely use the existing tools and found that APIs outside the core Qt string classes largely ignore them, and probably rightfully so, since there's a combinatorical explosion when you combine QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc. QStringView is not more efficient than any of those. It is an abstraction over them, so that you can use, say, QStringRef in more places. > >And please don't forget that QString fanboys can continue to massacre the > >heap if they so wish. After all, we've had QStringRef for a long time > >now, but most people still write str.mid(n).toInt() instead of > >str.midRef().toInt().
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Hi Martin After watching https://youtu.be/H9gAaNRoon4 I got a much better picture of string view. I really recommend to watch it. My understanding is that qstringview would be a qt implementation of string view and the conversation cost would be zero. I find that very promising because it sould be very easy to use every string manipulation which is written for utf16 string view or qstringview with no cost and quite conveniently. So it really improves interoperability. :-) There are other advantages of string view. For example instead to a pointer of characters and size in a function signature it could be better statically analyzed. I really think we should embrace it more openly. Sent from cellphone ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 19:42:27 Bubke Marco wrote: > > Last time I profiled Qt Creator startup and parsing of projects, the two > > most expensive calls in QtCore were qHash and the SHA1 calculator. The > > former I've already fixed. The latter I was hoping that some colleagues > > would fix by creating an optimised library[1] we'd use but they refused > > to add SHA1 support to it. > > Thiago, we will be using clang as the new code model and in my experiences > it is quite slow so we have to hide the latency. This is not only clang but > same parsing of the results too where we heavily use a QByteArray wrapper. > If I have time I will try out QValArray but it is not so urgent. I think > QtCreator is an example that string operations matter because every latency > as you type is quite unpleasant. Roberto's parser didn't use QByteArray either and performance with Creator was acceptable. If you replace that with the Clang parser and that's slower, we conclude that the fault lies with Clang. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 08:06:33 Marc Mutz wrote: > > The word 'fanboys' disturbs me (I know you don't mean it that way) because > > there are no 'hard' numbers on how 'bad' the current situation really is. > > It would really be helpful to have an idea how 'real-world' applications > > suffer from the current implementation of QString. That would give a very > > good context to decide how Qt can help to leverage these problems. > > If you want real-world numbers, follow the optimisations Milian did in moc. > IIRC, he documented the speedup.They are not (all) string-related, but I > also don't claim that QStringView will single-handedly make your apps go > faster. I wrote about QList already. There's more where these topics came > from, but essentially they all boil down to: "minimise allocations". On the other hand, note how real world numbers show that QtGui and QtWidgets consume a lot more CPU time than the combined time spent in malloc and QString operations. Last time I profiled Qt Creator startup and parsing of projects, the two most expensive calls in QtCore were qHash and the SHA1 calculator. The former I've already fixed. The latter I was hoping that some colleagues would fix by creating an optimised library[1] we'd use but they refused to add SHA1 support to it. [1] https://github.com/01org/tinycrypt -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Hi Marc They thread mostly concentrated on performance but what about statical analysis? string_view and array_view are specifically designed for that. Maybe you mention it already but how is QStringView working for it. Sent from cellphone ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
>Please understand my POV: I am of the firm belief that Qt has no business >creating inefficiencies for its users by sloppy implementation. Anywhere. I think you are overreacting here, way too much. You have discovered a more efficient way to handle strings, but that doesn't mean Trolltech created inefficiency by a sloppy implementation of QString. QString has been used productively by many users for a long time. The fact that a moere efficient implementation exists doesn't mean the current implementation is inefficient. >And please don't forget that QString fanboys can continue to massacre the heap >if they so wish. After all, we've had QStringRef for a long time now, but most >people still write str.mid(n).toInt() instead of str.midRef().toInt(). But that is because we don't explain in the documentation the most efficient ways to use these classes together. And the reason it isn't in the documentation is we who write the documentation don't know what you know. >I find it intolerable that todays software >takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs >done in less wallclock time than software 20 years ago. But the fact is, it usually IS tolerable, because the applications you are talking about mostly work in human time. Your efforts to improve Qt are most appreciated, but you make it sound like we should publicly name and shame all the engineers who ever worked on QString but who failed to find the holy grail of the most efficient implementation. I had no idea my heap was in such pain. martin From: development-bounces+martin.smith=theqtcompany@qt-project.org <development-bounces+martin.smith=theqtcompany@qt-project.org> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Saturday, October 17, 2015 1:18 AM To: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings Kurt, The mail you're replying to has nothing to do with QStringView. It was a tangent about std::string_view. If you need a real-world example of where QStringView would be handy: Some 3rd-party code returns you a char16_t *path, and you want to perform a QDir::cd(). Currently, you need to create a QString (which allocates). Had QDir::cd() taken a QStringView, you wouldn't have that allocation. Another example I already mentioned: > E.g. local QStrings could be replaced by QVarLengthArray, > > e.g. in code like this: > QString joinstr = QLatin1String("\"\n"); > joinstr += indent; > joinstr += indent; > joinstr += QLatin1Char('"'); > > QString rc(QLatin1Char('"')); > rc += result.join(joinstr); > rc += QLatin1Char('"'); > > Even when replaced with QStringBuilder: > QString joinstr = QLatin1String("\"\n") + indent + indent > >+ QLatin1Char('"'); > > QString rc = QLatin1Char('"') + result.join(joinstr) + > QLatin1Char('"'); > > But the main benefit would be that we only ever need one overload for a > QString-taking function, not N, N > 3. Final with QStringView (and a minor modification to QStringBuilder): QVarLengthArray joinstr // doesn't alloc (for most 'indent's) = QLatin1String("\"\n") + indent + indent + QLatin1Char('"'); // in this line, QStringList::join() doesn't know about QVarLengthArray. // but it takes a QStringView and therefore transparently accepts QVLA QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"'); Please understand my POV: I am of the firm belief that Qt has no business creating inefficiencies for its users by sloppy implementation. Anywhere. We cannot know whether one user is calling QString::multiArg() in a tight loop. It might be dominating her runtime. 99.9% of users probably couldn't care less about QString::multiArg() performance. But we need to care about the 0.1%, because that number is much closer to 100% if you integrate it over all users' critical paths (iow: every Qt user will have *some* part of Qt that she wished was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at some point in the past!). The inefficiencies I (and Sergio and Milian and, I'm happy to see, Friedemann, and others) care about are not ones you will find in a profiler in those 99.9% of cases, because they are not used in a critical path. But even for the 99.9% of users, if we make all of Qt 10% faster on average, their apps will run close to 10% faster, too. Automatically. So yes, I care about performance. I find it intolerable that todays software takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs done in less wallclock time than software 20 years ago. And people don't use C++ because it's pretty. You cannot compete with Java on pro
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 15:51:35 Smith Martin wrote: > >Please understand my POV: I am of the firm belief that Qt has no business > >creating inefficiencies for its users by sloppy implementation. Anywhere. > > I think you are overreacting here, way too much. You have discovered a more > efficient way to handle strings, but that doesn't mean Trolltech created > inefficiency by a sloppy implementation of QString. QString has been used > productively by many users for a long time. The fact that a moere > efficient implementation exists doesn't mean the current implementation is > inefficient. I have not discovered anything. I merely use the existing tools and found that APIs outside the core Qt string classes largely ignore them, and probably rightfully so, since there's a combinatorical explosion when you combine QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc. QStringView is not more efficient than any of those. It is an abstraction over them, so that you can use, say, QStringRef in more places. > >And please don't forget that QString fanboys can continue to massacre the > >heap if they so wish. After all, we've had QStringRef for a long time > >now, but most people still write str.mid(n).toInt() instead of > >str.midRef().toInt(). > > But that is because we don't explain in the documentation the most > efficient ways to use these classes together. And the reason it isn't in > the documentation is we who write the documentation don't know what you > know. Then documenters should ask. There's a class QStringRef. If you/they don't understand what the difference is to QString and why it exists, just ask. > >I find it intolerable that todays software > >takes 1000x the memory, 1000x the CPU capacity and doesn't get a given > >jobs done in less wallclock time than software 20 years ago. > > But the fact is, it usually IS tolerable, because the applications you are > talking about mostly work in human time. Again, if it _were_ tolerable, we would all be using Java by now. > Your efforts to improve Qt are most appreciated, but you make it sound like > we should publicly name and shame all the engineers who ever worked on > QString but who failed to find the holy grail of the most efficient > implementation. Erm, no, that's not what I'm saying. I'm said we shouldn't use sloppy implementation because people here claimed it was ok until proven otherwise. I merely pointed out (with many words) that that is not how C++ libraries are supposed to work. You can profile an application and you can profile a function. You cannot profile a library. What TT created is magnificient. It greatly eased my learning of C++. But it needs to evolve to stay relevant, because hardware doesn't stay the same. In the 2000s C++ was seen as a dying language by many. It is understandable that TT picked concepts from Java, the then-favourite language. But now C++ developers have gained back their self-esteem, because the industry has realised that they absolutely _need_ the efficiency of C++. Java doesn't cut it. We no longer look up to Java and C#. When Qt was created, and for a long time afterwards, CoW was a good optimisation. When multithreading came along, it ceased to be. That is now universally understood, except in Qt where things are sometimes done because they have been done that way since Qt 1 or 2. I try to shake thing up here and there, because I'm a C++ fanboy and want Qt to stay relevant. > I had no idea my heap was in such pain. Play around with heaptrack. And watch Milian's talks at QtWS once they become available. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 17, 2015 19:22:55 Thiago Macieirawrote: > On Saturday 17 October 2015 08:06:33 Marc Mutz wrote: >> > The word 'fanboys' disturbs me (I know you don't mean it that way) because >> > there are no 'hard' numbers on how 'bad' the current situation really is. >> > It would really be helpful to have an idea how 'real-world' applications >> > suffer from the current implementation of QString. That would give a very >> > good context to decide how Qt can help to leverage these problems. >> >> If you want real-world numbers, follow the optimisations Milian did in moc. >> IIRC, he documented the speedup.They are not (all) string-related, but I >> also don't claim that QStringView will single-handedly make your apps go >> faster. I wrote about QList already. There's more where these topics came >> from, but essentially they all boil down to: "minimise allocations". > > On the other hand, note how real world numbers show that QtGui and QtWidgets > consume a lot more CPU time than the combined time spent in malloc and > QString > operations. > > Last time I profiled Qt Creator startup and parsing of projects, the two most > expensive calls in QtCore were qHash and the SHA1 calculator. The former I've > already fixed. The latter I was hoping that some colleagues would fix by > creating an optimised library[1] we'd use but they refused to add SHA1 > support > to it. Thiago, we will be using clang as the new code model and in my experiences it is quite slow so we have to hide the latency. This is not only clang but same parsing of the results too where we heavily use a QByteArray wrapper. If I have time I will try out QValArray but it is not so urgent. I think QtCreator is an example that string operations matter because every latency as you type is quite unpleasant. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
2015-10-17 8:58 GMT+04:00 Thiago Macieira: > On Saturday 17 October 2015 07:14:58 Konstantin Ritt wrote: > > 2015-10-17 6:23 GMT+04:00 Thiago Macieira : > > > On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote: > > > > - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 > } */ > > > > > > - > > > > > > > explicitly means "no owning, deep-copy when necessary" > > > > > > And when is it necessary? > > > > In a given example, left() derives the original data, even when it isn't > > shared, whilst leftRef() produces a non-shared QString, just like > proposed > > QStringView. > > You missed the point. > > The whole problem of QString::fromRawData is that the method you called may > store the QString and thus keep referencing the string you had, even past > the > point where your string changed. > > In fact, that happens with QStringLiteral too. If you use QStringLiteral > in a > plugin and the plugin gets unloaded, the application may crash. > Indeed, it is the same issue. QStringView doesn't solve it either. So the best thing we can do, IMO, is to give the user a tool and describe how to use it properly to get some extra performance, on his own risk. > The whole point here was "deep-copy when necessary": when is it necessary? > If > we can't solve the two problems above, we are going to make the problem > worse. We could probably introduce a QString::ensureSharable() for the advanced user's goals and maybe call it in the copy constructor and the assignment operator to avoid storing a data we do not own. The latter would potentially kill all the extra performance we got in a first place, though. In any way, QString::fromRawData() currently has the issue you've described above, so I have some doubts we'll make the problem even worse. Regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 17, 2015 21:05:29 Marc Mutzwrote: > On Saturday 17 October 2015 15:51:35 Smith Martin wrote: >> >Please understand my POV: I am of the firm belief that Qt has no business >> >creating inefficiencies for its users by sloppy implementation. Anywhere. >> >> I think you are overreacting here, way too much. You have discovered a more >> efficient way to handle strings, but that doesn't mean Trolltech created >> inefficiency by a sloppy implementation of QString. QString has been used >> productively by many users for a long time. The fact that a moere >> efficient implementation exists doesn't mean the current implementation is >> inefficient. > > I have not discovered anything. I merely use the existing tools and found > that > APIs outside the core Qt string classes largely ignore them, and probably > rightfully so, since there's a combinatorical explosion when you combine > QLatin1String, QString, QStringRef, QChar, const char*, QByteArray, etc. > > QStringView is not more efficient than any of those. It is an abstraction > over > them, so that you can use, say, QStringRef in more places. > >> >And please don't forget that QString fanboys can continue to massacre the >> >heap if they so wish. After all, we've had QStringRef for a long time >> >now, but most people still write str.mid(n).toInt() instead of >> >str.midRef().toInt(). >> >> But that is because we don't explain in the documentation the most >> efficient ways to use these classes together. And the reason it isn't in >> the documentation is we who write the documentation don't know what you >> know. > > Then documenters should ask. There's a class QStringRef. If you/they don't > understand what the difference is to QString and why it exists, just ask. > >> >I find it intolerable that todays software >> >takes 1000x the memory, 1000x the CPU capacity and doesn't get a given >> >jobs done in less wallclock time than software 20 years ago. >> >> But the fact is, it usually IS tolerable, because the applications you are >> talking about mostly work in human time. > > Again, if it _were_ tolerable, we would all be using Java by now. > >> Your efforts to improve Qt are most appreciated, but you make it sound like >> we should publicly name and shame all the engineers who ever worked on >> QString but who failed to find the holy grail of the most efficient >> implementation. > > Erm, no, that's not what I'm saying. I'm said we shouldn't use sloppy > implementation because people here claimed it was ok until proven otherwise. > I > merely pointed out (with many words) that that is not how C++ libraries are > supposed to work. You can profile an application and you can profile a > function. > You cannot profile a library. > > What TT created is magnificient. It greatly eased my learning of C++. But it > needs to evolve to stay relevant, because hardware doesn't stay the same. > > In the 2000s C++ was seen as a dying language by many. It is understandable > that TT picked concepts from Java, the then-favourite language. But now C++ > developers have gained back their self-esteem, because the industry has > realised that they absolutely _need_ the efficiency of C++. Java doesn't cut > it. > We no longer look up to Java and C#. > > When Qt was created, and for a long time afterwards, CoW was a good > optimisation. When multithreading came along, it ceased to be. That is now > universally understood, except in Qt where things are sometimes done because > they have been done that way since Qt 1 or 2. I try to shake thing up here > and > there, because I'm a C++ fanboy and want Qt to stay relevant. Yes but you cannot change everything in one day. You are referencing to something like http://www.gotw.ca/publications/optimizations.htm? I still believe sharing mutable data between threads is not a good idea. In most cases you start to sterilize your threads. Working on the same cache line is not an good idea. I think coping or slicing data between different threads is much smarter. In that sense functional programming is much easier to program as you write your own loop. So I am not sure that our current CoW story is the answer. Not because it is inefficient but because it can lead to inefficient systems because it makes sharing so easy. But before we add new concepts we should agree in which direction we want to go. I have the feeling that we build an airplan but some want a jumbo jet, some a jet fighter and some, lets describe it so, they want to fly to the moon because earth will maybe burning down and build something completely different which doesn't looks like an airplane anymore which we built over many years. I don't think we will be succed before we find a common context. So it would be good to describe what is our story. >> I had no idea my heap was in such pain. > > Play around with heaptrack. And watch Milian's talks at QtWS once they become > available. > > Thanks, > Marc
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 17, 2015 15:52:08 Smith Martinwrote: >>Please understand my POV: I am of the firm belief that Qt has no business >>creating inefficiencies for its users by sloppy implementation. Anywhere. > > I think you are overreacting here, way too much. You have discovered a more > efficient way to handle strings, but that doesn't mean Trolltech created > inefficiency by a sloppy implementation of QString. QString has been used > productively by many users for a long time. The fact that a moere efficient > implementation exists doesn't mean the current implementation is inefficient. QString makes heap allocation and has atomic pointers. I think it was designed for convenience and not for maximum speed. That's okay because Qt is mostly a GUI and not text processing library. >>I find it intolerable that todays software >>takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs >>done in less wallclock time than software 20 years ago. > > But the fact is, it usually IS tolerable, because the applications you are > talking about mostly work in human time. > > Your efforts to improve Qt are most appreciated, but you make it sound like > we should publicly name and shame all the engineers who ever worked on > QString but who failed to find the holy grail of the most efficient > implementation. Like I wrote they were very understandable design decisions. Anyway I think a professional developer knows that he should has distance to his code so that he get criticism about it not personal. Understanding criticism and argumentation as a process to write code is, I think, the key to get a good developer. So why they should be feel ashamed if something better comes up. This is called progress und it happens all the time. I think I would be feel ashamed if I would fight other code because of personal reason. Sometimes I feel that way but I hopefully not stick to it. ;-) After reading some proposals about string view it has to my understanding this properties: 1. Makes string usage between different implementation more efficient 2. Improve static analysis - makes ownership clear 3. Gives the different string implementation a common ground You can use easily use a Qt function without creating a QString which can be a big advantage if performance matters. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 19:03:18 Marc Mutz wrote: > The problem is that I feel we can't wait for string_view because we already > drown in QString equivalents. Plus, we'd get the usual replies if anyone > suggested to use std::string_view as such a fundamental Qt type. Naturally, > QStringView would trivially convert to and from std::u16string_view with no > runtime overhead (unless compiler writers ruin the concept of string_view > by implementing it as (T*, size_t) in one compiler and (T*, T*) in > another) The conversion from one to the other is one instruction. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 19:31:50 Thiago Macieira wrote: > The conversion from one to the other is one instruction. Which is one instruction too much for tail-call optimisation, e.g. But the point was about std::string_view binary compatibility. As basically a C struct with no ownership semantics, it would be a perfect vehicle to pass strings around different programming languages and compilers, but only if the ABI (ie. the data members) were specified. -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Marc, It is clear that your main concern is performance (needless conversions) and convenience (being able to work efficiently with 3rd party libraries). Regarding performance, I think it would be good if we could come up with some numbers. How 'bad' is the current implementation compared to an 'ideal' situation? And then define some acceptable target keeping into account convenience. Otherwise I am afraid that discussions can keep going forever over 1 instruction more or less. Regarding convenience, and that is what Qt is all about, I think you made a tremendous proposal that can really bring Qt a big step forward. The idea is very sound and clear. Of course, there are technical challenges, but that is what we are supposed to solve. I also agree that we should aim for simplicity. QString and QStringView is simple to understand, iff we can get rid of all other string related classes. The ideal case would be just to have one QString class, although I doubt that is achievable. I think it would be good to come up with some real-world use-cases that show how a Qt user and a Qt application could benefit from your proposal. Personally, I never experienced a performance problem with QString; but then the applications I write are not very string oriented. I am a QString aficionado :-) After all, what I really like about Qt is convenience, simplicity and portability. I think this should be the main focus of Qt, and that also implies smooth collaboration with the outside world. Cheers, Kurt > On 16 Oct 2015, at 21:47, Marc Mutzwrote: > >> On Friday 16 October 2015 19:31:50 Thiago Macieira wrote: >> The conversion from one to the other is one instruction. > > Which is one instruction too much for tail-call optimisation, e.g. > > But the point was about std::string_view binary compatibility. As basically a > C struct with no ownership semantics, it would be a perfect vehicle to pass > strings around different programming languages and compilers, but only if the > ABI (ie. the data members) were specified. > > -- > Marc Mutz | Senior Software Engineer > KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company > Tel: +49-30-521325470 > KDAB - The Qt Experts > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Then after we add QStringView, we will have QString for containing a string and QStringView for passing its contents out of Qt? martin From: m...@kdab.com <m...@kdab.com> on behalf of Marc Mutz <marc.m...@kdab.com> Sent: Friday, October 16, 2015 10:07 AM To: Smith Martin Cc: development@qt-project.org Subject: Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings On Friday 16 October 2015 07:59:40 Smith Martin wrote: > Can you refocus the discussion for everyone? For me at least? > > QString is my favorite class of all time. I use it every day in every > program; it always works, and I never make a mistake. Then you can continue to be blissfully ignorant of reality as you are :) If you are a QString fanboy and don't care about 3rd-party code and/or efficiency, then you can just continue as before. > 1. What is the problem with QString that QStringView will solve? TL;DR: Inefficient conversion from/to QString, leading to inefficiencies within pure Qt-code and in particular when interfacing with 3rd-party code, BiC issues, > 2. Why can't QString be fixed? Because it is a container, not an interface type. An interface type needs to be BC across most of compilers and languages (incl. C and - potentially Java). A container needs to handle (as in own) memory. An interface type (quite obviously, if it's to work with C, too) cannot. > 3. What is a string view? A pimped { QChar *begin, QChar *end; } Thanks, Marc -- Marc Mutz <marc.m...@kdab.com> | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
> On Oct 16, 2015, at 9:10 AM, Poenitz Andre> wrote: > > > Marc Mutz wrote: >>> I think too we should embrace the standard library more and don't replicate >>> their features. >> >> So you think that QStringView is too experimental and _at the same time_ >> replicating the standard. Sounds paradoxal to me. > > It's not paradoxical at all. It would be a new implementation with the usual > need for maturing even if it implements a well-known standard concepts. > > And there is indeed room for actual experiments, e.g. when it comes > structure layout. Pointer+int offset is not the only possible choice. > > You cannot do that in Qt proper, due to BC promises. And there would also be some room in Qt Creator to experiment with and gather experience on how all this fits with corresponding API changes, which I also do not see how that could be done in Qt without lots of #ifdefs. > Compile-time opt-in > is a way to exempt it from BC, but then your main reason to have it in Qt > and not in Creator, namely 'more widespread use', is void, as barely anyone > will opt in. You'll get more public exposure (with less risk!) if it's used by > default in Creator code for a while before the then-matured implementation > goes to the library. > > Andre' > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development -- Eike Ziller, Senior Software Engineer - The Qt Company GmbH The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 16, 2015 08:53:33 Marc Mutzwrote: > On Friday 16 October 2015 01:32:26 Bubke Marco wrote: >> On October 16, 2015 00:20:22 Marc Mutz wrote: >> > Guys, this thread is for QStringView. Could we keep it on-topic, please? >> > There are more than enough bits floating around to create your own >> > threads (with a tip of the hat to Kai). >> >> Good argument but actually I think before we introduce something new to our >> string handling we should test it out. Why not add it Qt creator like >> André proposed and see how it works. > > I have already answered why I think this is a bad idea. Yes, I read it. That why I proposed the experimental playground. It is not so much about QStringView but about the following changes. What is so bad about to play around features before we see all the implications. My experiences whisper to me that if I like an idea too much I should be careful because I love to ignore unpleasant side effects. >> I think we don't want to end with something like our model view system or >> QtControls. Lets test it before we make changes. >> >> I think too we should embrace the standard library more and don't replicate >> their features. > > So you think that QStringView is too experimental and _at the same time_ > replicating the standard. Sounds paradoxal to me. Is it already in the standard? The standard has an experimental stage too. And I like this concept very much. >> A better process to add features would be helpful too. First they should >> be experimental so we can change them easily. Second we should be better >> at removing features. If we do not remove things we will getting slower >> and slower to add new interesting stuff. It is hard to find the balance >> but if you are too conservative you will getting slowly less used. We >> tried to be very innovative with Qml and we learned much about it. >> >> So the question is how can we maximize the usefulness of Qt with our man >> power. Is replicating the standard library really helping? > > My stance on replicating std functionality and the NIH syndrome should by now > be above suspicion, even for the casual reader. It was not about you or me. It was about us an organization. How we can work better together. How can we make a difference, how can we help other developers. > I have already argued why I think QStringView is needed, but QArrayView is > not. Sorry if you got the impression it was about you. I should be more careful with my arguments. I try to disconnect arguments from the self. I thinking the argument should stand for himself in his context and not so much by the person who speaks it. To make it clear, I find QStringView promising but still we should explore the concept with all their implications more. Actually I think we sould add more of this kind of changes in an experimental stage like #include . So we are forced to evaluate for every release how it works out and if it looks well we move it to the normal include path. So experimental would be an area in every repository so that everybody could try it out. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 01:32:26 Bubke Marco wrote: > On October 16, 2015 00:20:22 Marc Mutzwrote: > > Guys, this thread is for QStringView. Could we keep it on-topic, please? > > There are more than enough bits floating around to create your own > > threads (with a tip of the hat to Kai). > > Good argument but actually I think before we introduce something new to our > string handling we should test it out. Why not add it Qt creator like > André proposed and see how it works. I have already answered why I think this is a bad idea. > I think we don't want to end with something like our model view system or > QtControls. Lets test it before we make changes. > > I think too we should embrace the standard library more and don't replicate > their features. So you think that QStringView is too experimental and _at the same time_ replicating the standard. Sounds paradoxal to me. > A better process to add features would be helpful too. First they should > be experimental so we can change them easily. Second we should be better > at removing features. If we do not remove things we will getting slower > and slower to add new interesting stuff. It is hard to find the balance > but if you are too conservative you will getting slowly less used. We > tried to be very innovative with Qml and we learned much about it. > > So the question is how can we maximize the usefulness of Qt with our man > power. Is replicating the standard library really helping? My stance on replicating std functionality and the NIH syndrome should by now be above suspicion, even for the casual reader. I have already argued why I think QStringView is needed, but QArrayView is not. Thanks, Marc -- Marc Mutz | Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Marc Mutz wrote: > > I think too we should embrace the standard library more and don't replicate > > their features. > > So you think that QStringView is too experimental and _at the same time_ > replicating the standard. Sounds paradoxal to me. It's not paradoxical at all. It would be a new implementation with the usual need for maturing even if it implements a well-known standard concepts. And there is indeed room for actual experiments, e.g. when it comes structure layout. Pointer+int offset is not the only possible choice. You cannot do that in Qt proper, due to BC promises. Compile-time opt-in is a way to exempt it from BC, but then your main reason to have it in Qt and not in Creator, namely 'more widespread use', is void, as barely anyone will opt in. You'll get more public exposure (with less risk!) if it's used by default in Creator code for a while before the then-matured implementation goes to the library. Andre' ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
> > > You're misrepresenting the argument. QString doesn't support other > encodings > > because UTF-16 is the best for the task at hand and we have too much > legacy to > > support. Because of that, QCollator only supports UTF-16. > > I buy the legacy argument but I don't buy that utf 16 is the best > encoding. I think because of legacy we cannot change QString but I think we > should still support UTF 8 better. > So what? This doesn't change the fact UTF-16 has advantages over UTF-8 and thus QString's backing storage is UTF-16 - and won't be changed to anything else. Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Back to the topic. Marc, what usecases of QStringView are not covered with QString { ushort* data; int size; QStringData *refcount; } Not sure i understand that "interface" terminology. Would it be possible to pass QStringView to other (non-qt) APIs? ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 09:53:56 Smith Martin wrote: > Then after we add QStringView, we will have QString for containing a string > and QStringView for passing its contents out of Qt? Yes, except for "out of Qt". We already have - within Qt - lots of equivalent ways to provide a QChar-based string - and QString is only one (QStringRef and QVarLengthArray are others, as is QChar[]). As I said before, this is very similar to Q(Persistent)ModelIndex - QMI is the interface type, and it's transparent to the compiler and thus efficient to work with. QPMI is the storage type, intransparent to the compiler and very slow to work with. HTH, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 13:28:57 Иван Комиссаров wrote: > Back to the topic. > Marc, what usecases of QStringView are not covered with QString { ushort* > data; int size; QStringData *refcount; } > Not sure i understand that "interface" terminology. Would it be possible to > pass QStringView to other (non-qt) APIs? See my mail to Martin today. While not the first reason to create a QStringView, yes, a (ushort*, ushort*) can allow zero-copy string transfer between at least C, C++ and Java, possibly also JavaScript. So it may have a direct benefit for QML, though I don't know that code at all. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
I guess it's not by incidence that there's also a std::string_view coming, see e.g. https://www.youtube.com/watch?v=H9gAaNRoon4 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3762.html So, is this really the same, except non-templated and for QString (with conversion from/to Qt types)? Or do you have other differences in mind? Regards Kai > -Original Message- > From: development-bounces+kai.koehne=theqtcompany@qt-project.org > [mailto:development-bounces+kai.koehne=theqtcompany@qt-project.org] > On Behalf Of Marc Mutz > Sent: Tuesday, October 13, 2015 10:47 PM > To: development@qt-project.org > Subject: [Development] RFC: Proposal for a semi-radical change in Qt APIs > taking strings > > Hi, > > After looking quite a bit into the current state of string handling in Qt for > my > QtWS talk last week, I have become frustrated by the state of string handling > in > Qt. > > We have such powerful tools for string handling (QStringRef, QStringBuilder), > but all APIs outside QString and its immediate surroundings only deal in > QString. > The correct way would be to overload every function taking QString with > QLatin1String and QStringRef versions, and then, for some other rare cases, > const QChar *, int size. Let alone std::basic_string. > > I would therefore like to propose to abandon QString for new API (and over > time > phase it out of existing API), and only provide (const QChar*, size_t) as the > most > general form. I would propose to package the two into a class, called > - you guessed it - QStringView. > > =FAQ= > > Q: Why not just use QStringRef? > > A: QStringRef is tied to QString. E.g. you can't create a QStringRef from a > pair of > QChar*, int. It also is kind of stuck in historic mistakes making it > undesireable as > a cheap-to-pass parameter type. > > Q: What mistakes do you refer to? > > A: The fact that it has copy ctor and assignment operator, so it's not a > trivally- > copyable type and thus cannot efficiently passed by-value. It may also be too > large for pass-by-value due to the rather useless QString pointer (should have > been QStringData*, if any). Neither can be fixed before Qt 6. > > Q: Why size_t? > > A: The intent of QStringView (and std::experimental::string_view) is to act > as an > interface between modules written with different compilers and different > flags. > A std::string will never be compatible between compilers or even just > different > flags, but a simple struct {char*, size_t} will always be, by way of it's C > compatibility. > > So the goal is not just to accept QString, QStringRef, and (QChar*,int) (and > QVarLengthArray!) as input to QStringView, but also > std::basic_string and std::vector. > > Q: What about the plans to make QString UTF-8-backed? > > A: QStringView-using code will need to be ported just as QString-using code > will. > > Q: What future do you have in mind for QStringRef? > > A: None in particular, though I have found a need for an owning QStringRef in > some places. But I expect Qt 6' QString to be able to provide a restricted > view on > shared data, such that it would subsume QStringRef completely. > > Q: What about QLatin1String? > > A: Once QString is backed by UTF-8, latin-1 ceases to be a special charset. We > might want something like QUsAsciiString, but it would just be a UTF-8 > string, so > it could be packed into QStringView. > > Q: What about QByteArray, QVector? > > A: I'm unsure about QByteArrayView. It might not pull its weight compared to > std::(experimental::)string_view, but I also note that we're currently > missing a > QByteArrayRef, so a QBAView might make sense while we wait for the std one > to become available to us. > > I'm actively opposed to a QArrayView, because I don't think it provides us > with > anything std::(experimental::)array_view doesn't already. > > Q: What about a rope? > > A: A rope is a more complex string that can provide complex views on existing > data as well as store rules for generating stretches of data (as opposed to > the > data itself). > > A rope is a very complex data structure and would not work as a universal > interface type. It would be cool if Qt had a rope, but that is outside the > scope of > my proposal. > > Q: What do you mean when you say "abandon QString"? > > A: I mean that functions should not take QStrings as arguments, but > QStringViews. Then users can transparently pass QString, QStringRef and any of > a number of other "string" types without overloading the function on each of > them. > > I do not mean to abandon QString, the class. Only QString, the interface type. > > Q: What API should QStringView have? > > A: Since it's mainly an interface type, it should have implicit conversions > from all > kinds of "string" types, but explicit conversion _to_ those string types. It > should > carry all the API from QString that can be implemented on just a (QChar*, > size_t) > (e.g. trimmed(),
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 17:03:41 Koehne Kai wrote: > I guess it's not by incidence that there's also a std::string_view coming, > see e.g. > > https://www.youtube.com/watch?v=H9gAaNRoon4 > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3762.html > > So, is this really the same, except non-templated and for QString (with > conversion from/to Qt types)? Or do you have other differences in mind? Nothing qualitatively different. Just std::basic_string_view with a subset of QString API, yes. The problem is that I feel we can't wait for string_view because we already drown in QString equivalents. Plus, we'd get the usual replies if anyone suggested to use std::string_view as such a fundamental Qt type. Naturally, QStringView would trivially convert to and from std::u16string_view with no runtime overhead (unless compiler writers ruin the concept of string_view by implementing it as (T*, size_t) in one compiler and (T*, T*) in another) Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 01:18:06 Marc Mutz wrote: > If you need a real-world example of where QStringView would be handy: Some > 3rd-party code returns you a char16_t *path, and you want to perform a > QDir::cd(). Currently, you need to create a QString (which allocates). Had > QDir::cd() taken a QStringView, you wouldn't have that allocation. Bad example. You can use QString::fromRawData. While that today allocates the d pointer, for Qt 6 it wouldn't (null d pointer) and there would be no code change for the user. There is, however, the danger of fromRawData: you need to be sure the called function won't try to store the QString. With a QStringView or the unsharable QString that Brano proposed in the other email, it would be clear that a deep copy needs to happen. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Thiago described Qt6 QString as { QArrayData *d; ushort* b; qsize size }. That is pretty close to QStringView, just extra *d makes it slightly bigger. This is how I understand this class: QString will have (typical) case when b points into d. QString own one reference in d in this case. QString will have special case when b points to text section and d does not own anything. QString will have special case for SSO. Can QString have special case for not owned, non persistent b pointer (non-owning / view mode)? Non-owning QString can be created with special static QString method. It stays non-owning only while being passed through const &. Code that cares about keeping QString in view mode, must stick to const QString & all the time. Copy assignment on non-owning QString results in owning QString - it triggers deep copy into newly created shared data. It solves the queued connection problem that existed with QStringView class. I suppose that queued connection internally performs QString copy. I have no idea about performance implications for code paths that do not care about this use case. But having just one string class would be very convenient. Kind regards Brano ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Friday 16 October 2015 22:08:59 Branislav Katreniak wrote: > Thiago described Qt6 QString as { QArrayData *d; ushort* b; qsize size }. > That is pretty close to QStringView, just extra *d makes it slightly bigger. > > This is how I understand this class: > QString will have (typical) case when b points into d. QString own one > reference in d in this case. > QString will have special case when b points to text section and d does not > own anything. > QString will have special case for SSO. > > Can QString have special case for not owned, non persistent b pointer > (non-owning / view mode)? How is that different from your second case? If it doesn't own, it doesn't need a d pointer. > Non-owning QString can be created with special static QString method. It > stays non-owning only while being passed through const &. Code that cares > about keeping QString in view mode, must stick to const QString & all the > time. Copy assignment on non-owning QString results in owning QString - it > triggers deep copy into newly created shared data. Ok, I understand the difference, but why do you want this behaviour? BTW, this is the unsharable strings that we're getting rid of. This would make sense to view a string of ephemeral data. But if the data is ephemeral, why can't we use the refcounter? In other words, if someone owns this data, why isn't that QString? > It solves the queued connection problem that existed with QStringView > class. I suppose that queued connection internally performs QString copy. True, but I think that having QString share refcount of substrings solves the problem that QStringView was proposed for in the first place. > I have no idea about performance implications for code paths that do not > care about this use case. But having just one string class would be very > convenient. That is the problem. That's the reason we're trying to get rid of unsharable containers: performance impact on the code paths that don't need it, which are the majority of cases. There are two possibilities for the unsharable flag: 1) it's inside the d pointer, which means the d pointer must be non-null. That means even the non-owning QString needs to allocate memory for the d pointer block. 2) it's outside the d pointer, which means that a) it doesn't need to allocate memory b) we need to extend QString to be: { QArrayData *d; ushort *b; qsize size; int flags; } (24 bytes if qsize is 32-bit on 64-bit platforms; 32 if it is 64-bit; always 16 bytes on 32-bit platforms) Either way, the existence of the flag means that every QString copy-assignment needs to check this flag, instead of simply incrementing the refcount if d is not null and being done with it. That means code bloat for the deep copy showing up everywhere. I really don't want the unsharable flag. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Kurt, The mail you're replying to has nothing to do with QStringView. It was a tangent about std::string_view. If you need a real-world example of where QStringView would be handy: Some 3rd-party code returns you a char16_t *path, and you want to perform a QDir::cd(). Currently, you need to create a QString (which allocates). Had QDir::cd() taken a QStringView, you wouldn't have that allocation. Another example I already mentioned: > E.g. local QStrings could be replaced by QVarLengthArray, > > e.g. in code like this: > QString joinstr = QLatin1String("\"\n"); > joinstr += indent; > joinstr += indent; > joinstr += QLatin1Char('"'); > > QString rc(QLatin1Char('"')); > rc += result.join(joinstr); > rc += QLatin1Char('"'); > > Even when replaced with QStringBuilder: > QString joinstr = QLatin1String("\"\n") + indent + indent > >+ QLatin1Char('"'); > > QString rc = QLatin1Char('"') + result.join(joinstr) + > QLatin1Char('"'); > > But the main benefit would be that we only ever need one overload for a > QString-taking function, not N, N > 3. Final with QStringView (and a minor modification to QStringBuilder): QVarLengthArray joinstr // doesn't alloc (for most 'indent's) = QLatin1String("\"\n") + indent + indent + QLatin1Char('"'); // in this line, QStringList::join() doesn't know about QVarLengthArray. // but it takes a QStringView and therefore transparently accepts QVLA QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"'); Please understand my POV: I am of the firm belief that Qt has no business creating inefficiencies for its users by sloppy implementation. Anywhere. We cannot know whether one user is calling QString::multiArg() in a tight loop. It might be dominating her runtime. 99.9% of users probably couldn't care less about QString::multiArg() performance. But we need to care about the 0.1%, because that number is much closer to 100% if you integrate it over all users' critical paths (iow: every Qt user will have *some* part of Qt that she wished was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at some point in the past!). The inefficiencies I (and Sergio and Milian and, I'm happy to see, Friedemann, and others) care about are not ones you will find in a profiler in those 99.9% of cases, because they are not used in a critical path. But even for the 99.9% of users, if we make all of Qt 10% faster on average, their apps will run close to 10% faster, too. Automatically. So yes, I care about performance. I find it intolerable that todays software takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs done in less wallclock time than software 20 years ago. And people don't use C++ because it's pretty. You cannot compete with Java on programmer efficiency. Its standard library is orders of magnitude larger than C++'s. People choose C++ (and accept the associated loss in programmer efficiency), because they require the bare metal efficiency of C++. And if Qt gets in the way, they will look for alternatives and find them in places that make Qt look like paradise. That said, QStringView is as much about - slim interface type vs. thick, fat, storage type - interop with 3rd-party and native code than it is about efficiency. Bottomline: I don't need a fancy anylysis to tell me that less allocations = faster programs = more happy Qt users. And please don't forget that QString fanboys can continue to massacre the heap if they so wish. After all, we've had QStringRef for a long time now, but most people still write str.mid(n).toInt() instead of str.midRef().toInt(). QStringView won't change that: A function taking QStringView accepts a QString, too (either by implicit conversion, or where it makes sence, by QString overload). Thanks, Marc On Friday 16 October 2015 21:42:23 Kurt Pattyn wrote: > Marc, > > It is clear that your main concern is performance (needless conversions) > and convenience (being able to work efficiently with 3rd party libraries). > Regarding performance, I think it would be good if we could come up with > some numbers. How 'bad' is the current implementation compared to an > 'ideal' situation? And then define some acceptable target keeping into > account convenience. Otherwise I am afraid that discussions can keep going > forever over 1 instruction more or less. Regarding convenience, and that > is what Qt is all about, I think you made a tremendous proposal that can > really bring Qt a big step forward. The idea is very sound and clear. Of > course, there are technical challenges, but that is what we are supposed > to solve. > > I also agree that we should aim for simplicity. QString and QStringView is > simple to understand, iff we can get rid of all other string related > classes. The ideal case would be just to have one QString class, although > I doubt that is achievable. > > I think it
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
If QString in Qt6 could be { QArrayData *d; ushort *b; int s }, then it supersedes QStringView in all aspects: - QString(u"hello world", 5) /* { d=.., b=.., s=5 } */ - still has a superpower of COW, etc - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */ - explicitly means "no owning, deep-copy when necessary" then we could get rid of QStringRef (or typedef it to QString) and do something like: private: QString(QArrayData *dd, ushort *unicode, int size) { if (dd) { dd->ref(); } d = dd; b = unicode; s = size; } /* not that trivial, of course, just to describe the idea */ public: QString QString::left(int n) const { return QString(d, b, n); } QString QString::leftRef(int n) const { return QString(nullptr, b, n); } > That said, QStringView is as much about > > - slim interface type vs. thick, fat, storage type > - interop with 3rd-party and native code So as QString now... i.e. DWORD dwBufferSize = 0; if (::GetUserProfileDirectory(token, NULL, ) == 0 && dwBufferSize != 0) { QVarLengthArray userDirectory(dwBufferSize); if (::GetUserProfileDirectory(token, userDirectory.data(), ) == 0) { QString dirName = QString::fromRawData(userDirectory.constData(), dwBufferSize); // still unsafe, but it is clearly documented for years and nothing has really changed for its users anyways return QDir().exists( dirName.rightRef(dirName.lastIndexOf(QLatin1Char('\\'; } } Regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
> On 17 Oct 2015, at 01:18, Marc Mutzwrote: > > Kurt, > > The mail you're replying to has nothing to do with QStringView. It was a > tangent about std::string_view. Yes, but this thread is diverging on a lot of different things than the original, genuine intent you have. So, please forgive me on replying on a 'side-branch' :-) > > If you need a real-world example of where QStringView would be handy: Some > 3rd-party code returns you a char16_t *path, and you want to perform a > QDir::cd(). Currently, you need to create a QString (which allocates). Had > QDir::cd() taken a QStringView, you wouldn't have that allocation. > > Another example I already mentioned: > >> E.g. local QStrings could be replaced by QVarLengthArray, >> >> e.g. in code like this: >>QString joinstr = QLatin1String("\"\n"); >>joinstr += indent; >>joinstr += indent; >>joinstr += QLatin1Char('"'); >> >>QString rc(QLatin1Char('"')); >>rc += result.join(joinstr); >>rc += QLatin1Char('"'); >> >> Even when replaced with QStringBuilder: >>QString joinstr = QLatin1String("\"\n") + indent + indent >> >> + QLatin1Char('"'); >> >>QString rc = QLatin1Char('"') + result.join(joinstr) + >>QLatin1Char('"'); >> >> But the main benefit would be that we only ever need one overload for a >> QString-taking function, not N, N > 3. > > Final with QStringView (and a minor modification to QStringBuilder): > > QVarLengthArray joinstr // doesn't alloc (for most 'indent's) > = QLatin1String("\"\n") + indent + indent + QLatin1Char('"'); > > // in this line, QStringList::join() doesn't know about QVarLengthArray. > // but it takes a QStringView and therefore transparently accepts QVLA > QString rc = QLatin1Char('"') + result.join(joinstr) + QLatin1Char('"'); > > Please understand my POV: I am of the firm belief that Qt has no business > creating inefficiencies for its users by sloppy implementation. Anywhere. We > cannot know whether one user is calling QString::multiArg() in a tight loop. > It might be dominating her runtime. 99.9% of users probably couldn't care > less > about QString::multiArg() performance. But we need to care about the 0.1%, > because that number is much closer to 100% if you integrate it over all > users' > critical paths (iow: every Qt user will have *some* part of Qt that she > wished > was faster, heck, iirc KMail ran into QObject::connect as a bottleneck at > some > point in the past!). Don't misunderstand me, I completely understand your POV. You are right: Qt IS about convenience and performance. Only, if I see the 'heated' discussions, it would be convenient if some baseline could be defined. > > The inefficiencies I (and Sergio and Milian and, I'm happy to see, > Friedemann, > and others) care about are not ones you will find in a profiler in those > 99.9% > of cases, because they are not used in a critical path. But even for the > 99.9% > of users, if we make all of Qt 10% faster on average, their apps will run > close to 10% faster, too. Automatically. > > So yes, I care about performance. I find it intolerable that todays software > takes 1000x the memory, 1000x the CPU capacity and doesn't get a given jobs > done in less wallclock time than software 20 years ago. These are 'numbers'. Is it like that? Really 1000x the memory, 1000x the CPU capacity? I tend to say 'yes you are right', but then we still need some concrete measurements. > And people don't use > C++ because it's pretty. You cannot compete with Java on programmer > efficiency. > Its standard library is orders of magnitude larger than C++'s. People choose > C++ (and accept the associated loss in programmer efficiency), because they > require the bare metal efficiency of C++. And if Qt gets in the way, they > will > look for alternatives and find them in places that make Qt look like paradise. But Qt IS like paradise :-) > > That said, QStringView is as much about > > - slim interface type vs. thick, fat, storage type > - interop with 3rd-party and native code > > than it is about efficiency. > > Bottomline: I don't need a fancy anylysis to tell me that less allocations = > faster programs = more happy Qt users. But to what extent? If I run my application on a workstation, the effect is negligible, as my application has more to do than handling strings. If I have an embedded application, every piece of performance gain could benefit, but my experience tells me (for MY use cases) that the graphics stack is more a bottleneck than string handling. > > And please don't forget that QString fanboys can continue to massacre the > heap > if they so wish. After all, we've had QStringRef for a long time now, but > most > people still write str.mid(n).toInt() instead of str.midRef().toInt(). > QStringView won't change that: A function taking QStringView accepts a > QString, too (either by implicit conversion, or where
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote: > - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */ - > explicitly means "no owning, deep-copy when necessary" And when is it necessary? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 01:52:27 Kurt Pattyn wrote: > > Bottomline: I don't need a fancy anylysis to tell me that less > > allocations = faster programs = more happy Qt users. > > But to what extent? If I run my application on a workstation, the effect is > negligible, as my application has more to do than handling strings. If I > have an embedded application, every piece of performance gain could > benefit, but my experience tells me (for MY use cases) that the graphics > stack is more a bottleneck than string handling. I object to the notion that a desktop would not benefit from more efficient string handling. Hardware doesn't get faster the way it used to. If the speed of C++ wasn't needed, your workstation would only run Java apps (and Sun tried, but failed). > > > > And please don't forget that QString fanboys can continue to massacre the > > heap if they so wish. After all, we've had QStringRef for a long time > > now, but most people still write str.mid(n).toInt() instead of > > str.midRef().toInt(). QStringView won't change that: A function taking > > QStringView accepts a QString, too (either by implicit conversion, or > > where it makes sence, by QString overload). > > The word 'fanboys' disturbs me (I know you don't mean it that way) because > there are no 'hard' numbers on how 'bad' the current situation really is. > It would really be helpful to have an idea how 'real-world' applications > suffer from the current implementation of QString. That would give a very > good context to decide how Qt can help to leverage these problems. If you want real-world numbers, follow the optimisations Milian did in moc. IIRC, he documented the speedup.They are not (all) string-related, but I also don't claim that QStringView will single-handedly make your apps go faster. I wrote about QList already. There's more where these topics came from, but essentially they all boil down to: "minimise allocations". > That said, I like the idea of QStringView. Thanks, Marc -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Saturday 17 October 2015 07:14:58 Konstantin Ritt wrote: > 2015-10-17 6:23 GMT+04:00 Thiago Macieira: > > On Saturday 17 October 2015 03:34:51 Konstantin Ritt wrote: > > > - QString::fromRawData(u"hello world", 5) /* { d=nullptr, b=.., s=5 } */ > > > > - > > > > > explicitly means "no owning, deep-copy when necessary" > > > > And when is it necessary? > > In a given example, left() derives the original data, even when it isn't > shared, whilst leftRef() produces a non-shared QString, just like proposed > QStringView. You missed the point. The whole problem of QString::fromRawData is that the method you called may store the QString and thus keep referencing the string you had, even past the point where your string changed. In fact, that happens with QStringLiteral too. If you use QStringLiteral in a plugin and the plugin gets unloaded, the application may crash. The whole point here was "deep-copy when necessary": when is it necessary? If we can't solve the two problems above, we are going to make the problem worse. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
2015-10-15 11:00 GMT+03:00 Bubke Marco: > On October 15, 2015 08:45:30 Knoll Lars > wrote: > > > On 14/10/15 23:51, "Bubke Marco" wrote: > > > >>On October 14, 2015 23:10:26 Thiago Macieira > >>wrote: > >>> Qt does not have to provide a comparator that operates on something > >>>other than > >>> its native string type. > >> > >>Isn't Qt a framework to help developers? Sorry your argumentation is > >>sounds not very empirical. > > > > Of course our aim should be to help developers. But there will always be > > some use cases which we will not cover. The question is whether this is > > one of them or not. > > Most file and network content is in utf 8, databases too. It has simply a > size and performance advantage for most cases. You have not so many cases > where you have pure Chinese signs in an text. Mostly it is an mixture. In > Linux, which is very important in embedded, utf 8 dominates ťhe APIs. Ask > your self if we don't want support that. We could start simply and expand > slowly. If the standard library would support utf 8 collations on all > platforms very well we could skip it but today you have to do your own > solutions again and again. > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel free to compare their complexity against UTF-32). And why "pure Chinese signs" again? Did you ever look into the Unicode's Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all spoken languages, without any performance hits (in compare to UTF-8), and all we have to pay is an extra byte per every Base Latin character (in compare to UTF-8, again). [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt Regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
> On Oct 14, 2015, at 5:14 PM, Matthew Woehlkewrote: > > On 2015-10-14 10:30, André Somers wrote: >> Op 14-10-2015 om 15:59 schreef Matthew Woehlke: >>> STL should change. In Qt and Python, you can use negative indices to >>> refer to a distance (length) relative to the end (length) of the string. >>> In STL you can't do that, which is a significant limitation by >>> comparison. Please don't drop this useful functionality! >> >> I'm not so sure anymore. Do you really think that for instance passing >> in a negative _from_ in QString::indexOf to search from the back of the >> string is intuitive API? I don't. > > Huh? Of course it is. > > s.indexOf('c', 5); // find 'c', forward, starting at offset 5 > s.indexOf('c', -5); // find 'c', forward, starting at offset N-5 So from where does s.indexOf(‘c’, i-2) search? This is similar to integer overflow, and I think utilizing that in an API leads to less readable and potentially unexpectedly behaving code. Anyhow this seems to be only vaguely related to the things that are discussed in this thread. Br, Eike > A negative offset -K is exactly the same as N + 1 - K (N = length of > string). It just saves having to write that out yourself. It *doesn't* > change the behavior of the function. (I think you are confusing > tail-relative offset with reverse operation, which is totally different > and orthogonal.) > > Even STL supports this, partially, for -1; string::npos is generally > equivalent to -1 in Qt. > > Bah. Okay, apparently Qt actually *doesn't* support tail-relative, but > just treats n<0 like string::npos. That could be improved for Qt 6 > though, but only if n is signed. > > -- > Matthew > > ___ > Development mailing list > Development@qt-project.org > http://lists.qt-project.org/mailman/listinfo/development -- Eike Ziller, Senior Software Engineer - The Qt Company GmbH The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Geschäftsführer: Mika Pälsi, Juha Varelius, Tuula Haataja Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Thursday 15 October 2015 06:05:34 Thiago Macieira wrote: > On Thursday 15 October 2015 02:22:50 Marc Mutz wrote: > > On Thursday 15 October 2015 00:27:14 Thiago Macieira wrote: > > > Way too much code would break if we did that because we allow people > > > access > > > to the data pointer in QString and to iterate directly > > > (std::{,w,u16}string don't allow that, which makes parsing them > > > actually a lot more cumbersome). > > > > > > > > Just chiming in to say: It does: > > http://en.cppreference.com/w/cpp/string/basic_string/data > > Ah, right. The mutable pointer is the one missing... char *data = &*str.begin(); There might not be explicit API for it, but it's not forbidden to use that idiom (if !str.empty(), of course). -- Marc Mutz| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 14/10/15 23:51, "Bubke Marco"wrote: >On October 14, 2015 23:10:26 Thiago Macieira >wrote: > >> On Wednesday 14 October 2015 20:52:12 Bubke Marco wrote: >>> On October 14, 2015 22:13:11 Thiago Macieira >>> >> wrote: >>> And I don't want an utf 8 baked >>> QString. For my use cases implicit sharing is overkill. Move semantics >>> would be enough. I want localAwareCompare(const char *s1, const char >>>*s2). >> >> Do it on your own. You just said that ICU has the function you want, so >>use >> it. > >So Qt is always shipping with ICU? No, we wanted to do this at some point, but it turns out that it’s not possible to rely on it on all platforms. > > >> Qt does not have to provide a comparator that operates on something >>other than >> its native string type. > >Isn't Qt a framework to help developers? Sorry your argumentation is >sounds not very empirical. Of course our aim should be to help developers. But there will always be some use cases which we will not cover. The question is whether this is one of them or not. > > >> >>> Maybe windows and mac os will bring support to the standard library so >>>we >>> don't need it but in the mean time it would be very helpful. >>> >>> A utf 8 based QTextDocument would be maybe nice too. >> >> What for? It needs to keep a lot of extra structures, so the cost of >> conversion and extra memory is minimal. And besides, QTextDocument >>really >> needs a seekable string, not UTF-8. > >Is UTF 16 seekable? You still have surrogates and you can merge merge >code points. For the most parts. When it comes to positioning cursors inside the text, you’ll always need to take care of complex text layouting, diacritics and (in the case of utf16) surrogates. Still, a lot of the seeking is probably easier with utf16 than with utf8. > > >Lets describe an example. I send the QTextDocument content to an library >which expect utf8 content and gives me back positions. This gets >interesting if you use non >ASCII signs. Actually the new clang code model works that way. We also have the opposite case, where we need to send utf16 to a 3rd party or system library, and get back positions. Unfortunately, not all APIs take the same encoding. > >> >> Even if we provide UTF-8 support classes, those will not propagate to >>the GUI. >> Forget it. > >What about compressing UTF 16 like python is doing it for UTF 32. If you >are only using ascii you set a flag and you can remove all that useless >zeros. It would be have implications for data() but maybe we should not >provide access to the internal representation. If you use UTF 32 as a >base you don't need anymore surrogates. That’s back to a mixed representation in QString. I personally think that combines the worst of both worlds. Cheers, Lars ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 15, 2015 08:45:30 Knoll Larswrote: > On 14/10/15 23:51, "Bubke Marco" wrote: > >>On October 14, 2015 23:10:26 Thiago Macieira >>wrote: >> >>> On Wednesday 14 October 2015 20:52:12 Bubke Marco wrote: On October 14, 2015 22:13:11 Thiago Macieira >>> wrote: And I don't want an utf 8 baked QString. For my use cases implicit sharing is overkill. Move semantics would be enough. I want localAwareCompare(const char *s1, const char *s2). >>> >>> Do it on your own. You just said that ICU has the function you want, so >>>use >>> it. >> >>So Qt is always shipping with ICU? > > No, we wanted to do this at some point, but it turns out that it’s not > possible to rely on it on all platforms. Is there an other utf 8 backend on Windows? I heard that Microsoft is using it now on some plattforms too. >> >> >>> Qt does not have to provide a comparator that operates on something >>>other than >>> its native string type. >> >>Isn't Qt a framework to help developers? Sorry your argumentation is >>sounds not very empirical. > > Of course our aim should be to help developers. But there will always be > some use cases which we will not cover. The question is whether this is > one of them or not. Most file and network content is in utf 8, databases too. It has simply a size and performance advantage for most cases. You have not so many cases where you have pure Chinese signs in an text. Mostly it is an mixture. In Linux, which is very important in embedded, utf 8 dominates ťhe APIs. Ask your self if we don't want support that. We could start simply and expand slowly. If the standard library would support utf 8 collations on all platforms very well we could skip it but today you have to do your own solutions again and again. >> >> >>> Maybe windows and mac os will bring support to the standard library so we don't need it but in the mean time it would be very helpful. A utf 8 based QTextDocument would be maybe nice too. >>> >>> What for? It needs to keep a lot of extra structures, so the cost of >>> conversion and extra memory is minimal. And besides, QTextDocument >>>really >>> needs a seekable string, not UTF-8. >> >>Is UTF 16 seekable? You still have surrogates and you can merge merge >>code points. > > For the most parts. When it comes to positioning cursors inside the text, > you’ll always need to take care of complex text layouting, diacritics and > (in the case of utf16) surrogates. Still, a lot of the seeking is probably > easier with utf16 than with utf8. Actually I would like to see measurements. Does anybody have seen anything about it. What is Chrome and Mozilla doing? >> >> >>Lets describe an example. I send the QTextDocument content to an library >>which expect utf8 content and gives me back positions. This gets >>interesting if you use non >>ASCII signs. Actually the new clang code model works that way. > > We also have the opposite case, where we need to send utf16 to a 3rd party > or system library, and get back positions. Unfortunately, not all APIs > take the same encoding. Yes, but in my experience C APIs are using UTF 8 or all UTF standards. There some older which use wchar but that is different anyway. It may be different for Java etc. but I don't think that people who use Qt use that much java APIs. ;-) >> >>> >>> Even if we provide UTF-8 support classes, those will not propagate to >>>the GUI. >>> Forget it. >> >>What about compressing UTF 16 like python is doing it for UTF 32. If you >>are only using ascii you set a flag and you can remove all that useless >>zeros. It would be have implications for data() but maybe we should not >>provide access to the internal representation. If you use UTF 32 as a >>base you don't need anymore surrogates. > > That’s back to a mixed representation in QString. I personally think that > combines the worst of both worlds. > I am not so sure eighter but I like to have an open discussion about it with examples of other use cases. How other programs like mozilla, chrome handling it, what does other languages support. Especially Internet centric solutions handle it would be interesting. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 2015-10-15 02:38, Ziller Eike wrote: > So from where does 's.indexOf(‘c’, i-2)' search? > > This is similar to integer overflow, and I think utilizing that in an > API leads to less readable and potentially unexpectedly behaving > code. It depends on the value of i, of course. And you're not going to get any better results with an unsigned offset. > Anyhow this seems to be only vaguely related to the things that are discussed > in this thread. There was a question whether to use signed or unsigned as the size type, that somewhere turned into Qt's use of signed for string sizes/offsets being inconsistent with STL/CSL. I suppose it has wandered off a bit on a tangent, though. -- Matthew ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 15, 2015 14:53:29 Konstantin Rittwrote: > 2015-10-15 11:00 GMT+03:00 Bubke Marco > >: > On October 15, 2015 08:45:30 Knoll Lars > > wrote: > >> On 14/10/15 23:51, "Bubke Marco" >> > wrote: >> >>>On October 14, 2015 23:10:26 Thiago Macieira >>> > >>>wrote: Qt does not have to provide a comparator that operates on something other than its native string type. >>> >>>Isn't Qt a framework to help developers? Sorry your argumentation is >>>sounds not very empirical. >> >> Of course our aim should be to help developers. But there will always be >> some use cases which we will not cover. The question is whether this is >> one of them or not. > > Most file and network content is in utf 8, databases too. It has simply a > size and performance advantage for most cases. You have not so many cases > where you have pure Chinese signs in an text. Mostly it is an mixture. In > Linux, which is very important in embedded, utf 8 dominates ?he APIs. Ask > your self if we don't want support that. We could start simply and expand > slowly. If the standard library would support utf 8 collations on all > platforms very well we could skip it but today you have to do your own > solutions again and again. > > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel > free to compare their complexity against UTF-32). Do you have mesured it? Please no theoretical discussions. If you have larger texts you have annotations in ascii like XML. > And why "pure Chinese signs" again? Did you ever look into the Unicode's > Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all > spoken languages, without any performance hits (in compare to UTF-8), and all > we have to pay is an extra byte per every Base Latin character (in compare to > UTF-8, again). Okay, again. Most text today is embedded in meta information and this information are very often ascii. Why do you think utf8 is used so widely? ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Thursday 15 October 2015 14:52:46 Konstantin Ritt wrote: > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 (feel > free to compare their complexity against UTF-32). > And why "pure Chinese signs" again? Did you ever look into the Unicode's > Scripts.txt [1], for example? It clearly shows UTF-16 covers [almost] all > spoken languages, without any performance hits (in compare to UTF-8), and > all we have to pay is an extra byte per every Base Latin character (in > compare to UTF-8, again). > > [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt What everyone is referring to here is this: Google html { background: #fff; margin: 0 1em; } body { font: .8125em/1.5 arial, sans-serif; text-align: center; } h1 { font-size: 1.5em; font-weight: normal; margin: 1em 0 0; } p#footer { color: #767676; font-size: .77em; } p#footer a { background: url(//www.google.cn/intl/zh- CN_cn/images/cn_icp.gif) top right no-repeat; padding: 5px 20px 5px 0; } ul { margin: 2em; padding: 0; } li { display: inline; padding: 0 2em; } div { -moz-border-radius: 20px; -webkit-border-radius: 20px; border: 1px solid #ccc; border-radius: 20px; margin: 2em auto 1em; max-width: 650px; min- width: 544px; } div:hover, div:hover * { cursor: pointer; } div:hover { border-color: #999; } div p { margin: .5em 0 1.5em; } img { border: 0; } http://www.google.com.hk/webhp?hl=zh-CNsourceid=cnhp;> http://www.google.com.hk/webhp?hl=zh- CNsourceid=cnhp">google.com.hk 请收藏我们的网址 http://translate.google.cn/?sourceid=cnhp;>翻译 2011 - http://www.miibeian.gov.cn/;>ICP证合字 B2-20070004号 var gcn=gcn|| {};gcn.IS_IMAGES=(/images\.google\.cn/.exec(window.location)|| window.location.hash=='#images'|| window.location.hash=='images');gcn.HOMEPAGE_DEST='http://www.google.com.hk/webhp?hl=zh- CN&sourceid=cnhp';gcn.IMAGES_DEST='http://images.google.com.hk/imghp?'+'hl=zh- CN&sourceid=cnhp';gcn.DEST_URL=gcn.IS_IMAGES?gcn.IMAGES_DEST:gcn.HOMEPAGE_DEST;gcn.READABLE_HOMEPAGE_URL='google.com.hk';gcn.READABLE_IMAGES_URL='images.google.com.hk';gcn.redirectIfLocationHasQueryParams=function() {if(window.location.search&&/google\.cn/.exec(window.location)&&!/webhp/.exec(window.location)) {window.location=String(window.location).replace('google.cn','google.com.hk')}} ();gcn.replaceHrefsWithImagesUrl=function(){if(gcn.IS_IMAGES){var a=document.getElementsByTagName('a');for(var i=0,len=a.length;i| Senior Software Engineer KDAB (Deutschland) GmbH & Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 2015-10-14 18:27, Thiago Macieira wrote: > On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote: >> Is UTF 16 seekable? You still have surrogates and you can merge merge code >> points. > > Seekable enough. It's much easier to deal with than UTF-8. A surrogate pair, > as its name says, appears *only* in pairs, so you always know if you're on > the > first or on the second. Moreover, all living languages are encoded in the > Basic > Multilingual Plane, so no surrogate pairs are required for any of them. > Handling of surrogate pairs can be moved to non-critical codepaths. Oh? -- Matthew ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 15, 2015 15:56:35 Matthew Woehlkewrote: > On 2015-10-15 02:38, Ziller Eike wrote: >> So from where does 's.indexOf(‘c’, i-2)' search? >> >> This is similar to integer overflow, and I think utilizing that in an >> API leads to less readable and potentially unexpectedly behaving >> code. > > It depends on the value of i, of course. And you're not going to get any > better results with an unsigned offset. You could get an assert or warning because you are out of range. >> Anyhow this seems to be only vaguely related to the things that are >> discussed in this thread. > > There was a question whether to use signed or unsigned as the size type, > that somewhere turned into Qt's use of signed for string sizes/offsets > being inconsistent with STL/CSL. I suppose it has wandered off a bit on > a tangent, though. > I would prefer we would be more in line with the standard library. All the casts to silence warnings are not very beautiful. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Op 15-10-2015 om 14:52 schreef Konstantin Ritt: > > > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 > (feel free to compare their complexity against UTF-32). > And why "pure Chinese signs" again? Did you ever look into the > Unicode's Scripts.txt [1], for example? It clearly shows UTF-16 covers > [almost] all spoken languages, without any performance hits (in > compare to UTF-8), and all we have to pay is an extra byte per every > Base Latin character (in compare to UTF-8, again). > > [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt > "All we have to pay"? Isn't that quite a significant cost, if your every other byte in your data is going to be null? Doesn't that impact cache lines? Doesn't that impact how many characters you can stuff into a string with SSO (as planned for Qt6?) and thus when you need to start allocating? I would certainly think that there is an impact on things like XML parsing speed. But true enough, that is without measuring that. I also think it should be the default for saving QStrings to files. Many years ago, I was writing with an application on a very constrained mobile device dealing with maps. Converting the map format to use UTF8 instead of UTF16 for its contained strings (which were many) caused a very significant reduction of file size (impacting the map area the users could take with them on the road) and loading speed (impacting UX very positively). André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On October 16, 2015 00:20:22 Marc Mutzwrote: > Guys, this thread is for QStringView. Could we keep it on-topic, please? > There > are more than enough bits floating around to create your own threads (with a > tip of the hat to Kai). > Good argument but actually I think before we introduce something new to our string handling we should test it out. Why not add it Qt creator like André proposed and see how it works. I think we don't want to end with something like our model view system or QtControls. Lets test it before we make changes. I think too we should embrace the standard library more and don't replicate their features. A better process to add features would be helpful too. First they should be experimental so we can change them easily. Second we should be better at removing features. If we do not remove things we will getting slower and slower to add new interesting stuff. It is hard to find the balance but if you are too conservative you will getting slowly less used. We tried to be very innovative with Qml and we learned much about it. So the question is how can we maximize the usefulness of Qt with our man power. Is replicating the standard library really helping? > > On Thursday 15 October 2015 23:02:09 Bubke Marco wrote: >> On October 15, 2015 00:27:45 Thiago Macieira > wrote: >> > On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote: >> >> On October 14, 2015 23:10:26 Thiago Macieira >> > >> > wrote: >> >> > Do it on your own. You just said that ICU has the function you want, >> >> > so use >> >> > it. >> >> >> >> So Qt is always shipping with ICU? >> > >> > It can be disabled on Windows. On OS X there's no point since it's part >> > of the system. On Linux, if you disable it, you're going to have some >> > other features reduced, so don't disable it. >> > >> >> > Qt does not have to provide a comparator that operates on something >> >> > other than its native string type. >> >> >> >> Isn't Qt a framework to help developers? Sorry your argumentation is >> >> sounds not very empirical. >> > >> > Yes, it is. But Qt's goal is not to support every single use-case and >> > corner- case out there. Qt should make 90% easy and 9% possible. That >> > means there's a 1% of the realm of possibilities that Qt does not >> > address. If your use-case calls into this group, use the fact that Qt is >> > native code and just call other libraries. >> >> Actually I think Qt is not main developing library people use. It is there >> to make the boring stuff easy, to hide the different interfaces between >> different platforms. That is why many people use Qt, they want to have a >> GUI but don't want to invest to much time in it. The interesting stuff >> which is differentiating you from others is mostly home grown in >> connection with much more specialized libraries. And this libraries are >> much more important to the users. So we should support them, their >> interfaces and not force our interfaces on them. How many users use the >> standard library too, especially the new features, why don't we support >> them not much better. Why do we have to reinvent the wheel again and >> again. I know binary compatibility is important for you but is it really >> that important outside of the special linux distribution cocoon. Is it >> important under Windows, is it important under Mac, is it important under >> embedded Linux? I think the advantages are smaller than the drawbacks. >> >> > That's one of the two main advantages of native code. There's no sandbox >> > to escape from. >> > >> > Qt already supports doing locale-aware comparison. We even have a class >> > for it, so it can be done efficiently: QCollator and it supports our >> > native string type (QString). >> >> Do you like to live on a native island? >> >> > Providing extra support for a character encoding that is not what QString >> > uses falls in that 1%. Just use ICU. >> >> You arguments sounds very tautological. Because it is unimportant we don't >> have it a string class for it. It is unimportant because QString is not >> supporting it. >> >> I know you love plationian argumentation but it would be much more >> effective if you would try to get in the context of other and understand >> their arguments in their context. Showing in your own context that their >> arguments "makes no sense" is not very useful. >> >> >> Lets describe an example. I send the QTextDocument content to an library >> >> which expect utf8 content and gives me back positions. This gets >> >> interesting if you use non ASCII signs. Actually the new clang code >> >> model works that way. >> > >> > That example shows how UTF-16 is better. See above on seekability of >> > UTF-16 vs UTF-8. >> > >> > The solution for this is to fix the library to accept UTF-16. When we >> > were doing Qt 5.0, we needed PCRE to support UTF-16. Their developers >> > were very
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Guys, this thread is for QStringView. Could we keep it on-topic, please? There are more than enough bits floating around to create your own threads (with a tip of the hat to Kai). Thanks, Marc On Thursday 15 October 2015 23:02:09 Bubke Marco wrote: > On October 15, 2015 00:27:45 Thiago Macieirawrote: > > On Wednesday 14 October 2015 21:51:23 Bubke Marco wrote: > >> On October 14, 2015 23:10:26 Thiago Macieira > > > > wrote: > >> > Do it on your own. You just said that ICU has the function you want, > >> > so use > >> > it. > >> > >> So Qt is always shipping with ICU? > > > > It can be disabled on Windows. On OS X there's no point since it's part > > of the system. On Linux, if you disable it, you're going to have some > > other features reduced, so don't disable it. > > > >> > Qt does not have to provide a comparator that operates on something > >> > other than its native string type. > >> > >> Isn't Qt a framework to help developers? Sorry your argumentation is > >> sounds not very empirical. > > > > Yes, it is. But Qt's goal is not to support every single use-case and > > corner- case out there. Qt should make 90% easy and 9% possible. That > > means there's a 1% of the realm of possibilities that Qt does not > > address. If your use-case calls into this group, use the fact that Qt is > > native code and just call other libraries. > > Actually I think Qt is not main developing library people use. It is there > to make the boring stuff easy, to hide the different interfaces between > different platforms. That is why many people use Qt, they want to have a > GUI but don't want to invest to much time in it. The interesting stuff > which is differentiating you from others is mostly home grown in > connection with much more specialized libraries. And this libraries are > much more important to the users. So we should support them, their > interfaces and not force our interfaces on them. How many users use the > standard library too, especially the new features, why don't we support > them not much better. Why do we have to reinvent the wheel again and > again. I know binary compatibility is important for you but is it really > that important outside of the special linux distribution cocoon. Is it > important under Windows, is it important under Mac, is it important under > embedded Linux? I think the advantages are smaller than the drawbacks. > > > That's one of the two main advantages of native code. There's no sandbox > > to escape from. > > > > Qt already supports doing locale-aware comparison. We even have a class > > for it, so it can be done efficiently: QCollator and it supports our > > native string type (QString). > > Do you like to live on a native island? > > > Providing extra support for a character encoding that is not what QString > > uses falls in that 1%. Just use ICU. > > You arguments sounds very tautological. Because it is unimportant we don't > have it a string class for it. It is unimportant because QString is not > supporting it. > > I know you love plationian argumentation but it would be much more > effective if you would try to get in the context of other and understand > their arguments in their context. Showing in your own context that their > arguments "makes no sense" is not very useful. > > >> Lets describe an example. I send the QTextDocument content to an library > >> which expect utf8 content and gives me back positions. This gets > >> interesting if you use non ASCII signs. Actually the new clang code > >> model works that way. > > > > That example shows how UTF-16 is better. See above on seekability of > > UTF-16 vs UTF-8. > > > > The solution for this is to fix the library to accept UTF-16. When we > > were doing Qt 5.0, we needed PCRE to support UTF-16. Their developers > > were very welcoming and wrote the version that supports UTF-16, so Qt > > does not need to reallocate. > > You have ever heard of Pippi Longstocking: "Widdiwiddiwitt, we make the > world like we wish it should be. " or how it is translated to english. > You really think that you can force other larger projects to use utf16 > instead of utf8 if it has disadvantages for them. > > And the PCRE is now supporting both at runtime? Especially for large text > it would be very helpful if you don't need to convert them to QString > before you use regular expressions on it. > > >> > Even if we provide UTF-8 support classes, those will not propagate to > >> > the GUI. Forget it. > >> > >> What about compressing UTF 16 like python is doing it for UTF 32. If you > >> are only using ascii you set a flag and you can remove all that useless > >> zeros. It would be have implications for data() but maybe we should not > >> provide access to the internal representation. If you use UTF 32 as a > >> base you don't need anymore surrogates. > > > > That's what Lars called a "hybrid solution" and vetoed. I second that. > > > > Way too
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Thursday 15 October 2015 21:02:09 Bubke Marco wrote: > Actually I think Qt is not main developing library people use. It is there > to make the boring stuff easy, to hide the different interfaces between > different platforms. That is why many people use Qt, they want to have a > GUI but don't want to invest to much time in it. The interesting stuff > which is differentiating you from others is mostly home grown in connection > with much more specialized libraries. And this libraries are much more > important to the users. So we should support them, their interfaces and not > force our interfaces on them. That's a slippery slope. If we have to support every library's interface, we'll have a horrible mish-mash of an API that tries to support everything and doesn't support anything really well. I'd rather we supported the "One Qt Way" really well. Supporting other ways is possible, as long as we don't cause too much maintenance trouble. > How many users use the standard library too, > especially the new features, why don't we support them not much better. Why > do we have to reinvent the wheel again and again. I won't bother repeating the arguments of why we can't use some of the standard library features. And see the discussion on std::chrono as a replacement for QTime(Span). > I know binary > compatibility is important for you but is it really that important outside > of the special linux distribution cocoon. Yes. Lots of users don't recompile the world when Qt issues a new release, but still try to upgrade. And besides, if we have to maintain binary compatibility for 99% of the API anyway because of Linux distributions, then we might as well go the extra mile and keep it for everyone. > Is it important under Windows, > is it important under Mac, is it important under embedded Linux? I think > the advantages are smaller than the drawbacks. I disagree. Not to mention that "embedded Linux" these days is not different from "regular Linux". Even devices with 32 MB of RAM have package managers and install software from a central repository. > > That's one of the two main advantages of native code. There's no sandbox > > to > > escape from. > > > > Qt already supports doing locale-aware comparison. We even have a class > > for > > it, so it can be done efficiently: QCollator and it supports our native > > string type (QString). > > Do you like to live on a native island? Yes. I love writing native code. That's one of the biggest powers of Qt compared to any other cross-platform solution out there. So, yeah, native island is good. Did you mean to ask if I like constraining myself to only Qt-style APIs? The answer is also yes. I hate the Standard Library API because it's confusing (to me, obviously not so much to the people who wrote it), limited in convenience forcing me to write more code than focusing on getting stuff done. I often feel that the Standard Library tries to achieve 100% support of some tiny feature, overengineering it and not focusing on making developers' lives easier. The example is std::chrono. > > Providing extra support for a character encoding that is not what QString > > uses falls in that 1%. Just use ICU. > > You arguments sounds very tautological. Because it is unimportant we don't > have it a string class for it. It is unimportant because QString is not > supporting it. You're misrepresenting the argument. QString doesn't support other encodings because UTF-16 is the best for the task at hand and we have too much legacy to support. Because of that, QCollator only supports UTF-16. > I know you love plationian argumentation but it would be much more effective > if you would try to get in the context of other and understand their > arguments in their context. Showing in your own context that their > arguments "makes no sense" is not very useful. You're dangerously close to attacking me instead of attacking my arguments. > > That example shows how UTF-16 is better. See above on seekability of > > UTF-16 vs UTF-8. > > > > The solution for this is to fix the library to accept UTF-16. When we were > > doing Qt 5.0, we needed PCRE to support UTF-16. Their developers were very > > welcoming and wrote the version that supports UTF-16, so Qt does not need > > to reallocate. > > You have ever heard of Pippi Longstocking: "Widdiwiddiwitt, we make the > world like we wish it should be. " or how it is translated to english. You > really think that you can force other larger projects to use utf16 instead > of utf8 if it has disadvantages for them. And why should we support UTF-8 instead of UTF-16 if it has disadvantages for us? We'll have to agree to disagree with those projects. We've chosen UTF-16 and we're aligned with a lot of other important API. > And the PCRE is now supporting both at runtime? Especially for large text > it would be very helpful if you don't need to convert them to QString > before you use regular expressions on it.
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
2015-10-15 17:52 GMT+03:00 André Somers: > Op 15-10-2015 om 14:52 schreef Konstantin Ritt: > > > > > > For everything but US-ASCII / Latin-1, UTF-8 isn't faster than UTF-16 > > (feel free to compare their complexity against UTF-32). > > And why "pure Chinese signs" again? Did you ever look into the > > Unicode's Scripts.txt [1], for example? It clearly shows UTF-16 covers > > [almost] all spoken languages, without any performance hits (in > > compare to UTF-8), and all we have to pay is an extra byte per every > > Base Latin character (in compare to UTF-8, again). > > > > [1] http://www.unicode.org/Public/8.0.0/ucd/Scripts.txt > > > "All we have to pay"? Isn't that quite a significant cost, if your every > other byte in your data is going to be null? Only for US-ASCII / Latin-1. Regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
Op 14-10-2015 om 15:59 schreef Matthew Woehlke: > >>> Yes, signed please. We can discuss whether it should be 64bit for Qt 6. >> The current std API uses size_t. Do you (= both of you) expect that ever to >> change? If it doesn't, Qt will forever be the odd one out, until we finally >> drop QVector etc for std::vector etc and then porting will be a horror >> because >> of MSVC's annoying warnings. > STL should change. In Qt and Python, you can use negative indices to > refer to a distance (length) relative to the end (length) of the string. > In STL you can't do that, which is a significant limitation by > comparison. Please don't drop this useful functionality! I'm not so sure anymore. Do you really think that for instance passing in a negative _from_ in QString::indexOf to search from the back of the string is intuitive API? I don't. I would rather have a specific indexOfBackwards or something like that. Or one could just use the iterator API with a standard algorithm I guess. André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On 2015-10-14 10:30, André Somers wrote: > Op 14-10-2015 om 15:59 schreef Matthew Woehlke: >> STL should change. In Qt and Python, you can use negative indices to >> refer to a distance (length) relative to the end (length) of the string. >> In STL you can't do that, which is a significant limitation by >> comparison. Please don't drop this useful functionality! > > I'm not so sure anymore. Do you really think that for instance passing > in a negative _from_ in QString::indexOf to search from the back of the > string is intuitive API? I don't. Huh? Of course it is. s.indexOf('c', 5); // find 'c', forward, starting at offset 5 s.indexOf('c', -5); // find 'c', forward, starting at offset N-5 A negative offset -K is exactly the same as N + 1 - K (N = length of string). It just saves having to write that out yourself. It *doesn't* change the behavior of the function. (I think you are confusing tail-relative offset with reverse operation, which is totally different and orthogonal.) Even STL supports this, partially, for -1; string::npos is generally equivalent to -1 in Qt. Bah. Okay, apparently Qt actually *doesn't* support tail-relative, but just treats n<0 like string::npos. That could be improved for Qt 6 though, but only if n is signed. -- Matthew ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Wednesday 14 October 2015 09:59:28 Matthew Woehlke wrote: > On 2015-10-14 06:16, Marc Mutz wrote: > > First, afaiu from what Thiago mentions in reviews, Q6String will have SSO > > (small-string-optimisation) which makes many short strings expensive to > > copy (if you think that copying 24 bytes is slower than upping an atomic > > int through an indirection) or cheap to copy (if you think the opposite). > > In any case, small strings will be very cheap to create (no allocation), > > so for many strings there will be not much difference between passing a > > QStringView or passing a QString. > > Atomic operations are expensive (I think I heard once 'on the order of > 100 instruction cycles', but that's highly apocryphal), mainly I would > guess due to the need to maintain cache coherency. A small copy might > happen entirely in local hot cache. 24 bytes is a whole three registers > on a modern 64-bit machine. That's probably not going to be very slow. This discussion is a red herring. It's not a choice between copying and atomically incrementing a reference counter. It's a choice between copying and between copying plus incrementing the reference counter. QString s2 = s1; needs to copy those bytes that are sizeof(QString) *anyway*, regardless of whether in addition to that it will increment the refcount. > >> Yes, signed please. We can discuss whether it should be 64bit for Qt 6. > > > > The current std API uses size_t. Do you (= both of you) expect that ever > > to > > change? If it doesn't, Qt will forever be the odd one out, until we > > finally > > drop QVector etc for std::vector etc and then porting will be a horror > > because of MSVC's annoying warnings. > > STL should change. In Qt and Python, you can use negative indices to > refer to a distance (length) relative to the end (length) of the string. > In STL you can't do that, which is a significant limitation by > comparison. Please don't drop this useful functionality! And see discussions in the std-discussions and std-proposals mailing list. I repeat what I said: the current *committee* stance is that you should use signed for everything, except when you need well-defined overflow behaviour. The only problem, which I raised there and did not get resolved, is which type to use. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Wednesday 14 October 2015 08:51:11 Thiago Macieira wrote: > The separation of the string itself from the size and the d pointer allows > the compiler, if it wants to, to share strings. In fact, disassembly of > > f(QStringLiteral("foo"), QStringLiteral("foo")) > > produces one copy of u"foo" only. Let me expand on this. Current Qt5 QStringLiteral("foo") produces a data block of size sizeof(QArrayData) + sizeof(u"foo") = 24 + 8 = 32 bytes and the fact that QStringLiterals don't share will cause the innocent-looking above code require 64 bytes of read-only data. My current code expands to 8 bytes of read-only data, at the expense of a little more code. Current code: leaq_ZZZ1fvENKUlvE0_clEvE15qstring_literal(%rip), %rax movq%rax, 16(%rsp) leaq_ZZZ1fvENKUlvE_clEvE15qstring_literal(%rip), %rax movq%rax, (%rsp) ; followed by the call: movq%rsp, %rdi leaq16(%rsp), %rsi call_Z1fRK7QStringS1_@PLT My code: leaq.LC0(%rip), %rcx; u"foo" movq_ZN10QArrayData18shared_static_dataE@GOTPCREL(%rip), %rax movq%rcx, 40(%rsp) movl$3, 48(%rsp) movq%rcx, 8(%rsp) movq%rax, 32(%rsp) movq%rax, (%rsp) movl$3, 16(%rsp) ; the call itself is unchanged: movq%rsp, %rdi leaq32(%rsp), %rsi call_Z1fRK7QStringS1_@PLT -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] RFC: Proposal for a semi-radical change in Qt APIs taking strings
On Wednesday 14 October 2015 12:16:47 Marc Mutz wrote: > > >But as a condition to be even considered, it needs to be only for the > > >methods > > >that do not hold a copy of the string. That is, methods that immediately > > >consume the string and no longer need to reference its contents. > > Thiago, I think it would help the discussion if you quickly summarised your > planned changes to QString in Qt 6. > > AFAIK, the size and offset will move into the object, so I expected that > Q6String would subsume QStringRef, because each QString could provide a > separate view on the shared underlying data. I also was led to believe that > Q6String would use SSO, which, given its inceased sizeof(), would make a > lot of sense, imo. Indeed, that's the biggest gain. QString will contain a QStringPrivate, which is struct QStringPrivate { QArrayData *d; ushort *b; qsize size; // let's bikeshed what qsize is later }; My current code initialises a QStringLiteral like so: # define QStringLiteral(str) \ ([]() -> QString { \ QStringPrivate holder = { \ QArrayData::sharedStatic(), \ reinterpret_cast(const_cast(QT_UNICODE_LITERAL(str))), \ sizeof(QT_UNICODE_LITERAL(str))/2 - 1 }; \ return QString(holder); \ }()) \ The separation of the string itself from the size and the d pointer allows the compiler, if it wants to, to share strings. In fact, disassembly of f(QStringLiteral("foo"), QStringLiteral("foo")) produces one copy of u"foo" only. Like you said, QString can become its own QStringView/QStringRef/QSubString. QString::left/mid/right can simply copy the d pointer, increment the refcount, then adjust b and size. This solves the issue I had with your proposal: passing a QStringView to a method that decides to copy it, so it wouldn't participate in reference counting. The drawback with this is the pathological case where a short substring is holding a large data block hostage. My next objective, not yet achieved due to lack of time, is to make that QArrayData::sharedStatic() actually be a null pointer. That is, for anything that we didn't allocate memory for, the d pointer should be null. That implies a much faster loading of constant QStringLiterals and much faster handling of the decrement case. The biggest pain point in the code above in my current version is what happens after the call to f(): the compiler generates 2x bit testing of d->flags and calls to QArrayData::deallocate(), which are dead code and will never be run. After that, implement SSO, which should hold 11 UTF-16 characters, including the null terminator. If we benchmark and find that we could use more, we can simply artificially increase sizeof(QString) to 32, which may have some extra benefits of its own, including the fact that the 24-byte short QString will be at odds with the null d pointer -- the if (d) check instead becomes if (quintptr(d) & ~quintptr(1)) [also note how the order of the members in QStringPrivate needs to change for big-endian architectures] [and note everything I say about QString also applies to QByteArray and QVector] > And then I thought, QString would be converted to hold UTF-8. I saw > wip/qstring-utf8 fly by on gerrit, but ok, that hasn't received any updates > since 2012. That was when we converted the QString methods taking const char* from Latin1 to UTF-8. The backing store has never changed. My version of QString stores an extra flag that indicates whether the string is US-ASCII, in which case we can run the unchecked to-Latin1 algorithm in both toLatin1 and toUtf8. Another idea I had but haven't investigated is to cache that result, which requires the returned QByteArray to share the d pointer with the QString. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development