Re: [Development] Important recent changes in QList/QString/QByteArray

Cc: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Thu, 10 Sep 2020 at 16:04, Andrei Golubev wrote: > > Should I expect to do a reserve call with a *smaller* value than my > current size is before removing elements from &

Re: [Development] Important recent changes in QList/QString/QByteArray

On 10/09/2020 15:38, Ville Voutilainen wrote: The "flag" is basically "a call to reserve was made"? How do I clear the flag? Yes (CapacityReserved, in Qt 5). squeeze() / shrink_to_fit() were supposed to clear it. HTH, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software

Re: [Development] Important recent changes in QList/QString/QByteArray

On Thu, 10 Sep 2020 at 16:04, Andrei Golubev wrote: > > Should I expect to do a reserve call with a *smaller* value than my > current size is before removing elements from > a container, in order to set a "target size"? For any standard > container, such a reserve is a complete noop because > it

Re: [Development] Important recent changes in QList/QString/QByteArray

nt: Thursday, September 10, 2020 3:43 PM To: Andrei Golubev Cc: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Thu, 10 Sep 2020 at 11:46, Andrei Golubev wrote: > > That's interesting. So the container remembers what sort

Re: [Development] Important recent changes in QList/QString/QByteArray

On Thu, 10 Sep 2020 at 11:46, Andrei Golubev wrote: > > That's interesting. So the container remembers what sort of a reserve > request I made on it, and uses > that as the preferred size whenever the element count of the container > changes? > > Yes. Calling reserve typically means that you do

Re: [Development] Important recent changes in QList/QString/QByteArray

lopment@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray Il 10/09/20 08:48, Andrei Golubev ha scritto: > > That's the dilemma that Andrei was talking about. Either solution is > valid and > both have a way for you to tell QList to do

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 10/09/20 08:48, Andrei Golubev ha scritto: That's the dilemma that Andrei was talking about. Either solution is valid and both have a way for you to tell QList to do what you want. Oh, but it's not a question of "if". It is already done in latest dev this way, the shrinking

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 10/09/20 04:31, Thiago Macieira ha scritto: This is another against automatic shrinking; it may invalidate everything rather than just the erased area. I don't understand the "detach" part, is it about holding references across a detach, so they refer to the original container, not the

Re: [Development] Important recent changes in QList/QString/QByteArray

changes in QList/QString/QByteArray On Wed, 9 Sep 2020 at 16:38, Andrei Golubev wrote: > > I don't understand what this means. Am I supposed to reserve a > container to its current size before erasing elements > from it, if I don't want the erase to shrink it? > > Yes. That

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wed, 9 Sep 2020 at 16:38, Andrei Golubev wrote: > > I don't understand what this means. Am I supposed to reserve a > container to its current size before erasing elements > from it, if I don't want the erase to shrink it? > > Yes. That's interesting. So the container remembers what sort of a

Re: [Development] Important recent changes in QList/QString/QByteArray

t sure who was an author of the original change). -- Best Regards, Andrei From: Development on behalf of Thiago Macieira Sent: Thursday, September 10, 2020 5:33 AM To: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/

Re: [Development] Important recent changes in QList/QString/QByteArray

Le mer. 9 sept. 2020 à 15:38, Andrei Golubev a écrit : > > I don't understand what this means. Am I supposed to reserve a > container to its current size before erasing elements > from it, if I don't want the erase to shrink it? > > Yes. For what it's worth, as a user, I think this would violate

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wednesday, 9 September 2020 06:38:01 PDT Andrei Golubev wrote: > > I don't understand what this means. Am I supposed to reserve a > > container to its current size before erasing elements > > from it, if I don't want the erase to shrink it? > Yes. If we implement this behaviour. That's the

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wednesday, 9 September 2020 02:17:52 PDT Giuseppe D'Angelo via Development wrote: > > *- except when detach is necessary or shrinking happens > > This is another against automatic shrinking; it may invalidate > everything rather than just the erased area. I don't understand the > "detach"

Re: [Development] Important recent changes in QList/QString/QByteArray

To: Andrei Golubev Cc: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Wed, 9 Sep 2020 at 11:58, Andrei Golubev wrote: > On the other hand, "please do not free memory, I still need it" use-case is > also justified.

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wed, 9 Sep 2020 at 11:58, Andrei Golubev wrote: > On the other hand, "please do not free memory, I still need it" use-case is > also justified. However, chances are that when you really need a certain > memory to be allocated/preserved, there is a call to QList::reserve() prior > to

Re: [Development] Important recent changes in QList/QString/QByteArray

ay, September 9, 2020 12:17 PM To: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On 09/09/2020 10:55, Andrei Golubev wrote: > Small update on QList::erase: the iterator invalidation model will be > aligned with std::vector's erase (in pro

Re: [Development] Important recent changes in QList/QString/QByteArray

On 09/09/2020 10:55, Andrei Golubev wrote: Small update on QList::erase: the iterator invalidation model will be aligned with std::vector's erase (in progress of merging the change to dev and it got approved already). This means that, when erasing, only the iterators that correspond to erased

Re: [Development] Important recent changes in QList/QString/QByteArray

Best Regards, Andrei From: Development on behalf of Иван Комиссаров Sent: Thursday, September 3, 2020 2:34 PM To: Jaroslaw Kobus Cc: Qt development mailing list Subject: Re: [Development] Important recent changes in QList/QString/QByteArray In the awful world of the standard library we call

Re: [Development] Important recent changes in QList/QString/QByteArray

л(а): > >> >> From: Development on behalf of Giuseppe >> D'Angelo via Development >> Sent: Wednesday, September 2, 2020 9:37 PM >> To: Andrei Golubev; development@qt-project.org; Ville Voutilainen >> Subject:

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wed, Sep 02, 2020 at 09:37:49PM +0200, Giuseppe D'Angelo via Development wrote: > On 02/09/2020 21:18, Andrei Golubev wrote: > > Also not sure whether it is an implementation detail or the behavior > > that should always be anticipated. > > People build performance sensitive code assuming the

Re: [Development] Important recent changes in QList/QString/QByteArray

On 02/09/2020 21:18, Andrei Golubev wrote: Also not sure whether it is an implementation detail or the behavior that should always be anticipated. People build performance sensitive code assuming the cost of certain operations -- like, assuming that erasing elements from a vector never

Re: [Development] Important recent changes in QList/QString/QByteArray

_ From: Development on behalf of Giuseppe D'Angelo via Development Sent: Wednesday, September 2, 2020 8:21 PM To: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray Il 02/09/20 17:38, Andrei Golubev ha scritto: > >

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wed, 2 Sep 2020 at 17:16, Andrei Golubev wrote: > > With this setup, one might be tempted to optimize erasure in the first > half of the container by shifting elements towards the end (rather than > from the end towards the beginning), as it would be cheaper. I guess > that's what's happening

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 02/09/20 17:38, Andrei Golubev ha scritto: But yes, good job bringing this up. In Qt 6 code base we also do the shrinking in erase. So this already is not aligned with std::vector. Even if the capacity has been reserved by the user? Thanks, -- Giuseppe D'Angelo |

Re: [Development] Important recent changes in QList/QString/QByteArray

From: Development on behalf of Thiago Macieira Sent: Wednesday, September 2, 2020 5:37 PM To: development@qt-project.org Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Wednesday, 2 September 2020 05:28:33 PDT Ville Voutilainen wrote

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wednesday, 2 September 2020 05:28:33 PDT Ville Voutilainen wrote: > No. I mean "before", not "after". That's how std::vector works. An > erase is a shrinking operation > that keeps everything before the erased position untouched and > left-shifts everything after it. If you > want a prepend

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wednesday, 2 September 2020 00:39:04 PDT Иван Комиссаров wrote: > If you write a vector and your vector doesn’t do what the standard says (and > cppref literally quotes the paper here > http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2011/n3242.pdf >

Re: [Development] Important recent changes in QList/QString/QByteArray

: Qt development mailing list Subject: Re: [Development] Important recent changes in QList/QString/QByteArray Il 02/09/20 10:04, Ville Voutilainen ha scritto: > Interesting. I'm curious what sort of repacking happens on erase, and why > it wasn't done in such a way that e.g. QVector is 4 po

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 02/09/20 10:04, Ville Voutilainen ha scritto: Interesting. I'm curious what sort of repacking happens on erase, and why it wasn't done in such a way that e.g. QVector is 4 pointers instead of 3, so that the element storage wouldn't necessarily be at the beginning of the allocated block; in

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 02/09/20 15:32, Dmitriy Purgin ha scritto: As far as I remember, until recently, for years we were told in this mailing list to move away from QList and use QVector as default, because QList will be deprecated/removed in Qt 6 anyway. Search in this ML's archives. The short story: * QList

Re: [Development] Important recent changes in QList/QString/QByteArray

Again, any exceptions should be documented. Please do not assume any >> specific behavior otherwise. >> >> This seems like a potentially quite significant compatibility break, >> that chopping a vector invalidates all references >> and iterators to parts of the vector that

Re: [Development] Important recent changes in QList/QString/QByteArray

that chopping a vector invalidates all references > and iterators to parts of the vector that the chopping didn't touch before. > > > -- > Best Regards, > Andrei > > ---------- > *From:* Ville Voutilainen > *Sent:* Wednesday, September 2, 2020 3:28 PM &g

Re: [Development] Important recent changes in QList/QString/QByteArray

__ From: Ville Voutilainen Sent: Wednesday, September 2, 2020 3:28 PM To: Andrei Golubev Cc: Giuseppe D'Angelo ; development@qt-project.org ; Ville Voutilainen Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Wed, 2 Sep 2020 at 15:19, Andrei Golubev wrote: &

Re: [Development] Important recent changes in QList/QString/QByteArray

On Wed, 2 Sep 2020 at 15:19, Andrei Golubev wrote: > Ville: > > Interesting. I'm curious what sort of repacking happens on erase > > The iterators before or after may be invalidated. Exactly which of the two > (before or after) is going to happen depends on the position of the > to-be-erased

Re: [Development] Important recent changes in QList/QString/QByteArray

'Angelo Cc: Qt development mailing list Subject: Re: [Development] Important recent changes in QList/QString/QByteArray On Tue, 1 Sep 2020 at 20:44, Giuseppe D'Angelo via Development wrote: > > Il 01/09/20 19:33, Thiago Macieira ha scritto: > > All non-const functions that may detach s

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tue, 1 Sep 2020 at 20:44, Giuseppe D'Angelo via Development wrote: > > Il 01/09/20 19:33, Thiago Macieira ha scritto: > > All non-const functions that may detach should be coded so they DO detach. > > That is, after any and all non-const functions, the refcount of the > > container > > should

Re: [Development] Important recent changes in QList/QString/QByteArray

If you write a vector and your vector doesn’t do what the standard says (and cppref literally quotes the paper here http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2011/n3242.pdf ) than you have to have slightly better

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tuesday, 1 September 2020 13:06:39 PDT Иван Комиссаров wrote: > Sorry, but you’re wrong here. Cppreference clearly says Cppreference is irrelevant. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel DPG Cloud Engineering

Re: [Development] Important recent changes in QList/QString/QByteArray

Sorry, but you’re wrong here. Cppreference clearly says Invalidates iterators and references at or after the point of the erase, including the end() iterator. https://en.cppreference.com/w/cpp/container/vector/erase

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tuesday, 1 September 2020 10:41:33 PDT Giuseppe D'Angelo via Development wrote: > This is the contract, and it's OK. I don't think however that this is > what was intended by OP. Rather, that calling a non-const function may > repack/reallocate a QCC (*), causing invalidation of all references

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tuesday, 1 September 2020 10:46:21 PDT Giuseppe D'Angelo via Development wrote: > Pedantically: if any and all non-const function will invalidate > iterators, then calling any algorithm that takes an iterator range > becomes formally impossible (unless one obtains an iterator range in one >

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 01/09/20 19:31, Thiago Macieira ha scritto: There's no violation. Your code was incorrect in the past, it just happened to work. This isn't true... Assume any and all non-const function will invalidate iterators. ... because this is sketchy. It has never been 100% documented, but

Re: [Development] Important recent changes in QList/QString/QByteArray

> There's no violation. Your code was incorrect in the past, it just happened to work. I have to remind you about the Qt5.5 and the QDebug behaviour change that had to be rolled back because of the backlash. This has ramifications way bigger than a simple logging breakage. If a mission critical

Re: [Development] Important recent changes in QList/QString/QByteArray

Il 01/09/20 19:33, Thiago Macieira ha scritto: All non-const functions that may detach should be coded so they DO detach. That is, after any and all non-const functions, the refcount of the container should be 1. This is the contract, and it's OK. I don't think however that this is what was

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tuesday, 1 September 2020 08:53:36 PDT Giuseppe D'Angelo via Development wrote: > So, now, formally, std::sort(v.begin(), v.end()) risks undefined > behavior? E.g. begin() returns the begin iterator without touching > anything, but end() decides to invalidate all the iterators. We've seen

Re: [Development] Important recent changes in QList/QString/QByteArray

On Tuesday, 1 September 2020 09:05:48 PDT NIkolai Marchenko wrote: > Wait what? The switch to Qt6 was supposed to become a painless process yet > you're introducing important and hard to notice in real code changes that > can result in undefined behaviour? > What? WHAT?! There's no violation.

Re: [Development] Important recent changes in QList/QString/QByteArray

> A more subtle issue concerns the users of QVector. Wait what? The switch to Qt6 was supposed to become a painless process yet you're introducing important and hard to notice in real code changes that can result in undefined behaviour? What? WHAT?! On Tue, Sep 1, 2020 at 6:58 PM Giuseppe

Re: [Development] Important recent changes in QList/QString/QByteArray

Hi, Thanks for the heads up! Il 31/08/20 13:50, Andrei Golubev ha scritto: The invalidation existed before for cases when a container could detach (due to copy-on-write) or reallocate (due to growing or squeezing). This sounds incorrect? Which invalidation did happen due to COW? Now this

Re: [Development] Important recent changes in QList/QString/QByteArray

From: Andrei Golubev To: "development@qt-project.org" Sent: 8/31/2020 2:01 PM Subject: [Development] Important recent changes in QList/QString/QByteArray Hello all, Over the last week I pushed several patches to enable prepend optimization in some core

[Development] Important recent changes in QList/QString/QByteArray

Hello all, Over the last week I pushed several patches to enable prepend optimization in some core containers, namely, QList, QString, QByteArray. In the light of these changes, there may occur new subtle issues in your code. New model --- In a nutshell, prepend optimization code does some

[Development] Important recent changes in QList/QString/QByteArray

Hello all, Over the last week I pushed several patches to enable prepend optimization in some core containers, namely, QList, QString, QByteArray. In the light of these changes, there may occur new subtle issues in your code. New model --- In a nutshell, prepend optimization code does some