Re: [Development] QUtf8String{, View} (was: Re: QString and related changes for Qt 6)

[Thiago Macieira]

On Thursday, 14 May 2020 07:41:45 PDT Marc Mutz via Development wrote:
> Also, given a function like
> 
> setFoo(const QByteArray &);
> 
> what does this actually expect? An UTF-8 string? A local 8-bit string?
> An octet stream? A Latin-1 string? QByteArray is the jack of all these,
> master of none.

Like that, it's just "array of bytes of an arbitrary encoding (or none)". 
There's still a reason to have QByteArray and it'll need to exist in 
networking and file I/O code. That means the string classes, if any, need to 
be convertible to QByteArray anyway.

> So, assuming the premiss that QByteArray should not be string-ish
> anymore, what do we want to have as the result type of QString::toUtf8()
> and QString::toLatin1()? Do we really want mere bytes?
> 
> I don't think so.

Since for Qt, String = UTF-16, then anything in another encoding is "a bag of 
bytes". QByteArray does serve that purpose.

> If Unicode succeeds, most I/O will be in the form of UTF-8. File names
> on Unix are UTF-8 (for all intents and purposes these days), not UTF-16
> (as they are on Windows). It makes a _ton_ of sense to have a container
> for this, and C++20 tempts us with char8_t to do exactly that. I'd love
> to do string processing in UTF-8 without potentially doubling the
> storage requirements by first converting it to UTF-16, then doing the
> processing, then converting it back.

Unless you're processing Cyrillic or Greek text, in which case your memory 
usage will be about the same. Or if you're processing CJK, in which case 
UTF-16 is a 33% reduction in memory use.

> Qt should have a strong story not just for UTF-16, but also for UTF-8.

So long as it's not confusing on which class to use, sure. If that means a 
proliferation of overloads everywhere, we've gone wrong somewhere.

> I'm not sure we need the utf32 one, and I'm ok with dropping the L1 one,
> provided a) we can depend on char8_t (ie. Qt 7) and b) utf-8 <-> utf16
> operations are not much slower than L1 <-> utf16 ones (I heard Lars'
> team has them down to within 5% of each other, not sure that's
> possible). 

The conversion of US-ASCII content using either fromUtf8 or fromLatin1 is 
within 5% of the other. The UTF-8 codec is optimised towards US-ASCII. The 
difference in performance is the need to check if the high bit is set. Both 
codecs are vectorised with both SSE2 and AVX2 implementations. There are also 
Neon implementations, but I don't know their benchmark numbers (note: the 
UTF-8 Neon code is AArch64 only, while the Latin1 also runs on 32-bit).

For non-US-ASCII Latin1 text, the performance is more than 5% worse, depending 
on how dense the non-ASCII characters are in the string. But given that we 
want our files to be encoded in UTF-8 anyway, decoding of non-ASCII Latin1 
should be rare.

I also have an implementation of UTF-16 to ASCII codec, which is the same as 
UTF-16 to Latin1, but without error checking. That requires that the string 
class store whether it contains only US-ASCII. I've never pushed this to Qt.

> Anyway, we'd have two class templates, and they'd just be
> instantiated with different Char types to flesh out all of the above,
> with the exception of the byte array ones:
> 
>using QUtf8String = QBasicString;
>using QString = QBasicString;
>using QLatin1String = QBasicString;
>(using QByteArray = QVector;)

BTW, I've said this before: QVector should over-allocate by one element and 
memset it to zero, if the element is small enough (4 or 8 bytes). This should 
be done behind the scenes, so the API would never notice it. But it would 
allow transferring the ownership of a QByteArray's payload to any of the other 
classes and still have a null-terminated string.

I don't mind having a QUtf8String{,View} but there needs to be a limit into 
how much we add to its API. Do we have indexOf(char32_t) optimised with 
vectorisation? Do we have indexOf(QRegularExpression)? The latter would make 
us link to libpcre2-8 in addition to libpcre2-16 or would require on-the-fly 
conversions and memory allocations. If your objective is to speed things up, 
having too many methods may actually make it worse.

And then there's the overload set for generic functions. I'm going to insist a 
single, clear rule that does not depend on implementation details and is 
reasonably future-proof. It has to be about *what* the function does, not 
*how* it does that.

> If, after getting all of the above runnig, we _then_ want The One String
> (View) To Rule Them All, then I'd suggest QAnyString{,View} (not sure we
> need a QAnyString), which can contain any of the 2-4 string (view)
> classes above (but not QByteArray(View)), but which doesn't have
> string-ish API. Instead, you need to inspect it to extract the actual
> string class (QLatin1String, QUtf8String, QString) contained, or simply
> ask for the one you want, and it will convert, if necessary.

Excluding QLatin1String since I don't think we need that, I'm willing to see 
this 

Re: [Development] Views in APIs

[Thiago Macieira]

On Thursday, 14 May 2020 12:34:54 PDT Matthew Woehlke wrote:
> On 14/05/2020 14.58, Thiago Macieira wrote:
> > Which is:
> > b) misspelling "iteratable"
> 
> Be that as it may, that ship has sailed:
> 
>https://www.google.com/search?q=iteratable
> 
> Even Google thinks so, and if you insist otherwise, 12k results instead
> of 2M.

The majority of the results in the first and second pages are about a company 
called Iterable.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Using dwz on Qt

[Thiago Macieira]

On Thursday, 14 May 2020 13:24:06 PDT Lisandro Damián Nicanor Pérez Meyer 
wrote:
> > I recently discovered there is a binutils tools for size optimizing dwarf
> > debug symbols, and it really works. When applied to Qt debug symbol
> > binaries it makes them about 25% smaller on average (some up to 40%
> > smaller). And that is just the simple optimization. We can save more by
> > combining duplicates from multiple debug symbol binaries into one shared
> > one, but that would change the files we ship.
> 
> For what is worth Debian packages have been doing this with Qt without
> issues (of course as a post process after building Qt and before
> packaging them).

No distro relies on Qt's stripping routines. They all build with -no-strip and 
then figure out themselves using their own techniques.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Using dwz on Qt

[Lisandro Damián Nicanor Pérez Meyer]

Hi!

On Thu, 14 May 2020 at 07:33, Allan Sandfeld Jensen  wrote:
>
> Hi Qt
>
> I recently discovered there is a binutils tools for size optimizing dwarf
> debug symbols, and it really works. When applied to Qt debug symbol binaries
> it makes them about 25% smaller on average (some up to 40% smaller). And that
> is just the simple optimization. We can save more by combining duplicates from
> multiple debug symbol binaries into one shared one, but that would change the
> files we ship.

For what is worth Debian packages have been doing this with Qt without
issues (of course as a post process after building Qt and before
packaging them).

-- 
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[Jean-Michaël Celerier]

> To have type-erased containers, we need to go back to something like
java.lang.Collection.

C++20 offers coroutines which allows the same things without java-like type
hierarchies though
(and with much more flexibility as you don't even need to have an actual
container existing somewhere - your function's stack
can serve as one just fine :p).

see e.g. https://godbolt.org/z/cEv7Gp - in simple cases it optimizes as
well as as a std::vector.

Well, for Qt 7 maybe :-)


On Thu, May 14, 2020 at 9:00 PM Thiago Macieira 
wrote:

> On Thursday, 14 May 2020 02:33:43 PDT Marc Mutz via Development wrote:
> > On 2020-05-14 02:15, Thiago Macieira wrote:
> > > On quarta-feira, 13 de maio de 2020 01:41:26 PDT Иван Комиссаров wrote:
> > >> std::span Project::productsSpan() const & { return
> > >> d->products; }
> > >> std::span Project::productsSpan() const && = delete;
> > >
> > > Now implement either or both of these functions if internally you have
> > > a
> > > QSet.
> >
> > Now implement QHash roleNames() when
> > internally you have a std::pmr::unordered_map > std::pmr::string>, with all the data allocated from one
> > std::pmr::monotonic_buffer_resource.
> >
> > This does both ways.
>
> No, it doesn't. In both you created a copy on demand.
>
> But you can only *return* said copy if the return type is an owning
> container.
> std::span and the views are not.
>
> > And: you can also have views over associated data structures, they just
> > can't have inline functions because they need to type-erase the
> > container. For an extreme example, consider QAssociativeIterable.
>
> Which is:
> a) a horrible API
> b) misspelling "iteratable"
>
> To have type-erased containers, we need to go back to something like
> java.lang.Collection.
>
> --
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel System Software Products
>
>
>
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
>
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[Matthew Woehlke]

On 14/05/2020 14.58, Thiago Macieira wrote:

Which is:
b) misspelling "iteratable"


Be that as it may, that ship has sailed:

  https://www.google.com/search?q=iteratable

Even Google thinks so, and if you insist otherwise, 12k results instead 
of 2M.


--
Matthew
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Bug in either moc or VS Tools

[Thiago Macieira]

On Thursday, 14 May 2020 02:21:01 PDT David C. Partridge wrote:
> I'm not sure whether this problem is caused by a bug in moc, or a bug in VS
> Tools.
> 
> Problem description is here:
>  s>

It's not moc.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs (was: Re: QString and related changes for Qt 6)

[Thiago Macieira]

On Thursday, 14 May 2020 04:20:43 PDT Иван Комиссаров wrote:
> I don’t see the problem here. Ranges library operates on views only, so
> there *should be* (I am just not aware of it’s name) the type that
> corresponds to the «forward access iterator» (well, in case of QSet, that
> would be bidirectional but not random access iterator) range.
> 
> If there’s no such view, it’s quite easy to implement it - it’s simply a
> pair of iterators.

You're missing the point. I want you to implement it without changing your 
return type.

Or, in different words: choose one return type for your non-inline function 
that you can be sure to support for the next 7 years, regardless of internal 
modifications to your data store.

> The disadvantage here is that such a view doesn’t gives the same flexibility
> as std::span does. i.e. it is still not possible to replace QSet with
> std::unordered_set, so we don’t have much gain compared to returning QSet /
> const QSet &.
> 
> So, as I said, I do not see the problem. Can we always return views? Yes. Do
> they gives us benefits in every case? No.

Should we do it? Almost never.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[Thiago Macieira]

On Thursday, 14 May 2020 02:33:43 PDT Marc Mutz via Development wrote:
> On 2020-05-14 02:15, Thiago Macieira wrote:
> > On quarta-feira, 13 de maio de 2020 01:41:26 PDT Иван Комиссаров wrote:
> >> std::span Project::productsSpan() const & { return
> >> d->products; }
> >> std::span Project::productsSpan() const && = delete;
> > 
> > Now implement either or both of these functions if internally you have
> > a
> > QSet.
> 
> Now implement QHash roleNames() when
> internally you have a std::pmr::unordered_map std::pmr::string>, with all the data allocated from one
> std::pmr::monotonic_buffer_resource.
> 
> This does both ways.

No, it doesn't. In both you created a copy on demand.

But you can only *return* said copy if the return type is an owning container. 
std::span and the views are not.

> And: you can also have views over associated data structures, they just
> can't have inline functions because they need to type-erase the
> container. For an extreme example, consider QAssociativeIterable.

Which is:
a) a horrible API
b) misspelling "iteratable"

To have type-erased containers, we need to go back to something like 
java.lang.Collection.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel System Software Products



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[André Pönitz]

On Thu, May 14, 2020 at 11:46:29AM +0200, Marc Mutz via Development wrote:
> > Please stop with this crusade of yours to end all CoW, get rid of
> > QList, etc. It is misguided and harmful to the ecosystem at large.
> 
> You are entitled to your opinions just as I am. The difference is that I put
> time (much more than KDAB pays me for, esp. when it comes to my "crusade")
> where my mouth is.

The problem here is not what personal or company time you invest yourself.
You can spent as much of that as you wish or are allowed to.

The problem is the effort these activities cause for others, especially
for those that do not share your opinions, and are still forced to adapt
their code to your preferences.

Andre'
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] QString and related changes for Qt 6

[Elvis Stansvik]

Den tors 14 maj 2020 15:46Marc Mutz via Development <
development@qt-project.org> skrev:

> On 2020-05-13 17:17, Matthew Woehlke wrote:
> [...]
> > Non-owning QString is dangerous. QStringLiteral is less dangerous
> > because it is almost never used with non-rodata storage (and indeed, I
> > would consider any such usage highly suspect, if not outright broken).
> > QString::fromRawData is dangerous, but "obviously" so.
> >
> > We should not implement any way of creating a non-owning QString that
> > is not explicit, and if we adhere to that, I don't see us *not*
> > wanting QStringView in many instances.
>
> I must be crazy, but ... +1!
>

*chuckle* :)


> Thanks,
> Marc
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development
>
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

[Development] QUtf8String{, View} (was: Re: QString and related changes for Qt 6)

[Marc Mutz via Development]

Hi Lars,

On 2020-05-12 09:49, Lars Knoll wrote:
[...]

One open question is whether we should add a QUtf8String with a
char8_t. I am not yet convinced that we actually need the class
though.

[...]

I positively want to stop using QByteArray as the QUtf8String that it 
currently is. QByteArray should lose all notion of string-ness 
(deprecate toLower() etc, remove in Qt 7) and be a QVector. 
Not sure we'll get there for Qt 6, not sure we'll get there with the 
name QByteArray, but that should be the end game for this class.


The networking code is full of uses of QByteArray and due to the lack of 
QByteArrayRef (QStringRef) or QByteArrayView (QStringView), it's 
splitting and substringing is much less performant than it could be.


Also, given a function like

   setFoo(const QByteArray &);

what does this actually expect? An UTF-8 string? A local 8-bit string? 
An octet stream? A Latin-1 string? QByteArray is the jack of all these, 
master of none.


So, assuming the premiss that QByteArray should not be string-ish 
anymore, what do we want to have as the result type of QString::toUtf8() 
and QString::toLatin1()? Do we really want mere bytes?


I don't think so.

If Unicode succeeds, most I/O will be in the form of UTF-8. File names 
on Unix are UTF-8 (for all intents and purposes these days), not UTF-16 
(as they are on Windows). It makes a _ton_ of sense to have a container 
for this, and C++20 tempts us with char8_t to do exactly that. I'd love 
to do string processing in UTF-8 without potentially doubling the 
storage requirements by first converting it to UTF-16, then doing the 
processing, then converting it back.


Qt should have a strong story not just for UTF-16, but also for UTF-8.

I've talked about this on QtWS, but here's TL;DV: of it:

value_type  container  viewstring-ish 
API?


char / QLatin1Char— QLatinString — QLatin1StringView — yes
char8_t / qchar8  — QUtf8String  — QUtf8StringView   — yes
char16_t / QChar  — QString  — QStringView   — yes
(char32_t ­— QUtf32String — QUtf32StringView  — yes)

std::byte — QByteArray   — QByteArrayView­— NO

I'm not sure we need the utf32 one, and I'm ok with dropping the L1 one, 
provided a) we can depend on char8_t (ie. Qt 7) and b) utf-8 <-> utf16 
operations are not much slower than L1 <-> utf16 ones (I heard Lars' 
team has them down to within 5% of each other, not sure that's 
possible). Anyway, we'd have two class templates, and they'd just be 
instantiated with different Char types to flesh out all of the above, 
with the exception of the byte array ones:


  using QUtf8String = QBasicString;
  using QString = QBasicString;
  using QLatin1String = QBasicString;
  (using QByteArray = QVector;)

If, after getting all of the above runnig, we _then_ want The One String 
(View) To Rule Them All, then I'd suggest QAnyString{,View} (not sure we 
need a QAnyString), which can contain any of the 2-4 string (view) 
classes above (but not QByteArray(View)), but which doesn't have 
string-ish API. Instead, you need to inspect it to extract the actual 
string class (QLatin1String, QUtf8String, QString) contained, or simply 
ask for the one you want, and it will convert, if necessary.


With this, your typical Qt function taking strings would look like this:

   QLineEdit::setText(QAnyStringView text)
   {
   Q_D(QLineEdit);
   if (text == d->text) // mixed-mode comparisons are supported out 
of the box

   return;
   d->text = text.toString(); // centralized conversion to QString 
(in library, not user code)
  // also available: toLatin1(), 
toUtf8()

   update();
   }

Callers now have total freedom in what to pass:

   le->setText("Hi");
   le->setText(u"Hi");
   le->setText(u8"Hi");
   le->setText(u"Hi"s);
   le->setText(u8"Hi"sv);
   le->setText(QVarLengthArray{'H', 'i'});
   le->setText("Hello" % ", World"); // QStringBuilder

and they'd all result in optimal code, because QAnyStringView is a 
trivial type (in the C++ sense), which means, unlike QString, it can be 
passed in CPU registers instead of on the stack.


Likewise, parsing code could do

   Meep parseMeep(QAnyStringView str)
   {
   return str.visit([](auto str) {
   Meep meep;
   for (auto me : str.tokenize(u'\n'))
  meep += parse(me);
   return meep;
   });
   }

iow: instead of a bunch of overloads, you write your code as a template 
and let QAnyStringView instantiate your lambda with the actual type of 
string view passed.


As a further example, here's op== for QAnyStringView (provided by Qt):

   bool operator==(QAnyStringView lhs, QAnyStringView rhs) noexcept
   {
   return lhs.visit([rhs](auto lhs) {
   return rhs.visit([lhs](auto rhs) {
   return lhs == rhs;
   });
   });
   }

Last year, I heard someone (don't remember whom) suggest this for 
QString. That is: allow 

Re: [Development] QString and related changes for Qt 6

[Marc Mutz via Development]

On 2020-05-13 17:17, Matthew Woehlke wrote:
[...]

Non-owning QString is dangerous. QStringLiteral is less dangerous
because it is almost never used with non-rodata storage (and indeed, I
would consider any such usage highly suspect, if not outright broken).
QString::fromRawData is dangerous, but "obviously" so.

We should not implement any way of creating a non-owning QString that
is not explicit, and if we adhere to that, I don't see us *not*
wanting QStringView in many instances.


I must be crazy, but ... +1!

Thanks,
Marc
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] QString and related changes for Qt 6

[Marc Mutz via Development]

On 2020-05-13 20:48, Jaroslaw Kobus wrote:
From: Development  on behalf of 
Thiago Macieira 

Sent: Wednesday, May 13, 2020 6:21 PM
To: development@qt-project.org
Subject: Re: [Development] QString and related changes for Qt 6

On terça-feira, 12 de maio de 2020 22:57:31 PDT Jaroslaw Kobus wrote:
> That's why I've mentioned the better option: aggregation: QStringView could
> be a member of QString. However, the downside would be that every time you
> want to call a const method for QString, you would need to first get access
> to the QStringView member. The advantage is that in this way you may easily
> integrate different interfaces inside one class.

This is more or less what we want to do. QString in Qt 6 is {begin, 
size, d}
and QStringView has always been {begin, size}. So, yeah, it can be 
done.


The idea is indeed to offload the majority of the non-mutating methods 
to the

same functions, from inline code. There's no reason to have both
QString::indexOf and QStringView::indexOf entry points in the library.


Good to hear. And I hope that Marc will resurrect soon after his veto.


Had you looked into qstring.cpp (I know it hurts!), you'd've seen that 
it's already implemented that way. But Neither does QString aggregate a 
QStringView nor does it inherit it.


So, there's no resurrection coming because no death was caused.

Thanks,
Marc
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs (was: Re: QString and related changes for Qt 6)

[Иван Комиссаров]

I don’t see the problem here. Ranges library operates on views only, so there 
*should be* (I am just not aware of it’s name) the type that corresponds to the 
«forward access iterator» (well, in case of QSet, that would be bidirectional 
but not random access iterator) range.

If there’s no such view, it’s quite easy to implement it - it’s simply a pair 
of iterators.

The disadvantage here is that such a view doesn’t gives the same flexibility as 
std::span does. i.e. it is still not possible to replace QSet with 
std::unordered_set, so we don’t have much gain compared to returning QSet / 
const QSet &.

So, as I said, I do not see the problem. Can we always return views? Yes. Do 
they gives us benefits in every case? No.

Ivan

> 14 мая 2020 г., в 02:15, Thiago Macieira  
> написал(а):
> 
> On quarta-feira, 13 de maio de 2020 01:41:26 PDT Иван Комиссаров wrote:
>> std::span Project::productsSpan() const & { return d->products; }
>> std::span Project::productsSpan() const && = delete;
> 
> Now implement either or both of these functions if internally you have a 
> QSet.
> 
> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>  Software Architect - Intel System Software Products
> 
> 
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Using dwz on Qt

[Alexandru Croitor]

Hi,

I haven't looked too much into the details, but quickly looking over 
Konstantin's code in OptionsQt.cmake, we already do something similar
in 
https://code.qt.io/cgit/qt/qtbase.git/tree/cmake/QtSeparateDebugInfo.cmake#n62

So i assume it should be straightforward to add dwz support.

> On 14. May 2020, at 12:39, Konstantin Tokarev  wrote:
> 
> 
> 
> 14.05.2020, 13:34, "Allan Sandfeld Jensen" :
>> Hi Qt
>> 
>> I recently discovered there is a binutils tools for size optimizing dwarf
>> debug symbols, and it really works. When applied to Qt debug symbol binaries
>> it makes them about 25% smaller on average (some up to 40% smaller). And that
>> is just the simple optimization. We can save more by combining duplicates 
>> from
>> multiple debug symbol binaries into one shared one, but that would change the
>> files we ship.
> 
> FWIW, I've already reported it at https://bugreports.qt.io/browse/QTBUG-82681
> 
>> 
>> In any case. While too late for 5.15 and written for qmake so not suitable 
>> for
>> qt6, I would like to share my minimal change with you here, in case anybody
>> would like to ship smaller debug binaries.
>> 
>> Is there an existing cmake way to trigger the same? Though we might also want
>> to consider triggering compressed dwarf symbols at the same time, as that is
>> another great reduction, but ups the minimum tool versions required.
> 
> FWIW, I'm using this code for separating and compressing debug info:
> https://github.com/qtwebkit/qtwebkit/blob/qtwebkit-dev-wip/Source/cmake/OptionsQt.cmake#L152-L167
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Using dwz on Qt

[Konstantin Tokarev]

14.05.2020, 13:34, "Allan Sandfeld Jensen" :
> Hi Qt
>
> I recently discovered there is a binutils tools for size optimizing dwarf
> debug symbols, and it really works. When applied to Qt debug symbol binaries
> it makes them about 25% smaller on average (some up to 40% smaller). And that
> is just the simple optimization. We can save more by combining duplicates from
> multiple debug symbol binaries into one shared one, but that would change the
> files we ship.

FWIW, I've already reported it at https://bugreports.qt.io/browse/QTBUG-82681

>
> In any case. While too late for 5.15 and written for qmake so not suitable for
> qt6, I would like to share my minimal change with you here, in case anybody
> would like to ship smaller debug binaries.
>
> Is there an existing cmake way to trigger the same? Though we might also want
> to consider triggering compressed dwarf symbols at the same time, as that is
> another great reduction, but ups the minimum tool versions required.

FWIW, I'm using this code for separating and compressing debug info:
https://github.com/qtwebkit/qtwebkit/blob/qtwebkit-dev-wip/Source/cmake/OptionsQt.cmake#L152-L167
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

[Development] Using dwz on Qt

[Allan Sandfeld Jensen]

Hi Qt

I recently discovered there is a binutils tools for size optimizing dwarf 
debug symbols, and it really works. When applied to Qt debug symbol binaries 
it makes them about 25% smaller on average (some up to 40% smaller). And that 
is just the simple optimization. We can save more by combining duplicates from 
multiple debug symbol binaries into one shared one, but that would change the 
files we ship.

In any case. While too late for 5.15 and written for qmake so not suitable for 
qt6, I would like to share my minimal change with you here, in case anybody 
would like to ship smaller debug binaries.

Is there an existing cmake way to trigger the same? Though we might also want 
to consider triggering compressed dwarf symbols at the same time, as that is 
another great reduction, but ups the minimum tool versions required.

Best regards
'Allandiff --git a/mkspecs/features/unix/separate_debug_info.prf b/mkspecs/features/unix/separate_debug_info.prf
index 9070cf33f0..7b7106bb80 100644
--- a/mkspecs/features/unix/separate_debug_info.prf
+++ b/mkspecs/features/unix/separate_debug_info.prf
@@ -100,10 +100,11 @@ have_target:!static:if(darwin|!isEmpty(QMAKE_OBJCOPY)) {
 mkdir_debug_info = $$QMAKE_MKDIR $$shell_quote($$debug_info_target_dir)
 QMAKE_POST_LINK = $$mkdir_debug_info && $$copy_debug_info && $$strip_debug_info $$QMAKE_POST_LINK
 } else {
+opt_debug_info = dwz $$shell_target
 link_debug_info = $$QMAKE_OBJCOPY --add-gnu-debuglink=$$shell_target_debug_info $$shell_target
 !contains(QMAKE_HOST.os, Windows): \
 QMAKE_POST_LINK = && chmod -x $$shell_target_debug_info $$QMAKE_POST_LINK
-QMAKE_POST_LINK = $$copy_debug_info && $$strip_debug_info && $$link_debug_info $$QMAKE_POST_LINK
+QMAKE_POST_LINK = $$opt_debug_info && $$copy_debug_info && $$strip_debug_info && $$link_debug_info $$QMAKE_POST_LINK
 }
 silent:QMAKE_POST_LINK = @echo creating $@.$$debug_info_suffix && $$QMAKE_POST_LINK
 
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[Marc Mutz via Development]

On 2020-05-13 17:27, Matthew Woehlke wrote:

On 12/05/2020 12.59, Marc Mutz via Development wrote:
AsidE: If you think that CoW is still a thing today: no. SSO is a 
thing these days, and it seems that QString will not have it in Qt 6, 
either. NOI favours SSO, QString-everywhere cements the naïve CoW 
world of the 1990s for yet another decade.


I am really, *really* sick of this.

Okay, for "most" *strings*, you may have a point.


I thought this thread was about strings. Did I miss something?


However, CoW is
*absolutely* still a useful tool for a lot of other applications, and
will continue to be so;


*Optional* *explicit* sharing, as per shared_ptr is a thing, 
yes. But that's completely different from the broken Qt "implicit 
sharing" that we have now. Have you read the 22year-old Sutter articles? 
After having read the second, have you checked Qt's implementation for 
the pitfalls mentioned therein? Continue discussing after you did.



the combination of implicit value semantics
and "cheap" copies (an atomic increment may be relatively expensive,
but so are memory allocations, especially for large data structures)
is not going away any time soon.

Please stop with this crusade of yours to end all CoW, get rid of
QList, etc. It is misguided and harmful to the ecosystem at large.


You are entitled to your opinions just as I am. The difference is that I 
put time (much more than KDAB pays me for, esp. when it comes to my 
"crusade") where my mouth is.


Thanks,
Marc
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Re: [Development] Views in APIs

[Marc Mutz via Development]

On 2020-05-14 02:15, Thiago Macieira wrote:

On quarta-feira, 13 de maio de 2020 01:41:26 PDT Иван Комиссаров wrote:
std::span Project::productsSpan() const & { return 
d->products; }

std::span Project::productsSpan() const && = delete;


Now implement either or both of these functions if internally you have 
a

QSet.


Now implement QHash roleNames() when 
internally you have a std::pmr::unordered_mapstd::pmr::string>, with all the data allocated from one 
std::pmr::monotonic_buffer_resource.


This does both ways.

And: you can also have views over associated data structures, they just 
can't have inline functions because they need to type-erase the 
container. For an extreme example, consider QAssociativeIterable.


Thanks,
Marc
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

[Development] Bug in either moc or VS Tools

[David C. Partridge]

I'm not sure whether this problem is caused by a bug in moc, or a bug in VS
Tools.

Problem description is here:


David


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development