Re: [Development] qsizetype and classes working with QStrings or QList
Hi, Il 27/08/20 16:47, Thiago Macieira ha scritto: So, can someone take a look at what it would take to make the models use 64- bit and come up with a proper guide for how to maintain code that compiles and works on both Qt5 and Qt6? The latter is very important: if you can't easily maintain for both, we just add to users' pain. It might be as simple as a typedef added to QAbstractItemModel. Finger in the wind: it's going to be painful. int-based APIs are used - in virtuals in QAbstractItemModel (rowCount(), but also things like moveRows() which take int as parameter, so changing it is a straight API break) - in Q(Persistent)ModelIndex - in convenience subclasses like QStandardItemModel (and of course in the convenience views) - in selection handling - in the views' convenience handlers (e.g. QAbstractItemView::rowsInserted) Long story short, I don't see this happening without also some configure-time switch for a "qaimsizetype" or somesuch. My 2 c, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] QHelpEngineCore::documentsForIdentifier
Martin Koller (14 August 2020 17:06) wrote: > I found myself getting an empty list when using > QHelpEngineCore::documentsForIdentifier(id) but getting 1 element when > using the deprecated QHelpEngineCore::linksForIdentifier(id). Definitely sounds like a bug, although I find no QHelpEngineCore::linksForIdentifier() - did you mean QHelpCollectionHandler::linksForIdentifier() ? > Checking the code I stumbled over something which I believe is a bug: > > QHelpEngineCore::documentsForIdentifier(const QString , const > QString ) does > > if (!d->setup() || !d->usesFilterEngine) > return QList(); > > so it checks if the new filter engine is active, but I think this is > not needed here, since in this code, the filter engine is not used at > all, since the filterName is already given as argument. Sounds plausible. Also, QHelpEngineCore::documentsForIdentifier(const QString ) calls documentsForIdentifier(const QString , const QString ) passing a filterName that it determines based on d->usesFilterEngine, which rather suggests it thinks d->usesFilterEngine isn't a prerequisite of getting any entries. > Since I ported older code to the new api but did not call > setUsesFilterEngine(true) (which I believe should not be needed when I > don't work with filters), this check was hit and I get no results. > > Am I right ? I don't know this code at all but it sounds like reasonable grounds to submit a patch to Gerrit and ask for review by folk implicated in the review that added this code (this March): https://codereview.qt-project.org/c/qt/qttools/+/291144 Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Codereview maintenance break is over
> On 25 Aug 2020, at 18:52, Thiago Macieira wrote: > > Can we configure it so the commit message is at least 80 columns wide? Since > our messages are wrapped at 72 right now but the display is narrower, it's > causing line-wrapping. > As far as I can find, the commit message box automatically resizes itself to adjust for what's required by the "related commits" field. There's a bug reported about this behavior upstream, requesting the behavior of the old ui (where the commit message box apparently had a 75 character width) be reintroduced in Gerrit >= 3. However, as the upstream bug report is almost 2 years old, I fear the short answer is "no”. > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Software Architect - Intel DPG Cloud Engineering > > > > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
On Thursday, 27 August 2020 00:52:10 PDT Lars Knoll wrote: > > On 27 Aug 2020, at 09:12, Philippe wrote: > >>> * QAbstractItemModel > >>> * QModelIndex > If I were do design these from scratch, I would certainly use qsizetype > here, maybe even a qint64 (because one can handle those large data sets on > 32bit systems as well with on demand loading of the data). Good point. File sizes are qint64, so I agree any quantity that is tied to data, not to memory, should either use int or qint64. So, can someone take a look at what it would take to make the models use 64- bit and come up with a proper guide for how to maintain code that compiles and works on both Qt5 and Qt6? The latter is very important: if you can't easily maintain for both, we just add to users' pain. It might be as simple as a typedef added to QAbstractItemModel. Like the q6sizetype typedef that was proposed and I never implemented. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel DPG Cloud Engineering ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] bugreports.qt.io back online
Hi, Let me start with the good news. bugreports.qt.io is back in business since a few minutes ago. We had server issues which prevented Jira from being usable for the first half of today. Despite our best attempts we were not able fix the corruption. Therefore we rolled the system back to our latest backup from August 26, 2020 at 11:00:51 PM UTC+3. Unfortunately this means that all changes between backup and now have been lost. We are very sorry for the inconvenience and loss you may have experienced. I would encourage everybody to check their Jira email notifications as they may provide hints to lost data. -- Alex ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
Il 25/08/20 07:49, Thiago Macieira ha scritto: >> But how about models? This is an honest question. Does it make sense >> for tables and lists that big? Note that an item*view* has a purpose >> of being viewed, so how does one display such a huge list, tree or >> table? Giuseppe D'Angelo (25 August 2020 11:58) replied: > Just another thought -- models may not necessarily used directly with > views but as data sources for other business logic parts of the > application (including but not necessarily limited to proxy models). > Given the underlying data sources are 64-bit capable, such models > should be as well. and, at a more brutal level: in the late '90s, I ported a Geographic Information System to cope with database files > 2GB in size (despite the fact that the underlying OSen didn't; I had to use a virtual file system), because we had customers whose datasets were big enough to need it. I consequently cannot believe, more than two decades later, that 2GB size limits on containers make sense. Even if we didn't have a use-case we could think of, some user out there almost certainly has, or shall within Qt6's life-time have, such use-cases. Coding to cope with them gracefully may, of course, require care: but we must support them. Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
On 27 Aug 2020, at 10:45, Lars Knoll mailto:lars.kn...@qt.io>> wrote: On 27 Aug 2020, at 08:32, Lars Knoll mailto:lars.kn...@qt.io>> wrote: On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development mailto:development@qt-project.org>> wrote: Hi, Il 27/08/20 02:46, Thiago Macieira ha scritto: A QListView of 2 billion lines with where each line is a QString one to 7 characters in length would be 2G * (24 + 32) = 96 GB of memory use. QListWidget's overhead is much worse. This isn't accurate; QListView (with the default delegate) doesn't cache data, and only fetches and shows what's visible in its viewport. So the actual consumption is pretty much constant no matter how big is the underlying model. Correct. But the argument for QTextDocument still holds. If you have such huge files, I believe you want to load them using some custom editing component that mmaps the file and only loads the relevant parts on the fly. QTextDocument allows for rich text editing and requires to hold the content in memory in a QString because of that. QTextDocument should be able to handle very large regular documents this without larger issues. Just as a comparison: All Harry Potter books together are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 times larger document) into a QTextDocument should work without issues. But a 5G set of machine generated data? I would believe you want a special class handling that and loading data on the fly, so you have some memory left for the rest of your system. Having said that, I don’t mind changing our API to use qsizetype for QTextDocument, I just don’t believe you will find the class to be very snappy and usable with such large documents. The piece table itself should be able to handle it if you have enough RAM, but the default layout engine probably can’t (you could try using your own custom layout though). I’ll give it a try and let’s see. There are very few to no virtual functions that would be affected, so changing the classes to use qsizetype might be worthwhile. Did a quick trial, and I don’t think it’s worth it. It’s quite a lot of work and will break user code in some places for very little gain. Little gain not because there aren’t use cases that need to handle very large documents, but because QTextEdit and QTextDocument are probably not the right classes to do so given that they load everything into memory at once. Cheers, Lars ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
On 27 Aug 2020, at 08:32, Lars Knoll mailto:lars.kn...@qt.io>> wrote: On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development mailto:development@qt-project.org>> wrote: Hi, Il 27/08/20 02:46, Thiago Macieira ha scritto: A QListView of 2 billion lines with where each line is a QString one to 7 characters in length would be 2G * (24 + 32) = 96 GB of memory use. QListWidget's overhead is much worse. This isn't accurate; QListView (with the default delegate) doesn't cache data, and only fetches and shows what's visible in its viewport. So the actual consumption is pretty much constant no matter how big is the underlying model. Correct. But the argument for QTextDocument still holds. If you have such huge files, I believe you want to load them using some custom editing component that mmaps the file and only loads the relevant parts on the fly. QTextDocument allows for rich text editing and requires to hold the content in memory in a QString because of that. QTextDocument should be able to handle very large regular documents this without larger issues. Just as a comparison: All Harry Potter books together are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 times larger document) into a QTextDocument should work without issues. But a 5G set of machine generated data? I would believe you want a special class handling that and loading data on the fly, so you have some memory left for the rest of your system. Having said that, I don’t mind changing our API to use qsizetype for QTextDocument, I just don’t believe you will find the class to be very snappy and usable with such large documents. The piece table itself should be able to handle it if you have enough RAM, but the default layout engine probably can’t (you could try using your own custom layout though). I’ll give it a try and let’s see. There are very few to no virtual functions that would be affected, so changing the classes to use qsizetype might be worthwhile. Cheers, Lars ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
> On 27 Aug 2020, at 09:12, Philippe wrote: > > On Wed, 26 Aug 2020 06:36:27 + > Lars Knoll wrote: > >>> * QAbstractItemModel >>> * QModelIndex >> >> I dont think we should port these to use qsizetype. > > If you mean, the effort would be too big, or there is lack of resources > to do it, or too many API breaks for Qt 5 to 6, then I can agree. > > But if you mean there is no usage need, I disagree. If I were do design these from scratch, I would certainly use qsizetype here, maybe even a qint64 (because one can handle those large data sets on 32bit systems as well with on demand loading of the data). Cheers, Lars ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
On Wed, 26 Aug 2020 06:36:27 + Lars Knoll wrote: > > * QAbstractItemModel > > * QModelIndex > > I dont think we should port these to use qsizetype. If you mean, the effort would be too big, or there is lack of resources to do it, or too many API breaks for Qt 5 to 6, then I can agree. But if you mean there is no usage need, I disagree. Philippe ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
> On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development > wrote: > > Hi, > > Il 27/08/20 02:46, Thiago Macieira ha scritto: >> A QListView of 2 billion lines with where each line is a QString one to 7 >> characters in length would be 2G * (24 + 32) = 96 GB of memory use. >> QListWidget's overhead is much worse. > > This isn't accurate; QListView (with the default delegate) doesn't cache > data, and only fetches and shows what's visible in its viewport. So the > actual consumption is pretty much constant no matter how big is the > underlying model. Correct. But the argument for QTextDocument still holds. If you have such huge files, I believe you want to load them using some custom editing component that mmaps the file and only loads the relevant parts on the fly. QTextDocument allows for rich text editing and requires to hold the content in memory in a QString because of that. QTextDocument should be able to handle very large regular documents this without larger issues. Just as a comparison: All Harry Potter books together are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 times larger document) into a QTextDocument should work without issues. But a 5G set of machine generated data? I would believe you want a special class handling that and loading data on the fly, so you have some memory left for the rest of your system. Having said that, I don’t mind changing our API to use qsizetype for QTextDocument, I just don’t believe you will find the class to be very snappy and usable with such large documents. The piece table itself should be able to handle it if you have enough RAM, but the default layout engine probably can’t (you could try using your own custom layout though). Cheers, Lars ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] qsizetype and classes working with QStrings or QList
Hi, Il 27/08/20 02:46, Thiago Macieira ha scritto: A QListView of 2 billion lines with where each line is a QString one to 7 characters in length would be 2G * (24 + 32) = 96 GB of memory use. QListWidget's overhead is much worse. This isn't accurate; QListView (with the default delegate) doesn't cache data, and only fetches and shows what's visible in its viewport. So the actual consumption is pretty much constant no matter how big is the underlying model. My 2 c, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - The Qt, C++ and OpenGL Experts smime.p7s Description: Firma crittografica S/MIME ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
