Re: [Interest] Models and aborting structural changes

[Elvis Stansvik]

Den ons 3 juni 2020 10:38Jonathan Purol  skrev:

> Hello everyone,
>
> I've been in a bit of a tough spot recently with `QAbstractItemModel`.
> I'll try to delineate my situation, then my problem, and then the
> potential solutions and why I think they're all inferior to what I wish
> to have.
>
> We have a desktop applications using QtWidgets, however instead of a
> data model we obtain our data from an "external source" (this is a
> little complicated in our use case, but you can think of it as a remote
> server for all intents and purposes).
> Now the server emits events for changes in the underlying data model.
> For example there is an event for `itemAdded` with all appropriate data
> points being passed to us via a web socket.
> This is all fine and dandy.
>
> The problem arises when I try to now connect this to the model
> representing all `Item` objects.
> Qt requires both `beginInsertRows` and `endInsertRows` to be called when
> these changes are made. Furthermore it requires the underlying data to
> really only be changed in-between those calls. I.e. `rowCount` should
> return something different at the time `beginInsertRows` is called than
> at the time `endInsertRows` is called. So far so good. But we don't
> really have that in-between point. We only get notified when the change
> has already happened.
>
> There are a few solutions I could see here:
> 1. Keep a local cache of all of the data, update it incrementally when
> changes come through with this pattern:
> - change events arrives
> - emit "begin change"
> - update internal data
> - emit "end change"
>
> 2. Keep a local cache of all the data on a per-model basis then repeat
> as can be seen above
>
> I dislike both of these changes, even if they do solve the issue
> flawlessly. The reason for that is that it is quite a bit of overhead.
> Both regarding code complexity as well as regarding memory usage and
> performance. Don't get me wrong, having a cache is important, and we
> will have one as well. What doesn't stick right with me is a cache that
> in turn keeps track of all the data all the time.
>
> Now you might say "why not just have the server emit both a "before" and
> a "after" change event for whatever happens.
> Yeah, I would love to do that - but with Qt's current setup it just
> isn't possible.
> What Qt is missing is an `abortInsertRows` function to reset the model's
> internal state.
> There are many, many things that can go wrong in our setup, even IF the
> data was successfully updated. And if we cannot tell Qt to revert the
> changes in case such an error occurs we cannot possible use this approach.
> The only solution I would see here would be to call
> `begin/endResetModel` in case an error occurs, but that sounds like
> trying to put a nail into a coffin with a wrecking ball.
>

I understand your problem. I've not been in your situation, but how about

- Change server to emit before and after events, plus a cancel event.

- Upon receiving a before event, call beginInsertRows,

- Upon receiving an after event, call endInsertRows,

- Upon receiving a cancel event, call endInsertRows, then call
beginRemoveRows, remove the cancelled rows, then call endRemoveRows.

Could that work? I don't think the models would ever show the cancelled
data, as control never returned to the event loop during the dance in the
last step.

Cheers,
Elvis


> My questions now are as follows:
> Is there any chance `abort*` methods would be added in the future?
> Could I perhaps add such a method myself?
> Is it safe to just call `endInsertRows` when no changes were made?
> Is there a simple solution I overlooked?
>
> Sincerely,
> Jonathan Purol
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Models and aborting structural changes

[Elvis Stansvik]

Den tors 4 juni 2020 kl 09:00 skrev Elvis Stansvik :
>
> Den ons 3 juni 2020 10:38Jonathan Purol  skrev:
>>
>> Hello everyone,
>>
>> I've been in a bit of a tough spot recently with `QAbstractItemModel`.
>> I'll try to delineate my situation, then my problem, and then the
>> potential solutions and why I think they're all inferior to what I wish
>> to have.
>>
>> We have a desktop applications using QtWidgets, however instead of a
>> data model we obtain our data from an "external source" (this is a
>> little complicated in our use case, but you can think of it as a remote
>> server for all intents and purposes).
>> Now the server emits events for changes in the underlying data model.
>> For example there is an event for `itemAdded` with all appropriate data
>> points being passed to us via a web socket.
>> This is all fine and dandy.
>>
>> The problem arises when I try to now connect this to the model
>> representing all `Item` objects.
>> Qt requires both `beginInsertRows` and `endInsertRows` to be called when
>> these changes are made. Furthermore it requires the underlying data to
>> really only be changed in-between those calls. I.e. `rowCount` should
>> return something different at the time `beginInsertRows` is called than
>> at the time `endInsertRows` is called. So far so good. But we don't
>> really have that in-between point. We only get notified when the change
>> has already happened.
>>
>> There are a few solutions I could see here:
>> 1. Keep a local cache of all of the data, update it incrementally when
>> changes come through with this pattern:
>> - change events arrives
>> - emit "begin change"
>> - update internal data
>> - emit "end change"
>>
>> 2. Keep a local cache of all the data on a per-model basis then repeat
>> as can be seen above
>>
>> I dislike both of these changes, even if they do solve the issue
>> flawlessly. The reason for that is that it is quite a bit of overhead.
>> Both regarding code complexity as well as regarding memory usage and
>> performance. Don't get me wrong, having a cache is important, and we
>> will have one as well. What doesn't stick right with me is a cache that
>> in turn keeps track of all the data all the time.
>>
>> Now you might say "why not just have the server emit both a "before" and
>> a "after" change event for whatever happens.
>> Yeah, I would love to do that - but with Qt's current setup it just
>> isn't possible.
>> What Qt is missing is an `abortInsertRows` function to reset the model's
>> internal state.
>> There are many, many things that can go wrong in our setup, even IF the
>> data was successfully updated. And if we cannot tell Qt to revert the
>> changes in case such an error occurs we cannot possible use this approach.
>> The only solution I would see here would be to call
>> `begin/endResetModel` in case an error occurs, but that sounds like
>> trying to put a nail into a coffin with a wrecking ball.
>
>
> I understand your problem. I've not been in your situation, but how about
>
> - Change server to emit before and after events, plus a cancel event.
>
> - Upon receiving a before event, call beginInsertRows,
>
> - Upon receiving an after event, call endInsertRows,
>
> - Upon receiving a cancel event, call endInsertRows, then call 
> beginRemoveRows, remove the cancelled rows, then call endRemoveRows.
>
> Could that work? I don't think the models would ever show the cancelled data, 
> as control never returned to the event loop during the dance in the last step.

Thinking a bit more (and I'm no expert), if I understand your setup
correctly, I still think there's a race that could confuse your views.

If I understand you right, your model always reports "live" data from
the server, which I presume means your data(..) returns data fetched
from the server.

If so, then I think this scenario could happen (?):

1. View requests some data at index X, and gets back data Y.
2. Something happens on the server so the data changes.
3. The server sends out a change event.
4. View again requests some data at index X, but gets back data Z (!).
5. The change event arrives at your application.
...

This is a situation in which the data returned at some index changes
from the POV of the view, with no beginInsertRows/endInsertRows in
between, which I think is a breach of the Qt model/view contract.

I may be wrong though, so maybe someone more knowledgeable can step in
an confirm/deny.

Cheers,
Elvis


>
> Cheers,
> Elvis
>
>>
>> My questions now are as follows:
>> Is there any chance `abort*` methods would be added in the future?
>> Could I perhaps add such a method myself?
>> Is it safe to just call `endInsertRows` when no changes were made?
>> Is there a simple solution I overlooked?
>>
>> Sincerely,
>> Jonathan Purol
>>
>> ___
>> Interest mailing list
>> Interest@qt-project.org
>> https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Int

Re: [Interest] Qt3D - Deploying renderer plugins introduced in Qt 5.15

[Paul Lemire via Interest]

Hi,

From what I can see the macdeployqt and windeployqt scripts weren't 
updated to deploy the renderers plugin.


Hopefully these patches would fix that:

https://codereview.qt-project.org/c/qt/qttools/+/302945/1

https://codereview.qt-project.org/c/qt/qttools/+/302946/1

Paul

On 5/28/20 6:41 PM, Esch Lorenz TU Ilmenau wrote:


Hello,

I am using windeployqt to deploy an application facilitating Qt3D via 
specifying QT+=3dcore3drender3dinput3dextras in its .pro file. Since 
Qt 5.15 the opengl renderer is isolated as a separated plugin 
(openglrenderer.dll). This new plugin is somehow not caught by 
windeployqt and thus not deployed to the binary’s folder.


Any idea why? Maybe I am missing something here? Do I always need to 
deploy the renderer manually for versions > 5.15?


Thanks,

Lorenz


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest


smime.p7s
Description: S/MIME Cryptographic Signature
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Models and aborting structural changes

[Jonathan Purol]

> I understand your problem. I've not been in your situation, but how about
> 
> - Change server to emit before and after events, plus a cancel event.
> 
> - Upon receiving a before event, call beginInsertRows,
> 
> - Upon receiving an after event, call endInsertRows,
> 
> - Upon receiving a cancel event, call endInsertRows, then call
> beginRemoveRows, remove the cancelled rows, then call endRemoveRows.
> 
> Could that work? I don't think the models would ever show the cancelled
> data, as control never returned to the event loop during the dance in
> the last step.


I have thought about that and believe that in theory it could work
although it would require some testing.
What should definitely work is to call `begin/endResetModel` in case an
error occurs, though that has obvious drawbacks as well. I'd have to
give it a test run. It does seem like a very hacky solution however.
I've spoken with my partners and for now we'll just go with a cache that
duplicates all necessary data - from what I understand we only really
have to duplicate structural information, since the normal "data
changes" don't require the begin-end system (i.e. `emit dataChanged`).

> Thinking a bit more (and I'm no expert), if I understand your setup
> correctly, I still think there's a race that could confuse your views.
> 
> If I understand you right, your model always reports "live" data from
> the server, which I presume means your data(..) returns data fetched
> from the server.
> 
> If so, then I think this scenario could happen (?):
> 
> 1. View requests some data at index X, and gets back data Y.
> 2. Something happens on the server so the data changes.
> 3. The server sends out a change event.
> 4. View again requests some data at index X, but gets back data Z (!).
> 5. The change event arrives at your application.
> ...
> 
> This is a situation in which the data returned at some index changes
> from the POV of the view, with no beginInsertRows/endInsertRows in
> between, which I think is a breach of the Qt model/view contract.
> 
> I may be wrong though, so maybe someone more knowledgeable can step in
> an confirm/deny.

Yeah this would definitely break the contract. Furthermore - I believe
Qt wouldn't even poll for any information about index X if it hasn't
been established that index X exists yet (either by a remove, insert, or
move command). Anything else would seem like a false implementation of
what I expect the view-classes to do.

I genuinely believe that the `abort*` functions are just... missing - it
seems like such an inherent feature to me. Perhaps the complexity of
implementing it was considered too large to be worth the hassle. In the
forums people say "just verify everything beforehand" - but not every
situation warrants that.


Thank you for your time and effort! If I try out the "just call the
reverse if it fails" and it works I'll let you know!

cheers,
Jonathan Purol
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Qt3D - Deploying renderer plugins introduced in Qt 5.15

[Esch Lorenz TU Ilmenau]

Hi,

thanks for the patches. I created a bug report a couple of days ago: 
https://bugreports.qt.io/browse/QTBUG-84576

In terms of linuxeployqt the following worked for me:

linuxdeployqt-continuous-x86_64.AppImage yourQt3DAppName 
-extra-plugins=renderers

Best,

Lorenz

Von: Paul Lemire 
Gesendet: Donnerstag, 4. Juni 2020 04:57
An: Esch Lorenz TU Ilmenau ; interest@qt-project.org
Betreff: Re: [Interest] Qt3D - Deploying renderer plugins introduced in Qt 5.15


Hi,

>From what I can see the macdeployqt and windeployqt scripts weren't updated to 
>deploy the renderers plugin.

Hopefully these patches would fix that:

https://codereview.qt-project.org/c/qt/qttools/+/302945/1

https://codereview.qt-project.org/c/qt/qttools/+/302946/1
Paul

On 5/28/20 6:41 PM, Esch Lorenz TU Ilmenau wrote:
Hello,

I am using windeployqt to deploy an application facilitating Qt3D via 
specifying QT += 3dcore 3drender 3dinput 3dextras in its .pro file. Since Qt 
5.15 the opengl renderer is isolated as a separated plugin 
(openglrenderer.dll). This new plugin is somehow not caught by windeployqt and 
thus not deployed to the binary's folder.

Any idea why? Maybe I am missing something here? Do I always need to deploy the 
renderer manually for versions > 5.15?

Thanks,

Lorenz



___

Interest mailing list

Interest@qt-project.org

https://lists.qt-project.org/listinfo/interest
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

[Interest] how do I load massive table views instantly?

[David M. Cotter]

say i have a few "playlists" that are really big, like 200k items, with many 
columns
i'd like to be able to switch to them and have them be instantly loaded.  

by that i mean the entire list, such that i can take the scroll thumb and drag 
it to the bottom and see the last item, ie: no "loading by pages"

alternately, if "loading only what you see" is just "how it's done", then can i 
just set the scroll bar height to be the total height?  so CAN still grab the 
scroll thumb, and drag it to the end, and see the data at the end, without 
waiting for all the unseen data to load?

i know other apps do this... how is it done?

-dave
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

[Interest] Using qrc/qss files with Qt WASM

[Esch Lorenz TU Ilmenau]

Hi all,

I have a question regarding Qt WASM. I would like to change between different 
style sheets during run time in a WASM application. In the desktop version of 
my application I just use QFile to access the .qss file included in my .qrc 
file.

if(QApplication *pApp = qobject_cast(QApplication::instance())) 
{
   QFile file;
   file.setFileName(":/styles/dark.qss");
  file.open(QFile::ReadOnly | QFile::Text);
  QTextStream stream(&file);
  pApp->setStyleSheet(stream.readAll());
}

However, under Qt WASM QFile is not supported. Instead one can use 
QFileDialog::getOpenFileContent to read the complete data as a QByteArray. But 
this prompts me with a QFileDialog first, which I do not want/need. My 
question: Is there a way to read from a file included in a .qrc file directly 
into a QByteArray? I had a look at QResource, but it seems that this class can 
only be used to read from an external qrc/rcc file during runtime. Any ideas?

Thanks,

Lorenz


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] how do I load massive table views instantly?

[Giuseppe D'Angelo via Interest]

Il 04/06/20 21:12, David M. Cotter ha scritto:

alternately, if "loading only what you see" is just "how it's done", then can i 
just set the scroll bar height to be the total height?  so CAN still grab the scroll thumb, and 
drag it to the end, and see the data at the end, without waiting for all the unseen data to load?

i know other apps do this... how is it done?




What table view are you talking about, specifically?

What did you try so far?

--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] how do I load massive table views instantly?

[David M. Cotter]

>> alternately, if "loading only what you see" is just "how it's done", then 
>> can i just set the scroll bar height to be the total height?  so CAN still 
>> grab the scroll thumb, and drag it to the end, and see the data at the end, 
>> without waiting for all the unseen data to load?
>> i know other apps do this... how is it done?
> 
> What table view are you talking about, specifically?

QTableView with QSortFilterProxyModel

> What did you try so far?
when showing the view for the proxy model:

while (modelP->canFetchMore(modelIndex)) {
modelP->fetchMore(modelIndex);

if (dlgP) {
dlgP->Inc(false);
}
}

but that is way slow

i'd rather just quick-calc the number of items and set the scroll bar to max at 
the start
then let the viewer fetch what it needs to when the rows get revealed?

is this possible?

or can i pre-load the proxyModel at startup and just "swap it in" when the user 
switches playlists?

-dave___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] how do I load massive table views instantly?

[David M. Cotter]

> What did you try so far?
the other thing tried was showing the table without pre-loading it, but then 
loading the rest in the background

but this has the problem that the user can't then pick up the scroll thumb and 
drag it to the bottom, because not all the data is loaded.  as soon as the user 
lets go, more data keeps getting loaded and the scroll thumb slides out from 
under their mouse as more data comes in.

uck

:)
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] how do I load massive table views instantly?

[Constantin Makshin]

Here is my idea:
1) quickly calculate/get the total number of items and return that value from 
rowCount() to let views properly configure their scroll bars;
2) load data in the background;
3) return some placeholder values from data() for items that have not been 
loaded yet;
4) when a new batch of items is available, add it to the model's internal 
storage and emit the dataChanged() signal with appropriate index range.

On 06/05/2020 12:39 AM, David M. Cotter wrote:
>> What did you try so far?
> the other thing tried was showing the table without pre-loading it, but then 
> loading the rest in the background
> 
> but this has the problem that the user can't then pick up the scroll thumb 
> and drag it to the bottom, because not all the data is loaded.  as soon as 
> the user lets go, more data keeps getting loaded and the scroll thumb slides 
> out from under their mouse as more data comes in.
> 
> uck
> 
> :)



signature.asc
Description: OpenPGP digital signature
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] how do I load massive table views instantly?

[David M. Cotter]

> 1) quickly calculate/get the total number of items and return that value from 
> rowCount() to let views properly configure their scroll bars;
is that a method one overrides? where is "rowCount()" defined?
do i override it by subclassing QSortFilterProxyModel?

> 2) load data in the background;
i think this is already happening

> 3) return some placeholder values from data() for items that have not been 
> loaded yet;
> 4) when a new batch of items is available, add it to the model's internal 
> storage and emit the dataChanged() signal with appropriate index range.

wouldn't the view ask the model for whatever data is revealed as visible, all 
by itself?
so no need to show mock data? the point is i want to show the real data, even 
if the user jumps to the last rows immediately

do i *need* to manually do this?

thanks for the suggestions, i'm trying to understand

-dave___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest