Something seems off.
But without looking at the actual code that is allocation 60k tiles and the
constructor itself, it just seems like a very expensive construction if the
"new" + "moving a pointer" is taking 3ms each.
Scott
-----Original Message-----
From: Interest <interest-boun...@qt-project.org> On Behalf Of Murphy, Sean
Sent: Tuesday, February 1, 2022 7:32 AM
To: interest@qt-project.org
Subject: Re: [Interest] [External]Re: How to get QtConcurrent to do what I want?
> Not knowing if a partial value makes any sense to your system.
> Qt::Concurrent::mappedReduced might make more sense, if its purely a
> speedup you are looking for, and not a "keep the GUI alive during it"
> possibly blockingMappedReduced.
I don't think mappedReduced would help me until after the remapping step when I
want to assemble the individual tiles into a QImage - although since the
ultimate destination for the image is inside a QGraphicsView, I'm tempted to
just leave it as individual tiles, but I'm not there yet as far as testing.
Regarding the keep GUI alive portion, my tileManager is already running in a
separate thread from the UI thread, so I do have the option of making blocking
calls in the tileManager class with the exception that I do need to fire
progress signals out of tileManager back to MainWindow to provide progress to
the user
>
> If you need the gui, setting up a qfuturewatcher on the results of the
> mapped call, would be my approach
>
> QFutureWatcher< XXX > watcher;
> connect( &watcher, &finished, manager, &handleFinished);
>
> auto future = QtConcurrent::mapReduced(tiles, processTile,
> mergeFunction) watcher.setFuture( future );
I actually spent yesterday refactoring the tileManager class as you've just
described, as well as changing out the tile class to no longer inherit from
QObject. I've got a couple more things I want to try/clean up today but I still
seem to be having trouble speeding up the allocation & assignment of the tiles
themselves.
As my code currently stands, I now have to vectors, each of which will have
60,000 items in them once they're populated:
QVector<int> mTileIndices;
QList<tile> mTiles;
The mTileIndices vector is implementing Andrei's idea of quickly generating a
list of unique tile indices, which can then be fed to a QtConcurrent::map()
call to create & uniquely assign the tile items in parallel. The " mTiles "
vector is obviously the tiles that will do the work.
As I build the vectors up, these are the timings I get:
1. resizing tile index vector to 60000 took 0.1514 ms
a. This is just calling QVector<int>::resize(60000) on the ID. This takes
less than a
millisecond, so no complaints here.
2. allocated 60000 indices in 0.0207 ms
a. This is calling std::iota(mTileIndices.begin(), mTileIndices.end(),
0). Also takes less than
a millisecond, still no complaints
3. assigning 60000 tiles took 18087.1 ms
a. This timing is the result of QtConcurrent::mapped(mTiles, initTile)
where the initTile
function takes in an integer from mTileIndices, and calls the tile
constructor using the
combination of the tile index and tile size to do the assignment. The
assigned tiles end
up in the mTiles
b. This step takes 18 seconds, which seems excessive to me and I'd love
to continue to
reduce that time.
4. load finished in 37507.1 ms
a. this is calling tile::load() on each tile. Right now that is just a
dummy function that calls
msleep for a random amount milliseconds to simulate doing the actual
work
5. remapping 60000 tiles took 48519.3 ms
b. this is calling tile::remap() on each tile. Right now that is just a
dummy function that
calls msleep for a random amount milliseconds to simulate doing the
actual work
So the only step in this process that still bothers me is step 3 - creating and
assigning each tile object takes 18 seconds. I log the total time by each tile
spent in steps 4 & 5 and compare how long steps 4 & 5 actually take vs. the sum
of how long each tile spent sleep and I routinely get a speedup factor of about
7.5. QThread::idealThreadCount() reports 8 on my machine, so I think I'm
getting what I should expect from those steps on this machine.
I'm not sure what else to try at this point. One thing I was thinking about
measuring is that even though my tile class no longer inherits from QObject, it
still is a class with a constructor and some getter functions. And when I look
at the combination of steps 1 & 2 where I both resize a vector of integers AND
assign each one a unique ID in less than a millisecond total, but then it takes
me 18 seconds to create and assign each tile, I keep wondering if there isn't
room for improvement there still?
And as I was typing this whole thing up, Konstantin gave me a different
approach that I'll probably pursue, but I would like to better understand how
to solve the question of "if you absolutely need to have a lot of items
(whatever type an "item" needs to be), what's the right design approach to be
able to create and populate them quickly..."
Sean
_______________________________________________
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
_______________________________________________
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest
