Re: [Interest] Best way to threaded render (images) going forward?

[Wesley Krasko]

I solved this problem, thanks all. So our previous video surface WAS
QPainter on a separate thread already, it is possible. It was also cross
platform. But, it was the deprecated QGLWidget.
It turns out even if I could have gotten this all working in QOpenGLWidget,
that newer widget had a bug so we can't use it.
https://bugreports.qt.io/browse/QTBUG-49657
Our app does run in full screen and that bug still exists in 5.13.x
So, I went to a new, simpler model and it's working. I subclassed the
regular QWidget for the surface, and made a worker thread that uses a
QPainter on a QImage. I do all the resizing of the frames, compositing them
together, etc in that thread, then just pass the final QImage via signal to
the QWidget to draw in the UI thread. It's working fine, on all platforms
again, at 60fps with no apparent impact on the rest of the UI.
If it helps anyone, here is the final thread with some example code:
https://forum.qt.io/topic/109480/proper-way-to-pass-qimage-from-thread-through-signal/2
So in the final though, in Renderer::render(), I'm actually doing all the
work stated above, not just drawImage.


On Fri, Dec 6, 2019 at 8:56 AM Uwe Rathmann 
wrote:

> On 11/22/19 2:14 PM, Roland Hughes wrote:
>
> > There have been many requests over the years to allow QPainter
> > operations outside of the primary thread. In particular for running
> > graphs like heart rate monitors which are just a brick on the screen the
> > user generally cannot interact with.
>
> With X11 it is at least possible to draw outside of the paint event -
> see Qt::WA_PaintOutsidePaintEvent. This type of operation is used in the
> Qwt plot package for exactly the use case you have mentioned.
>
> I'm not 100% sure if it wouldn't be possible to the same in an extra
> thread.
>
> With Qt3 it was even possible to draw in XOR mode, what was at that time
> the way how rubberbands were implemented. This was an operation where
> you could erase content without having to repaint from scratch. But this
> one seems to be gone forever.
>
> Uwe
>
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>


-- 
Wes Krasko
www.worldwidewes.com
www.kraskofamily.com
"Stay away from negative people. They have a problem for every solution."
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Uwe Rathmann]

On 11/22/19 2:14 PM, Roland Hughes wrote:

There have been many requests over the years to allow QPainter 
operations outside of the primary thread. In particular for running 
graphs like heart rate monitors which are just a brick on the screen the 
user generally cannot interact with.


With X11 it is at least possible to draw outside of the paint event - 
see Qt::WA_PaintOutsidePaintEvent. This type of operation is used in the 
Qwt plot package for exactly the use case you have mentioned.


I'm not 100% sure if it wouldn't be possible to the same in an extra thread.

With Qt3 it was even possible to draw in XOR mode, what was at that time 
the way how rubberbands were implemented. This was an operation where 
you could erase content without having to repaint from scratch. But this 
one seems to be gone forever.


Uwe


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Roland Hughes]

On 11/22/19 5:00 AM, Wesley Krasko wrote:

Thanks. For now anyways I'm trying to just move from QGLWidget to
QOpenGLWidget, buy time.
Can I get some help? In the QGLWidget one I currently have there is a
separate thread to render and it uses QPainter.
I'm trying to do the same for the new class but I can't seem to get a
QPainter working no matter what! The app runs but as soon as drawing starts
it crashes because, but I"m using similar methods to what I did before. If
I comment out just the attempt to create QPainter, it works fine, renders
fast just doing the glclear part. Hopefully I can attach here. There is a
lot of "Extra" from our previous surface/class, but it's not yet used
anyways. As mentioned, this is working as is, but where you find "//TODO
HERE" in the code, I tried many ways to use a QPainter in the thread
without success.


Honestly, I'm shocked you ever got it to work. The original Achilles' 
Heel of Qt was the fact the GUI thread had to be the primary thread and 
all GUI (QPainter) operations had to occur there. Over the years they've 
added some non-rendered data classes for loading of images which could 
be threaded off, but they had to be moved into an object in the GUI 
thread to be rendered.


There have been many requests over the years to allow QPainter 
operations outside of the primary thread. In particular for running 
graphs like heart rate monitors which are just a brick on the screen the 
user generally cannot interact with.


https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Ftse1.mm.bing.net%2Fth%3Fid%3DOIP.N-43HpXNXBnwYyOgweb19wHaEL%26pid%3DApi=1


--
Roland Hughes, President
Logikal Solutions
(630)-205-1593

http://www.theminimumyouneedtoknow.com
http://www.infiniteexposure.net
http://www.johnsmith-book.com
http://www.logikalblog.com
http://www.interestingauthors.com/blog

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Wesley Krasko]

Anybody? I've been on this one for a week now and I just can't seem to
figure out what I'm doing wrong or missing.

On Thu, Nov 21, 2019 at 3:04 PM Wesley Krasko  wrote:

> Thanks. For now anyways I'm trying to just move from QGLWidget to
> QOpenGLWidget, buy time.
> Can I get some help? In the QGLWidget one I currently have there is a
> separate thread to render and it uses QPainter.
> I'm trying to do the same for the new class but I can't seem to get a
> QPainter working no matter what! The app runs but as soon as drawing starts
> it crashes because, but I"m using similar methods to what I did before. If
> I comment out just the attempt to create QPainter, it works fine, renders
> fast just doing the glclear part. Hopefully I can attach here. There is a
> lot of "Extra" from our previous surface/class, but it's not yet used
> anyways. As mentioned, this is working as is, but where you find "//TODO
> HERE" in the code, I tried many ways to use a QPainter in the thread
> without success.
>
>
> On Tue, Nov 19, 2019 at 11:18 AM Christoph Feck  wrote:
>
>> On 11/19/19 19:52, Wesley Krasko wrote:
>> > Yes, I've read about RHI but it all seems to imply it's for QT Quick,
>> what
>> > about Qt Widget based apps going forward?
>>
>> Let me quote what is written there: "We will need to base all our
>> rendering infrastructure (QPainter, the Qt Quick Scenegraph, and our 3D
>> support) on top of that layer."
>>
>> QWidgets use QPainter for rendering.
>>
>> > Is my best bet QOpenGLWidget until 6.x comes out then and then re-write
>> again?
>>
>> If you are using OpenGL, RHI won't magically change your code. Use
>> QPainter, the Qt Quick scene graph, or Qt3D, depending on your case.
>>
>> ___
>> Interest mailing list
>> Interest@qt-project.org
>> https://lists.qt-project.org/listinfo/interest
>>
>
>
> --
> Wes Krasko
> www.worldwidewes.com
> www.kraskofamily.com
> "Stay away from negative people. They have a problem for every solution."
>
>

-- 
Wes Krasko
www.worldwidewes.com
www.kraskofamily.com
"Stay away from negative people. They have a problem for every solution."
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Wesley Krasko]

Thanks. For now anyways I'm trying to just move from QGLWidget to
QOpenGLWidget, buy time.
Can I get some help? In the QGLWidget one I currently have there is a
separate thread to render and it uses QPainter.
I'm trying to do the same for the new class but I can't seem to get a
QPainter working no matter what! The app runs but as soon as drawing starts
it crashes because, but I"m using similar methods to what I did before. If
I comment out just the attempt to create QPainter, it works fine, renders
fast just doing the glclear part. Hopefully I can attach here. There is a
lot of "Extra" from our previous surface/class, but it's not yet used
anyways. As mentioned, this is working as is, but where you find "//TODO
HERE" in the code, I tried many ways to use a QPainter in the thread
without success.


On Tue, Nov 19, 2019 at 11:18 AM Christoph Feck  wrote:

> On 11/19/19 19:52, Wesley Krasko wrote:
> > Yes, I've read about RHI but it all seems to imply it's for QT Quick,
> what
> > about Qt Widget based apps going forward?
>
> Let me quote what is written there: "We will need to base all our
> rendering infrastructure (QPainter, the Qt Quick Scenegraph, and our 3D
> support) on top of that layer."
>
> QWidgets use QPainter for rendering.
>
> > Is my best bet QOpenGLWidget until 6.x comes out then and then re-write
> again?
>
> If you are using OpenGL, RHI won't magically change your code. Use
> QPainter, the Qt Quick scene graph, or Qt3D, depending on your case.
>
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>


-- 
Wes Krasko
www.worldwidewes.com
www.kraskofamily.com
"Stay away from negative people. They have a problem for every solution."
#ifndef GLWIDGET_H
#define GLWIDGET_H

#include "framequeue.h"
#include "hapiinterface.h"

#include 
#include 
#include 
#include 
#include 
#include 
#include 

class GLWidget;

class Renderer : public QObject
{
Q_OBJECT

public:
Renderer(GLWidget *);
~Renderer();
void lockRenderer() { m_renderMutex.lock(); }
void unlockRenderer() { m_renderMutex.unlock(); }
QMutex *grabMutex() { return _grabMutex; }
QWaitCondition *grabCond() { return _grabCond; }
void prepareExit() { m_exiting = true; m_grabCond.wakeAll(); }

enum {
FULLVL = 0,//"Full Video Local"
FULLVR = 1,//"Full Video Remote"
PIP1 = 2,//PIP one line
PIP2 = 3,//PIP 2 line
LSV = 4,//Left Side Video
RSV = 5//Right Side Video
};
FrameQueue *selfView;
QList remoteViewList;

signals:
void contextWanted();

public slots:
void start();
void stop();
void pause();
void resume();
void resumeToClose();
void setVidLayout(int layout);
void setPrivacyMode(bool isPrivacy);
void setNoRemoteVideo(int lineHandle, bool noRemote);
void scaleUp();
void scaleDown();
void disableEvents();
void render();
void eventHandler(int event, long param1, long param2);

private:
static const int animationFPS = 30;
static const int pipWidth = 160;
static const int pipHeight = 120;

bool m_inited;
GLWidget *m_glwidget;
QMutex m_renderMutex;
QMutex m_grabMutex;
QWaitCondition m_grabCond;
bool m_exiting;

bool skipEvents;
QImage selfImageToRender;
//I'm putting this in temporarily.
//I'm hoping to find a more generic way so I can use lines available from 
Xyclops like everywhere else.
QImage remoteImageToRender1;
QImage remoteImageToRender2;
QImage privacyImage;
QImage holdImage;
QImage audioOnlyImage;
QImage noRemoteVideoImage;
bool pauseRendering;
int localCWidth;
int localCHeight;
int scalingFactor;

bool m_privacyMode;
int m_currentVideoLayout;
int m_sessionVideoLayout;

QRect sizeFrameToFit(int frameWidth, int frameHeight, int position);
};

class GLWidget : public QOpenGLWidget
{
Q_OBJECT
public:
GLWidget(QWidget *parent);
~GLWidget();
void stopRendering();
void startRendering();
void pauseRendering();
void resumeRendering();
void resumeForClose();
bool isStarted();
void setVideoLayout(int layout);
//int getVideoLayout();
void setPrivacyMode(bool isPrivacy);
void setNoRemoteVideo(int lineHandle, bool noRemote);
void scaleUp();
void scaleDown();
void disableEvents();

enum {
NORMALMODE = 0,
SIDEBYSIDE = 1,
REMOTEONLY = 2
};

protected:
void paintEvent(QPaintEvent *) Q_DECL_OVERRIDE { }

signals:
void renderRequested();
void vidLayoutChange(int);
void privacyModeChange(bool);
void setNoRemote(int, bool);

public slots:
void grabContext();

private slots:
void onAboutToCompose();
void onFrameSwapped();
void onAboutToResize();
void onResized();

private:
QThread *m_thread;
Renderer *m_renderer;
bool m_isStarted;
};

#endif // GLWIDGET_H
#include 

Re: [Interest] Best way to threaded render (images) going forward?

[Christoph Feck]

On 11/19/19 19:52, Wesley Krasko wrote:

Yes, I've read about RHI but it all seems to imply it's for QT Quick, what
about Qt Widget based apps going forward?


Let me quote what is written there: "We will need to base all our
rendering infrastructure (QPainter, the Qt Quick Scenegraph, and our 3D
support) on top of that layer."

QWidgets use QPainter for rendering.


Is my best bet QOpenGLWidget until 6.x comes out then and then re-write again?


If you are using OpenGL, RHI won't magically change your code. Use 
QPainter, the Qt Quick scene graph, or Qt3D, depending on your case.


___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Wesley Krasko]

Yes, I've read about RHI but it all seems to imply it's for QT Quick, what
about Qt Widget based apps going forward? Also, AFAIK 6 will not be
released until 11 of 2020 which will likely be too late for Mac OS
considering they'll probably have another release before that doing away
completely with OpenGL. What about now? Is my best bet QOpenGLWidget until
6.x comes out then and then re-write again?

On Tue, Nov 19, 2019 at 10:46 AM Christoph Feck  wrote:

> On 11/19/19 19:33, David M. Cotter wrote:
> > is someone working on something like QNativeGraphics, which picks the
> best impl under the hood? eg: metal for macos, opengl for linux, and
> whatever it is now for windows?
>
> See https://www.qt.io/blog/2019/08/07/technical-vision-qt-6
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest
>


-- 
Wes Krasko
www.worldwidewes.com
www.kraskofamily.com
"Stay away from negative people. They have a problem for every solution."
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[Christoph Feck]

On 11/19/19 19:33, David M. Cotter wrote:

is someone working on something like QNativeGraphics, which picks the best impl 
under the hood? eg: metal for macos, opengl for linux, and whatever it is now 
for windows?


See https://www.qt.io/blog/2019/08/07/technical-vision-qt-6
___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest

Re: [Interest] Best way to threaded render (images) going forward?

[David M. Cotter]

+1 on this please

is someone working on something like QNativeGraphics, which picks the best impl 
under the hood? eg: metal for macos, opengl for linux, and whatever it is now 
for windows?

the writing is on the wall for OpenGL on mac, what are the future plans for 
that?

> On Nov 19, 2019, at 9:32 AM, Wesley Krasko  wrote:
> 
> Hello all, I was sent here from the Qt forum for more "technical" questions.
> We have an app, it runs on Linux, Windows, and Mac OS. It is a video relay 
> app. The video is supplied via an SDK, I subclass one of their classes and 
> re-implement a lock and unlock function. On lock the SDK writes the current 
> frame/image into a buffer I supply (raw image data, I'm using a QImage at the 
> moment) then on unlock it says "ready" and I can draw it. This is happening 
> about 60fps.
> So, at the moment I have 2 methods. One is a QGLWidget based class where I 
> use a QPainter in another thread to draw the QImage when ready. Its very 
> fast, no impact on GUI thread, on all platforms. However, there's a known bug 
> in Qt causing loss of style under Windows so I then had to do a backup D2D 
> method, non-threaded so it does not perform nearly as well.
> Fast forward to today. We ran into all kinds of issues with Mac OS Catalina 
> and the beginning of their loss of support for OpenGL. We were able to 
> mitigate this for now by sticking with Mojave + Xcode 10 to build.
> I would like to solve this once and for all hopefully. One threaded rendering 
> method I can use across all platforms. I know RHI is coming but it looks like 
> it's for Quick. I know that QGLWidget is deprecated and moving to 
> QOpenGLWidget might solve things but will it be future proof on Mac? Also, in 
> my attempts to move to this thus far, I can thread if I use open gl commands 
> directly but am no longer able to use QPainter in the thread.
> Any suggestions on the best method going forward? I think I've read on 
> everything, QGraphicsView/Scene, RHI, etc and either the 
> documentation/examples are lacking or what I need to do doesn't seem doable 
> which I find odd, this seems like it would be a common use case to render 
> video frames! Thanks.
> 
> -- 
> Wes Krasko
> www.worldwidewes.com  
> www.kraskofamily.com 
> "Stay away from negative people. They have a problem for every solution."
> 
> ___
> Interest mailing list
> Interest@qt-project.org
> https://lists.qt-project.org/listinfo/interest

___
Interest mailing list
Interest@qt-project.org
https://lists.qt-project.org/listinfo/interest