Re: [Paraview] Catalyst dlopen issues with OpenGL2 support for off-screen Mesa

[Andy Bauer]

Hi Yvan,

Thanks for the update.

I haven't seen anything like this yet but if I'm able to reproduce it I'll
see if I can track down the issue.

Best,
Andy

On Mon, Jan 22, 2018 at 9:16 PM, <yvan.fourn...@free.fr> wrote:

> Hello,
>
> I made further progress on this old thread, so am answering myself to
> close this issue...
>
> With recent versions of OSMesa (17.2 or 17.8), I observed that using
> LD_PRELOAD with OSMesa.so.8 also avoided the incorrect initialization issue
> of OSMesa leading to Catalyst complaining aboit OpenGL2 features not being
> available.
>
> On an Arch Linux system, I had also reproduced the reported issue at one
> time, but do not reproduce it anymore with Mesa 17.3.
> On a Debian 8 based system, I still reproduce it, but running new tests,
> instead of LD_PRELOAD, using RTLD_LAZY | RTLD_GLOBAL as dlopen flags solves
> the issue (either RTLD_LAZY or RTLD_NOW work with RTLD_GLOBAL, none work
> wth RTLD_LOCAL). I am pretty sure I had run similar tests a few month ago
> with older versions, with less success.
>
> I assume the improvements come from Mesa updates, though as the behavior
> of dlopen is concerned, it might also be due to lower level patches.
>
> In any case, I started building a small test case to reproduce this, but
> it was not working yet (trying to get the
> correct libraries to link in a minimal setting, or other options correct
> while not being familiar with CMake), so as I have a good work, I guess
> I'll leave it there and hope the issue does not resurfaces.
>
> Best regards,
>
>   Yvan
>
> - Mail original -
> De: "Yvan Fournier" <yvan.fourn...@free.fr>
> À: paraview@paraview.org
> Envoyé: Jeudi 25 Mai 2017 02:18:04
> Objet: Re: [Paraview] Catalyst dlopen issues with OpenGL2 support for
> off-screen Mesa
>
> Hello,
>
> I reproduced the issue described 2 months ago relative to detection of
> OSMesa
> OpenGL2 support using dynamically loaded shared libraries on another
> machine.
>
> I had reproduced this issue on Debian-9 based systems, but not with the
> preconfigured Mesa/OSMesa from Arch Linux. Recently, I had other issues
> with the
>  packaged library on Arch (Mesa 17.1.0, including both off-screen and
> on-screen
> drivers), as glGetString and a few other symbols were not found.
>
> So I recompiled Mesa for off-screen mode, using the recommendations from
> the
> ParaView Wiki (with the packaged LLVM 4.0), and encounter the following
> error
> again when starting Catalyst:
>
> 
> ERROR: In
> /home/yvan/src/ParaView/VTK/Rendering/OpenGL2/vtkOpenGLRenderWindow.cxx,
> line
> 831
>
> vtkOSOpenGLRenderWindow (0x72e3c00): GL version 2.1 with the gpu_shader4
> extension is not supported by your graphics driver but is required fo
> r the new OpenGL rendering backend. Please update your OpenGL driver. If
> you are
> using Mesa please make sure you have version 10.6.5 or later
> and make sure your driver in Mesa supports OpenGL 3.2.
> 
>
> Trying to force usage of OpenSWR (using "export GALLIUM_DRIVER=swr") I have
> another error:
>
> 
> SWR detected AVX2
>
> SWR library load failure: /home/yvan/opt/osmesa/lib/libswrAVX2.so:
> undefined
> symbol: _glapi_tls_Dispatch
> 
>
> So this seems quite reproducible on various systems, and both with LLVM
> 3.9 or
> LLVM 4, Mesa 13 or 17.
>
> When linking ParaView (Catalyst) normally (rather than linking them with a
> smaller module loaded as a plugin with dlopen), both the default LLVMpipe
> and
> OpenSwr errors dissapear, and things work fine
>
> Does anybody have an idea what I may be missing when loading with dlopen ?
>
> Best regards,
>
>   Yvan
>
>
> From: Yvan Fournier <yvan.fourn...@free.fr>
> To: paraview@paraview.org
> Objet: Re: [Paraview] Issues with OpenGL2 support for off-screen Mesa
> Date: Sat, 11 Mar 2017 00:36:00 +0100
>
> > Hello,
> >
> > I made some progress using OpenGL2 for offs-screen Mesa on a Debian
> 8-based
> > system:
> >
> > I don't need to compile my code or ParaView in static: compiling
> everything
> > with
> > dyanmic libraries works, as long as I link everything together instead of
> > loading a plugin (consisting of some Code_Saturne libraries, ParaView
> > Libraries,
> > and OSMesa/Gallium).
> >
> > So it seems some things re not initialized correctly when I load
> ParaView and
> > OSMesa as a plugin, so I'm not took sure the problem comes from ParaView,
> > OSMesa, or LLVM. I tried OSMesa 13.0.3 and 17.0.1, and LLVM

Re: [Paraview] Catalyst dlopen issues with OpenGL2 support for off-screen Mesa

[yvan . fournier]

Hello,

I made further progress on this old thread, so am answering myself to close 
this issue...

With recent versions of OSMesa (17.2 or 17.8), I observed that using LD_PRELOAD 
with OSMesa.so.8 also avoided the incorrect initialization issue of OSMesa 
leading to Catalyst complaining aboit OpenGL2 features not being available.

On an Arch Linux system, I had also reproduced the reported issue at one time, 
but do not reproduce it anymore with Mesa 17.3.
On a Debian 8 based system, I still reproduce it, but running new tests, 
instead of LD_PRELOAD, using RTLD_LAZY | RTLD_GLOBAL as dlopen flags solves the 
issue (either RTLD_LAZY or RTLD_NOW work with RTLD_GLOBAL, none work wth 
RTLD_LOCAL). I am pretty sure I had run similar tests a few month ago with 
older versions, with less success.

I assume the improvements come from Mesa updates, though as the behavior of 
dlopen is concerned, it might also be due to lower level patches.

In any case, I started building a small test case to reproduce this, but it was 
not working yet (trying to get the
correct libraries to link in a minimal setting, or other options correct while 
not being familiar with CMake), so as I have a good work, I guess I'll leave it 
there and hope the issue does not resurfaces.

Best regards,

  Yvan

- Mail original -
De: "Yvan Fournier" <yvan.fourn...@free.fr>
À: paraview@paraview.org
Envoyé: Jeudi 25 Mai 2017 02:18:04
Objet: Re: [Paraview] Catalyst dlopen issues with OpenGL2 support for 
off-screen Mesa

Hello,

I reproduced the issue described 2 months ago relative to detection of OSMesa
OpenGL2 support using dynamically loaded shared libraries on another machine.

I had reproduced this issue on Debian-9 based systems, but not with the
preconfigured Mesa/OSMesa from Arch Linux. Recently, I had other issues with the
 packaged library on Arch (Mesa 17.1.0, including both off-screen and on-screen
drivers), as glGetString and a few other symbols were not found.

So I recompiled Mesa for off-screen mode, using the recommendations from the
ParaView Wiki (with the packaged LLVM 4.0), and encounter the following error
again when starting Catalyst:


ERROR: In
/home/yvan/src/ParaView/VTK/Rendering/OpenGL2/vtkOpenGLRenderWindow.cxx, line
831

vtkOSOpenGLRenderWindow (0x72e3c00): GL version 2.1 with the gpu_shader4
extension is not supported by your graphics driver but is required fo
r the new OpenGL rendering backend. Please update your OpenGL driver. If you are
using Mesa please make sure you have version 10.6.5 or later 
and make sure your driver in Mesa supports OpenGL 3.2.


Trying to force usage of OpenSWR (using "export GALLIUM_DRIVER=swr") I have
another error:


SWR detected AVX2

SWR library load failure: /home/yvan/opt/osmesa/lib/libswrAVX2.so: undefined
symbol: _glapi_tls_Dispatch


So this seems quite reproducible on various systems, and both with LLVM 3.9 or
LLVM 4, Mesa 13 or 17.

When linking ParaView (Catalyst) normally (rather than linking them with a
smaller module loaded as a plugin with dlopen), both the default LLVMpipe and
OpenSwr errors dissapear, and things work fine

Does anybody have an idea what I may be missing when loading with dlopen ?

Best regards,

  Yvan


From: Yvan Fournier <yvan.fourn...@free.fr>
To: paraview@paraview.org
Objet: Re: [Paraview] Issues with OpenGL2 support for off-screen Mesa
Date: Sat, 11 Mar 2017 00:36:00 +0100

> Hello,
> 
> I made some progress using OpenGL2 for offs-screen Mesa on a Debian 8-based
> system:
> 
> I don't need to compile my code or ParaView in static: compiling everything
> with
> dyanmic libraries works, as long as I link everything together instead of
> loading a plugin (consisting of some Code_Saturne libraries, ParaView
> Libraries,
> and OSMesa/Gallium).
> 
> So it seems some things re not initialized correctly when I load ParaView and
> OSMesa as a plugin, so I'm not took sure the problem comes from ParaView,
> OSMesa, or LLVM. I tried OSMesa 13.0.3 and 17.0.1, and LLVM 3.5 (packaged with
> Debian 8) and 3.9 (local build), with identical results.
> 
> I also tried using RTLD_NOW and even RTLD_NOW | RTLD_GLOBAL instead of the
> usual
>  RTLD_LAZY when using dlopen, with no difference in behavior (I have not tried
> RTLD_DEEPBIND).
> 
> Has anyone encountered similar issues or does anyone have an idea what could
> be
> missing in the initialization stage using dlopen for a plugin linked with
> Catalyst ? I do not encounter this issue on an Arch-Linux-based system, but
> encounter it on a Debian 8 ("Jessie") system ?
> 
> Using a plugin is not an absolute must, but it would be preferable if I
> managed
> to get it working again with that build option.
> 
> Best regards,
> 
>   Yva

Re: [Paraview] Catalyst dlopen issues with OpenGL2 support for off-screen Mesa

[Yvan Fournier]

Hello,

I reproduced the issue described 2 months ago relative to detection of OSMesa
OpenGL2 support using dynamically loaded shared libraries on another machine.

I had reproduced this issue on Debian-9 based systems, but not with the
preconfigured Mesa/OSMesa from Arch Linux. Recently, I had other issues with the
 packaged library on Arch (Mesa 17.1.0, including both off-screen and on-screen
drivers), as glGetString and a few other symbols were not found.

So I recompiled Mesa for off-screen mode, using the recommendations from the
ParaView Wiki (with the packaged LLVM 4.0), and encounter the following error
again when starting Catalyst:


ERROR: In
/home/yvan/src/ParaView/VTK/Rendering/OpenGL2/vtkOpenGLRenderWindow.cxx, line
831

vtkOSOpenGLRenderWindow (0x72e3c00): GL version 2.1 with the gpu_shader4
extension is not supported by your graphics driver but is required fo
r the new OpenGL rendering backend. Please update your OpenGL driver. If you are
using Mesa please make sure you have version 10.6.5 or later 
and make sure your driver in Mesa supports OpenGL 3.2.


Trying to force usage of OpenSWR (using "export GALLIUM_DRIVER=swr") I have
another error:


SWR detected AVX2

SWR library load failure: /home/yvan/opt/osmesa/lib/libswrAVX2.so: undefined
symbol: _glapi_tls_Dispatch


So this seems quite reproducible on various systems, and both with LLVM 3.9 or
LLVM 4, Mesa 13 or 17.

When linking ParaView (Catalyst) normally (rather than linking them with a
smaller module loaded as a plugin with dlopen), both the default LLVMpipe and
OpenSwr errors dissapear, and things work fine

Does anybody have an idea what I may be missing when loading with dlopen ?

Best regards,

  Yvan


From: Yvan Fournier 
To: paraview@paraview.org
Objet: Re: [Paraview] Issues with OpenGL2 support for off-screen Mesa
Date: Sat, 11 Mar 2017 00:36:00 +0100

> Hello,
> 
> I made some progress using OpenGL2 for offs-screen Mesa on a Debian 8-based
> system:
> 
> I don't need to compile my code or ParaView in static: compiling everything
> with
> dyanmic libraries works, as long as I link everything together instead of
> loading a plugin (consisting of some Code_Saturne libraries, ParaView
> Libraries,
> and OSMesa/Gallium).
> 
> So it seems some things re not initialized correctly when I load ParaView and
> OSMesa as a plugin, so I'm not took sure the problem comes from ParaView,
> OSMesa, or LLVM. I tried OSMesa 13.0.3 and 17.0.1, and LLVM 3.5 (packaged with
> Debian 8) and 3.9 (local build), with identical results.
> 
> I also tried using RTLD_NOW and even RTLD_NOW | RTLD_GLOBAL instead of the
> usual
>  RTLD_LAZY when using dlopen, with no difference in behavior (I have not tried
> RTLD_DEEPBIND).
> 
> Has anyone encountered similar issues or does anyone have an idea what could
> be
> missing in the initialization stage using dlopen for a plugin linked with
> Catalyst ? I do not encounter this issue on an Arch-Linux-based system, but
> encounter it on a Debian 8 ("Jessie") system ?
> 
> Using a plugin is not an absolute must, but it would be preferable if I
> managed
> to get it working again with that build option.
> 
> Best regards,
> 
>   Yvan
> 
> 
> Friday Feb 24 février 2017 at 18:25 +0100, yvan.fourn...@free.fr wrote:
> 
> > Hi Chuck,
> > 
> > here is what I have in /home/D43345/opt/Mesa-13.0/arch/calibre9/lib:
> > 
> > -rwxr-xr-x 1 D43345 rdusers 1098 Feb 22 15:24 libOSMesa.la
> > lrwxrwxrwx 1 D43345 rdusers   18 Feb 22 15:24 libOSMesa.so ->
> > libOSMesa.so.8.0.0
> > lrwxrwxrwx 1 D43345 rdusers   18 Feb 22 15:24 libOSMesa.so.8 ->
> > libOSMesa.so.8.0.0
> > -rwxr-xr-x 1 D43345 rdusers 47309888 Feb 22 15:24 libOSMesa.so.8.0.0
> > -rwxr-xr-x 1 D43345 rdusers  962 Feb 22 15:24 libglapi.la
> > lrwxrwxrwx 1 D43345 rdusers   17 Feb 22 15:24 libglapi.so ->
> > libglapi.so.0.0.0
> > lrwxrwxrwx 1 D43345 rdusers   17 Feb 22 15:24 libglapi.so.0 ->
> > libglapi.so.0.0.0
> > -rwxr-xr-x 1 D43345 rdusers  1400520 Feb 22 15:24 libglapi.so.0.0.0
> > -rwxr-xr-x 1 D43345 rdusers 1018 Feb 22 15:24 libswrAVX.la
> > lrwxrwxrwx 1 D43345 rdusers   18 Feb 22 15:24 libswrAVX.so ->
> > libswrAVX.so.0.0.0
> > lrwxrwxrwx 1 D43345 rdusers   18 Feb 22 15:24 libswrAVX.so.0 ->
> > libswrAVX.so.0.0.0
> > -rwxr-xr-x 1 D43345 rdusers 97879312 Feb 22 15:24 libswrAVX.so.0.0.0
> > -rwxr-xr-x 1 D43345 rdusers 1024 Feb 22 15:24 libswrAVX2.la
> > lrwxrwxrwx 1 D43345 rdusers   19 Feb 22 15:24 libswrAVX2.so ->
> > libswrAVX2.so.0.0.0
> > lrwxrwxrwx 1 D43345 rdusers   19 Feb 22 15:24 libswrAVX2.so.0 ->
> > libswrAVX2.so.0.0.0
> > -rwxr-xr-x 1 D43345 rdusers 96655416 Feb 22 15:24 libswrAVX2.so.0.0.0
> > drwxr-xr-x 2 D43345 rdusers 4096 Feb 22 15:24 pkgconfig
> > 
> > 
> > And here is the summary after configuration of Mesa:
> > 
> > prefix: