The main problem with this idea is that there is no universally available hardware rendering backend in JavaFX. There's OpenGL on Linux and macOS, Direct3D on Windows, and potentially a software renderer on all platforms.
One approach might be to create a new ANGLE rendering backend, which would allow you to use the OpenGL API with any kind of platform-specific API. If such a rendering backend is ever created, it might actually be sensible to deprecate the existing D3D and GLES backends. Since ANGLE also supports Vulkan, it might be unnecessary to implement a new backend for Vulkan support specifically. As far as JavaFX API is concerned, it should be relatively straightforward to create a control that can host a custom GL surface similar to D3DImage in WPF. It is generally not safe to expose the OpenGL rendering context that is used internally by JavaFX, because users might inadvertently change the GL state machine. A better approach would be to create a new shared context that can share surfaces with the main JavaFX GL context. Of course, having all of that might still not be really useful without a Java-based API for OpenGL. Perhaps JOGL could be adapted to work for this scenario? Am Di., 16. Feb. 2021 um 21:42 Uhr schrieb Mark Raynsford < org.open...@io7m.com>: > On 2021-02-16T16:09:04 +0000 > Neil C Smith <n...@codelerity.com> wrote: > > > > I agree with you, and have certain similar requirements, like being > > able to allow GStreamer and JavaFX to share GPU contexts. In fact, > > was bugging Johan about this in the chat around his FOSDEM talk, and > > promised to follow up here, so might as well pop my head above the > > parapet. :-) > > I do think it ultimately boils down to "provide an image to JavaFX when > it asks for one, and don't allow the image to leave the GPU". I've used > GStreamer outside of Java quite a bit for aggregating feeds from > network cameras... In your case, is it that you want to consume one or > more video streams from GStreamer inside a JavaFX UI? We'd be pretty > close in terms of requirements with regards to not copying across > CPU/GPU boundaries, and requiring low latency. :) > > > I certainly don't know what such an API should look like, but in some > > ways to me parallels the differences between io file and nio2 files - > > hidden by abstraction vs type-safe queryable capabilities / profiles. > > In fact, given above, also something slightly akin to GStreamer's > > context querying. > > > > Incidentally, reading your initial post about PixelBuffer reminds me > > that I should also follow up on a point about concurrency issues with > > that from last year. The API has (or had?) issues with accessing the > > buffer from the rendering thread after it's been removed in the event > > thread, which particularly with externally allocated buffers makes it > > hard to safely mark as invalid to allow them to be freed. > > Yeah, that is a problem. It's why I said in my original email that I > was resigned to the fact that a copy was probably going to have to > happen somewhere. Without going into API specifics, I was sort of > picturing JavaFX maintaining two images on the GPU; at any given time > one is being displayed (ie. used as a texture whilst rendering a scene) > and the other is being offered to the user as a destination for a GPU > copy operation. It would be the responsibility of the user to copy an > image in the right format into the offered buffer. There would need to > be the appropriate memory barriers inserted, and I don't know how this > could/would be handled given that it might be two completely different > graphics APIs involved (there isn't a Vulkan JavaFX backend, for > example,so in my case I'm guaranteed to be talking to something that isn't > Vulkan). When an image had been copied in as required, and JavaFX had > been told that this had happened, the two images would be swapped on the > GPU. > > -- > Mark Raynsford | https://www.io7m.com > >
