At minimum the Media Player must support H.264, MP3, AAC (so there is a guarantee of a cross-platform format) and then defer to the default platform media decoder for everything else. I.e. *anything* that the standard OS provided media system can decode - I.e. QuickTime on Mac, GStreamer on Linux, DirectShow on Windows. But all that is needed is the hook and the community can make the bindings to QuickTime, VLC, etc.
The current implementation appears to be too unstable to rely on at the moment anyway though: https://javafx-jira.kenai.com/browse/RT-31412 I have heard rumors of people being able to play HD video via Canvas. I have tried everything and can't come close. (Yes, I have been careful about the pixel format.) I mean, it looks like it is working for a few seconds, but then as the memory fills with the Canvas backlog it can lead to the GC using a lot more CPU, thus reducing the ability for Canvas to process its command queue even further, well it just collapses in on itself and dies. Is your app able to do *anything* else while the video is playing? The slightest delay to the rendering and that Canvas buffering bug kills the app. Not that it would matter if it could keep up, because the off-screen thing is also a deal breaker. Of course 25fps is well below the 60fps required for full-speed video. I suspect it is the frame rate more than the frame size that matters here. Plain old, standard definition, interlaced, 60 fields per second will probably kill most apps with the current Canvas implementation. We need something better. I proposed having at least a JNI method so we could get native window handles from Stages but didn't get any traction either. Security was brought up as a concern, which I totally do NOT understand as the use of JNI means you are out of the Java sandbox already. If we had native window handles we might be able get our own window for rendering to at least interact nicely with the Stage. (We already did this successfully in Swing via JAWT and a heavyweight component that we paint to from native code) Regards, Scott On Thu, Aug 8, 2013 at 9:06 PM, John Hendrikx <hj...@xs4all.nl> wrote: > On 9/08/2013 02:10, Scott Palmer wrote: > >> The Media APIs are mostly useless in their current state. Other than >> demoing that you can play a video, they don't go far enough to be of >> practical value. I tried to get someone to pay attention to them back in >> the JavaFX 1.0 days >> https://javafx-jira.kenai.com/**browse/RT-2684<https://javafx-jira.kenai.com/browse/RT-2684> >> > > Unfortunately, I have to agree here. I'd love to use the Media APIs of > JavaFX, but they are way to limited. At a minimum I'd need support for MKV > containers and some common audio formats used with these. Decoding AVI > containers would be rather important as well. Without those, JavaFX video > is basically limited to "bring your own" videos only and completely > unsuitable for playing back videos that end users might have... > > Instead, I've been using VLC + VLCJ from day one while working with JavaFX > as a work-around. > > > at least someone listened to the request to get H.264 support in there, >> but >> that is just a workaround. We need to be able to get our data into the >> media pipeline. This would allow those of us that have attempted to do a >> video window to have a fighting chance. Canvas can't keep up and will >> likely crash the app with out of memory errors. Support for drawing into >> a >> native surface (OpenGL or D3D context) has been talked about, but doesn't >> appear to be on the horizon yet. If we just had a hook to get the dang >> pixel data into the media pipeline so we could supply the "next frame" >> with >> whatever we want - either from any native codec via JNI, or dynamically >> generated from Java code, whatever... that would be just so dang useful... >> (to me at least) >> > This would atleast allow me to decode the MKV container myself and supply > data... not looking forward to having to write all that code, but I would > if it meant I'd be able to go "native" JavaFX with video. > > Canvas however is working for me even with HD video (copying 25 frames/sec > of HD video from VLCJ to Canvas). Playback is smooth even with 1920x1080 > video for hours long video, and I can't tell the difference with a native > player or copying frame by frame using pixelWriter anymore. There is some > CPU penalty but on a my system it is well below 5%. This is basically how > that looks with VLCJ: > > new RenderCallback() { > @Override > public void display(DirectMediaPlayer mp, Memory[] memory, final > BufferFormat bufferFormat) { > final ByteBuffer byteBuffer = memory[0].getByteBuffer(0, > memory[0].size()); > > Platform.runLater(new Runnable() { > @Override > public void run() { > pixelWriter.setPixels(0, 0, bufferFormat.getWidth(), > bufferFormat.getHeight(), byteBgraInstance, byteBuffer, > bufferFormat.getPitches()[0]); > } > }); > } > } > > However, the bug where Canvas keeps buffering data when off-screen > sometimes bites me, resulting in out of memory errors -- even slight > display delays can fairly easily cause this when video is running at 25 fps > (takes about a second to go through 256 MB of memory). > > --John > > >> Regards, >> >> Scott >> >> >> >> On Thu, Aug 8, 2013 at 5:04 PM, Fabrizio Giudici< >> fabrizio.giud...@tidalwave.it> wrote: >> >> On Thu, 08 Aug 2013 22:57:51 +0200, Joe McGlynn<joe.mcgl...@oracle.com** >>> > >>> wrote: >>> >>> I don't know why FX Media isn't in the FX 8 API docs, but that's >>> clearly >>> >>>> an error. Please file a bug on that. >>>> >>>> In the meantime, you should look at the FX 2 media docs, there isn't a >>>> lot of change from FX2 media in FX8. Buffering and streaming (HTTP Live >>>> Streaming) are both supported, as is playback from a URL. >>>> >>>> What is the strategy for codecs? I mean, now we have ImageIO (there is >>> also JAI but it seems basically dead). ImageIO provides many image codecs >>> and there's a SPI that can be used to support more formats. Will it be >>> replaced by FX2 media or co-exist with it? >>> >>> >>> -- >>> Fabrizio Giudici - Java Architect @ Tidalwave s.a.s. >>> "We make Java work. Everywhere." >>> http://tidalwave.it/fabrizio/****blog<http://tidalwave.it/fabrizio/**blog> >>> <http://tidalwave.it/**fabrizio/blog <http://tidalwave.it/fabrizio/blog>> >>> - >>> fabrizio.giud...@tidalwave.it >>> >>> >
