Hello here is the work I have pushed in a fork of FFmpeg integrated in source code of xuggler (a Java/C++ wrapper of FFmpeg) https://github.com/olivierayache/xuggle-xuggler/commit/98469ebea59c921f46a114bd954f7a27a7c95a41
For the limitations of NDK I agree with you and that is why I have made some backport to be able to support all needed features for FFmpeg (when I worked on that it was FFmpeg 3.4.x version). This backport allows support to these functions for Android devices since API 21 (which is more than 99% of devices). - AMediaCodec_createInputSurface - AMediaCodec_setParameters - ANativeWindow_toSurface For testing reviewing the patch I will make it as soon as I update FFmpeg to the current version. I will be able to do it this week. Olivier Le dim. 13 nov. 2022, 12:49, Zhao Zhili <quinkbl...@foxmail.com> a écrit : > > > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of > Olivier Ayache > > Sent: 2022年11月12日 18:00 > > To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> > > Cc: matthieu.bou...@stupeflix.com; a...@tmm1.net > > Subject: Re: [FFmpeg-devel] [Internet][PATCH 00/12] Add MediaCodec > encoder and NDK MediaCodec support > > > > Hello there I implemented that few years ago in > > https://github.com/olivierayache/xuggle-xuggler/ > > I can't find where is the patch or source code of FFmpeg. > > > And when I tried to submit a patch to FFmpeg for add support for NDK I > had > > received this answer. > > I think using NDK directly rather than JNI can be better in case of > > developing native application on Android. > > NDK is still missing some important features, and some features only > supported recently. > So the NDK implementation doesn't meant to be a replacement of the JNI > version: > 1. JNI version has more features and supports more devices > 2. NDK implementation combined with some JNI functions has almost the same > feature > As JNI version on new devices. > 3. In the environment where JVM is unavailable, the NDK version is the > only choice. > > > If I can help on these subjects I would be really happy > > Welcome to review and/or test the patch. > > > > > Olivier Ayache > > > > > > > > ---------- Forwarded message --------- > > From: Olivier Ayache <olivier.aya...@gmail.com> > > Date: Sun, Jun 28, 2020 at 2:48 PM > > Subject: Re: [FFmpeg-devel] [PATCH] add support to Android ndk MediaCodec > > for encoding/decoding > > To: <ffmpeg-devel@ffmpeg.org> > > > > > > Thank for your replies I choose the NDK approach in order to be able to > be > > independant from the JVM, to maximize performance and to be sure to > detect > > compatibility issues (using JNI approach can compile and crash at > runtime). > > Concerning the version of Android it is compatible from API 21 (94% of > > devices) > > I agree that JNI overhead is not big but in combinaison with Xuggler it > > seems to me a little weird because Xuggler (Java part) already use JNI to > > interact with Xuggler (C++) and FFmpeg. If FFmpeg uses JNI in the other > way > > (from c to Java) it seems to me an anti pattern. > > > > If you think it is better to begin to use JNI I could transform my > encoder > > and we could discuss about NDK after > > > > Olivier > > > > > > > > Le sam. 27 juin 2020 à 20:55, Martin Storsjö <mar...@martin.st> a écrit > : > > > > > On Sat, 27 Jun 2020, Olivier Ayache wrote: > > > > > > > Hi everyone this is the first time I post on this mailing list. I am > > > > working since several years on a fork of Xuggler for manipulating > ffmpeg > > > > API with Java/Kotlin. > > > > This work leads me to develop encoder and decoder based on Android > NDK > > > > MediaCodec. > > > > > > > > This work can be found on my Github repository > > > > > > > > https://github.com/olivierayache/xuggle-xuggler > > > > > > > > > > > > I know that FFmpeg already integrate MediaCodec for decoding via Jni > > > > wrappers since version 3.1. I began this work on FFmpeg 2.8.x and I > > > choose > > > > the NDK in order to achieve optimum performance. > > > > > > If you mean you used the NDK MediaCodec API, I'd suggest you use the > same > > > JNI wrappers as ffmpeg has already, for consistency. The overhead of a > few > > > JNI calls per encoded frame is generally negligible. If it, at a later > > > time, is decided to drop support for older versions at some point, it > > > should be straightforward to convert it to use the NDK MediaCodec API > > > instead. > > > > > > // Martin > > > > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > > > > On Sat, Nov 12, 2022 at 6:07 AM Zhao Zhili <quinkbl...@foxmail.com> > wrote: > > > > > > > > > > > > -----Original Message----- > > > > From: ffmpeg-devel <ffmpeg-devel-boun...@ffmpeg.org> On Behalf Of > Chema > > > Gonzalez > > > > Sent: 2022年11月12日 0:55 > > > > To: FFmpeg development discussions and patches < > ffmpeg-devel@ffmpeg.org> > > > > Cc: matthieu.bou...@stupeflix.com; a...@tmm1.net > > > > Subject: Re: [FFmpeg-devel] [Internet][PATCH 00/12] Add MediaCodec > > > encoder and NDK MediaCodec support > > > > > > > > On Thu, Nov 10, 2022 at 7:36 PM "zhilizhao(赵志立)" < > quinkbl...@foxmail.com> > > > wrote: > > > > > > > > > > Ping for review. > > > > > > > > > > > On Oct 24, 2022, at 11:16, Zhao Zhili <quinkbl...@foxmail.com> > > > wrote: > > > > > > > > > > > > From: Zhao Zhili <zhiliz...@tencent.com> > > > > > > > > > > > > Firstly, some bugs were fixed (patch 1-4). > > > > > > > > > > > > Patch 5 and 6 make mediacodec_wrapper support Java MediaCodec > and NDK > > > > > > MediaCodec. The use case I'm considering is run FFmpeg on cmdline > > > without JVM, > > > > > > for example, run FFmpeg inside of termux (an Android terminal > > > emulator). It's > > > > > > well known that NDK MediaCodec missing some important functions, > > > like get the > > > > > > list of codecs, but still useable. > > > > > > > > > > > > Patch 7 add NDK MediaCodec decoder support. It can be enabled via > > > options, > > > > > > and enabled automatically if no JVM is available. > > > > > > > > > > > > Patch 8 add ANativeWindow support to hwcontext_mediacodec. It > can be > > > set by > > > > > > user, and can be created via > AMediaCodec_createPersistentInputSurface > > > > > > automatically. This is a preparation for encoder. > > > > > > > > > > > > Patch 9 makes MediaCodec decoder to support ANativeWindow > directly. > > > It worth > > > > > > to note that AVMediaCodecContext has only surface. Although we > > > provided > > > > > > av_mediacodec_alloc_context(), we didn't strictly prevent users > to > > > allocate > > > > > > AVMediaCodecContext on stack. I'm not sure if it's OK to add new > > > field to > > > > > > AVMediaCodecContext. > > > > > > > > > > > > Patch 10 add MediaCodec encoder support. Frame can be feed to > > > encoder via > > > > > > buffer, or via Surface/ANativeWindow. If Surface/ANativeWindow is > > > used, and > > > > > > the frames come from our MediaCodec decoder wrapper, we can > control > > > it's > > > > > > 'render' (send to encoder's surface) via > > > av_mediacodec_release_buffer(). A DTS > > > > > > generation strategy works in this case. However, if frames comes > > > from other > > > > > > sources, like a camera, there is no way to control the 'render' > yet, > > > so DTS is > > > > > > missing in this case. > > > > > > > > > > > > Finally, we can do mediacodec transcoding with FFmpeg cmdline on > > > Android. > > > > > > More importantly, we can do MediaCodec decoder to encoder without > > > copy frames, > > > > > > although it's very limited since most of avfilters doesn't work. > For > > > example: > > > > > > > > > > > > ./ffmpeg -hwaccel mediacodec -hwaccel_output_format mediacodec -i > > > /sdcard/test.mp4 -an -c:v h264_mediacodec -y > > > > /sdcard/out.mp4 > > > > > > > > How do you build the ffmpeg binary you're using for experiments in > the > > > > android device? > > > > > > Just configure with --enable-mediacodec and --enable-jni. Here is the > build > > > script I used for the development. > > > > > > https://github.com/quink-black/ffmpeg-ci/blob/master/android_ffmpeg.sh > > > > > > Here is the Android Studio project which can be used to test the > mediacodec > > > wrapper implemented via JNI. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Since there is no real AVHWFrameContext implementation in > > > hwcontext_mediacodec. > > > > > > there is no hwframe_ctx for mediacodec and > > > av_hwframe_transfer_data() doesn't > > > > > > work. So if -hwaccel_output_format isn't being specified like: > > > > > > > > > > > > ./ffmpeg -hwaccel mediacodec -i /sdcard/test.mp4 -an -c:v > > > h264_mediacodec -y /sdcard/out.mp4 > > > > > > > > > > > > It will trigger a crash in av_hwframe_transfer_data. Patch 11 > add a > > > check on > > > > > > hwframe_ctx. Patch 12 set hwaccel_output_format automatically to > > > avoid such > > > > > > case. > > > > > > > > > > > > > > > > > > Zhao Zhili (12): > > > > > > avcodec/mediacodec: fix incorrect crop info > > > > > > avcodec/mediacodecdec: don't break out if both input and output > port > > > > > > return try again > > > > > > avcodec/mediacodecdec_common: fix misuse av_free/av_freep > > > > > > avcodec/mediacodecdec_common: fix useless av_buffer_unref > > > > > > avcodec/mediacodec_wrapper: separate implementation from > interface > > > > > > avcodec/mediacodec: add NDK media codec wrapper > > > > > > avcodec/mediacodecdec: enable NDK mediacodec > > > > > > avutil/hwcontext_mediacodec: add ANativeWindow support > > > > > > avcodec/mediacodec: add ANativeWindow support > > > > > > avcodec: add MediaCodec encoder > > > > > > avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc > > > > > > fftools/ffmpeg_opt: set default hwaccel_output_format for > mediacodec > > > > > > > > > > > > Changelog | 2 + > > > > > > configure | 6 + > > > > > > fftools/ffmpeg_opt.c | 4 + > > > > > > libavcodec/Makefile | 2 + > > > > > > libavcodec/allcodecs.c | 2 + > > > > > > libavcodec/mediacodec_surface.c | 46 +- > > > > > > libavcodec/mediacodec_surface.h | 8 +- > > > > > > libavcodec/mediacodec_wrapper.c | 942 > > > +++++++++++++++++++++++++++--- > > > > > > libavcodec/mediacodec_wrapper.h | 275 +++++++-- > > > > > > libavcodec/mediacodecdec.c | 21 +- > > > > > > libavcodec/mediacodecdec_common.c | 35 +- > > > > > > libavcodec/mediacodecdec_common.h | 1 + > > > > > > libavcodec/mediacodecenc.c | 495 ++++++++++++++++ > > > > > > libavcodec/version.h | 2 +- > > > > > > libavutil/hwcontext.c | 6 +- > > > > > > libavutil/hwcontext_mediacodec.c | 56 +- > > > > > > libavutil/hwcontext_mediacodec.h | 11 + > > > > > > libavutil/version.h | 4 +- > > > > > > 18 files changed, 1780 insertions(+), 138 deletions(-) > > > > > > create mode 100644 libavcodec/mediacodecenc.c > > > > > > > > > > > > -- > > > > > > 2.25.1 > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > ffmpeg-devel mailing list > > > > > ffmpeg-devel@ffmpeg.org > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > > > > > To unsubscribe, visit link above, or email > > > > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > > _______________________________________________ > > > > ffmpeg-devel mailing list > > > > ffmpeg-devel@ffmpeg.org > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > > > To unsubscribe, visit link above, or email > > > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > > > > _______________________________________________ > > > ffmpeg-devel mailing list > > > ffmpeg-devel@ffmpeg.org > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > > > To unsubscribe, visit link above, or email > > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".