On Thu, May 22, 2025 at 7:03 PM Timothée <timothee.informati...@regaud-chapuy.fr> wrote: > The 2025-05-22T14:33:22.000+02:00, Ramiro Polla > <ramiro.po...@gmail.com> wrote : > > On Thu, May 22, 2025 at 1:59 PM Ronald S. Bultje <rsbul...@gmail.com> wrote: > >> On Wed, May 21, 2025 at 9:34 AM Timothée < > >> timothee.informati...@regaud-chapuy.fr> wrote: > >>> I am interested in expanding ffmpeg's capabilities to extract > >>> low-level data from video codecs. Specifically, I'd like to > >>> implement functionality that would allow exporting frame data, > >>> macroblock information, quantization tables, and similar > >>> codec-specific elements to binary files for further analysis. > >>> After searching through the documentation and existing features, > >>> I haven't found similar functionality, though I may have missed > >>> something. Has this been implemented before, or are there > >>> related features I should examine? > >> > >> Some older codecs implement minor variants for this, e.g. grep > >> for AV_FRAME_DATA_MOTION_VECTORS, which attaches a frame's motion > >> vectors to the picture data. I believe there's an example app and > >> possibly a filter to overlay MVs on top of the video frame based > >> on this concept. You could extend this to cover other (macro)block > >> info. There used to be a variant of this for quant-tables also but > >> I can't find it, maybe it was removed. > > > > If I recall correctly, we wanted to move away from exporting more > > of this kind of codec-specific information as picture data. > > Timothée, this kind of feature depends a lot on the codec that you > > want to work with. It will also depend on which syntax elements you > > need from the codec. In FFmpeg, there is the CBS code which > > describes the bitstream for a few codecs, which might help you. I > > have a separate project (called FFglitch), which lets you export > > some elements (such as quantization tables, dct coefficients, and > > motion vectors) from some codecs (jpeg, mpeg2, mpeg4) to JSON > > format, and even modify them in the bitstream. In the end, it will > > depend on what you want to do with the extracted low-level data, and > > for which codecs. Ramiro > > Thank you for your response and for mentioning FFglitch - it looks > like an excellent project that's very relevant to what I'm trying to > accomplish. > > I have a question about FFglitch's capabilities: is it possible to > extract quantization parameter (QP) tables using ffedit? I've reviewed > the documentation but may have missed this functionality if it exists.
You can get the dqt for jpeg and qscale for mpeg-2, but that's about it for the moment. > Ideally I would want to extract data for all existing codecs, but I > know this is impossible. My current focus is on implementing this > feature for H.264, with plans to extend support to H.265 and AVI > formats as well. I want to extract this data for scientific use. I see a few qp fields in libavcodec/cbs_h264.h and libavcodec/cbs_h265.h. I've never used this functionality before, but it might help you. OTOH, if you want to add h264 support to FFglitch I would be very happy :) Ramiro _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
