> > I am interested in storing custom metadata on a per frame level in a mov > or mp4 file. It looks like using user data unregistered (UDU) > supplemental enhancement information (SEI) may be a suitable approach for > this. I am however struggling to get this to work as I am always reading > back the custom message I used for the first frame.
Turns out I was wrongly assuming from the API that there can be only one item of AV_FRAME_DATA_SEI_UNREGISTERED. I was accumulating messages in a single frame and only displaying the first one. This is now fixed in my gist by calling `av_frame_remove_side_data`: https://gist.github.com/tvercaut/94c68e46d0d0321a2c83b88024d9cd69 It seems a bit unclear to me though why `av_frame_get_side_data` only returns the first item rather than a list of items. > I am however struggling to get this to work as I am always reading back > the custom message I used for the first frame. > > You can't, at least using FFmpeg Tools. I am not sure if there is an > libavXXX function which would allow you to, but FFMpeg tools themselves > cannot. Looking at the FFmpeg patches for Apple's ambient viewing > brightness, and from Discussions in the FFmpeg IRC, I understand that to > extract that data you need to register a UUID for your usecase in > libavcodec to be able to pull information, which means modifying > libavcodec. Don't know how accurate my understanding is though. > > You can follow <https://stackoverflow.com/questions/67283568 to get > FFMpeg to announce the presence of an unrecognized custom SEI Message, but > there is no function to actually display it. > > In my project I resorted to reading the H264 Stream binary as a whole, > scanning for the UUID / Magic Number of the SEI Message and extracting the > JSON Data that way. Performance wise this was actually really good, as I > could export nice Charts of my Data without interfacing with FFmpeg, but > that is really more of a hackye workaround. > > I think injecting custom data through SEI messages is really cool, as you > can Livestream sensor data in tandem with video, without sync issues. Hope > this workflow can become more elegant and less hacky. > Thanks. I found a way with libav as mentioned above but would still be interested in being able to extract that SEI from the ffmpeg command line.
_______________________________________________ Libav-user mailing list Libav-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/libav-user To unsubscribe, visit link above, or email libav-user-requ...@ffmpeg.org with subject "unsubscribe".
