On 05/02/2024 01:19, Tomas Härdin wrote:
[...] Which entry in the table would the provided file correspond to? To me it seems none of them fit. There's two fields, meaning two j2k codestreams, in each corresponding essence element KLV packet (I think, unless CP packets get reassembled somewhere else). Entry I2 seems closest but it specifies FULL_FRAME. I1 is otherwise tempting, but there SampleRate should equal the field rate whereas the file has SampleRate = 30000/1001.
Other examples I have (not shareable) with 2 jp2k pictures per KLV have identification from an old version of AmberFin iCR, I have no file with the I2 correctly signaled, with my first example it isI2 (2 fields per KLV) with I1 Header Metadata Property Values **but** with I2 essence container label which has a content byte (byte 15 of the UL) of 0x04 = I2. The AmberFin iCR files have the generic essence container label with content byte of 0x01 = FU (Unspecified) so for my main use case we could activate the search of the 2nd jp2k only if I2 is explicitly signaled by the essence container label but it would prevent to catch the 2nd field when this signaling is unspecified and buggy Frame layout + sample rate + edit rate.
I agree that this is not what is defined in ST 422 in full, but note that frame layout and height are not required by ST 377 (only "best effort") so IMO we should not rely much on them, and at least we should handle what is in the wild, correct me if I am wrong but handling non conforming content seems an acceptable policy in FFmpeg (I think to e.g. DPX and non conforming EOLs of some scanners, their names are directly written in FFmpeg source code). Video Line Map is also best effort but without it it is not possible to know the field_order, I wonder what should be done in that case. Currently I rely on current implementation in FFmpeg for catching field_order and don't try to find the 2nd field if field_order is AV_FIELD_UNKNOWN (not important for me as all files I have have field_order related metadata).
Also if I manually edit the MXF for having a conforming I2 property values, FFmpeg behaves same (still indicating half height and still silently discarding the 2nd field), so in my opinion the handling of 2 jp2k pictures per KLV is still relevant for handling correctly I2 conforming files and tolerating wrong property values may be relevant (checking essence container label only? to be discussed).
On 03/02/2024 20:58, Tomas Härdin wrote:
The fastest way, in a player, is probably to do it with a shader. That should be the least amount of copies and the most cache coherent.
As far a I know the player is not aware that the AVFrame actually contains a field so it can not apply a shader or something else, which AVFrame field indicates that this is a a field to be interleaved with the next AVFrame before display? Currently for I1 files ffplay or VLC show double rate half height so it seems that they don't catch that AVFrame contains a field.
On 03/02/2024 21:04, Tomas Härdin wrote:
It should also be said that what this patch effectively does is silently convert SEPARATE_FIELDS to MIXED_FIELDS. What if I want to transcode J2K to lower bitrate but keep it SEPARATE_FIELDS?
I don't get what is the expected behavior of FFmpeg: what is the meaning of "243" in "Stream #0:0: Video: jpeg2000, yuv422p10le(bottom coded first (swapped)), 720x243, lossless, SAR 9:20 DAR 4:3, 29.97 fps, 29.97 tbr, 29.97 tbn"
My understanding is that a frame height is expected, never a field height, and there is no indication in the current output that 243 is a field height for both I1 & I2, so the "silent conversion" would be expected by the user in order to have a frame in the output and never a field, am I wrong?
Also it seems that there is no way to signal that the outputted picture is a field and not a frame, and FFmpeg handles I1 (1 field per KLV) as if it ignores that this is a field and not a frame, so when a I1 file is converted to another format without an interleave filter manually added the output is an ugly flipping double rate half height content. Silently converting a field to a frame seems to me a worse behavior than silently converting SEPARATE_FIELDS to MIXED_FIELDS because the output is not what is expected by the person who created the file as well as the person watching the output of FFmpeg.
What if I want to transcode J2K to lower bitrate but keep it SEPARATE_FIELDS?
Interlacing the lines then encoding separately the fields? It is more a matter of default behavior (deinterlace or not) and who would need to apply a filter, my issue is that I see no way to signal in FFmpeg that "got_frame" means "got frame or field" and that AVFrame contains a field so I would prefer that the default behavior is to handle frames in AVFrame rather than fields. Is it acceptable?Additionally the MXF container indicates (for conforming files) that I2 edit rate is for a frame even if there are 2 separate fields in the KLV, do you expect that FFmpeg forces separate fields in separate AVFrame like I1 even when the MXF muxer explicitly said that it is expected that edit is by frame? We have here 3 cases (separated fields in separated KLV, 2 separated fields in 1 KLV, mixed fields in 1 KLV), my understanding of current AVFrame is that only mixed fields (so 1 frame) in 1 AVFrame is supported, and both other MXF methods are silently converted to it (fields handled as frame for separated fields in separated KLV, 2nd field discarded for 2 separated fields in 1 KLV). My planned next step is to handle I1 files automatically in the right output (a frame rather than fields in AVFrame and output) so users don't have the surprise to have a double rate half height content in MKV or MP4 without any signaling about that and badly handled by any player, including ffplay. Wouldn't it be acceptable? Should I split the 2 I2 fields and provide 2 AVFrame?
In other words, I would like to know if AVFrame is intended at long term to handle also fields in addition to frames, and if so is there a way to signal that the AVFrame structure actually contains a field so players/transcoders (including FFmpeg) can do the interleave before showing (ffplay) something or converting to a format not supporting separated fields, if not what is the prefered approach (my current proposal, decode both fields then interleave, adding an interleave filter, splitting a KLV into 2 AVPacket, other?).
Jerome _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
