Hello,I'm building an industry grade streaming platform where I use ffmpeg and ffprobe to convert and parse information from media objects. I'm using ffprobe to read all available information from a mp4 file using the following command:
ffprobe -print_format json -export_all true -show_streams -show_format -loglevel quiet -hide_banner /path/to/file.mp4
But if I take a closer look at the output of it, I'm not able to check if a subtitle is enforced, marked as SDH or auxiliary content. The only thing I really see is when the subtitle is marked as a default track which does not help at all, e.g.:
{
"index": 1,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x2",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "44",
"nb_frames": "8",
"extradata_size": 48,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
You might wonder what I mean by a subtitle is marked as SDH, enforced or
auxiliary content. These are extended information of a specific track
element of a subtitle embedded into a mp4 container, in most cases these
embedded subtitles come in tx3g format. The most advanced tool I know of
to write metadata and tracks into a mp4 container is called "Subler" and
it's sadly only available on MAC OS since many years.
Subler is extremely conform with apple's definition of mp4 containers
and how to "metarise" them, by that it very nicely integrates with
QuickTime, iOS etc. Subler also allows me to set a subtitle track of a
mp4 as SDH, enforced, auxiliary content etc. But I'm not able to read
this information later using ffprobe, which is absolutely mandatory for
me as I repack media from these highly metarised mp4 containers into
HLS/Dash streams where I really want to know if a subtitle is marked as
enforced or SDH. Otherwise, I cannot display this information to the
end-user using e.g. VideoJS, as all the Video players out there parse
their subtitle track names from a m3u8/mpd manifest you pass them. So
the only information I can rely on while building a HLS/DASH stream from
a mp4 input is the ffprobe's output, which currently does not cover
these fields I have with subler and would really solve this issue. The
information is there, but I can't read it using ffprobe to put it in
more simple words. The only thing I can do at the moment is to compare
the nb_frames value and if I have 2 subtitles in the same language.
Maybe I have 2x English subtitles, one is shorter than the other, than I
maybe know what of the two is the enforced subtitle track and which is
the full subtitle track ... This workaround kinda sucks, and I still
have no clue what track is marked as SDH.
I created an example mp4 file with 4 subtitle tracks in it, each of the subtitle tracks covers a specific type of metadata setup which I want to be able to read using ffprobe. I also included screenshots where I show how each subtitle track is metarised under the use of Subler.
https://drive.google.com/file/d/1kShvOA6_Emx3COrMudn3V2__zErj8_-c/view?usp=sharingIf I parse the example file using the command I showd above, I get backe the following which does not reflect the metadata I set befor using subler, execpt of one single field "default": 1, this is the only one that works for me:
{
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part
10",
"profile": "High",
"codec_type": "video",
"codec_tag_string": "avc1",
"codec_tag": "0x31637661",
"width": 1920,
"height": 1080,
"coded_width": 1920,
"coded_height": 1080,
"closed_captions": 0,
"film_grain": 0,
"has_b_frames": 2,
"pix_fmt": "yuv420p",
"level": 40,
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "true",
"nal_length_size": "4",
"id": "0x1",
"r_frame_rate": "24000/1001",
"avg_frame_rate": "24000/1001",
"time_base": "1/24000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 677677,
"duration": "28.236542",
"bit_rate": "10833691",
"bits_per_raw_sample": "8",
"nb_frames": "677",
"extradata_size": 41,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"language": "eng",
"handler_name": "VideoHandler",
"vendor_id": "[0][0][0][0]"
}
},
{
"index": 1,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x2",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "44",
"nb_frames": "8",
"extradata_size": 48,
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 2,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x3",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "74",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 3,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x4",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "82",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "eng"
}
},
{
"index": 4,
"codec_name": "mov_text",
"codec_long_name": "MOV text",
"codec_type": "subtitle",
"codec_tag_string": "tx3g",
"codec_tag": "0x67337874",
"width": 1920,
"height": 162,
"id": "0x5",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 28237,
"duration": "28.237000",
"bit_rate": "94",
"nb_frames": "14",
"extradata_size": 48,
"disposition": {
"default": 0,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0,
"captions": 0,
"descriptions": 0,
"metadata": 0,
"dependent": 0,
"still_image": 0
},
"tags": {
"creation_time": "2022-09-17T11:23:47.000000Z",
"language": "deu"
}
}
],
"format": {
"filename": "subtitle_sample_1920x1080.mp4",
"nb_streams": 5,
"nb_programs": 0,
"format_name": "mov,mp4,m4a,3gp,3g2,mj2",
"format_long_name": "QuickTime / MOV",
"start_time": "0.000000",
"duration": "28.237000",
"size": "38253389",
"bit_rate": "10837805",
"probe_score": 100,
"tags": {
"major_brand": "mp42",
"minor_version": "512",
"compatible_brands": "isomiso2avc1mp41",
"tagc": "public.main-program-content"
}
}
}
It would really be awesome if somebody could add compatibility here, as
in the end I just need to be able to read if the subtitle is enforced or
SDH, everything else is just "Main Program" to me. Adding this feature
would also make it much simpler for other developers to build proper
streaming outputs.
-- Kind regards, Robin Linden
0x257F9C8D.asc
Description: application/pgp-keys
_______________________________________________ ffmpeg-user mailing list ffmpeg-user@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email ffmpeg-user-requ...@ffmpeg.org with subject "unsubscribe".
