PR #24056 opened by Qingzheng Li (iSoldLeo) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24056 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24056.patch
## Problem When VideoToolbox emits its own H.264 SEI NAL, the current A53 path tries to append the caption payload to that NAL. The insertion code scans already escaped EBSP as if it were RBSP, so emulation-prevention bytes can move the apparent message boundary. In longer caption-bearing streams this can corrupt the existing SEI, lose A53 data, or make following NAL units undecodable. ## Changes The first patch moves H.264-without-A53 and HEVC length-prefix conversion to a checked reader-based helper. It uses identical count and write passes, allocates the exact Annex B packet size, accepts only 1-, 2-, and 4-byte length prefixes, and rejects zero, truncated, or overlong NAL units. It also confines codec- and A53-specific code to the corresponding build configurations and releases an allocated packet if the adjacent raw CoreMedia copy path fails. Its only cosmetic change removes braces from the nearby single-statement keyframe flag check. The second patch keeps every VideoToolbox-produced H.264 NAL body byte-exact and writes A53 data in a standalone `user_data_registered_itu_t_t35` SEI before the first Annex A VCL NAL. The A53 path requires exactly one `CMSampleBuffer` sample and rejects samples without VCL or with a definite new-access-unit boundary after VCL. H.264 without A53 and HEVC use the generic converter; ProRes retains its raw-copy path. Portable FATE coverage checks NAL ordering, preservation of existing NAL bodies, 1/2/4-byte length prefixes, SEI size fields and escaping, exact and undersized capacities, malformed samples, definite post-VCL AU boundaries, and reader failures. ## Testing The broader matrix, sanitizer/fuzz/stress, and build-configuration results below were collected from the pre-rebase validation build (`N-125927-g25663e6924`). The relevant patched production and test sources used by that build are byte-identical to this series. The focused six-case VideoToolbox smoke and selected Homebrew reproduction were then rerun with the committed two-patch head (`N-125996-g458fb81bc7f`). - `make fate-videotoolboxenc-rewrite` - `make checkheaders` - ASan+UBSan unit test, 500,000-case independent-reference fuzz test, 32 MiB NAL copy stress, and 16 MiB synthetic SEI-writer capacity stress - H.264-only, HEVC-only, ProRes-only, static, shared, LTO, no-VideoToolbox, minimal, and full build configurations - A broader VideoToolbox matrix ran 179 cases per engine. The candidate passed all 157 reached cases. Reached coverage included caption-bearing AV1/H.264/HEVC/MPEG-2 inputs, A53 on/off, Baseline/Constrained Baseline/Main/High/Constrained High H.264, MP4/MKV/MOV/MPEG-TS/raw/fMP4/HLS, 23.976/59.94 fps, all-intra, B-frames, 360p through 4K, HEVC, and ProRes including alpha. Twenty Extended-profile cases, one real-multi-slice probe, and one HEVC-alpha probe were not reached and are not counted as passes. - On the committed two-patch head, a focused six-case VideoToolbox rerun passed H.264 at 60000/1001 fps, H.264 B-frames, H.264 with A53 disabled, a full-duration A53 input, HEVC, and ProRes 4444 with alpha. Homebrew reproduced the three selected old H.264/A53 failures. - Across the 150 cases where both candidate and Homebrew succeeded, normalized coded-picture content was identical (109 H.264, 15 HEVC, 26 ProRes). The seven candidate-only successes that reproduced the baseline A53/SEI failures contained 1,881/1,881 matching standalone A53 events under an independent AVCC/RBSP parser. For those seven files, AVFoundation frame counts, decoded NV12 active-pixel hashes, and microsecond-normalized PTS hashes were identical before and after removing all H.264 type-6 SEI NAL units. >From 556c626f0b2404aed8436b33592141559ad242f3 Mon Sep 17 00:00:00 2001 From: Qingzheng Li <[email protected]> Date: Sat, 8 Aug 2026 23:07:21 +0800 Subject: [PATCH 1/2] avcodec/videotoolboxenc: use checked NAL length conversion The current packet path counts NAL units by advancing over unchecked length prefixes, then estimates the Annex B allocation from that count. A malformed sample may therefore pass the count stage and fail only after allocation. For packets without A53 data, move length-prefix conversion to a reader-based helper. Use the same checked parser for count and write passes, allocate the exact packet size, accept only 1-, 2-, and 4-byte prefixes, and reject zero, truncated, or overlong NAL units. Keep the existing A53 insertion path unchanged for now. Add portable FATE coverage for prefix widths, malformed samples, capacity limits, and reader failures. Signed-off-by: Qingzheng Li <[email protected]> --- libavcodec/Makefile | 6 +- libavcodec/tests/videotoolboxenc_rewrite.c | 154 +++++++++++++++++ libavcodec/videotoolboxenc.c | 190 +++++++++++++++------ libavcodec/videotoolboxenc_rewrite.c | 123 +++++++++++++ libavcodec/videotoolboxenc_rewrite.h | 39 +++++ tests/fate/libavcodec.mak | 5 + 6 files changed, 465 insertions(+), 52 deletions(-) create mode 100644 libavcodec/tests/videotoolboxenc_rewrite.c create mode 100644 libavcodec/videotoolboxenc_rewrite.c create mode 100644 libavcodec/videotoolboxenc_rewrite.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index e464811af6..c194a1d11f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -454,7 +454,7 @@ OBJS-$(CONFIG_H264_VAAPI_ENCODER) += vaapi_encode_h264.o h264_levels.o \ OBJS-$(CONFIG_H264_VULKAN_ENCODER) += vulkan_encode.o vulkan_encode_h264.o \ hw_base_encode.o hw_base_encode_h264.o \ h264_levels.o h2645data.o -OBJS-$(CONFIG_H264_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o +OBJS-$(CONFIG_H264_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o videotoolboxenc_rewrite.o OBJS-$(CONFIG_H264_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_H264_V4L2M2M_ENCODER) += v4l2_m2m_enc.o OBJS-$(CONFIG_HAP_DECODER) += hapdec.o hap.o @@ -486,7 +486,7 @@ OBJS-$(CONFIG_HEVC_VULKAN_ENCODER) += vulkan_encode.o vulkan_encode_h265.o \ h265_profile_level.o h2645data.o OBJS-$(CONFIG_HEVC_V4L2M2M_DECODER) += v4l2_m2m_dec.o OBJS-$(CONFIG_HEVC_V4L2M2M_ENCODER) += v4l2_m2m_enc.o -OBJS-$(CONFIG_HEVC_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o +OBJS-$(CONFIG_HEVC_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o videotoolboxenc_rewrite.o OBJS-$(CONFIG_HNM4_VIDEO_DECODER) += hnm4video.o OBJS-$(CONFIG_HQ_HQA_DECODER) += hq_hqa.o hq_hqadsp.o hq_common.o canopus.o OBJS-$(CONFIG_HQX_DECODER) += hqx.o hqxdsp.o hq_common.o canopus.o @@ -1382,6 +1382,7 @@ TESTPROGS = avcodec \ htmlsubtitles \ jpeg2000dwt \ mathops \ + videotoolboxenc_rewrite \ TESTPROGS-$(CONFIG_APV_DECODER) += apv TESTPROGS-$(CONFIG_AV1_VAAPI_ENCODER) += av1_levels @@ -1417,6 +1418,7 @@ HOSTPROGS = aacps_tablegen \ CLEANFILES = *_tables.c *_tables.h *_tablegen$(HOSTEXESUF) $(SUBDIR)tests/dct$(EXESUF): $(SUBDIR)dctref.o $(SUBDIR)aandcttab.o +$(SUBDIR)tests/videotoolboxenc_rewrite$(EXESUF): $(SUBDIR)videotoolboxenc_rewrite.o $(SUBDIR)dv_tablegen$(HOSTEXESUF): $(SUBDIR)dvdata_host.o ifdef CONFIG_SMALL diff --git a/libavcodec/tests/videotoolboxenc_rewrite.c b/libavcodec/tests/videotoolboxenc_rewrite.c new file mode 100644 index 0000000000..c5fa53d150 --- /dev/null +++ b/libavcodec/tests/videotoolboxenc_rewrite.c @@ -0,0 +1,154 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdio.h> +#include <string.h> + +#include "libavutil/error.h" + +#include "libavcodec/videotoolboxenc_rewrite.h" + +typedef struct BufferReader { + const uint8_t *data; + size_t size; + int calls; + int fail_at; + int error; +} BufferReader; + +static int read_buffer(void *opaque, size_t offset, uint8_t *dst, size_t size) +{ + BufferReader *reader = opaque; + int call = reader->calls++; + + if (reader->fail_at >= 0 && call == reader->fail_at) + return reader->error; + if (offset > reader->size || size > reader->size - offset) + return AVERROR_INVALIDDATA; + if (size) + memcpy(dst, reader->data + offset, size); + return 0; +} + +static int rewrite_lengths(const uint8_t *src, size_t src_size, + size_t length_size, uint8_t *dst, + size_t capacity, size_t *written) +{ + BufferReader reader = { src, src_size, 0, -1, 0 }; + + return ff_vt_rewrite_length_prefixes(read_buffer, &reader, src_size, + length_size, dst, capacity, written); +} + +static int test_plain_converter(void) +{ + static const uint8_t src1[] = { 1, 0x65, 2, 0x61, 0xaa }; + static const uint8_t src2[] = { 0, 1, 0x65, 0, 2, 0x61, 0xaa }; + static const uint8_t src4[] = { + 0, 0, 0, 1, 0x65, 0, 0, 0, 2, 0x61, 0xaa, + }; + static const uint8_t expected[] = { + 0, 0, 0, 1, 0x65, + 0, 0, 0, 1, 0x61, 0xaa, + }; + static const uint8_t zero[] = { 0 }; + static const uint8_t truncated[] = { 0, 1 }; + static const uint8_t overlong[] = { 2, 0x65 }; + static const uint8_t trailing[] = { 1, 0x65, 0 }; + const uint8_t *sources[] = { src1, src2, src4 }; + const size_t source_sizes[] = { sizeof(src1), sizeof(src2), sizeof(src4) }; + const size_t length_sizes[] = { 1, 2, 4 }; + uint8_t dst[sizeof(expected) + 16]; + size_t needed = 0, written = 0, capacity; + int i; + + for (i = 0; i < 3; i++) { + if (rewrite_lengths(sources[i], source_sizes[i], length_sizes[i], + NULL, 0, &needed) < 0 || + needed != sizeof(expected) || + rewrite_lengths(sources[i], source_sizes[i], length_sizes[i], + dst, needed, &written) < 0 || written != needed || + memcmp(dst, expected, sizeof(expected))) { + fprintf(stderr, "plain converter length size %zu failed\n", + length_sizes[i]); + return 1; + } + } + + for (capacity = 0; capacity < sizeof(expected); capacity++) { + size_t j; + + memset(dst, 0xa5, sizeof(dst)); + if (rewrite_lengths(src1, sizeof(src1), 1, dst, capacity, + &written) != AVERROR_BUFFER_TOO_SMALL) + return 1; + for (j = capacity; j < sizeof(dst); j++) { + if (dst[j] != 0xa5) { + fprintf(stderr, "plain converter canary changed at %zu\n", j); + return 1; + } + } + } + + if (rewrite_lengths(NULL, 0, 1, NULL, 0, &written) < 0 || written || + rewrite_lengths(zero, sizeof(zero), 1, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(truncated, sizeof(truncated), 2, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(overlong, sizeof(overlong), 1, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(trailing, sizeof(trailing), 1, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(src1, sizeof(src1), 0, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(src1, sizeof(src1), 3, NULL, 0, + &written) != AVERROR_INVALIDDATA || + rewrite_lengths(src1, sizeof(src1), 5, NULL, 0, + &written) != AVERROR_INVALIDDATA || + ff_vt_rewrite_length_prefixes(NULL, NULL, 0, 1, NULL, 0, + &written) != AVERROR(EINVAL) || + ff_vt_rewrite_length_prefixes(read_buffer, NULL, 0, 1, NULL, 0, + NULL) != AVERROR(EINVAL)) { + fprintf(stderr, "plain converter malformed policy mismatch\n"); + return 1; + } + return 0; +} + +static int test_reader_failures(void) +{ + static const uint8_t src[] = { 1, 0x65 }; + BufferReader reader = { src, sizeof(src), 0, 0, AVERROR_EXTERNAL }; + uint8_t dst[8]; + size_t written; + + if (ff_vt_rewrite_length_prefixes(read_buffer, &reader, sizeof(src), 1, + NULL, 0, &written) != AVERROR_EXTERNAL) + return 1; + reader = (BufferReader) { src, sizeof(src), 0, 1, AVERROR_EXTERNAL }; + if (ff_vt_rewrite_length_prefixes(read_buffer, &reader, sizeof(src), 1, + dst, sizeof(dst), + &written) != AVERROR_EXTERNAL) + return 1; + return 0; +} + +int main(void) +{ + return test_plain_converter() || test_reader_failures(); +} diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 85561c0479..7563c8ba0c 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -26,6 +26,8 @@ #include <TargetConditionals.h> #include <VideoToolbox/VideoToolbox.h> +#include "config_components.h" + #include "libavutil/attributes.h" #include "libavutil/avassert.h" #include "libavutil/imgutils.h" @@ -42,6 +44,10 @@ #include "hwconfig.h" #include "internal.h" +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER +#include "videotoolboxenc_rewrite.h" +#endif + #if !HAVE_KCMVIDEOCODECTYPE_HEVC enum { kCMVideoCodecType_HEVC = 'hvc1' }; #endif @@ -475,6 +481,7 @@ static void vtenc_q_push(VTEncContext *vtctx, BufNode *info) pthread_mutex_unlock(&vtctx->lock); } +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER static int count_nalus(size_t length_code_size, CMSampleBufferRef sample_buffer, int *count) @@ -517,6 +524,7 @@ static int count_nalus(size_t length_code_size, *count = nalu_ct; return 0; } +#endif static CMVideoCodecType get_cm_codec_type(AVCodecContext *avctx, int profile, @@ -780,6 +788,7 @@ static void vtenc_output_callback( vtenc_q_push(vtctx, info); } +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER static int get_length_code_size( AVCodecContext *avctx, CMSampleBufferRef sample_buffer, @@ -810,6 +819,7 @@ static int get_length_code_size( *size = isize; return 0; } +#endif /* * Returns true on success. @@ -1833,6 +1843,7 @@ static void vtenc_get_frame_info(CMSampleBufferRef buffer, bool *is_key_frame) } } +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER static int is_post_sei_nal_type(int nal_type){ return nal_type != H264_NAL_SEI && nal_type != H264_NAL_SPS && @@ -2204,6 +2215,26 @@ static int get_sei_msg_bytes(const ExtraSEI* sei, int type){ return copied_size + sei->size / 255 + 1 + type / 255 + 1; } +typedef struct VTEncByteReader { + AVCodecContext *avctx; + CMBlockBufferRef block; +} VTEncByteReader; + +static int vtenc_read_bytes(void *opaque, size_t offset, + uint8_t *dst, size_t size) +{ + VTEncByteReader *reader = opaque; + OSStatus status = CMBlockBufferCopyDataBytes(reader->block, offset, + size, dst); + + if (status != kCMBlockBufferNoErr) + av_log(reader->avctx, AV_LOG_ERROR, + "Cannot copy VideoToolbox sample data: %d\n", status); + + return status == kCMBlockBufferNoErr ? 0 : AVERROR_EXTERNAL; +} +#endif + static int vtenc_cm_to_avpacket( AVCodecContext *avctx, CMSampleBufferRef sample_buffer, @@ -2214,26 +2245,31 @@ static int vtenc_cm_to_avpacket( int status; bool is_key_frame; - bool add_header; - size_t length_code_size; - size_t header_size = 0; - size_t in_buf_size; - size_t out_buf_size; - size_t sei_nalu_size = 0; int64_t dts_delta; - int nalu_count; CMTime pts; CMTime dts; - CMVideoFormatDescriptionRef vid_fmt; +#if !CONFIG_H264_VIDEOTOOLBOX_ENCODER && !CONFIG_HEVC_VIDEOTOOLBOX_ENCODER + (void)sei; +#endif vtenc_get_frame_info(sample_buffer, &is_key_frame); +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER if (vtctx->get_param_set_func) { + CMVideoFormatDescriptionRef vid_fmt = NULL; + size_t length_code_size; + size_t header_size = 0; + size_t in_buf_size = CMSampleBufferGetTotalSampleSize(sample_buffer); + bool add_header; + status = get_length_code_size(avctx, sample_buffer, &length_code_size); - if (status) return status; + if (status < 0) + return status; + if (length_code_size != 1 && length_code_size != 2 && + length_code_size != 4) + return AVERROR_INVALIDDATA; add_header = is_key_frame && !(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER); - if (add_header) { vid_fmt = CMSampleBufferGetFormatDescription(sample_buffer); if (!vid_fmt) { @@ -2242,49 +2278,95 @@ static int vtenc_cm_to_avpacket( } status = get_params_size(avctx, vid_fmt, &header_size); - if (status) return status; + if (status < 0) + return status; } - status = count_nalus(length_code_size, sample_buffer, &nalu_count); - if(status) - return status; + if (!sei) { + CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sample_buffer); + VTEncByteReader reader; + size_t body_size; + size_t out_buf_size; + size_t written; - if (sei) { - size_t msg_size = get_sei_msg_bytes(sei, - SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35); + if (!block) + return AVERROR_EXTERNAL; + reader = (VTEncByteReader) { avctx, block }; - sei_nalu_size = sizeof(start_code) + 1 + msg_size + 1; + status = ff_vt_rewrite_length_prefixes( + vtenc_read_bytes, &reader, in_buf_size, length_code_size, + NULL, 0, &body_size); + if (status < 0) + return status; + if (body_size > SIZE_MAX - header_size) + return AVERROR_INVALIDDATA; + out_buf_size = header_size + body_size; + if (out_buf_size > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE) + return AVERROR_INVALIDDATA; + + status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); + if (status < 0) + return status; + + if (add_header) { + status = copy_param_sets(avctx, vid_fmt, pkt->data, + header_size); + if (status < 0) + goto fail_packet; + } + + status = ff_vt_rewrite_length_prefixes( + vtenc_read_bytes, &reader, in_buf_size, length_code_size, + pkt->data + header_size, body_size, &written); + if (status < 0) { + av_log(avctx, AV_LOG_ERROR, + "Error copying packet data: %d\n", status); + goto fail_packet; + } + if (written != body_size) { + status = AVERROR_BUG; + goto fail_packet; + } + } else { + size_t sei_nalu_size; + size_t out_buf_size; + int nalu_count; + + status = count_nalus(length_code_size, sample_buffer, &nalu_count); + if (status < 0) + return status; + + sei_nalu_size = sizeof(start_code) + 1 + + get_sei_msg_bytes( + sei, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35) + 1; + out_buf_size = header_size + in_buf_size + sei_nalu_size + + nalu_count * ((int)sizeof(start_code) - + (int)length_code_size); + + status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); + if (status < 0) + return status; + + if (add_header) { + status = copy_param_sets(avctx, vid_fmt, pkt->data, + header_size); + if (status < 0) + goto fail_packet; + } + + status = copy_replace_length_codes( + avctx, length_code_size, sample_buffer, sei, + pkt->data + header_size, pkt->size - header_size); + if (status < 0) { + av_log(avctx, AV_LOG_ERROR, + "Error copying packet data: %d\n", status); + goto fail_packet; + } } - - in_buf_size = CMSampleBufferGetTotalSampleSize(sample_buffer); - out_buf_size = header_size + - in_buf_size + - sei_nalu_size + - nalu_count * ((int)sizeof(start_code) - (int)length_code_size); - - status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); - if (status < 0) - return status; - - if (add_header) { - status = copy_param_sets(avctx, vid_fmt, pkt->data, out_buf_size); - if(status) return status; - } - - status = copy_replace_length_codes( - avctx, - length_code_size, - sample_buffer, - sei, - pkt->data + header_size, - pkt->size - header_size - ); - - if (status) { - av_log(avctx, AV_LOG_ERROR, "Error copying packet data: %d\n", status); - return status; - } - } else { + } else +#endif + { size_t len; CMBlockBufferRef buf = CMSampleBufferGetDataBuffer(sample_buffer); if (!buf) { @@ -2301,13 +2383,13 @@ static int vtenc_cm_to_avpacket( status = CMBlockBufferCopyDataBytes(buf, 0, len, pkt->data); if (status) { av_log(avctx, AV_LOG_ERROR, "Error copying packet data: %d\n", status); + av_packet_unref(pkt); return AVERROR_EXTERNAL; } } - if (is_key_frame) { + if (is_key_frame) pkt->flags |= AV_PKT_FLAG_KEY; - } pts = CMSampleBufferGetPresentationTimeStamp(sample_buffer); dts = CMSampleBufferGetDecodeTimeStamp (sample_buffer); @@ -2332,6 +2414,12 @@ static int vtenc_cm_to_avpacket( - dts_delta; return 0; + +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER +fail_packet: + av_packet_unref(pkt); + return status; +#endif } static int copy_avframe_to_pixel_buffer(AVCodecContext *avctx, @@ -2482,7 +2570,9 @@ static int vtenc_send_frame(AVCodecContext *avctx, CMTime time; CFDictionaryRef frame_dict = NULL; CVPixelBufferRef cv_img = NULL; +#if CONFIG_ATSC_A53 AVFrameSideData *side_data = NULL; +#endif BufNode *node = av_mallocz(sizeof(*node)); int status; diff --git a/libavcodec/videotoolboxenc_rewrite.c b/libavcodec/videotoolboxenc_rewrite.c new file mode 100644 index 0000000000..e23792df3f --- /dev/null +++ b/libavcodec/videotoolboxenc_rewrite.c @@ -0,0 +1,123 @@ +/* + * VideoToolbox length-prefixed sample rewriting helpers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> +#include <string.h> + +#include "libavutil/error.h" + +#include "videotoolboxenc_rewrite.h" + +typedef struct ByteWriter { + uint8_t *dst; + size_t capacity; + size_t position; +} ByteWriter; + +static int writer_reserve(const ByteWriter *writer, size_t size) +{ + if (size > SIZE_MAX - writer->position) + return AVERROR_INVALIDDATA; + if (writer->dst && + (writer->position > writer->capacity || + size > writer->capacity - writer->position)) + return AVERROR_BUFFER_TOO_SMALL; + return 0; +} + +static int writer_bytes(ByteWriter *writer, const uint8_t *src, size_t size) +{ + int ret; + + if (size && !src) + return AVERROR(EINVAL); + ret = writer_reserve(writer, size); + if (ret < 0) + return ret; + if (writer->dst && size) + memcpy(writer->dst + writer->position, src, size); + writer->position += size; + return 0; +} + +static int copy_source(ByteWriter *writer, FFVTByteReader read, void *opaque, + size_t offset, size_t size) +{ + int ret = writer_reserve(writer, size); + + if (ret < 0) + return ret; + if (writer->dst && size) { + ret = read(opaque, offset, writer->dst + writer->position, size); + if (ret < 0) + return ret; + } + + writer->position += size; + return 0; +} + +int ff_vt_rewrite_length_prefixes(FFVTByteReader read, void *opaque, + size_t src_size, size_t length_code_size, + uint8_t *dst, size_t dst_capacity, + size_t *written) +{ + static const uint8_t start_code[] = { 0, 0, 0, 1 }; + ByteWriter writer = { dst, dst_capacity, 0 }; + size_t src_offset = 0; + + if (!read || !written) + return AVERROR(EINVAL); + if (length_code_size != 1 && length_code_size != 2 && + length_code_size != 4) + return AVERROR_INVALIDDATA; + + while (src_offset < src_size) { + uint8_t length_bytes[4]; + size_t nal_size = 0; + size_t remaining = src_size - src_offset; + size_t i; + int ret; + + if (length_code_size > remaining) + return AVERROR_INVALIDDATA; + ret = read(opaque, src_offset, length_bytes, length_code_size); + if (ret < 0) + return ret; + src_offset += length_code_size; + remaining -= length_code_size; + + for (i = 0; i < length_code_size; i++) + nal_size = (nal_size << 8) | length_bytes[i]; + if (!nal_size || nal_size > remaining) + return AVERROR_INVALIDDATA; + + ret = writer_bytes(&writer, start_code, sizeof(start_code)); + if (ret < 0) + return ret; + ret = copy_source(&writer, read, opaque, src_offset, nal_size); + if (ret < 0) + return ret; + src_offset += nal_size; + } + + *written = writer.position; + return 0; +} diff --git a/libavcodec/videotoolboxenc_rewrite.h b/libavcodec/videotoolboxenc_rewrite.h new file mode 100644 index 0000000000..7db9bc221b --- /dev/null +++ b/libavcodec/videotoolboxenc_rewrite.h @@ -0,0 +1,39 @@ +/* + * VideoToolbox length-prefixed sample rewriting helpers + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_VIDEOTOOLBOXENC_REWRITE_H +#define AVCODEC_VIDEOTOOLBOXENC_REWRITE_H + +#include <stddef.h> +#include <stdint.h> + +typedef int (*FFVTByteReader)(void *opaque, size_t offset, + uint8_t *dst, size_t size); + +/* + * Rewrite length-prefixed NAL units as Annex B without interpreting their + * contents. Passing a null dst performs a checked count-only pass. + */ +int ff_vt_rewrite_length_prefixes(FFVTByteReader read, void *opaque, + size_t src_size, size_t length_code_size, + uint8_t *dst, size_t dst_capacity, + size_t *written); + +#endif /* AVCODEC_VIDEOTOOLBOXENC_REWRITE_H */ diff --git a/tests/fate/libavcodec.mak b/tests/fate/libavcodec.mak index e2d616e307..5e400bda8b 100644 --- a/tests/fate/libavcodec.mak +++ b/tests/fate/libavcodec.mak @@ -96,6 +96,11 @@ fate-mathops: libavcodec/tests/mathops$(EXESUF) fate-mathops: CMD = run libavcodec/tests/mathops$(EXESUF) fate-mathops: CMP = null +FATE_LIBAVCODEC-yes += fate-videotoolboxenc-rewrite +fate-videotoolboxenc-rewrite: libavcodec/tests/videotoolboxenc_rewrite$(EXESUF) +fate-videotoolboxenc-rewrite: CMD = run libavcodec/tests/videotoolboxenc_rewrite$(EXESUF) +fate-videotoolboxenc-rewrite: CMP = null + FATE_LIBAVCODEC-$(CONFIG_JPEG2000_ENCODER) += fate-j2k-dwt fate-j2k-dwt: libavcodec/tests/jpeg2000dwt$(EXESUF) fate-j2k-dwt: CMD = run libavcodec/tests/jpeg2000dwt$(EXESUF) -- 2.52.0 >From 458fb81bc7f39361aa7f90c6d719471fe06228b6 Mon Sep 17 00:00:00 2001 From: Qingzheng Li <[email protected]> Date: Sat, 8 Aug 2026 23:10:45 +0800 Subject: [PATCH 2/2] avcodec/videotoolboxenc: write A53 data in standalone SEI The current A53 path appends caption data to the first H.264 SEI NAL after converting its length prefix. Existing data is EBSP, while the append code locates the insertion point as if it were RBSP. This can corrupt the SEI and lose captions when VideoToolbox supplies its own SEI. Keep every VideoToolbox NAL body byte-exact instead and insert A53 data in a standalone user_data_registered_itu_t_t35 SEI before the first Annex A VCL NAL. Require exactly one CMSampleBuffer sample, and reject samples without VCL or with a definite new-access-unit boundary after VCL. Add FATE coverage for NAL ordering, preservation of existing NAL bodies, SEI size fields and escaping, malformed samples, capacity limits, and reader failures. Signed-off-by: Qingzheng Li <[email protected]> --- libavcodec/tests/videotoolboxenc_rewrite.c | 356 +++++++++++++- libavcodec/videotoolboxenc.c | 541 +++------------------ libavcodec/videotoolboxenc_rewrite.c | 166 ++++++- libavcodec/videotoolboxenc_rewrite.h | 15 + 4 files changed, 592 insertions(+), 486 deletions(-) diff --git a/libavcodec/tests/videotoolboxenc_rewrite.c b/libavcodec/tests/videotoolboxenc_rewrite.c index c5fa53d150..e7299f5899 100644 --- a/libavcodec/tests/videotoolboxenc_rewrite.c +++ b/libavcodec/tests/videotoolboxenc_rewrite.c @@ -17,9 +17,12 @@ */ #include <stdio.h> +#include <stdlib.h> #include <string.h> #include "libavutil/error.h" +#include "libavutil/macros.h" +#include "libavutil/mem.h" #include "libavcodec/videotoolboxenc_rewrite.h" @@ -45,6 +48,17 @@ static int read_buffer(void *opaque, size_t offset, uint8_t *dst, size_t size) return 0; } +static int rewrite(const uint8_t *src, size_t src_size, size_t length_size, + const uint8_t *a53, size_t a53_size, + uint8_t *dst, size_t capacity, size_t *written) +{ + BufferReader reader = { src, src_size, 0, -1, 0 }; + + return ff_vt_h264_rewrite_annexb(read_buffer, &reader, src_size, + length_size, a53, a53_size, + dst, capacity, written); +} + static int rewrite_lengths(const uint8_t *src, size_t src_size, size_t length_size, uint8_t *dst, size_t capacity, size_t *written) @@ -55,6 +69,46 @@ static int rewrite_lengths(const uint8_t *src, size_t src_size, length_size, dst, capacity, written); } +static int check_equal(const char *name, const uint8_t *actual, + const uint8_t *expected, size_t size) +{ + if (!memcmp(actual, expected, size)) + return 0; + fprintf(stderr, "%s: byte mismatch\n", name); + return 1; +} + +static int test_length_sizes(void) +{ + static const uint8_t src1[] = { 1, 0x65 }; + static const uint8_t src2[] = { 0, 1, 0x65 }; + static const uint8_t src4[] = { 0, 0, 0, 1, 0x65 }; + static const uint8_t a53[] = { 0x7f }; + const uint8_t *sources[] = { src1, src2, src4 }; + const size_t sizes[] = { sizeof(src1), sizeof(src2), sizeof(src4) }; + const size_t lengths[] = { 1, 2, 4 }; + static const uint8_t expected[] = { + 0, 0, 0, 1, 0x06, 0x04, 0x01, 0x7f, 0x80, + 0, 0, 0, 1, 0x65, + }; + int i; + + for (i = 0; i < 3; i++) { + uint8_t dst[32]; + size_t needed = 0, written = 0; + int ret = rewrite(sources[i], sizes[i], lengths[i], a53, sizeof(a53), + NULL, 0, &needed); + if (ret < 0 || needed > sizeof(dst) || + rewrite(sources[i], sizes[i], lengths[i], a53, sizeof(a53), + dst, needed, &written) < 0 || written != needed || + needed != sizeof(expected) || memcmp(dst, expected, needed)) { + fprintf(stderr, "length size %zu failed\n", lengths[i]); + return 1; + } + } + return 0; +} + static int test_plain_converter(void) { static const uint8_t src1[] = { 1, 0x65, 2, 0x61, 0xaa }; @@ -119,18 +173,14 @@ static int test_plain_converter(void) rewrite_lengths(src1, sizeof(src1), 3, NULL, 0, &written) != AVERROR_INVALIDDATA || rewrite_lengths(src1, sizeof(src1), 5, NULL, 0, - &written) != AVERROR_INVALIDDATA || - ff_vt_rewrite_length_prefixes(NULL, NULL, 0, 1, NULL, 0, - &written) != AVERROR(EINVAL) || - ff_vt_rewrite_length_prefixes(read_buffer, NULL, 0, 1, NULL, 0, - NULL) != AVERROR(EINVAL)) { + &written) != AVERROR_INVALIDDATA) { fprintf(stderr, "plain converter malformed policy mismatch\n"); return 1; } return 0; } -static int test_reader_failures(void) +static int test_plain_converter_reader_failures(void) { static const uint8_t src[] = { 1, 0x65 }; BufferReader reader = { src, sizeof(src), 0, 0, AVERROR_EXTERNAL }; @@ -148,7 +198,299 @@ static int test_reader_failures(void) return 0; } +static int test_order_and_existing_bodies(void) +{ + static const uint8_t src[] = { + 1, 0x09, + 2, 0x67, 0xaa, + 2, 0x68, 0xbb, + 2, 0x6d, 0xcc, + 2, 0x6f, 0xdd, + 2, 0x70, 0xee, + 2, 0x71, 0xe1, + 2, 0x72, 0xe2, + 5, 0x06, 0x00, 0x00, 0x03, 0x01, + 2, 0x6e, 0xf0, + 2, 0x61, 0xf1, + 2, 0x61, 0xf2, + }; + static const uint8_t a53[] = { 0x00, 0x00, 0x01 }; + static const uint8_t expected[] = { + 0, 0, 0, 1, 0x09, + 0, 0, 0, 1, 0x67, 0xaa, + 0, 0, 0, 1, 0x68, 0xbb, + 0, 0, 0, 1, 0x6d, 0xcc, + 0, 0, 0, 1, 0x6f, 0xdd, + 0, 0, 0, 1, 0x70, 0xee, + 0, 0, 0, 1, 0x71, 0xe1, + 0, 0, 0, 1, 0x72, 0xe2, + 0, 0, 0, 1, 0x06, 0x00, 0x00, 0x03, 0x01, + 0, 0, 0, 1, 0x06, 0x04, 0x03, + 0x00, 0x00, 0x03, 0x01, 0x80, + 0, 0, 0, 1, 0x6e, 0xf0, + 0, 0, 0, 1, 0x61, 0xf1, + 0, 0, 0, 1, 0x61, 0xf2, + }; + uint8_t dst[sizeof(expected)]; + size_t written = 0; + + if (rewrite(src, sizeof(src), 1, a53, sizeof(a53), + dst, sizeof(dst), &written) < 0 || written != sizeof(expected)) + return 1; + return check_equal("order/existing bodies", dst, expected, + sizeof(expected)); +} + +static int test_prefix_before_idr(void) +{ + static const uint8_t src[] = { 1, 0x6e, 1, 0x65 }; + static const uint8_t a53[] = { 0x7f }; + static const uint8_t expected[] = { + 0, 0, 0, 1, 0x06, 0x04, 0x01, 0x7f, 0x80, + 0, 0, 0, 1, 0x6e, + 0, 0, 0, 1, 0x65, + }; + uint8_t dst[sizeof(expected)]; + size_t written = 0; + + if (rewrite(src, sizeof(src), 1, a53, sizeof(a53), + dst, sizeof(dst), &written) < 0 || written != sizeof(expected)) + return 1; + return check_equal("prefix before IDR", dst, expected, sizeof(expected)); +} + +static int check_size_case(const char *name, size_t a53_size, + const uint8_t *prefix, size_t prefix_size) +{ + static const uint8_t src[] = { 1, 0x65 }; + uint8_t *a53 = av_malloc(a53_size); + uint8_t *dst = NULL; + size_t needed = 0, written = 0; + int result = 1; + + if (!a53) + return 1; + memset(a53, 0x7f, a53_size); + if (rewrite(src, sizeof(src), 1, a53, a53_size, + NULL, 0, &needed) < 0) + goto end; + dst = av_malloc(needed); + if (!dst) + goto end; + if (rewrite(src, sizeof(src), 1, a53, a53_size, + dst, needed, &written) < 0 || written != needed || + needed < 5 + prefix_size || memcmp(dst + 5, prefix, prefix_size)) { + fprintf(stderr, "%s: payload-size encoding mismatch\n", name); + goto end; + } + result = 0; +end: + av_free(dst); + av_free(a53); + return result; +} + +static int test_size_fields(void) +{ + static const uint8_t size1[] = { 0x04, 0x01, 0x7f }; + static const uint8_t size254[] = { 0x04, 0xfe, 0x7f }; + static const uint8_t size255[] = { 0x04, 0xff, 0x00, 0x7f }; + static const uint8_t size256[] = { 0x04, 0xff, 0x01, 0x7f }; + static const uint8_t size510[] = { 0x04, 0xff, 0xff, 0x00, 0x7f }; + + return check_size_case("size 1", 1, size1, sizeof(size1)) || + check_size_case("size 254", 254, size254, sizeof(size254)) || + check_size_case("size 255", 255, size255, sizeof(size255)) || + check_size_case("size 256", 256, size256, sizeof(size256)) || + check_size_case("size 510", 510, size510, sizeof(size510)); +} + +static int test_epb_and_trailing(void) +{ + static const uint8_t src[] = { 1, 0x65 }; + static const uint8_t a53[] = { + 0, 0, 0, 0, 0, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, + }; + static const uint8_t escaped_and_trailing[] = { + 0, 0, 3, 0, 0, 3, 0, 1, 0, 0, 3, 2, + 0, 0, 3, 3, 0, 0, 4, 0x80, + }; + uint8_t dst[64]; + size_t written; + + if (rewrite(src, sizeof(src), 1, a53, sizeof(a53), + dst, sizeof(dst), &written) < 0 || + written < 7 + sizeof(escaped_and_trailing) || + memcmp(dst + 7, escaped_and_trailing, + sizeof(escaped_and_trailing))) { + fprintf(stderr, "EPB/trailing mismatch\n"); + return 1; + } + return 0; +} + +static int test_fail_closed_and_malformed(void) +{ + static const uint8_t leading_only[] = { + 1, 0x09, 1, 0x67, 1, 0x68, 1, 0x06, 1, 0x6d, 1, 0x6f, 1, 0x70, + }; + static const uint8_t prefixed_vcl[] = { 1, 0x6e, 1, 0x61 }; + static const uint8_t auxiliary_only[] = { 1, 0x73 }; + static const uint8_t au_boundary_types[] = { + 6, 7, 8, 9, 14, 15, 16, 17, 18, + }; + static const uint8_t bad_zero[] = { 0 }; + static const uint8_t bad_long[] = { 2, 0x65 }; + static const uint8_t a53[] = { 0x7f }; + uint8_t dst[32]; + uint8_t multiple_aus[] = { 1, 0x65, 1, 0, 1, 0x61 }; + size_t written = 123; + size_t i; + + if (rewrite(leading_only, sizeof(leading_only), 1, a53, sizeof(a53), + NULL, 0, &written) != AVERROR_INVALIDDATA || + rewrite(bad_zero, sizeof(bad_zero), 1, a53, sizeof(a53), + NULL, 0, &written) != AVERROR_INVALIDDATA || + rewrite(bad_long, sizeof(bad_long), 1, a53, sizeof(a53), + NULL, 0, &written) != AVERROR_INVALIDDATA || + rewrite(auxiliary_only, sizeof(auxiliary_only), 1, a53, sizeof(a53), + NULL, 0, &written) != AVERROR_INVALIDDATA || + rewrite(prefixed_vcl, sizeof(prefixed_vcl), 1, a53, sizeof(a53), + dst, sizeof(dst), &written) < 0 || + rewrite(prefixed_vcl, sizeof(prefixed_vcl), 3, a53, sizeof(a53), + NULL, 0, &written) != AVERROR_INVALIDDATA || + rewrite(prefixed_vcl, sizeof(prefixed_vcl), 1, NULL, 0, + NULL, 0, &written) != AVERROR(EINVAL)) { + fprintf(stderr, "fail-closed/malformed policy mismatch\n"); + return 1; + } + + for (i = 0; i < FF_ARRAY_ELEMS(au_boundary_types); i++) { + multiple_aus[3] = au_boundary_types[i]; + if (rewrite(multiple_aus, sizeof(multiple_aus), 1, + a53, sizeof(a53), NULL, 0, + &written) != AVERROR_INVALIDDATA) { + fprintf(stderr, "post-VCL NAL type %u did not fail closed\n", + au_boundary_types[i]); + return 1; + } + } + return 0; +} + +static int test_capacity_edges(void) +{ + static const uint8_t src[] = { 1, 0x65 }; + static const uint8_t a53[] = { 0, 0, 1 }; + uint8_t dst[64]; + size_t needed = 0, written = 0, capacity; + + if (rewrite(src, sizeof(src), 1, a53, sizeof(a53), + NULL, 0, &needed) < 0) + return 1; + + for (capacity = 0; capacity < needed; capacity++) { + size_t i; + int ret; + + memset(dst, 0xa5, sizeof(dst)); + ret = rewrite(src, sizeof(src), 1, a53, sizeof(a53), + dst, capacity, &written); + if (ret != AVERROR_BUFFER_TOO_SMALL) { + fprintf(stderr, "capacity %zu returned %d\n", capacity, ret); + return 1; + } + for (i = capacity; i < sizeof(dst); i++) { + if (dst[i] != 0xa5) { + fprintf(stderr, "capacity canary changed at %zu\n", i); + return 1; + } + } + } + return 0; +} + +static int test_direct_large_nal_copy(void) +{ + static const uint8_t a53[] = { 0x7f }; + const size_t nal_size = 1024 * 1024 + 1; + const size_t src_size = 4 + nal_size; + BufferReader count_reader; + BufferReader write_reader; + uint8_t *src = av_malloc(src_size); + uint8_t *dst = NULL; + size_t needed = 0, written = 0; + int result = 1; + + if (!src) + return 1; + src[0] = (uint8_t)(nal_size >> 24); + src[1] = (uint8_t)(nal_size >> 16); + src[2] = (uint8_t)(nal_size >> 8); + src[3] = (uint8_t)nal_size; + src[4] = 0x65; + memset(src + 5, 0xa5, nal_size - 1); + + count_reader = (BufferReader) { src, src_size, 0, -1, 0 }; + if (ff_vt_h264_rewrite_annexb(read_buffer, &count_reader, + src_size, 4, a53, sizeof(a53), + NULL, 0, &needed) < 0) + goto end; + + dst = av_malloc(needed); + if (!dst) + goto end; + write_reader = (BufferReader) { src, src_size, 0, -1, 0 }; + if (ff_vt_h264_rewrite_annexb(read_buffer, &write_reader, + src_size, 4, a53, sizeof(a53), + dst, needed, &written) < 0 || + written != needed || needed < nal_size || + memcmp(dst + needed - nal_size, src + 4, nal_size)) { + fprintf(stderr, "large NAL copy mismatch\n"); + goto end; + } + + result = 0; +end: + av_free(dst); + av_free(src); + return result; +} + +static int test_reader_failures(void) +{ + static const uint8_t src[] = { 1, 0x65 }; + static const uint8_t a53[] = { 0x7f }; + BufferReader reader = { src, sizeof(src), 0, 0, AVERROR_EXTERNAL }; + uint8_t dst[32]; + size_t written; + + if (ff_vt_h264_rewrite_annexb(read_buffer, &reader, sizeof(src), 1, + a53, sizeof(a53), NULL, 0, + &written) != AVERROR_EXTERNAL) + return 1; + + reader = (BufferReader) { src, sizeof(src), 0, 2, AVERROR_EXTERNAL }; + if (ff_vt_h264_rewrite_annexb(read_buffer, &reader, sizeof(src), 1, + a53, sizeof(a53), dst, sizeof(dst), + &written) != AVERROR_EXTERNAL) { + fprintf(stderr, "body-reader failure mismatch\n"); + return 1; + } + return 0; +} + int main(void) { - return test_plain_converter() || test_reader_failures(); + return test_length_sizes() || + test_plain_converter() || + test_plain_converter_reader_failures() || + test_order_and_existing_bodies() || + test_prefix_before_idr() || + test_size_fields() || + test_epb_and_trailing() || + test_fail_closed_and_malformed() || + test_capacity_edges() || + test_direct_large_nal_copy() || + test_reader_failures(); } diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 7563c8ba0c..c109977276 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -30,6 +30,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/bswap.h" #include "libavutil/imgutils.h" #include "libavutil/mem.h" #include "libavutil/opt.h" @@ -39,8 +40,6 @@ #include "atsc_a53.h" #include "codec_internal.h" #include "encode.h" -#include "h264.h" -#include "h264_sei.h" #include "hwconfig.h" #include "internal.h" @@ -481,51 +480,6 @@ static void vtenc_q_push(VTEncContext *vtctx, BufNode *info) pthread_mutex_unlock(&vtctx->lock); } -#if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER -static int count_nalus(size_t length_code_size, - CMSampleBufferRef sample_buffer, - int *count) -{ - size_t offset = 0; - int status; - int nalu_ct = 0; - uint8_t size_buf[4]; - size_t src_size = CMSampleBufferGetTotalSampleSize(sample_buffer); - CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sample_buffer); - - if (length_code_size > 4) - return AVERROR_INVALIDDATA; - - while (offset < src_size) { - size_t curr_src_len; - size_t box_len = 0; - size_t i; - - status = CMBlockBufferCopyDataBytes(block, - offset, - length_code_size, - size_buf); - - if (status != kCMBlockBufferNoErr) { - return AVERROR_EXTERNAL; - } - - for (i = 0; i < length_code_size; i++) { - box_len <<= 8; - box_len |= size_buf[i]; - } - - curr_src_len = box_len + length_code_size; - offset += curr_src_len; - - nalu_ct++; - } - - *count = nalu_ct; - return 0; -} -#endif - static CMVideoCodecType get_cm_codec_type(AVCodecContext *avctx, int profile, double alpha_quality) @@ -1844,377 +1798,6 @@ static void vtenc_get_frame_info(CMSampleBufferRef buffer, bool *is_key_frame) } #if CONFIG_H264_VIDEOTOOLBOX_ENCODER || CONFIG_HEVC_VIDEOTOOLBOX_ENCODER -static int is_post_sei_nal_type(int nal_type){ - return nal_type != H264_NAL_SEI && - nal_type != H264_NAL_SPS && - nal_type != H264_NAL_PPS && - nal_type != H264_NAL_AUD; -} - -/* - * Finds the sei message start/size of type find_sei_type. - * If more than one of that type exists, the last one is returned. - */ -static int find_sei_end(AVCodecContext *avctx, - uint8_t *nal_data, - size_t nal_size, - uint8_t **sei_end) -{ - int nal_type; - size_t sei_payload_size = 0; - uint8_t *nal_start = nal_data; - *sei_end = NULL; - - if (!nal_size) - return 0; - - nal_type = *nal_data & 0x1F; - if (nal_type != H264_NAL_SEI) - return 0; - - nal_data++; - nal_size--; - - if (nal_data[nal_size - 1] == 0x80) - nal_size--; - - while (nal_size > 0 && *nal_data > 0) { - do{ - nal_data++; - nal_size--; - } while (nal_size > 0 && *nal_data == 0xFF); - - if (!nal_size) { - av_log(avctx, AV_LOG_ERROR, "Unexpected end of SEI NAL Unit parsing type.\n"); - return AVERROR_INVALIDDATA; - } - - do{ - sei_payload_size += *nal_data; - nal_data++; - nal_size--; - } while (nal_size > 0 && *nal_data == 0xFF); - - if (nal_size < sei_payload_size) { - av_log(avctx, AV_LOG_ERROR, "Unexpected end of SEI NAL Unit parsing size.\n"); - return AVERROR_INVALIDDATA; - } - - nal_data += sei_payload_size; - nal_size -= sei_payload_size; - } - - *sei_end = nal_data; - - return nal_data - nal_start + 1; -} - -/** - * Copies the data inserting emulation prevention bytes as needed. - * Existing data in the destination can be taken into account by providing - * dst with a dst_offset > 0. - * - * @return The number of bytes copied on success. On failure, the negative of - * the number of bytes needed to copy src is returned. - */ -static int copy_emulation_prev(const uint8_t *src, - size_t src_size, - uint8_t *dst, - ssize_t dst_offset, - size_t dst_size) -{ - int zeros = 0; - int wrote_bytes; - uint8_t* dst_start; - uint8_t* dst_end = dst + dst_size; - const uint8_t* src_end = src + src_size; - int start_at = dst_offset > 2 ? dst_offset - 2 : 0; - int i; - for (i = start_at; i < dst_offset && i < dst_size; i++) { - if (!dst[i]) - zeros++; - else - zeros = 0; - } - - dst += dst_offset; - dst_start = dst; - for (; src < src_end; src++, dst++) { - if (zeros == 2) { - int insert_ep3_byte = *src <= 3; - if (insert_ep3_byte) { - if (dst < dst_end) - *dst = 3; - dst++; - } - - zeros = 0; - } - - if (dst < dst_end) - *dst = *src; - - if (!*src) - zeros++; - else - zeros = 0; - } - - wrote_bytes = dst - dst_start; - - if (dst > dst_end) - return -wrote_bytes; - - return wrote_bytes; -} - -static int write_sei(const ExtraSEI *sei, - int sei_type, - uint8_t *dst, - size_t dst_size) -{ - uint8_t *sei_start = dst; - size_t remaining_sei_size = sei->size; - size_t remaining_dst_size = dst_size; - int header_bytes; - int bytes_written; - ssize_t offset; - - if (!remaining_dst_size) - return AVERROR_BUFFER_TOO_SMALL; - - while (sei_type && remaining_dst_size != 0) { - int sei_byte = sei_type > 255 ? 255 : sei_type; - *dst = sei_byte; - - sei_type -= sei_byte; - dst++; - remaining_dst_size--; - } - - if (!dst_size) - return AVERROR_BUFFER_TOO_SMALL; - - while (remaining_sei_size && remaining_dst_size != 0) { - int size_byte = remaining_sei_size > 255 ? 255 : remaining_sei_size; - *dst = size_byte; - - remaining_sei_size -= size_byte; - dst++; - remaining_dst_size--; - } - - if (remaining_dst_size < sei->size) - return AVERROR_BUFFER_TOO_SMALL; - - header_bytes = dst - sei_start; - - offset = header_bytes; - bytes_written = copy_emulation_prev(sei->data, - sei->size, - sei_start, - offset, - dst_size); - if (bytes_written < 0) - return AVERROR_BUFFER_TOO_SMALL; - - bytes_written += header_bytes; - return bytes_written; -} - -/** - * Copies NAL units and replaces length codes with - * H.264 Annex B start codes. On failure, the contents of - * dst_data may have been modified. - * - * @param length_code_size Byte length of each length code - * @param sample_buffer NAL units prefixed with length codes. - * @param sei Optional A53 closed captions SEI data. - * @param dst_data Must be zeroed before calling this function. - * Contains the copied NAL units prefixed with - * start codes when the function returns - * successfully. - * @param dst_size Length of dst_data - * @return 0 on success - * AVERROR_INVALIDDATA if length_code_size is invalid - * AVERROR_BUFFER_TOO_SMALL if dst_data is too small - * or if a length_code in src_data specifies data beyond - * the end of its buffer. - */ -static int copy_replace_length_codes( - AVCodecContext *avctx, - size_t length_code_size, - CMSampleBufferRef sample_buffer, - ExtraSEI *sei, - uint8_t *dst_data, - size_t dst_size) -{ - size_t src_size = CMSampleBufferGetTotalSampleSize(sample_buffer); - size_t remaining_src_size = src_size; - size_t remaining_dst_size = dst_size; - size_t src_offset = 0; - int wrote_sei = 0; - int status; - uint8_t size_buf[4]; - uint8_t nal_type; - CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sample_buffer); - - if (length_code_size > 4) { - return AVERROR_INVALIDDATA; - } - - while (remaining_src_size > 0) { - size_t curr_src_len; - size_t curr_dst_len; - size_t box_len = 0; - size_t i; - - uint8_t *dst_box; - - status = CMBlockBufferCopyDataBytes(block, - src_offset, - length_code_size, - size_buf); - if (status) { - av_log(avctx, AV_LOG_ERROR, "Cannot copy length: %d\n", status); - return AVERROR_EXTERNAL; - } - - status = CMBlockBufferCopyDataBytes(block, - src_offset + length_code_size, - 1, - &nal_type); - - if (status) { - av_log(avctx, AV_LOG_ERROR, "Cannot copy type: %d\n", status); - return AVERROR_EXTERNAL; - } - - nal_type &= 0x1F; - - for (i = 0; i < length_code_size; i++) { - box_len <<= 8; - box_len |= size_buf[i]; - } - - if (sei && !wrote_sei && is_post_sei_nal_type(nal_type)) { - //No SEI NAL unit - insert. - int wrote_bytes; - - memcpy(dst_data, start_code, sizeof(start_code)); - dst_data += sizeof(start_code); - remaining_dst_size -= sizeof(start_code); - - *dst_data = H264_NAL_SEI; - dst_data++; - remaining_dst_size--; - - wrote_bytes = write_sei(sei, - SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, - dst_data, - remaining_dst_size); - - if (wrote_bytes < 0) - return wrote_bytes; - - remaining_dst_size -= wrote_bytes; - dst_data += wrote_bytes; - - if (remaining_dst_size <= 0) - return AVERROR_BUFFER_TOO_SMALL; - - *dst_data = 0x80; - - dst_data++; - remaining_dst_size--; - - wrote_sei = 1; - } - - curr_src_len = box_len + length_code_size; - curr_dst_len = box_len + sizeof(start_code); - - if (remaining_src_size < curr_src_len) { - return AVERROR_BUFFER_TOO_SMALL; - } - - if (remaining_dst_size < curr_dst_len) { - return AVERROR_BUFFER_TOO_SMALL; - } - - dst_box = dst_data + sizeof(start_code); - - memcpy(dst_data, start_code, sizeof(start_code)); - status = CMBlockBufferCopyDataBytes(block, - src_offset + length_code_size, - box_len, - dst_box); - - if (status) { - av_log(avctx, AV_LOG_ERROR, "Cannot copy data: %d\n", status); - return AVERROR_EXTERNAL; - } - - if (sei && !wrote_sei && nal_type == H264_NAL_SEI) { - //Found SEI NAL unit - append. - int wrote_bytes; - int old_sei_length; - int extra_bytes; - uint8_t *new_sei; - old_sei_length = find_sei_end(avctx, dst_box, box_len, &new_sei); - if (old_sei_length < 0) - return old_sei_length; - - wrote_bytes = write_sei(sei, - SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, - new_sei, - remaining_dst_size - old_sei_length); - if (wrote_bytes < 0) - return wrote_bytes; - - if (new_sei + wrote_bytes >= dst_data + remaining_dst_size) - return AVERROR_BUFFER_TOO_SMALL; - - new_sei[wrote_bytes++] = 0x80; - extra_bytes = wrote_bytes - (dst_box + box_len - new_sei); - - dst_data += extra_bytes; - remaining_dst_size -= extra_bytes; - - wrote_sei = 1; - } - - src_offset += curr_src_len; - dst_data += curr_dst_len; - - remaining_src_size -= curr_src_len; - remaining_dst_size -= curr_dst_len; - } - - return 0; -} - -/** - * Returns a sufficient number of bytes to contain the sei data. - * It may be greater than the minimum required. - */ -static int get_sei_msg_bytes(const ExtraSEI* sei, int type){ - int copied_size; - if (sei->size == 0) - return 0; - - copied_size = -copy_emulation_prev(sei->data, - sei->size, - NULL, - 0, - 0); - - if ((sei->size % 255) == 0) //may result in an extra byte - copied_size++; - - return copied_size + sei->size / 255 + 1 + type / 255 + 1; -} - typedef struct VTEncByteReader { AVCodecContext *avctx; CMBlockBufferRef block; @@ -2248,7 +1831,10 @@ static int vtenc_cm_to_avpacket( int64_t dts_delta; CMTime pts; CMTime dts; -#if !CONFIG_H264_VIDEOTOOLBOX_ENCODER && !CONFIG_HEVC_VIDEOTOOLBOX_ENCODER +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER + bool rewrite_h264_a53 = avctx->codec_id == AV_CODEC_ID_H264 && + sei && sei->size > 0; +#else (void)sei; #endif @@ -2259,8 +1845,13 @@ static int vtenc_cm_to_avpacket( CMVideoFormatDescriptionRef vid_fmt = NULL; size_t length_code_size; size_t header_size = 0; + size_t body_size; size_t in_buf_size = CMSampleBufferGetTotalSampleSize(sample_buffer); + size_t out_buf_size; bool add_header; + CMBlockBufferRef block = NULL; + VTEncByteReader reader; + size_t written; status = get_length_code_size(avctx, sample_buffer, &length_code_size); if (status < 0) @@ -2269,6 +1860,11 @@ static int vtenc_cm_to_avpacket( length_code_size != 4) return AVERROR_INVALIDDATA; + block = CMSampleBufferGetDataBuffer(sample_buffer); + if (!block) + return AVERROR_EXTERNAL; + reader = (VTEncByteReader) { avctx, block }; + add_header = is_key_frame && !(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER); if (add_header) { vid_fmt = CMSampleBufferGetFormatDescription(sample_buffer); @@ -2282,39 +1878,62 @@ static int vtenc_cm_to_avpacket( return status; } - if (!sei) { - CMBlockBufferRef block = CMSampleBufferGetDataBuffer(sample_buffer); - VTEncByteReader reader; - size_t body_size; - size_t out_buf_size; - size_t written; - - if (!block) - return AVERROR_EXTERNAL; - reader = (VTEncByteReader) { avctx, block }; +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER + if (rewrite_h264_a53) { + /* Keep VideoToolbox's existing EBSP SEI NALs byte-exact. */ + if (CMSampleBufferGetNumSamples(sample_buffer) != 1) { + av_log(avctx, AV_LOG_ERROR, + "Cannot attach A53 data unless the sample buffer " + "contains exactly one sample.\n"); + return AVERROR_INVALIDDATA; + } + status = ff_vt_h264_rewrite_annexb( + vtenc_read_bytes, &reader, in_buf_size, length_code_size, + sei->data, sei->size, NULL, 0, &body_size); + if (status < 0) + return status; + } else { +#endif status = ff_vt_rewrite_length_prefixes( vtenc_read_bytes, &reader, in_buf_size, length_code_size, NULL, 0, &body_size); if (status < 0) return status; - if (body_size > SIZE_MAX - header_size) - return AVERROR_INVALIDDATA; - out_buf_size = header_size + body_size; - if (out_buf_size > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE) - return AVERROR_INVALIDDATA; +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER + } +#endif - status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); + if (body_size > SIZE_MAX - header_size) + return AVERROR_INVALIDDATA; + out_buf_size = header_size + body_size; + if (out_buf_size > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE) + return AVERROR_INVALIDDATA; + + status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); + if (status < 0) + return status; + + if (add_header) { + status = copy_param_sets(avctx, vid_fmt, pkt->data, header_size); if (status < 0) - return status; + goto fail_packet; + } - if (add_header) { - status = copy_param_sets(avctx, vid_fmt, pkt->data, - header_size); - if (status < 0) - goto fail_packet; +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER + if (rewrite_h264_a53) { + status = ff_vt_h264_rewrite_annexb( + vtenc_read_bytes, &reader, in_buf_size, length_code_size, + sei->data, sei->size, pkt->data + header_size, body_size, + &written); + if (status < 0) + goto fail_packet; + if (written != body_size) { + status = AVERROR_BUG; + goto fail_packet; } - + } else { +#endif status = ff_vt_rewrite_length_prefixes( vtenc_read_bytes, &reader, in_buf_size, length_code_size, pkt->data + header_size, body_size, &written); @@ -2327,43 +1946,9 @@ static int vtenc_cm_to_avpacket( status = AVERROR_BUG; goto fail_packet; } - } else { - size_t sei_nalu_size; - size_t out_buf_size; - int nalu_count; - - status = count_nalus(length_code_size, sample_buffer, &nalu_count); - if (status < 0) - return status; - - sei_nalu_size = sizeof(start_code) + 1 + - get_sei_msg_bytes( - sei, - SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35) + 1; - out_buf_size = header_size + in_buf_size + sei_nalu_size + - nalu_count * ((int)sizeof(start_code) - - (int)length_code_size); - - status = ff_get_encode_buffer(avctx, pkt, out_buf_size, 0); - if (status < 0) - return status; - - if (add_header) { - status = copy_param_sets(avctx, vid_fmt, pkt->data, - header_size); - if (status < 0) - goto fail_packet; - } - - status = copy_replace_length_codes( - avctx, length_code_size, sample_buffer, sei, - pkt->data + header_size, pkt->size - header_size); - if (status < 0) { - av_log(avctx, AV_LOG_ERROR, - "Error copying packet data: %d\n", status); - goto fail_packet; - } +#if CONFIG_H264_VIDEOTOOLBOX_ENCODER } +#endif } else #endif { diff --git a/libavcodec/videotoolboxenc_rewrite.c b/libavcodec/videotoolboxenc_rewrite.c index e23792df3f..0e7eb90ad3 100644 --- a/libavcodec/videotoolboxenc_rewrite.c +++ b/libavcodec/videotoolboxenc_rewrite.c @@ -23,12 +23,15 @@ #include "libavutil/error.h" +#include "h264.h" +#include "sei.h" #include "videotoolboxenc_rewrite.h" typedef struct ByteWriter { uint8_t *dst; size_t capacity; size_t position; + unsigned zero_count; } ByteWriter; static int writer_reserve(const ByteWriter *writer, size_t size) @@ -57,6 +60,97 @@ static int writer_bytes(ByteWriter *writer, const uint8_t *src, size_t size) return 0; } +static int writer_rbsp_byte(ByteWriter *writer, uint8_t byte) +{ + static const uint8_t emulation_prevention = 3; + int ret; + + if (writer->zero_count == 2 && byte <= 3) { + ret = writer_bytes(writer, &emulation_prevention, 1); + if (ret < 0) + return ret; + writer->zero_count = 0; + } + ret = writer_bytes(writer, &byte, 1); + if (ret < 0) + return ret; + + if (!byte) { + writer->zero_count++; + } else { + writer->zero_count = 0; + } + return 0; +} + +static int writer_ff_field(ByteWriter *writer, size_t value) +{ + int ret; + + while (value >= 255) { + ret = writer_rbsp_byte(writer, 255); + if (ret < 0) + return ret; + value -= 255; + } + + return writer_rbsp_byte(writer, (uint8_t)value); +} + +static int writer_a53_sei(ByteWriter *writer, const uint8_t *data, size_t size) +{ + static const uint8_t start_code[] = { 0, 0, 0, 1 }; + static const uint8_t nal_header = H264_NAL_SEI; + size_t i; + int ret; + + ret = writer_bytes(writer, start_code, sizeof(start_code)); + if (ret < 0) + return ret; + ret = writer_bytes(writer, &nal_header, 1); + if (ret < 0) + return ret; + + writer->zero_count = 0; + ret = writer_rbsp_byte(writer, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35); + if (ret < 0) + return ret; + ret = writer_ff_field(writer, size); + if (ret < 0) + return ret; + + for (i = 0; i < size; i++) { + ret = writer_rbsp_byte(writer, data[i]); + if (ret < 0) + return ret; + } + + return writer_rbsp_byte(writer, 0x80); +} + +static int is_leading_nal(unsigned type) +{ + return type == H264_NAL_SEI || type == H264_NAL_SPS || + type == H264_NAL_PPS || type == H264_NAL_AUD || + type == H264_NAL_SPS_EXT || type == H264_NAL_SUB_SPS || + type == H264_NAL_DPS || type == H264_NAL_RESERVED17 || + type == H264_NAL_RESERVED18; +} + +static int is_vcl_nal(unsigned type) +{ + /* VideoToolbox's public H.264 profiles produce Annex A VCL NALs. */ + return type >= H264_NAL_SLICE && type <= H264_NAL_IDR_SLICE; +} + +static int starts_new_access_unit(unsigned type) +{ + return type == H264_NAL_SEI || type == H264_NAL_SPS || + type == H264_NAL_PPS || type == H264_NAL_AUD || + (type >= H264_NAL_PREFIX && type <= H264_NAL_RESERVED18); +} + static int copy_source(ByteWriter *writer, FFVTByteReader read, void *opaque, size_t offset, size_t size) { @@ -80,7 +174,7 @@ int ff_vt_rewrite_length_prefixes(FFVTByteReader read, void *opaque, size_t *written) { static const uint8_t start_code[] = { 0, 0, 0, 1 }; - ByteWriter writer = { dst, dst_capacity, 0 }; + ByteWriter writer = { dst, dst_capacity, 0, 0 }; size_t src_offset = 0; if (!read || !written) @@ -121,3 +215,73 @@ int ff_vt_rewrite_length_prefixes(FFVTByteReader read, void *opaque, *written = writer.position; return 0; } + +int ff_vt_h264_rewrite_annexb(FFVTByteReader read, void *opaque, + size_t src_size, size_t length_code_size, + const uint8_t *a53_data, size_t a53_size, + uint8_t *dst, size_t dst_capacity, + size_t *written) +{ + static const uint8_t start_code[] = { 0, 0, 0, 1 }; + ByteWriter writer = { dst, dst_capacity, 0, 0 }; + size_t src_offset = 0; + int inserted = 0; + int saw_vcl = 0; + + if (!read || !written || !a53_data || !a53_size) + return AVERROR(EINVAL); + if (length_code_size != 1 && length_code_size != 2 && + length_code_size != 4) + return AVERROR_INVALIDDATA; + + while (src_offset < src_size) { + uint8_t length_bytes[4]; + uint8_t nal_header; + size_t nal_size = 0; + size_t remaining = src_size - src_offset; + size_t i; + unsigned nal_type; + int ret; + + if (length_code_size > remaining) + return AVERROR_INVALIDDATA; + ret = read(opaque, src_offset, length_bytes, length_code_size); + if (ret < 0) + return ret; + src_offset += length_code_size; + remaining -= length_code_size; + + for (i = 0; i < length_code_size; i++) + nal_size = (nal_size << 8) | length_bytes[i]; + if (!nal_size || nal_size > remaining) + return AVERROR_INVALIDDATA; + + ret = read(opaque, src_offset, &nal_header, 1); + if (ret < 0) + return ret; + nal_type = nal_header & 0x1f; + if (saw_vcl && starts_new_access_unit(nal_type)) + return AVERROR_INVALIDDATA; + saw_vcl |= is_vcl_nal(nal_type); + + if (!inserted && !is_leading_nal(nal_type)) { + ret = writer_a53_sei(&writer, a53_data, a53_size); + if (ret < 0) + return ret; + inserted = 1; + } + ret = writer_bytes(&writer, start_code, sizeof(start_code)); + if (ret < 0) + return ret; + ret = copy_source(&writer, read, opaque, src_offset, nal_size); + if (ret < 0) + return ret; + src_offset += nal_size; + } + + if (!inserted || !saw_vcl) + return AVERROR_INVALIDDATA; + + *written = writer.position; + return 0; +} diff --git a/libavcodec/videotoolboxenc_rewrite.h b/libavcodec/videotoolboxenc_rewrite.h index 7db9bc221b..8e100780dd 100644 --- a/libavcodec/videotoolboxenc_rewrite.h +++ b/libavcodec/videotoolboxenc_rewrite.h @@ -36,4 +36,19 @@ int ff_vt_rewrite_length_prefixes(FFVTByteReader read, void *opaque, uint8_t *dst, size_t dst_capacity, size_t *written); +/* + * Rewrite one VideoToolbox-produced length-prefixed H.264 compressed frame as + * Annex B and insert a standalone A53 closed-caption SEI NAL after leading + * AUD, SEI and parameter-set NALs. In particular, the SEI is written before + * any prefix NAL associated with the first Annex A VCL NAL. Existing NAL + * bodies are copied without parsing or re-encoding them. Passing a null dst + * performs a checked count-only pass. Samples without an Annex A VCL NAL, or + * with a definite new-access-unit boundary after VCL, fail closed. + */ +int ff_vt_h264_rewrite_annexb(FFVTByteReader read, void *opaque, + size_t src_size, size_t length_code_size, + const uint8_t *a53_data, size_t a53_size, + uint8_t *dst, size_t dst_capacity, + size_t *written); + #endif /* AVCODEC_VIDEOTOOLBOXENC_REWRITE_H */ -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
