[FFmpeg-devel] [PATCH 05/10] avcodec/d3d12va_encode: don't write garbage when there is no header

[Kacper Michajłow]

When codec->write_sequence_header is not defined, bit_len was undefined,
and while data bufer was zeroed we could just overread it. Do nothing
when we don't have anything to write.

Signed-off-by: Kacper Michajłow <kaspe...@gmail.com>
---
 libavcodec/d3d12va_encode.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c
index d99e960db6..64821da5f5 100644
--- a/libavcodec/d3d12va_encode.c
+++ b/libavcodec/d3d12va_encode.c
@@ -299,21 +299,20 @@ static int d3d12va_encode_issue(AVCodecContext *avctx,
                        "header: %d.\n", err);
                 goto fail;
             }
-        }
-
-        pic->header_size = (int)bit_len / 8;
-        pic->aligned_header_size = pic->header_size % 
ctx->req.CompressedBitstreamBufferAccessAlignment ?
-                                   FFALIGN(pic->header_size, 
ctx->req.CompressedBitstreamBufferAccessAlignment) :
-                                   pic->header_size;
+            pic->header_size = (int)bit_len / 8;
+            pic->aligned_header_size = pic->header_size % 
ctx->req.CompressedBitstreamBufferAccessAlignment ?
+                                    FFALIGN(pic->header_size, 
ctx->req.CompressedBitstreamBufferAccessAlignment) :
+                                    pic->header_size;
+
+            hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void 
**)&ptr);
+            if (FAILED(hr)) {
+                err = AVERROR_UNKNOWN;
+                goto fail;
+            }
 
-        hr = ID3D12Resource_Map(pic->output_buffer, 0, NULL, (void **)&ptr);
-        if (FAILED(hr)) {
-            err = AVERROR_UNKNOWN;
-            goto fail;
+            memcpy(ptr, data, pic->aligned_header_size);
+            ID3D12Resource_Unmap(pic->output_buffer, 0, NULL);
         }
-
-        memcpy(ptr, data, pic->aligned_header_size);
-        ID3D12Resource_Unmap(pic->output_buffer, 0, NULL);
     }
 
     d3d12_refs.NumTexture2Ds = base_pic->nb_refs[0] + base_pic->nb_refs[1];
-- 
2.50.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".