Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Timo Rothenpieler]

On 08/03/2019 00:57, Carl Eugen Hoyos wrote:

2019-03-06 15:57 GMT+01:00, Oliver Collyer :

Hi

I needed the dynamic resolution changing feature of NVENC to be accessible
through the ffmpeg libraries for a hobby project, so I added support and
here is a patch.

I will format this as a proper patch after any changes necessary following
feedback.

To use this feature you would need to:

1. Specify max_width and max_height before opening the encoder


Can't they be set to a maximum number to be as flexible as possible?


That'd be a bad idea, as it will allocate that amount of memory for 
every frame, no matter how large it actually ends up being.




smime.p7s
Description: S/MIME Cryptographic Signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Oliver Collyer]

> On 7 Mar 2019, at 23:57, Carl Eugen Hoyos  wrote:
> 
> 2019-03-06 15:57 GMT+01:00, Oliver Collyer :
>> Hi
>> 
>> I needed the dynamic resolution changing feature of NVENC to be accessible
>> through the ffmpeg libraries for a hobby project, so I added support and
>> here is a patch.
>> 
>> I will format this as a proper patch after any changes necessary following
>> feedback.
>> 
>> To use this feature you would need to:
>> 
>> 1. Specify max_width and max_height before opening the encoder
> 
> Can't they be set to a maximum number to be as flexible as possible?
> 

It would allocate the output surfaces to this size if we took that approach and 
I don’t think that is particularly sensible given this feature a pretty edge 
use case?

> 
>> +av_log(avctx, AV_LOG_VERBOSE,
>> +"resolution change: %d x %d -> %d x %d\n",
>> +params.reInitEncodeParams.encodeWidth,
>> +params.reInitEncodeParams.encodeHeight,
> 
>> +(uint32_t)avctx->width,
>> +(uint32_t)avctx->height);
> 
> These casts look strange and should be unneeded.
> 

No idea where those came from, I blame my fingers. Will remove.

> Carl Eugen
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Carl Eugen Hoyos]

2019-03-06 15:57 GMT+01:00, Oliver Collyer :
> Hi
>
> I needed the dynamic resolution changing feature of NVENC to be accessible
> through the ffmpeg libraries for a hobby project, so I added support and
> here is a patch.
>
> I will format this as a proper patch after any changes necessary following
> feedback.
>
> To use this feature you would need to:
>
> 1. Specify max_width and max_height before opening the encoder

Can't they be set to a maximum number to be as flexible as possible?


> +av_log(avctx, AV_LOG_VERBOSE,
> +"resolution change: %d x %d -> %d x %d\n",
> +params.reInitEncodeParams.encodeWidth,
> +params.reInitEncodeParams.encodeHeight,

> +(uint32_t)avctx->width,
> +(uint32_t)avctx->height);

These casts look strange and should be unneeded.

Carl Eugen
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Timo Rothenpieler]

looks good at first glance, will give it a test this weekend.



smime.p7s
Description: S/MIME Cryptographic Signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Oliver Collyer]

> 
>>> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
>>> index 0ce22ec4fa..7087f82ce1 100644
>>> --- a/libavcodec/avcodec.h
>>> +++ b/libavcodec/avcodec.h
>>> @@ -3357,6 +3357,12 @@ typedef struct AVCodecContext {
>>> * - encoding: unused
>>> */
>>>int discard_damaged_percentage;
>>> +
>>> +/*
>>> + * Video encoding only. Sets the maximum picture size for encoders that
>>> + * support adjusting the picture size dynamically during encoding.
>>> + */
>>> + int max_width, max_height;
>>> } AVCodecContext;
>>> 
>> 
>> I really don't like introducing public API fields for this. Maybe a
>> private nvenc option would be better at this point.
>> 
> 
> So would this be suitable?
> 
> The only thing was that I wasn't sure about exposing these options to ffmpeg 
> though, as they're useless to ffmpeg unless ffmpeg were to add features for 
> dynamically changing the encoded resolutionI just couldn't figure out how 
> you can add a private option that isn't exposed in this way though.
> 

So in the absence of any further feedback I've created a patch.



0001-nvenc-added-support-for-dynamic-resolution-encoding.patch
Description: Binary data

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Oliver Collyer]

>> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
>> index 0ce22ec4fa..7087f82ce1 100644
>> --- a/libavcodec/avcodec.h
>> +++ b/libavcodec/avcodec.h
>> @@ -3357,6 +3357,12 @@ typedef struct AVCodecContext {
>>  * - encoding: unused
>>  */
>> int discard_damaged_percentage;
>> +
>> +/*
>> + * Video encoding only. Sets the maximum picture size for encoders that
>> + * support adjusting the picture size dynamically during encoding.
>> + */
>> + int max_width, max_height;
>> } AVCodecContext;
>> 
> 
> I really don't like introducing public API fields for this. Maybe a
> private nvenc option would be better at this point.
> 

So would this be suitable?

The only thing was that I wasn't sure about exposing these options to ffmpeg 
though, as they're useless to ffmpeg unless ffmpeg were to add features for 
dynamically changing the encoded resolutionI just couldn't figure out how 
you can add a private option that isn't exposed in this way though.

>

diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index 304a684e0c..1a1331d468 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -384,6 +384,7 @@ static int nvenc_check_capabilities(AVCodecContext *avctx)
 #endif
 
 ctx->support_dyn_bitrate = nvenc_check_cap(avctx, 
NV_ENC_CAPS_SUPPORT_DYN_BITRATE_CHANGE);
+ctx->support_dyn_res = nvenc_check_cap(avctx, 
NV_ENC_CAPS_SUPPORT_DYN_RES_CHANGE);
 
 return 0;
 }
@@ -1121,6 +1122,8 @@ static av_cold int nvenc_setup_encoder(AVCodecContext 
*avctx)
 
 ctx->init_encode_params.encodeHeight = avctx->height;
 ctx->init_encode_params.encodeWidth = avctx->width;
+ctx->init_encode_params.maxEncodeHeight = ctx->max_height;
+ctx->init_encode_params.maxEncodeWidth = ctx->max_width;
 
 ctx->init_encode_params.encodeConfig = >encode_config;
 
@@ -1276,8 +1279,8 @@ static av_cold int nvenc_alloc_surface(AVCodecContext 
*avctx, int idx)
 }
 
 allocSurf.version = NV_ENC_CREATE_INPUT_BUFFER_VER;
-allocSurf.width = avctx->width;
-allocSurf.height = avctx->height;
+allocSurf.width = FFMAX(ctx->max_width, avctx->width);
+allocSurf.height = FFMAX(ctx->max_height, avctx->height);
 allocSurf.bufferFmt = ctx->surfaces[idx].format;
 
 nv_status = p_nvenc->nvEncCreateInputBuffer(ctx->nvencoder, 
);
@@ -1926,7 +1929,7 @@ static void reconfig_encoder(AVCodecContext *avctx, const 
AVFrame *frame)
 NV_ENC_RECONFIGURE_PARAMS params = { 0 };
 int needs_reconfig = 0;
 int needs_encode_config = 0;
-int reconfig_bitrate = 0, reconfig_dar = 0;
+int reconfig_bitrate = 0, reconfig_dar = 0, reconfig_res = 0;
 int dw, dh;
 
 params.version = NV_ENC_RECONFIGURE_PARAMS_VER;
@@ -1986,6 +1989,24 @@ static void reconfig_encoder(AVCodecContext *avctx, 
const AVFrame *frame)
 }
 }
 
+if (ctx->support_dyn_res && ctx->init_encode_params.maxEncodeWidth && 
ctx->init_encode_params.maxEncodeHeight) {
+if (params.reInitEncodeParams.encodeWidth != avctx->width || 
params.reInitEncodeParams.encodeHeight != avctx->height) {
+av_log(avctx, AV_LOG_VERBOSE,
+"resolution change: %d x %d -> %d x %d\n",
+params.reInitEncodeParams.encodeWidth,
+params.reInitEncodeParams.encodeHeight,
+(uint32_t)avctx->width,
+(uint32_t)avctx->height);
+
+params.reInitEncodeParams.encodeWidth = avctx->width;
+params.reInitEncodeParams.encodeHeight = avctx->height;
+params.forceIDR = 1;
+
+needs_reconfig = 1;
+reconfig_res = 1;
+}
+}
+
 if (!needs_encode_config)
 params.reInitEncodeParams.encodeConfig = NULL;
 
@@ -2005,6 +2026,10 @@ static void reconfig_encoder(AVCodecContext *avctx, 
const AVFrame *frame)
 ctx->encode_config.rcParams.vbvBufferSize = 
params.reInitEncodeParams.encodeConfig->rcParams.vbvBufferSize;
 }
 
+if (reconfig_res) {
+ctx->init_encode_params.encodeWidth = 
params.reInitEncodeParams.encodeWidth;
+ctx->init_encode_params.encodeHeight = 
params.reInitEncodeParams.encodeHeight;
+}
 }
 }
 }
diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
index ddd6168409..30a3aa6f6f 100644
--- a/libavcodec/nvenc.h
+++ b/libavcodec/nvenc.h
@@ -158,6 +158,7 @@ typedef struct NvencContext
 int first_packet_output;
 
 int support_dyn_bitrate;
+int support_dyn_res;
 
 void *nvencoder;
 
@@ -192,6 +193,8 @@ typedef struct NvencContext
 int coder;
 int b_ref_mode;
 int a53_cc;
+int max_width;
+int max_height;
 } NvencContext;
 
 int ff_nvenc_encode_init(AVCodecContext *avctx);
diff --git a/libavcodec/nvenc_h264.c b/libavcodec/nvenc_h264.c
index a6623f5f35..9c9a0c7b8d 100644
--- a/libavcodec/nvenc_h264.c
+++ b/libavcodec/nvenc_h264.c
@@ -138,6 +138,10 @@ static const AVOption options[] = {
 { 

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Oliver Collyer]

>> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
>> index 0ce22ec4fa..7087f82ce1 100644
>> --- a/libavcodec/avcodec.h
>> +++ b/libavcodec/avcodec.h
>> @@ -3357,6 +3357,12 @@ typedef struct AVCodecContext {
>>  * - encoding: unused
>>  */
>> int discard_damaged_percentage;
>> +
>> +/*
>> + * Video encoding only. Sets the maximum picture size for encoders that
>> + * support adjusting the picture size dynamically during encoding.
>> + */
>> + int max_width, max_height;
>> } AVCodecContext;
>> 
> 
> I really don't like introducing public API fields for this. Maybe a
> private nvenc option would be better at this point.
> 

Yeah, I just thought if you wanted to use the fields similarly for other 
encoders in future?

Happy to change it though.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Hendrik Leppkes]

On Wed, Mar 6, 2019 at 3:57 PM Oliver Collyer  wrote:
>
> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> index 0ce22ec4fa..7087f82ce1 100644
> --- a/libavcodec/avcodec.h
> +++ b/libavcodec/avcodec.h
> @@ -3357,6 +3357,12 @@ typedef struct AVCodecContext {
>   * - encoding: unused
>   */
>  int discard_damaged_percentage;
> +
> +/*
> + * Video encoding only. Sets the maximum picture size for encoders that
> + * support adjusting the picture size dynamically during encoding.
> + */
> + int max_width, max_height;
>  } AVCodecContext;
>

I really don't like introducing public API fields for this. Maybe a
private nvenc option would be better at this point.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [PATCH] avcodec/nvenc: Reconfigure resolution on-the-fly

[Oliver Collyer]

Hi

I needed the dynamic resolution changing feature of NVENC to be accessible 
through the ffmpeg libraries for a hobby project, so I added support and here 
is a patch.

I will format this as a proper patch after any changes necessary following 
feedback.

To use this feature you would need to:

1. Specify max_width and max_height before opening the encoder
2. Change the encoder context width/height during encoding

Note that it is the caller's responsibility to scale the input frames to match 
the new encoder size after changing it! In my test I just use sws_scale where 
needed.

Here is a link to a file that switches resolution a few times between 1920x1080 
and 1280x720, that was generated using this patch. This plays correctly in VLC.

https://www.dropbox.com/s/rvt94ai3jx5lbi9/test_switching_2.ts?dl=0 


Regards

Oliver


diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 0ce22ec4fa..7087f82ce1 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -3357,6 +3357,12 @@ typedef struct AVCodecContext {
  * - encoding: unused
  */
 int discard_damaged_percentage;
+
+/*
+ * Video encoding only. Sets the maximum picture size for encoders that
+ * support adjusting the picture size dynamically during encoding.
+ */
+ int max_width, max_height;
 } AVCodecContext;
 
 #if FF_API_CODEC_GET_SET
diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index 304a684e0c..414707dcfd 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -384,6 +384,7 @@ static int nvenc_check_capabilities(AVCodecContext *avctx)
 #endif
 
 ctx->support_dyn_bitrate = nvenc_check_cap(avctx, 
NV_ENC_CAPS_SUPPORT_DYN_BITRATE_CHANGE);
+ctx->support_dyn_res = nvenc_check_cap(avctx, 
NV_ENC_CAPS_SUPPORT_DYN_RES_CHANGE);
 
 return 0;
 }
@@ -1121,6 +1122,8 @@ static av_cold int nvenc_setup_encoder(AVCodecContext 
*avctx)
 
 ctx->init_encode_params.encodeHeight = avctx->height;
 ctx->init_encode_params.encodeWidth = avctx->width;
+ctx->init_encode_params.maxEncodeHeight = avctx->max_height;
+ctx->init_encode_params.maxEncodeWidth = avctx->max_width;
 
 ctx->init_encode_params.encodeConfig = >encode_config;
 
@@ -1276,8 +1279,8 @@ static av_cold int nvenc_alloc_surface(AVCodecContext 
*avctx, int idx)
 }
 
 allocSurf.version = NV_ENC_CREATE_INPUT_BUFFER_VER;
-allocSurf.width = avctx->width;
-allocSurf.height = avctx->height;
+allocSurf.width = FFMAX(avctx->max_width, avctx->width);
+allocSurf.height = FFMAX(avctx->max_height, avctx->height);
 allocSurf.bufferFmt = ctx->surfaces[idx].format;
 
 nv_status = p_nvenc->nvEncCreateInputBuffer(ctx->nvencoder, 
);
@@ -1926,7 +1929,7 @@ static void reconfig_encoder(AVCodecContext *avctx, const 
AVFrame *frame)
 NV_ENC_RECONFIGURE_PARAMS params = { 0 };
 int needs_reconfig = 0;
 int needs_encode_config = 0;
-int reconfig_bitrate = 0, reconfig_dar = 0;
+int reconfig_bitrate = 0, reconfig_dar = 0, reconfig_res = 0;
 int dw, dh;
 
 params.version = NV_ENC_RECONFIGURE_PARAMS_VER;
@@ -1986,6 +1989,24 @@ static void reconfig_encoder(AVCodecContext *avctx, 
const AVFrame *frame)
 }
 }
 
+if (ctx->support_dyn_res && ctx->init_encode_params.maxEncodeWidth && 
ctx->init_encode_params.maxEncodeHeight) {
+if (params.reInitEncodeParams.encodeWidth != avctx->width || 
params.reInitEncodeParams.encodeHeight != avctx->height) {
+av_log(avctx, AV_LOG_VERBOSE,
+"resolution change: %d x %d -> %d x %d\n",
+params.reInitEncodeParams.encodeWidth,
+params.reInitEncodeParams.encodeHeight,
+(uint32_t)avctx->width,
+(uint32_t)avctx->height);
+
+params.reInitEncodeParams.encodeWidth = avctx->width;
+params.reInitEncodeParams.encodeHeight = avctx->height;
+params.forceIDR = 1;
+
+needs_reconfig = 1;
+reconfig_res = 1;
+}
+}
+
 if (!needs_encode_config)
 params.reInitEncodeParams.encodeConfig = NULL;
 
@@ -2005,6 +2026,10 @@ static void reconfig_encoder(AVCodecContext *avctx, 
const AVFrame *frame)
 ctx->encode_config.rcParams.vbvBufferSize = 
params.reInitEncodeParams.encodeConfig->rcParams.vbvBufferSize;
 }
 
+if (reconfig_res) {
+ctx->init_encode_params.encodeWidth = 
params.reInitEncodeParams.encodeWidth;
+ctx->init_encode_params.encodeHeight = 
params.reInitEncodeParams.encodeHeight;
+}
 }
 }
 }
diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
index ddd6168409..f3b959b31a 100644
--- a/libavcodec/nvenc.h
+++ b/libavcodec/nvenc.h
@@ -158,6 +158,7 @@ typedef struct NvencContext
 int first_packet_output;
 
 int support_dyn_bitrate;
+int support_dyn_res;
 
 void *nvencoder;
 
diff