Re: [FFmpeg-devel] [PATCH v2] Add multiple padding method in dnn native
> -Original Message- > From: ffmpeg-devel [mailto:ffmpeg-devel-boun...@ffmpeg.org] On Behalf Of > Steven Liu > Sent: Wednesday, May 15, 2019 10:38 AM > To: FFmpeg development discussions and patches > Cc: Xuewei Meng > Subject: Re: [FFmpeg-devel] [PATCH v2] Add multiple padding method in dnn > native > > Xuewei Meng 于2019年5月11日周六 上午11:11 > 写道: > > > > --- > > libavfilter/dnn_backend_native.c | 52 > > libavfilter/dnn_backend_native.h | 3 ++ > > 2 files changed, 43 insertions(+), 12 deletions(-) @xuewei, we still need to mention the impact of sr filter, and explain why same_clamp_to_edge is needed. > > > > diff --git a/libavfilter/dnn_backend_native.c > b/libavfilter/dnn_backend_native.c > > index 06fbdf368b..171a756385 100644 > > --- a/libavfilter/dnn_backend_native.c > > +++ b/libavfilter/dnn_backend_native.c > > @@ -61,6 +61,12 @@ static DNNReturnType set_input_output_native(void > *model, DNNInputData *input, c > > return DNN_ERROR; > > } > > cur_channels = conv_params->output_num; > > + > > +if(conv_params->padding_method == VALID){ > > +int pad_size = conv_params->kernel_size - 1; > > +cur_height -= pad_size; > > +cur_width -= pad_size; > > +} > > break; > > case DEPTH_TO_SPACE: > > depth_to_space_params = (DepthToSpaceParams > *)network->layers[layer].params; > > @@ -77,6 +83,10 @@ static DNNReturnType set_input_output_native(void > *model, DNNInputData *input, c > > if (network->layers[layer].output){ > > av_freep(&network->layers[layer].output); > > } > > + > > +if(cur_height <= 0 || cur_width <= 0) > > +return DNN_ERROR; > > + > > network->layers[layer].output = av_malloc(cur_height * > cur_width * cur_channels * sizeof(float)); > > if (!network->layers[layer].output){ > > return DNN_ERROR; > > @@ -154,13 +164,14 @@ DNNModel *ff_dnn_load_model_native(const > char *model_filename) > > ff_dnn_free_model_native(&model); > > return NULL; > > } > > +conv_params->padding_method = > (int32_t)avio_rl32(model_file_context); > > conv_params->activation = > (int32_t)avio_rl32(model_file_context); > > conv_params->input_num = > (int32_t)avio_rl32(model_file_context); > > conv_params->output_num = > (int32_t)avio_rl32(model_file_context); > > conv_params->kernel_size = > (int32_t)avio_rl32(model_file_context); > > kernel_size = conv_params->input_num * > conv_params->output_num * > >conv_params->kernel_size * > conv_params->kernel_size; > > -dnn_size += 16 + (kernel_size + conv_params->output_num > << 2); > > +dnn_size += 20 + (kernel_size + conv_params->output_num > << 2); > > if (dnn_size > file_size || conv_params->input_num <= 0 || > > conv_params->output_num <= 0 || > conv_params->kernel_size <= 0){ > > avio_closep(&model_file_context); > > @@ -218,23 +229,35 @@ DNNModel *ff_dnn_load_model_native(const > char *model_filename) > > > > static void convolve(const float *input, float *output, const > ConvolutionalParams *conv_params, int width, int height) > > { > > -int y, x, n_filter, ch, kernel_y, kernel_x; > > int radius = conv_params->kernel_size >> 1; > > int src_linesize = width * conv_params->input_num; > > int filter_linesize = conv_params->kernel_size * > conv_params->input_num; > > int filter_size = conv_params->kernel_size * filter_linesize; > > +int pad_size = (conv_params->padding_method == VALID) ? > (conv_params->kernel_size - 1) / 2 : 0; > > > > -for (y = 0; y < height; ++y){ > > -for (x = 0; x < width; ++x){ > > -for (n_filter = 0; n_filter < conv_params->output_num; > ++n_filter){ > > +for (int y = pad_size; y < height - pad_size; ++y){ > > +for (int x = pad_size; x < width - pad_size; ++x){ > > +for (int n_filter = 0; n_filter < conv_params->output_num; > ++n_filter){ > > output[n_filter] = conv_params->biases[n_filter]; > > -for (ch = 0; ch < conv_params->input_num; ++ch){ > > -for (kernel_y = 0; kernel_y < > conv_params->kernel_size; ++kernel_y){ > > -for (kernel_x = 0; kernel_x < > conv_params->kernel_size; ++kernel_x){ > > -output[n_filter] += > input[CLAMP_TO_EDGE(y + kernel_y - radius, height) * src_linesize + > > - > CLAMP_TO_EDGE(x + kernel_x - radius, width) * conv_params->input_num + ch] > * > > - > conv_params->kernel[n_filter * filter_size + kernel_y * filter_linesize + > > - > kernel_x * conv_params->input_num + ch]; > > + > > +for (int ch = 0; ch < conv_params->input_num; ++ch){ > >
Re: [FFmpeg-devel] [PATCH v1] fftools/ffmpeg: Add support for per frame rotation and flip
On Mon, May 13, 2019 at 2:58 AM Michael Niedermayer wrote: > On Sat, May 11, 2019 at 10:57:01PM -0700, Jun Li wrote: > > On Sat, May 11, 2019 at 10:47 PM Jun Li wrote: > > > > > Fix #6945 > > > Current implementaion for autorotate works fine for stream > > > level rotataion but no support for frame level operation > > > and frame flip. This patch is for adding flip support and > > > per frame operations. > > > --- > > > fftools/cmdutils.c| 9 ++--- > > > fftools/cmdutils.h| 2 +- > > > fftools/ffmpeg.c | 21 +- > > > fftools/ffmpeg.h | 2 + > > > fftools/ffmpeg_filter.c | 81 --- > > > fftools/ffplay.c | 28 +++--- > > > libavutil/display.c | 14 +++ > > > libavutil/display.h | 14 +++ > > > libavutil/tests/display.c | 8 > > > tests/ref/fate/display| 4 ++ > > > 10 files changed, 166 insertions(+), 17 deletions(-) > [...] > > > > \ No newline at end of file > > > > diff --git a/libavutil/display.h b/libavutil/display.h > > > index 515adad795..23d71a0bc5 100644 > > > --- a/libavutil/display.h > > > +++ b/libavutil/display.h > > > @@ -106,6 +106,20 @@ void av_display_rotation_set(int32_t matrix[9], > > > double angle); > > > */ > > > void av_display_matrix_flip(int32_t matrix[9], int hflip, int vflip); > > > > > > +/** > > > + * Extract the rotation component and hflip status of the > transformation > > > matrix. > > > + * > > > + * @param matrix the transformation matrix > > > + * @param hflip wille be set to 1 if the matrix need horizontal > flipped > > > + * @return the angle (in degrees) by which the transformation rotates > the > > > frame > > > + * counterclockwise. The angle will be in range [-180.0, > 180.0], > > > + * or NaN if the matrix is singular. > > > + * > > > + * @note floating point numbers are inherently inexact, so callers are > > > + * recommended to round the return value to nearest integer > before > > > use. > > > + */ > > > +double av_display_rotation_hflip_get(const int32_t matrix[9], int* > hflip); > > > + > > > /** > > > * @} > > > * @} > > > diff --git a/libavutil/tests/display.c b/libavutil/tests/display.c > > > index 893ebb5543..65a0971e7b 100644 > > > --- a/libavutil/tests/display.c > > > +++ b/libavutil/tests/display.c > > > @@ -35,6 +35,8 @@ static void print_matrix(int32_t matrix[9]) > > > int main(void) > > > { > > > int32_t matrix[9]; > > > +int hflip = 0; > > > +double degree; > > > > > > // Set the matrix to 90 degrees > > > av_display_rotation_set(matrix, 90); > > > @@ -56,6 +58,12 @@ int main(void) > > > print_matrix(matrix); > > > printf("degrees: %f\n", av_display_rotation_get(matrix)); > > > > > > +// flip vertical > > > +av_display_matrix_flip(matrix, 0, 1); > > > +print_matrix(matrix); > > > +degree = av_display_rotation_hflip_get(matrix, &hflip); > > > +printf("degrees: %f, hflip: %i\n", degree, hflip); > > > + > > > return 0; > > > > > > } > > [...] > > > This change is to address the feedback of last patch using 1st frame's > > orientation as stream rotation. > > Thanks again for review. I added the support for per frame rotation. > > > > (I know it involves too many file changes and not easy to read. Sorry > about > > that I tried to > > separate them but failed since due to some dependencies). > > The libavutil changes should be in a seperate patch and need a version > and APIChanges update > Thanks Michael for the review. I updated the version to address the feedback: https://patchwork.ffmpeg.org/patch/13120/ https://patchwork.ffmpeg.org/patch/13119/ Thanks! -Jun > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > If a bugfix only changes things apparently unrelated to the bug with no > further explanation, that is a good sign that the bugfix is wrong. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 1/2] lavu/display: add av_display_rotation_hflip_get and bump version
Add av_display_rotation_hflip_get to get information whether the matrix indicates horizontal flip. --- doc/APIchanges| 3 +++ libavutil/display.c | 14 ++ libavutil/display.h | 14 ++ libavutil/tests/display.c | 8 libavutil/version.h | 2 +- tests/ref/fate/display| 4 6 files changed, 44 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index e75c4044ce..73c6809563 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,9 @@ libavutil: 2017-10-21 API changes, most recent first: +2019-05-13 - XX - lavu 56.28.100 - display.h + Add av_display_rotation_hflip_get + 2019-04-20 - 3153a6502a - lavc 58.52.100 - avcodec.h Add AV_CODEC_FLAG_DROPCHANGED to allow avcodec_receive_frame to drop frames whose parameters differ from first decoded frame in stream. diff --git a/libavutil/display.c b/libavutil/display.c index a0076e067b..f5a6a4a08d 100644 --- a/libavutil/display.c +++ b/libavutil/display.c @@ -71,3 +71,17 @@ void av_display_matrix_flip(int32_t matrix[9], int hflip, int vflip) for (i = 0; i < 9; i++) matrix[i] *= flip[i % 3]; } + +double av_display_rotation_hflip_get(const int32_t matrix[9], int *hflip) +{ +int32_t m[9]; +*hflip = 0; +memcpy(m, matrix, sizeof(m)); + +if (m[0] > 0 && m[4] < 0 || m[0] < 0 && m[4] > 0 || +m[1] > 0 && m[3] > 0 || m[1] < 0 && m[3] < 0) { +*hflip = 1; +av_display_matrix_flip(m, 1, 0); +} +return av_display_rotation_get(m); +} diff --git a/libavutil/display.h b/libavutil/display.h index 515adad795..c37d10c741 100644 --- a/libavutil/display.h +++ b/libavutil/display.h @@ -106,6 +106,20 @@ void av_display_rotation_set(int32_t matrix[9], double angle); */ void av_display_matrix_flip(int32_t matrix[9], int hflip, int vflip); +/** + * Extract the rotation component and hflip status of the transformation matrix. + * + * @param matrix the transformation matrix + * @param hflip will be set to 1 if the matrix indicates horizontal flip + * @return the angle (in degrees) by which the transformation rotates the frame + * counterclockwise. The angle will be in range [-180.0, 180.0], + * or NaN if the matrix is singular. + * + * @note floating point numbers are inherently inexact, so callers are + * recommended to round the return value to nearest integer before use. + */ +double av_display_rotation_hflip_get(const int32_t matrix[9], int* hflip); + /** * @} * @} diff --git a/libavutil/tests/display.c b/libavutil/tests/display.c index 893ebb5543..65a0971e7b 100644 --- a/libavutil/tests/display.c +++ b/libavutil/tests/display.c @@ -35,6 +35,8 @@ static void print_matrix(int32_t matrix[9]) int main(void) { int32_t matrix[9]; +int hflip = 0; +double degree; // Set the matrix to 90 degrees av_display_rotation_set(matrix, 90); @@ -56,6 +58,12 @@ int main(void) print_matrix(matrix); printf("degrees: %f\n", av_display_rotation_get(matrix)); +// flip vertical +av_display_matrix_flip(matrix, 0, 1); +print_matrix(matrix); +degree = av_display_rotation_hflip_get(matrix, &hflip); +printf("degrees: %f, hflip: %i\n", degree, hflip); + return 0; } diff --git a/libavutil/version.h b/libavutil/version.h index 12b4f9fc3a..91ab71604b 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 56 -#define LIBAVUTIL_VERSION_MINOR 27 +#define LIBAVUTIL_VERSION_MINOR 28 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ diff --git a/tests/ref/fate/display b/tests/ref/fate/display index 251e7e0cdf..14b4c7db7c 100644 --- a/tests/ref/fate/display +++ b/tests/ref/fate/display @@ -14,3 +14,7 @@ degrees: 135.00 -46340 -46340 0 0 0 1073741824 degrees: -135.00 +-46340 -46340 0 +-46340 46340 0 +0 0 1073741824 +degrees: 45.00, hflip: 1 -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2 2/2] fftools/ffmpeg: add support for per frame rotation and flip
Fix #6945 Current implementaion for autorotate works fine for stream level rotataion but no support for frame level operation and frame flip. This patch is for adding flip support and per frame operations. --- fftools/cmdutils.c | 9 ++--- fftools/cmdutils.h | 2 +- fftools/ffmpeg.c| 21 ++- fftools/ffmpeg.h| 2 + fftools/ffmpeg_filter.c | 81 ++--- fftools/ffplay.c| 28 +++--- 6 files changed, 126 insertions(+), 17 deletions(-) diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index 9cfbc45c2b..b8bb904419 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -2172,13 +2172,12 @@ void *grow_array(void *array, int elem_size, int *size, int new_size) return array; } -double get_rotation(AVStream *st) +double get_rotation_hflip(int32_t display_matrix[9], int* hflip) { -uint8_t* displaymatrix = av_stream_get_side_data(st, - AV_PKT_DATA_DISPLAYMATRIX, NULL); double theta = 0; -if (displaymatrix) -theta = -av_display_rotation_get((int32_t*) displaymatrix); + +if (display_matrix) +theta = -av_display_rotation_hflip_get(display_matrix, hflip); theta -= 360*floor(theta/360 + 0.9/360); diff --git a/fftools/cmdutils.h b/fftools/cmdutils.h index 6e2e0a2acb..dce44ed6e1 100644 --- a/fftools/cmdutils.h +++ b/fftools/cmdutils.h @@ -643,6 +643,6 @@ void *grow_array(void *array, int elem_size, int *size, int new_size); char name[128];\ av_get_channel_layout_string(name, sizeof(name), 0, ch_layout); -double get_rotation(AVStream *st); +double get_rotation_hflip(int32_t display_matrix[9], int* hflip); #endif /* FFTOOLS_CMDUTILS_H */ diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c index 01f04103cf..9ea1aaa930 100644 --- a/fftools/ffmpeg.c +++ b/fftools/ffmpeg.c @@ -2126,6 +2126,24 @@ static int ifilter_has_all_input_formats(FilterGraph *fg) return 1; } +static int ifilter_display_matrix_need_from_frame(InputFilter *ifilter, AVFrame* frame) +{ +int32_t* stream_matrix = (int32_t*)av_stream_get_side_data(ifilter->ist->st, +AV_PKT_DATA_DISPLAYMATRIX, NULL); +// if we already have display matrix from stream, use it instead of extracting +// from frame. +if (stream_matrix) { +memcpy(ifilter->display_matrix, stream_matrix, 4 * 9); +return 0; +} + +// cleanup the display matrix, may be from last frame +memset(ifilter->display_matrix, 0, 4 * 9); +av_display_rotation_set(ifilter->display_matrix, 0); + +return !!av_dict_get(frame->metadata, "Orientation", NULL, 0); +} + static int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame) { FilterGraph *fg = ifilter->graph; @@ -2141,7 +2159,8 @@ static int ifilter_send_frame(InputFilter *ifilter, AVFrame *frame) ifilter->channel_layout != frame->channel_layout; break; case AVMEDIA_TYPE_VIDEO: -need_reinit |= ifilter->width != frame->width || +need_reinit |= ifilter_display_matrix_need_from_frame(ifilter, frame) || + ifilter->width != frame->width || ifilter->height != frame->height; break; } diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index eb1eaf6363..8331720663 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -251,6 +251,8 @@ typedef struct InputFilter { int channels; uint64_t channel_layout; +int32_t display_matrix[9]; + AVBufferRef *hw_frames_ctx; int eof; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 72838de1e2..1894f30561 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -328,6 +328,7 @@ static void init_input_filter(FilterGraph *fg, AVFilterInOut *in) fg->inputs[fg->nb_inputs - 1]->format = -1; fg->inputs[fg->nb_inputs - 1]->type = ist->st->codecpar->codec_type; fg->inputs[fg->nb_inputs - 1]->name = describe_filter_link(fg, in, 1); +av_display_rotation_set(fg->inputs[fg->nb_inputs - 1]->display_matrix, 0); fg->inputs[fg->nb_inputs - 1]->frame_queue = av_fifo_alloc(8 * sizeof(AVFrame*)); if (!fg->inputs[fg->nb_inputs - 1]->frame_queue) @@ -807,22 +808,43 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter, last_filter = ifilter->filter; if (ist->autorotate) { -double theta = get_rotation(ist->st); +int hflip = 0; +double theta = get_rotation_hflip(ifilter->display_matrix, &hflip); -if (fabs(theta - 90) < 1.0) { +if (fabs(theta) < 1.0) { +if (hflip) +ret = insert_filter(&last_filter, &pad_idx, "hflip", NULL); +} else if (fabs(theta - 90) < 1.0) { ret = insert_filter(&last_filter, &pad_idx, "transpose", "clock"); -} else if (fabs(theta - 180) < 1.0) { -ret = insert_f
Re: [FFmpeg-devel] [PATCH v2] Add multiple padding method in dnn native
Xuewei Meng 于2019年5月11日周六 上午11:11写道: > > --- > libavfilter/dnn_backend_native.c | 52 > libavfilter/dnn_backend_native.h | 3 ++ > 2 files changed, 43 insertions(+), 12 deletions(-) > > diff --git a/libavfilter/dnn_backend_native.c > b/libavfilter/dnn_backend_native.c > index 06fbdf368b..171a756385 100644 > --- a/libavfilter/dnn_backend_native.c > +++ b/libavfilter/dnn_backend_native.c > @@ -61,6 +61,12 @@ static DNNReturnType set_input_output_native(void *model, > DNNInputData *input, c > return DNN_ERROR; > } > cur_channels = conv_params->output_num; > + > +if(conv_params->padding_method == VALID){ > +int pad_size = conv_params->kernel_size - 1; > +cur_height -= pad_size; > +cur_width -= pad_size; > +} > break; > case DEPTH_TO_SPACE: > depth_to_space_params = (DepthToSpaceParams > *)network->layers[layer].params; > @@ -77,6 +83,10 @@ static DNNReturnType set_input_output_native(void *model, > DNNInputData *input, c > if (network->layers[layer].output){ > av_freep(&network->layers[layer].output); > } > + > +if(cur_height <= 0 || cur_width <= 0) > +return DNN_ERROR; > + > network->layers[layer].output = av_malloc(cur_height * cur_width * > cur_channels * sizeof(float)); > if (!network->layers[layer].output){ > return DNN_ERROR; > @@ -154,13 +164,14 @@ DNNModel *ff_dnn_load_model_native(const char > *model_filename) > ff_dnn_free_model_native(&model); > return NULL; > } > +conv_params->padding_method = > (int32_t)avio_rl32(model_file_context); > conv_params->activation = (int32_t)avio_rl32(model_file_context); > conv_params->input_num = (int32_t)avio_rl32(model_file_context); > conv_params->output_num = (int32_t)avio_rl32(model_file_context); > conv_params->kernel_size = > (int32_t)avio_rl32(model_file_context); > kernel_size = conv_params->input_num * conv_params->output_num * >conv_params->kernel_size * > conv_params->kernel_size; > -dnn_size += 16 + (kernel_size + conv_params->output_num << 2); > +dnn_size += 20 + (kernel_size + conv_params->output_num << 2); > if (dnn_size > file_size || conv_params->input_num <= 0 || > conv_params->output_num <= 0 || conv_params->kernel_size <= > 0){ > avio_closep(&model_file_context); > @@ -218,23 +229,35 @@ DNNModel *ff_dnn_load_model_native(const char > *model_filename) > > static void convolve(const float *input, float *output, const > ConvolutionalParams *conv_params, int width, int height) > { > -int y, x, n_filter, ch, kernel_y, kernel_x; > int radius = conv_params->kernel_size >> 1; > int src_linesize = width * conv_params->input_num; > int filter_linesize = conv_params->kernel_size * conv_params->input_num; > int filter_size = conv_params->kernel_size * filter_linesize; > +int pad_size = (conv_params->padding_method == VALID) ? > (conv_params->kernel_size - 1) / 2 : 0; > > -for (y = 0; y < height; ++y){ > -for (x = 0; x < width; ++x){ > -for (n_filter = 0; n_filter < conv_params->output_num; > ++n_filter){ > +for (int y = pad_size; y < height - pad_size; ++y){ > +for (int x = pad_size; x < width - pad_size; ++x){ > +for (int n_filter = 0; n_filter < conv_params->output_num; > ++n_filter){ > output[n_filter] = conv_params->biases[n_filter]; > -for (ch = 0; ch < conv_params->input_num; ++ch){ > -for (kernel_y = 0; kernel_y < conv_params->kernel_size; > ++kernel_y){ > -for (kernel_x = 0; kernel_x < > conv_params->kernel_size; ++kernel_x){ > -output[n_filter] += input[CLAMP_TO_EDGE(y + > kernel_y - radius, height) * src_linesize + > - CLAMP_TO_EDGE(x + > kernel_x - radius, width) * conv_params->input_num + ch] * > -conv_params->kernel[n_filter > * filter_size + kernel_y * filter_linesize + > -kernel_x > * conv_params->input_num + ch]; > + > +for (int ch = 0; ch < conv_params->input_num; ++ch){ > +for (int kernel_y = 0; kernel_y < > conv_params->kernel_size; ++kernel_y){ > +for (int kernel_x = 0; kernel_x < > conv_params->kernel_size; ++kernel_x){ > +float input_pel; > +if(conv_params->padding_method == > SAME_CLAMP_TO_EDGE){ > +int y_pos = CLAMP_TO_EDGE(y + kernel_y -
[FFmpeg-devel] [PATCH] lavc/vaapi_hevc: extend parameter buffer to ParameterBufferHEVCExtension
Extend ParameterBufferHEVC to ParameterBufferHEVCExtension for both VAPicture and VASlice. Pass Range Extension flags to support the decode for HEVC REXT. Separate the behaviour of ff_vaapi_decode_make_slice_buffer for base and rext to avoid potential regression. Signed-off-by: Fu Linjie --- libavcodec/vaapi_hevc.c | 134 1 file changed, 112 insertions(+), 22 deletions(-) diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index c69d63d..f0b6f8e 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -29,8 +29,13 @@ #include "vaapi_decode.h" typedef struct VAAPIDecodePictureHEVC { +#if VA_CHECK_VERSION(1, 2, 0) +VAPictureParameterBufferHEVCExtension pic_param; +VASliceParameterBufferHEVCExtension last_slice_param; +#else VAPictureParameterBufferHEVC pic_param; VASliceParameterBufferHEVC last_slice_param; +#endif const uint8_t *last_buffer; size_t last_size; @@ -116,12 +121,17 @@ static int vaapi_hevc_start_frame(AVCodecContext *avctx, const HEVCSPS *sps = h->ps.sps; const HEVCPPS *pps = h->ps.pps; +#if VA_CHECK_VERSION(1, 2, 0) +VAPictureParameterBufferHEVC *pic_param = &pic->pic_param.base; +#else +VAPictureParameterBufferHEVC *pic_param = &pic->pic_param; +#endif + const ScalingList *scaling_list = NULL; int err, i; pic->pic.output_surface = ff_vaapi_get_surface_id(h->ref->frame); - -pic->pic_param = (VAPictureParameterBufferHEVC) { +*pic_param = (VAPictureParameterBufferHEVC) { .pic_width_in_luma_samples= sps->width, .pic_height_in_luma_samples = sps->height, .log2_min_luma_coding_block_size_minus3 = sps->log2_min_cb_size - 3, @@ -188,26 +198,54 @@ static int vaapi_hevc_start_frame(AVCodecContext *avctx, }, }; -fill_vaapi_pic(&pic->pic_param.CurrPic, h->ref, 0); -fill_vaapi_reference_frames(h, &pic->pic_param); +fill_vaapi_pic(&pic_param->CurrPic, h->ref, 0); +fill_vaapi_reference_frames(h, pic_param); if (pps->tiles_enabled_flag) { -pic->pic_param.num_tile_columns_minus1 = pps->num_tile_columns - 1; -pic->pic_param.num_tile_rows_minus1= pps->num_tile_rows - 1; +pic_param->num_tile_columns_minus1 = pps->num_tile_columns - 1; +pic_param->num_tile_rows_minus1= pps->num_tile_rows - 1; for (i = 0; i < pps->num_tile_columns; i++) -pic->pic_param.column_width_minus1[i] = pps->column_width[i] - 1; - +pic_param->column_width_minus1[i] = pps->column_width[i] - 1; for (i = 0; i < pps->num_tile_rows; i++) -pic->pic_param.row_height_minus1[i] = pps->row_height[i] - 1; +pic_param->row_height_minus1[i] = pps->row_height[i] - 1; } if (h->sh.short_term_ref_pic_set_sps_flag == 0 && h->sh.short_term_rps) { -pic->pic_param.st_rps_bits = h->sh.short_term_ref_pic_set_size; +pic_param->st_rps_bits = h->sh.short_term_ref_pic_set_size; } else { -pic->pic_param.st_rps_bits = 0; +pic_param->st_rps_bits = 0; } +#if VA_CHECK_VERSION(1, 2, 0) +if (sps->sps_range_extension_flag) { +pic->pic_param.rext = (VAPictureParameterBufferHEVCRext) { +.range_extension_pic_fields.bits = { +.transform_skip_rotation_enabled_flag = sps->transform_skip_rotation_enabled_flag, +.transform_skip_context_enabled_flag= sps->transform_skip_context_enabled_flag, +.implicit_rdpcm_enabled_flag= sps->implicit_rdpcm_enabled_flag, +.explicit_rdpcm_enabled_flag= sps->explicit_rdpcm_enabled_flag, +.extended_precision_processing_flag = sps->extended_precision_processing_flag, +.intra_smoothing_disabled_flag = sps->intra_smoothing_disabled_flag, +.high_precision_offsets_enabled_flag= sps->high_precision_offsets_enabled_flag, +.persistent_rice_adaptation_enabled_flag= sps->persistent_rice_adaptation_enabled_flag, +.cabac_bypass_alignment_enabled_flag= sps->cabac_bypass_alignment_enabled_flag, +.cross_component_prediction_enabled_flag= pps->cross_component_prediction_enabled_flag, +.chroma_qp_offset_list_enabled_flag = pps->chroma_qp_offset_list_enabled_flag, +}, +.diff_cu_chroma_qp_offset_depth = pps->diff_cu_chroma_qp_offset_depth, +.chroma_qp_offset_list_len_minus1 = pps->chroma_qp_offset_list_len_minus1, +.log2_sao_offset_scale_luma = pps->log2_sao_offset_scale_luma, +.log2_sao_offset_scale_chroma = pps->log2_sao_offset_scale_chroma, +.log2_max_t
Re: [FFmpeg-devel] [PATCH 3/3] avfilter/vf_scale_cuda: Simplify output plane addressing
Yes, this looks better and should have been this way. Thanks, Yogender -Original Message- From: Philip Langdale Sent: Tuesday, May 14, 2019 8:42 AM To: ffmpeg-devel@ffmpeg.org Cc: Yogender Gupta ; Philip Langdale Subject: [PATCH 3/3] avfilter/vf_scale_cuda: Simplify output plane addressing I'm not sure why this was written the way it was originally. We initialise the plane addresses correctly in hwcontext_cuda so why try and play games to calculate the plane offsets directly in this code? --- libavfilter/vf_scale_cuda.c | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/libavfilter/vf_scale_cuda.c b/libavfilter/vf_scale_cuda.c index a833dcd1a4..b7cdb81081 100644 --- a/libavfilter/vf_scale_cuda.c +++ b/libavfilter/vf_scale_cuda.c @@ -390,12 +390,12 @@ static int scalecuda_resize(AVFilterContext *ctx, out->data[0], out->width, out->height, out->linesize[0], 1); call_resize_kernel(ctx, s->cu_func_uchar, 1, - in->data[0]+in->linesize[0]*in->height, in->width/2, in->height/2, in->linesize[0]/2, - out->data[0]+out->linesize[0]*out->height, out->width/2, out->height/2, out->linesize[0]/2, + in->data[1], in->width/2, in->height/2, in->linesize[0]/2, + out->data[1], out->width/2, out->height/2, + out->linesize[0]/2, 1); call_resize_kernel(ctx, s->cu_func_uchar, 1, - in->data[0]+ ALIGN_UP((in->linesize[0]*in->height*5)/4, s->tex_alignment), in->width/2, in->height/2, in->linesize[0]/2, - out->data[0]+(out->linesize[0]*out->height*5)/4, out->width/2, out->height/2, out->linesize[0]/2, + in->data[2], in->width/2, in->height/2, in->linesize[0]/2, + out->data[2], out->width/2, out->height/2, + out->linesize[0]/2, 1); break; case AV_PIX_FMT_YUV444P: @@ -404,12 +404,12 @@ static int scalecuda_resize(AVFilterContext *ctx, out->data[0], out->width, out->height, out->linesize[0], 1); call_resize_kernel(ctx, s->cu_func_uchar, 1, - in->data[0]+in->linesize[0]*in->height, in->width, in->height, in->linesize[0], - out->data[0]+out->linesize[0]*out->height, out->width, out->height, out->linesize[0], + in->data[1], in->width, in->height, in->linesize[0], + out->data[1], out->width, out->height, + out->linesize[0], 1); call_resize_kernel(ctx, s->cu_func_uchar, 1, - in->data[0]+in->linesize[0]*in->height*2, in->width, in->height, in->linesize[0], - out->data[0]+out->linesize[0]*out->height*2, out->width, out->height, out->linesize[0], + in->data[2], in->width, in->height, in->linesize[0], + out->data[2], out->width, out->height, + out->linesize[0], 1); break; case AV_PIX_FMT_YUV444P16: @@ -433,7 +433,7 @@ static int scalecuda_resize(AVFilterContext *ctx, 1); call_resize_kernel(ctx, s->cu_func_uchar2, 2, in->data[1], in->width/2, in->height/2, in->linesize[1], - out->data[0] + out->linesize[0] * ((out->height + 31) & ~0x1f), out->width/2, out->height/2, out->linesize[1]/2, + out->data[1], out->width/2, out->height/2, + out->linesize[1]/2, 1); break; case AV_PIX_FMT_P010LE: @@ -443,7 +443,7 @@ static int scalecuda_resize(AVFilterContext *ctx, 2); call_resize_kernel(ctx, s->cu_func_ushort2, 2, in->data[1], in->width / 2, in->height / 2, in->linesize[1]/2, - out->data[0] + out->linesize[0] * ((out->height + 31) & ~0x1f), out->width / 2, out->height / 2, out->linesize[1] / 4, + out->data[1], out->width / 2, out->height / + 2, out->linesize[1] / 4, 2); break; case AV_PIX_FMT_P016LE: @@ -453,7 +453,7 @@ static int scalecuda_resize(AVFilterContext *ctx, 2); call_resize_kernel(ctx, s->cu_func_ushort2, 2, in->data[1], in->width / 2, in->height / 2, in->linesize[1] / 2, - out->data[0] + out->linesize[0] * ((out->height + 31) & ~0x1f), out->width / 2, out->height / 2, out->linesize[1] / 4, + out->data[1], out->width / 2, out->height / + 2, out->linesize[1] / 4, 2); break; default: -- 2.20.1 --
Re: [FFmpeg-devel] [PATCH] "assert(a && b)" --> "assert(a); assert(b)" for more precise diagnostics, except for libformat
On Wed, May 15, 2019 at 7:01 AM Hendrik Leppkes wrote: > > On Tue, May 14, 2019 at 11:25 PM Adam Richter wrote: > > > > Consider, for example, if you agree that columnization makes this range > > check > > more recognizable in a glance and makes it easier to spot what the bounds > > are > > (the sixteen space indentation is taken from the code in which it appeared): > > > > av_assert0(par->bits_per_coded_sample >= 0 && > > par->bits_per_coded_sample <= 8); > > > > ...vs... > > > > av_assert0(par->bits_per_coded_sample >= 0); > > av_assert0(par->bits_per_coded_sample <= 8); > > > > A possible counter-argument to this might be that, in a long sequence > > of assertions, can be informative to group related assertions > > together, which I think is true, but it is possible to get this other > > means, such as by using blank lines to separate express such grouping. > > > > So, Mark, if you decide you are OK with my complete patches, I encourage > > you to let me know. Otherwise, if there are any of those changes which you > > are OK with, I would like to just to to get those merged. Finally, if you > > would > > rather see none of those changes merged (one one to split the assertions in > > libavformat and one was for everywhere else in ffmpeg), please let me know > > about that too, in which case, if no one advocates for their > > inclusion, I'll drop > > my proposal to merge these changes. > > > > Unfortunately I have to agree with Mark. asserst that check the same > value or extremely closely related values should stay combined. > I agree this part > > > > Also after this, I may take a look at adding a branch hint to the av_assertN > > macros if nobody objects. > > > > Please don't, we don't do branch hints anywhere, and this is a bad > place to start. > If an assert is truely performance sensitive (as in, it makes a > measurable difference), it should probably be moved from assert0 to > assert1, and as such is only enabled in testing builds. > If assert0 or assert1 lead to performance drop, we need some profiling data, then try to fix it. > - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH 1/2] VP4 video decoder
On Tue, May 14, 2019 at 10:37:03PM +1000, Peter Ross wrote: > On Sun, May 12, 2019 at 01:24:56PM +0200, Reimar Döffinger wrote: > > On 12.05.2019, at 08:12, Peter Ross wrote: > > > +static int read_mb_value(GetBitContext *gb) > > > +{ > > > +int v = 1; > > > +int size; > > > + > > > +do { > > > +size = 0; > > > +if (!get_bits1(gb)) > > > +break; > > > +v++; > > > +do { > > > +if (!get_bits1(gb)) > > > +break; > > > +v += 1 << size++; > > > +} while (size < 8); > > > +} while (size == 8); > > > + > > > +if (size) > > > +v += get_bits(gb, size); > > > + > > > +return v; > > > +} > > > > Maybe not worth it performance wise, but did you check if this could be > > simplified? > > For example the get_bits1 cases that end up with size 0 could return > > directly. > > Or it could peek ahead 9 bits in the bitstream and count the leading 1s to > > get v and size without looping (i.e. loop only for the 9 bits of 1s > > specifically). > > Alternatively add a comment to clarify the encoding scheme it implements > > (like 9 consecutive 1s is a prefix encoding an offset of 257 etc). > > thanks for these suggestions. > > replacing get_bits() with OPEN_READER/UPDATE_CACHE/SHOW_UBITS/etc results in a > consistent 0.50 % speedup. Not sure how readable that gets, I actually just meant taking advantage of show_bits() While nice, I am not sure speed is much of a consideration for vp4 really. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] "assert(a && b)" --> "assert(a); assert(b)" for more precise diagnostics, except for libformat
On Tue, May 14, 2019 at 11:25 PM Adam Richter wrote: > > Consider, for example, if you agree that columnization makes this range check > more recognizable in a glance and makes it easier to spot what the bounds are > (the sixteen space indentation is taken from the code in which it appeared): > > av_assert0(par->bits_per_coded_sample >= 0 && > par->bits_per_coded_sample <= 8); > > ...vs... > > av_assert0(par->bits_per_coded_sample >= 0); > av_assert0(par->bits_per_coded_sample <= 8); > > A possible counter-argument to this might be that, in a long sequence > of assertions, can be informative to group related assertions > together, which I think is true, but it is possible to get this other > means, such as by using blank lines to separate express such grouping. > > So, Mark, if you decide you are OK with my complete patches, I encourage > you to let me know. Otherwise, if there are any of those changes which you > are OK with, I would like to just to to get those merged. Finally, if you > would > rather see none of those changes merged (one one to split the assertions in > libavformat and one was for everywhere else in ffmpeg), please let me know > about that too, in which case, if no one advocates for their > inclusion, I'll drop > my proposal to merge these changes. > Unfortunately I have to agree with Mark. asserst that check the same value or extremely closely related values should stay combined. > > Also after this, I may take a look at adding a branch hint to the av_assertN > macros if nobody objects. > Please don't, we don't do branch hints anywhere, and this is a bad place to start. If an assert is truely performance sensitive (as in, it makes a measurable difference), it should probably be moved from assert0 to assert1, and as such is only enabled in testing builds. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2 2/2] tests/fate: Add fate test for timestamp interpolation
Hello, On Tue, 14. May 17:54, Andriy Gelman wrote: > From: Andriy Gelman > > This test verifies that missing timestamps from mpegts are interpolated > for a HEVC stream. > --- > tests/fate/mpegts.mak | 7 ++ > tests/ref/fate/mpegts-probe-interptime | 154 + > 2 files changed, 161 insertions(+) > create mode 100644 tests/ref/fate/mpegts-probe-interptime > > diff --git a/tests/fate/mpegts.mak b/tests/fate/mpegts.mak > index bbcbfc47b2..71e40ef26f 100644 > --- a/tests/fate/mpegts.mak > +++ b/tests/fate/mpegts.mak > @@ -5,6 +5,10 @@ PROBE_CODEC_NAME_COMMAND = \ > ffprobe$(PROGSSUF)$(EXESUF) -show_entries stream=codec_name \ > -print_format default -bitexact -v 0 > > +PROBE_INTERP_TIME_COMMAND = \ > +ffprobe$(PROGSSUF)$(EXESUF) -show_entries packet=pts,dts \ > +-print_format default -bitexact -v 0 > + > FATE_MPEGTS_PROBE-$(call DEMDEC, MPEGTS, HEVC, AAC_LATM) += > fate-mpegts-probe-latm > fate-mpegts-probe-latm: SRC = $(TARGET_SAMPLES)/mpegts/loewe.ts > fate-mpegts-probe-latm: CMD = run $(PROBE_CODEC_NAME_COMMAND) -i "$(SRC)" > @@ -19,6 +23,9 @@ FATE_MPEGTS_PROBE-$(call DEMDEC, MPEGTS) += > fate-mpegts-probe-pmt-merge > fate-mpegts-probe-pmt-merge: SRC = $(TARGET_SAMPLES)/mpegts/pmtchange.ts > fate-mpegts-probe-pmt-merge: CMD = run $(PROBE_CODEC_NAME_COMMAND) > -merge_pmt_versions 1 -i "$(SRC)" > > +FATE_MPEGTS_PROBE-$(call DEMDEC, MPEGTS, HEVC) += > fate-mpegts-probe-interptime > +fate-mpegts-probe-interptime: SRC = $(TARGET_SAMPLES)/mpegts/foreman.ts I attach a file foreman.ts that is required for this fate test. Thanks, Andriy G}¾Ì®÷]í¢ÆDÀ0º æ;4úàý5¾Ôcf:½\{!b*Û Ñ(ýÄ@ rÏW¬ý£F)J¬[#¸h|°BûÚ¡ ÜVñ÷TRlºa2ôGv1ç2µÞÌ'v-N{ðϪHP47TÏXw×Ò½rå_Ìo¿Tc>PôÏDq;¦³©P¹/¸ÓÙ`ܯG 4- ½¡uGAܶíÙAɼïqd)lBÀdf¾ M*IÕZй_@{´Í3¢R2çiH 6£R}>~xýlÞ7§eW¼£äçÎ[â}ß Á2Â¥gÈâ,Ù©Î1K±ýÛaÝ4-mÜ0Ìvbi&ÑÓؤKõÌU°|E;GA à %ÃÛC« FP Ñ0ýÄ03çÙŬIíßâaþ2,P±÷¬|@w`Jg!b=ÆZpyzi+UY;Ï æ4³£ DxÇ8n XÖKB{-%uÖU>´<1Ëï?'Bhº©¶Ø£|É?±»'$®Ö:n# áÅ°\ ~gB·ßÂb/ÂßëähïÅåaG´°¿N W(íàG Ð,ק( D¢ð¾ Þ¥¼&U3⠤Πùá903ck¹8ã¹EªlJ¤ ªì¹¨¾¦ "Ý ´t½X|9áOú¾1óC:&}HEªéÀÎM*¸¶kyüé,9a Ñ8ýÄ03Ê 1JÁÓ#ÒT=iN#|RâÑ;A=¯òÅFë¢Á¦^X±L?ÔnÏFäƸMQtPF¶êÔ}÷o¡rG 7B ÿ¢ãU²3& b.ØïûQû1G Zå"_d>éȵãã.>¿@uY¤ÆHT7ö@ªÍÂ3á¡÷NlDfÉ÷EOhNÅþºejó®½êl& ¬½ÍâtC²dǯ¥Ý¹TÐo¶$ GA à %ÃÛó FP @ÿÿ` B` $Yd©$Êà DÁr´b@ *¬ M\%b|8À¡gî>-HÇÙÏiöic UM0X°F²p+å)Cø/Ú5 fOÖ^ `zu-ø¤Zzºþ ÍG eÕB¥Û íD1*h§Pì½±· ýÏô ÍÁ'ý¤¤c ¿Ï»w¼øgÛÝê__óuÁ^_ 2:IÂïß×®gÝÅ®³K& ¥\Xüê=+Ý»ôHûÛ¿èbËù[j4({ÞEýøÄ<ýVR©¡}sÔ[6Ï@«½×#¨§Ç¤aç-?ë?øÀÙK¾wîîjKª^.A,GvM B.m ÌÇæÅG zcÖµ;YÐu÷7"L|!6 +Êèë¥{zÃv:ÿ*jG8ø[Õ¬êJÙØÄþÿóízÞbÇr})"iú¾Îlø8ÓC?§ö2E2ëó\¢K8(· Nxmÿ¥ÎJ[æþd+8¼Ñhù²«,¼ÐñÝÈGÓò$ÚÃü¸ðÑÉ[ÿ¬<=x¡¿çà-of¹Þ/,®mFÅ*xâ ¶«ê/ÿ<ñ=G Ûoxà,¶/ßj¡³¬»WòÝ©üj>ÁV>NüT zfØì.¡.#ÁKË´Á×£è¬äÌéÇDzRw:;-M&=R½¹ì¦êYÔ'è%xW¸¿Ð¾ëâí¦éG] PªKerÛ0 ý ^îý>ì׺üÿ2aÞyÎGHÇ*ÑaÔ!¦·m®â¾SÒkÿqm(ÖÍKÝ*qæ^Q(EÜÍF0»G TUþÇÚÍOw»+ÈÄ«³&MÌøn|Úc+ñò`W³ù*£ô´ÆìÛYoâÛO[0 H'Ã;êCÌwxá)f Xù¦í%ë%åë¹íÕѪÇS&ôêÝLuè¦4¡p¸<F»»»6wë;Oé¯ÌNbÛåS-(|pÛ~2ÿ2Ô x´×L¨eûSEO>¦Ù¨.ÁÔã¾yBçêÔG a+«c·Ñ×Bè¨gãµcï Xû÷Ñä0"A|K »[óN-¶¼ÜAÂÚh½üÞRî×ã4B.Õ¢¦êï6DÅÌ:·÷KmÇty_ñ÷<¬O¼xJG&|«P*Rù#Hçè¬P#|,~ÆÖ$ûöÒüplð¡§t¾Ù?'1L³ë<òé§ì}f¹)é´q¢¸¡»WmÁÐCôo¥G >C ÿÿÉÒÑWë°OPÞPU÷ÕÑËk4¤Õ°Þ¶,mh8v[ò µø¼(}®z´Dc<þöz$fì¾,Á®fBNe.p\áXÖ©FÊÃx½í@ɸɲ°~üEÓ