Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-08 Thread Mark Thompson 
On 08/08/17 09:33, Jorge Ramirez wrote:
> On 08/08/2017 10:23 AM, Martin Storsjö wrote:
>
 This doesn't set extradata - you need to extract the codec global headers 
>>> (such as H.264 SPS and PPS) at init time to be able to write correct files 
>>> for some codecs (such as H.264) with muxers requiring global headers (such 
>>> as MP4).  It kindof works without it, but the files created will not 
>>> conform and will not be usable on some players.

>>> ah that might explain some things (when I play back the encoded video the 
>>> quality is pretty lousy)
>>> is there already some code I can use as a reference? I might be out of my 
>>> depth here so any help will be more than welcome
>>
>> This is exactly the thing I was trying to tell you about, off list, before.
> 
> Hi Martin,
> 
> yes sorry I was just thinking about that. I was going to follow up but little 
> details here on this patchset keep on piling up.
> 
> btw -just for info- the encoding issues I was seeing were related to the time 
> stamps not being properly passed to the encoder...still trying to figure out 
> that as well but at least I now get a proper encoded/decoded frame (even 
> though just one, that keeps on being played when I reproduce the encoded 
> video)
> 
> the encoder however does not complain during the process so I assume the 
> other frames are also there
> 
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.63e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.61e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.6e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.58e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.56e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.55e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.53e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.52e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.5e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.48e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.47e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.45e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.44e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.42e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.41e-07x
> [rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
> 1920trate=4923.1kbits/s speed=3.39e-07x
> [out_0_0 @ 0xe5e64be0] EOF on sink link 
> out_0_0:default.bitrate=4923.1kbits/s speed=3.36e-07x
> 
> No more output streams to write to, finishing.
> frame=  232 fps=1.0 q=-0.0 Lsize=  42kB time=00:00:00.00 
> bitrate=736989.3kbits/s speed=2.01e-06x
> video:41kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing 
> overhead: 1.783316%
> Input file #0 (/home/linaro/Videos/raw/freeway.yuv):
>   Input stream #0:0 (video): 232 packets read (320716800 bytes); 232 frames 
> decoded;
>   Total: 232 packets (320716800 bytes) demuxed
> Output file #0 (out/out.h264.mp4):
>   Output stream #0:0 (video): 232 frames encoded; 6 packets muxed (42449 
> bytes);
>   Total: 6 packets (42449 bytes) muxed
> 232 frames successfully decoded, 0 decoding errors
> 
>>
>> In the OMX driver used on android, this is requested on startup, via an 
>> ioctl with the following private ioctl value:
>> V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER
>>
>> See this code here:
>> https://android.googlesource.com/platform/hardware/qcom/media/+/63abe022/msm8996/mm-video-v4l2/vidc/venc/src/video_encoder_device_v4l2.cpp#2991
>>
>> This is a qcom specific, private ioctl. In the android kernel for qualcomm, 
>> this is handled correctly here:
>>
>> https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_venc.c#2987
>> https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_vidc_common.c#3767
>>
>> In the dragonboard kernel snapshot I had been testing, that I referred to 
>> you before, there are incomplete stubs of handling of this. In the 
>> debian-qcom-dragonboard410c-16.04 tag in the linaro kernel tree:
>>
>> http://git.linaro.org/landing-teams/working/qualcomm/kernel.git/tree/drivers/media/platform/msm/vidc/msm_venc-ctrls.c?h=debian-qcom-dragonboard410c-16.04&id=8205f603ceeb02d08a720676d9075c9e75e47b0f#n2116
>> This increments seq_hdr_reqs, just like in the android ke

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-08 Thread Jorge Ramirez 

On 08/08/2017 10:23 AM, Martin Storsjö wrote:


This doesn't set extradata - you need to extract the codec global 
headers 
(such as H.264 SPS and PPS) at init time to be able to write correct 
files for some codecs (such as H.264) with muxers requiring global 
headers (such as MP4).  It kindof works without it, but the files 
created will not conform and will not be usable on some players.


ah that might explain some things (when I play back the encoded video 
the quality is pretty lousy)
is there already some code I can use as a reference? I might be out 
of my depth here so any help will be more than welcome


This is exactly the thing I was trying to tell you about, off list, 
before.


Hi Martin,

yes sorry I was just thinking about that. I was going to follow up but 
little details here on this patchset keep on piling up.


btw -just for info- the encoding issues I was seeing were related to the 
time stamps not being properly passed to the encoder...still trying to 
figure out that as well but at least I now get a proper encoded/decoded 
frame (even though just one, that keeps on being played when I reproduce 
the encoded video)


the encoder however does not complain during the process so I assume the 
other frames are also there


[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.63e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.61e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.6e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.58e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.56e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.55e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.53e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.52e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.5e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.48e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.47e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.45e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.44e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.42e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.41e-07x
[rawvideo @ 0xe5e600a0] PACKET SIZE: 1382400, STRIDE: 
1920trate=4923.1kbits/s speed=3.39e-07x
[out_0_0 @ 0xe5e64be0] EOF on sink link 
out_0_0:default.bitrate=4923.1kbits/s speed=3.36e-07x


No more output streams to write to, finishing.
frame=  232 fps=1.0 q=-0.0 Lsize=  42kB time=00:00:00.00 
bitrate=736989.3kbits/s speed=2.01e-06x
video:41kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB 
muxing overhead: 1.783316%

Input file #0 (/home/linaro/Videos/raw/freeway.yuv):
  Input stream #0:0 (video): 232 packets read (320716800 bytes); 232 
frames decoded;

  Total: 232 packets (320716800 bytes) demuxed
Output file #0 (out/out.h264.mp4):
  Output stream #0:0 (video): 232 frames encoded; 6 packets muxed 
(42449 bytes);

  Total: 6 packets (42449 bytes) muxed
232 frames successfully decoded, 0 decoding errors



In the OMX driver used on android, this is requested on startup, via 
an ioctl with the following private ioctl value:

V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER

See this code here:
https://android.googlesource.com/platform/hardware/qcom/media/+/63abe022/msm8996/mm-video-v4l2/vidc/venc/src/video_encoder_device_v4l2.cpp#2991 



This is a qcom specific, private ioctl. In the android kernel for 
qualcomm, this is handled correctly here:


https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_venc.c#2987 

https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_vidc_common.c#3767 



In the dragonboard kernel snapshot I had been testing, that I referred 
to you before, there are incomplete stubs of handling of this. In the 
debian-qcom-dragonboard410c-16.04 tag in the linaro kernel tree:


http://git.linaro.org/landing-teams/working/qualcomm/kernel.git/tree/drivers/media/platform/msm/vidc/msm_venc-ctrls.c?h=debian-qcom-dragonboard410c-16.04&id=8205f603ceeb02d08a720676d9075c9e75e47b0f#n2116 

This increments seq_hdr_reqs, just like in the android kernel tree 
(where this is working). However in this kernel tree, nothing actually 
ever reads the seq_hdr_reqs, so it's a non-functional stub.


ok. thanks for the info



Now in the kernel tree you referred me to, in the 
release/db820c/qcomlt-4.11

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-08 Thread Martin Storsjö 

Hi Jorge,

On Mon, 7 Aug 2017, Jorge Ramirez wrote:


On 08/03/2017 01:53 AM, Mark Thompson wrote:

+default:
+return 0;
+}
+
+SET_V4L_EXT_CTRL(value, qmin, avctx->qmin, "minimum video quantizer 

scale");
+SET_V4L_EXT_CTRL(value, qmax, avctx->qmax, "maximum video quantizer 

scale");

+
+return 0;
+}
This doesn't set extradata - you need to extract the codec global headers 
(such as H.264 SPS and PPS) at init time to be able to write correct files 
for some codecs (such as H.264) with muxers requiring global headers (such as 
MP4).  It kindof works without it, but the files created will not conform and 
will not be usable on some players.


ah that might explain some things (when I play back the encoded video 
the quality is pretty lousy)
is there already some code I can use as a reference? I might be out of 
my depth here so any help will be more than welcome


This is exactly the thing I was trying to tell you about, off list, 
before.


In the OMX driver used on android, this is requested on startup, via an 
ioctl with the following private ioctl value:

V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER

See this code here:
https://android.googlesource.com/platform/hardware/qcom/media/+/63abe022/msm8996/mm-video-v4l2/vidc/venc/src/video_encoder_device_v4l2.cpp#2991

This is a qcom specific, private ioctl. In the android kernel for 
qualcomm, this is handled correctly here:


https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_venc.c#2987
https://android.googlesource.com/kernel/msm/+/android-7.1.2_r0.33/drivers/media/platform/msm/vidc/msm_vidc_common.c#3767

In the dragonboard kernel snapshot I had been testing, that I referred to 
you before, there are incomplete stubs of handling of this. In the 
debian-qcom-dragonboard410c-16.04 tag in the linaro kernel tree:


http://git.linaro.org/landing-teams/working/qualcomm/kernel.git/tree/drivers/media/platform/msm/vidc/msm_venc-ctrls.c?h=debian-qcom-dragonboard410c-16.04&id=8205f603ceeb02d08a720676d9075c9e75e47b0f#n2116
This increments seq_hdr_reqs, just like in the android kernel tree (where 
this is working). However in this kernel tree, nothing actually ever reads 
the seq_hdr_reqs, so it's a non-functional stub.


Now in the kernel tree you referred me to, in the 
release/db820c/qcomlt-4.11 branch, I don't see anything similar to 
V4L2_CID_MPEG_VIDC_VIDEO_REQUEST_SEQ_HEADER. I can't help you from there, 
you need to figure that out what alternative codepath there is, intended 
to replace it - if any. If there aren't any, you first need to fix the 
v4l2 driver before userspace apps can get what they need.


There is a clear need for this, as you witness in the android version of 
the kernel. It just seems to have been removed in the vanilla linux 
version of the driver.


// Martin
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Mark Thompson 
On 07/08/17 23:03, Jorge Ramirez wrote:
> On 08/07/2017 11:55 PM, Mark Thompson wrote:
>>> would you know where can I get this information from (qmin/qmx for the 
>>> other encoders)?
>>  From the relevant language standards.
>>
>> H.263 / MPEG-4 part 2:  [1, 31]  (: 
>> PQUANT, GQUANT.)
>> H.264 / MPEG-4 part 10 / AVC:  [48 - 6 * bitdepth, 51]  
>> (: QpBdOffsety, pic_init_qp_minus26, 
>> slice_qp_delta.)
>> H.265 / MPEG-H part 2 / HEVC:  [48 - 6 * bitdepth, 51]  
>> (: QpBdOffsety, init_qp_minus26, 
>> slice_qp_delta.)

(Possibly relevant: many implementations normalise these to be nonnegative on 
the user side rather than using the standard values, making them [0, 3 + 
bitdepth * 6].  If you have 10-bit video then you should check what range the 
codecs actually use here.)

>> VP8:  [0, 127]  (: q_index.)
>> VP9:  [0, 255]  
>> (:
>>  base_q_idx.)
> ah thanks a lot.
> 
> btw, how come that that this information is not already present in some 
> ffmpeg table?

Most files only deal with one codec.  It's only when you've delegated almost 
everything to somewhere else that you run into the problem of all of these 
together :)

> Just trying to decide if I should just import the relevant data into the 
> v4l2_m2m_enc file or somewhere else?
Put them in the v4l2_m2m_enc file, I think.  How you want do that probably 
depends on how V4L2 acts without the options: if it acts sensibly when you 
don't pass them, then make a defaults table with sentinel values and don't pass 
anything if you see them (so they are only passed to the codec if the user sets 
them manually).  If they are required, then you'll need to make a defaults 
table for each codec with the correct values in.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Mark Thompson 
On 07/08/17 20:28, Jorge Ramirez wrote:
> On 08/03/2017 01:53 AM, Mark Thompson wrote:
>>> +AV_OPT_TYPE_INT,\
>>> +{.i64 = V4L2_MEMORY_MMAP},\
>>> +0, INT_MAX, FLAGS },\
>>> +{ "output_memory",\
>>> +"Output memory model: See V4L2_MEMORY_* in videodev2.h. This 
>>> depends on the HW but default should work with most.",\
>>> +OFFSET(capture_pool.memory),\
>>> +AV_OPT_TYPE_INT,\
>>> +{.i64 = V4L2_MEMORY_MMAP},\
>>> +0, INT_MAX, FLAGS },\
>>> +{ "num_output_pool_buffers",\
>>> +"Number of buffers in the output pool",\
>>> +OFFSET(output_pool.num_buffers),\
>>> +AV_OPT_TYPE_INT,\
>>> +{ .i64 = 16 },\
>>> +4, INT_MAX, FLAGS }
>> Can we attempt to set this automatically based on the codec?  16 won't be 
>> enough for worst-case H.264/H.265.
>>
> 
> please could you suggest some defaults so I can prepare the table?

If, as you've suggested elsewhere, the decoder doesn't actually use these 
frames for reference and instead has its own copies internally then this won't 
actually matter.

If it turns out that you do need the buffering here then the numbers of 
surfaces used by other hardware decoders are probably good - e.g. 
.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Mark Thompson 
On 07/08/17 20:24, Jorge Ramirez wrote:
> On 08/03/2017 01:53 AM, Mark Thompson wrote:
>>> +default:
>>> +return 0;
>>> +}
>>> +
>>> +SET_V4L_EXT_CTRL(value, qmin, avctx->qmin, "minimum video quantizer 
>>> scale");
>>> +SET_V4L_EXT_CTRL(value, qmax, avctx->qmax, "maximum video quantizer 
>>> scale");
>>> +
>>> +return 0;
>>> +}
>> This doesn't set extradata - you need to extract the codec global headers 
>> (such as H.264 SPS and PPS) at init time to be able to write correct files 
>> for some codecs (such as H.264) with muxers requiring global headers (such 
>> as MP4).  It kindof works without it, but the files created will not conform 
>> and will not be usable on some players.
>>
> ah that might explain some things (when I play back the encoded video the 
> quality is pretty lousy)
> is there already some code I can use as a reference? I might be out of my 
> depth here so any help will be more than welcome

Methods used by other codecs to extract the headers:

Something to call directly to get them:
* x264_encoder_headers in libx264: 
.
* nvEncGetSequenceParams in nvenc: 
.

They appear on the front of the stream and can be read from there at init time:
* OMX_BUFFERFLAG_CODECCONFIG in OpenMAX IL: 
.

Know the answer magically in advance (probably not applicable here):
* VAAPI makes the user write their own headers, so we can just write those to 
the extradata directly as well: 
.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/07/2017 11:55 PM, Mark Thompson wrote:

would you know where can I get this information from (qmin/qmx for the other 
encoders)?

 From the relevant language standards.

H.263 / MPEG-4 part 2:  [1, 31]  (: 
PQUANT, GQUANT.)
H.264 / MPEG-4 part 10 / AVC:  [48 - 6 * bitdepth, 51]  
(: QpBdOffsety, pic_init_qp_minus26, 
slice_qp_delta.)
H.265 / MPEG-H part 2 / HEVC:  [48 - 6 * bitdepth, 51]  
(: QpBdOffsety, init_qp_minus26, 
slice_qp_delta.)
VP8:  [0, 127]  (: q_index.)
VP9:  [0, 255]  
(:
 base_q_idx.)

ah thanks a lot.

btw, how come that that this information is not already present in some 
ffmpeg table?
Just trying to decide if I should just import the relevant data into the 
v4l2_m2m_enc file or somewhere else?

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/08/2017 12:01 AM, Mark Thompson wrote:

great, thanks for that.
The issue I have is that the V4L2 driver I am using to test - yes this needs to 
be detected by the decoder- has not implemented the feature yet. Will keep it 
in mind before sending the last series of the patchset.

Are they actually expecting the user to parse the stream externally to detect this 
case?  That wouldn't be too horrible to implement if many drivers are likely to have 
this problem - we already do essentially the same thing in the QSV decoder - see the 
parser use 
at.


the detection if I understood it correctly is handled and flagged by the 
hardware codec (the v4l2 driver) and the user (ie ffmpeg) is then 
notified and need to do some corrections.


But yes, I need to look this in detail (will not be in the next v4 
patchset, probably in v5)





also, I have rewritten the patchset to add clarity and clean some of the 
sedimentation from testing/hacking. is it ok with you if I post v4 now? it will 
simplify the discussion - the driver now much easier to follow.

Sure.  I'll go through it again within a few days.


thanks a lot for your help on this.



- Mark


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Mark Thompson 
On 07/08/17 19:57, Jorge Ramirez wrote:
> On 08/04/2017 03:32 PM, Mark Thompson wrote:
>> On 04/08/17 14:06, Jorge Ramirez wrote:
>>> On 08/04/2017 01:51 PM, Mark Thompson wrote:
> +if (ret)
> +return 0;
> +
> +return avpriv_v4l_dequeue(cap_pool, frame, NULL, timeout);
> +}
 What happens to the decoder if some parameters like resolution change 
 mid-stream?  (Try the stream h264/reinit-large_420_8-to-small_420_8.h264 
 in the fate samples.)
>>> yes, it breaks - the player still shows and image but it is corrupted.
>>> I am not sure how to handle this situation to be honest...so need to look 
>>> into this.
>> There should be some way to detect it from the decoder, I think?  (I don't 
>> actually know how this works in V4L2.)
>>
>> To compare, if you look at MMAL (perhaps the most similar API in current 
>> lavc) there is a special FORMAT_CHANGED return when dequeuing a frame, and 
>> the decoder is reconfigured with new buffers when that 
>> happens:.
>>
> 
> great, thanks for that.
> The issue I have is that the V4L2 driver I am using to test - yes this needs 
> to be detected by the decoder- has not implemented the feature yet. Will keep 
> it in mind before sending the last series of the patchset.

Are they actually expecting the user to parse the stream externally to detect 
this case?  That wouldn't be too horrible to implement if many drivers are 
likely to have this problem - we already do essentially the same thing in the 
QSV decoder - see the parser use at 
.

> also, I have rewritten the patchset to add clarity and clean some of the 
> sedimentation from testing/hacking. is it ok with you if I post v4 now? it 
> will simplify the discussion - the driver now much easier to follow.

Sure.  I'll go through it again within a few days.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Mark Thompson 


On 07/08/17 19:51, Jorge Ramirez wrote:
> On 08/04/2017 01:51 PM, Mark Thompson wrote:
>>> +break;
>>> +case AV_CODEC_ID_H263:
>>> +qmin = MPEG_CID(H263_MIN_QP);
>>> +qmax = MPEG_CID(H263_MAX_QP);
>>> +break;
>>> +case AV_CODEC_ID_VP8:
>>> +case AV_CODEC_ID_VP9:
>>> +qmin = MPEG_CID(VPX_MIN_QP);
>>> +qmax = MPEG_CID(VPX_MAX_QP);
>>> +break;
>>> +default:
>>> +return 0;
>>> +}
>>> +
>>> +SET_V4L_EXT_CTRL(value, qmin, avctx->qmin, "minimum video quantizer 
>>> scale");
>>> +SET_V4L_EXT_CTRL(value, qmax, avctx->qmax, "maximum video quantizer 
>>> scale");
>> The default values of qmin/qmax in options_table.h are set for the MPEG-4 
>> part 2 encoder, and will do nasty things to others.  (Certainly for VP9 
>> (assuming it maps to qindex) it's a very low value, so it probably won't 
>> ever get anywhere near the bitrate target for normal video.  H.26[45] and 
>> VP8 also, but to a lesser degree.)
> 
> would you know where can I get this information from (qmin/qmx for the other 
> encoders)?

From the relevant language standards.

H.263 / MPEG-4 part 2:  [1, 31]  (: 
PQUANT, GQUANT.)
H.264 / MPEG-4 part 10 / AVC:  [48 - 6 * bitdepth, 51]  
(: QpBdOffsety, pic_init_qp_minus26, 
slice_qp_delta.)
H.265 / MPEG-H part 2 / HEVC:  [48 - 6 * bitdepth, 51]  
(: QpBdOffsety, init_qp_minus26, 
slice_qp_delta.)
VP8:  [0, 127]  (: q_index.)
VP9:  [0, 255]  
(:
 base_q_idx.)


- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+AV_OPT_TYPE_INT,\
+{.i64 = V4L2_MEMORY_MMAP},\
+0, INT_MAX, FLAGS },\
+{ "output_memory",\
+"Output memory model: See V4L2_MEMORY_* in videodev2.h. This depends on the 
HW but default should work with most.",\
+OFFSET(capture_pool.memory),\
+AV_OPT_TYPE_INT,\
+{.i64 = V4L2_MEMORY_MMAP},\
+0, INT_MAX, FLAGS },\
+{ "num_output_pool_buffers",\
+"Number of buffers in the output pool",\
+OFFSET(output_pool.num_buffers),\
+AV_OPT_TYPE_INT,\
+{ .i64 = 16 },\
+4, INT_MAX, FLAGS }

Can we attempt to set this automatically based on the codec?  16 won't be 
enough for worst-case H.264/H.265.



please could you suggest some defaults so I can prepare the table?
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+}
+
+dequeue:
+
+ret = avpriv_v4l_dequeue(cap_pool, NULL, pkt, timeout);
+if (ret == AVERROR(EAGAIN))
+return 0;
+if (!ret)
+*got_packet = 1;
+
+return ret;
+}

Does this actually work for a long stream?  You won't necessarily return a 
packet for every frame, but always need to enqueue a frame to be able to 
attempt to dequeue a packet.

It might be better to use the send/receive API here as well.


how long does it have to be? I have only tried a couple of minutes

ok will check the API
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+default:
+return 0;
+}
+
+SET_V4L_EXT_CTRL(value, qmin, avctx->qmin, "minimum video quantizer 
scale");
+SET_V4L_EXT_CTRL(value, qmax, avctx->qmax, "maximum video quantizer 
scale");
+
+return 0;
+}

This doesn't set extradata - you need to extract the codec global headers (such 
as H.264 SPS and PPS) at init time to be able to write correct files for some 
codecs (such as H.264) with muxers requiring global headers (such as MP4).  It 
kindof works without it, but the files created will not conform and will not be 
usable on some players.

ah that might explain some things (when I play back the encoded video 
the quality is pretty lousy)
is there already some code I can use as a reference? I might be out of 
my depth here so any help will be more than welcome

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+if (ret)
+return 0;
+
+return avpriv_v4l_dequeue(cap_pool, frame, NULL, timeout);

I feel like if the stream has a lot of delay then you will end up needing to 
wait for the timeout to get the required EAGAIN, which at 1s is very large.  
(Though I think you won't in common cases because of the min_queued_buffers 
constraint in dequeue.)


+}

yes, you are correct. fixed on v4
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+
+/* this will report the size of the frame back (see a4lbuf_to_avframe) */
+cap_pool->height = avctx->coded_height;
+cap_pool->width = avctx->coded_width;

Does coded_height/width need to be set to anything sensible here?  That won't 
be true in general.


not at this point. this code has been removed.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+return ret;
+}
+
+av_log(log_ctx, AV_LOG_INFO, "Using device %s\n", tmpbuf);
+ret = probe_and_set(s, log_ctx, 1);
+s->devname = devname_save;

This probing structure feels very fragile and possibly harmful (opening random 
devices and poking them).  It might be better to just force the user to supply 
the right device path?


+
I cleaned it up on v4 but I dont think it can be done much differently: 
the user needs to query the capabilities of every /dev/videoX node..


btw the same probing is already done in  libavdevice/v4l2.c
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+static int probe_and_set(V4Lm2mContext* s, void *log_ctx, int set)
+{
+int fail_log_level = ( set ? AV_LOG_ERROR : AV_LOG_DEBUG);
+int ret;
+
+s->fd = open(s->devname, O_RDWR, 0);

O_NONBLOCK, given DQBUF is used as if it is?


ack
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+return AVERROR(errno);
+
+bp->streamon = (cmd == VIDIOC_STREAMON);
+
+return 0;
+}
+
+int avpriv_v4l_enqueue(V4LBufferPool* bp, const AVFrame* f, const AVPacket* 
pkt)

Just make separate functions.


done in v4
thanks
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

+
+static int v4lbuf_to_avframe(AVFrame *frame, V4LBuffer *avbuf)
+{
+int i, ret;
+
+av_frame_unref(frame);
+
+for (i = 0; i < avbuf->num_planes; i++) {
+ret = v4l2bufref(avbuf, i, &frame->buf[i]);> +if (ret)
+return ret;
+
+frame->linesize[i] = avbuf->bytesperline[i];
+frame->data[i] = frame->buf[i]->data;
+
+if (avbuf->num_planes == 1) {
+if (avbuf->pool->av_pix_fmt == AV_PIX_FMT_NV12) {

Special-casing like this rather inspires doubt that other formats are handled 
correctly.

I dont disagree with you but this was the only way I could play the 
decoded videos with ffplay. do you think this could be an ffplay issue?


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/04/2017 01:51 PM, Mark Thompson wrote:

+break;
+case AV_CODEC_ID_H263:
+qmin = MPEG_CID(H263_MIN_QP);
+qmax = MPEG_CID(H263_MAX_QP);
+break;
+case AV_CODEC_ID_VP8:
+case AV_CODEC_ID_VP9:
+qmin = MPEG_CID(VPX_MIN_QP);
+qmax = MPEG_CID(VPX_MAX_QP);
+break;
+default:
+return 0;
+}
+
+SET_V4L_EXT_CTRL(value, qmin, avctx->qmin, "minimum video quantizer 
scale");
+SET_V4L_EXT_CTRL(value, qmax, avctx->qmax, "maximum video quantizer 
scale");

The default values of qmin/qmax in options_table.h are set for the MPEG-4 part 
2 encoder, and will do nasty things to others.  (Certainly for VP9 (assuming it 
maps to qindex) it's a very low value, so it probably won't ever get anywhere 
near the bitrate target for normal video.  H.26[45] and VP8 also, but to a 
lesser degree.)


would you know where can I get this information from (qmin/qmx for the 
other encoders)?



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-07 Thread Jorge Ramirez 

On 08/04/2017 03:32 PM, Mark Thompson wrote:

On 04/08/17 14:06, Jorge Ramirez wrote:

On 08/04/2017 01:51 PM, Mark Thompson wrote:

+if (ret)
+return 0;
+
+return avpriv_v4l_dequeue(cap_pool, frame, NULL, timeout);
+}

What happens to the decoder if some parameters like resolution change 
mid-stream?  (Try the stream h264/reinit-large_420_8-to-small_420_8.h264 in the 
fate samples.)

yes, it breaks - the player still shows and image but it is corrupted.
I am not sure how to handle this situation to be honest...so need to look into 
this.

There should be some way to detect it from the decoder, I think?  (I don't 
actually know how this works in V4L2.)

To compare, if you look at MMAL (perhaps the most similar API in current lavc) there 
is a special FORMAT_CHANGED return when dequeuing a frame, and the decoder is 
reconfigured with new buffers when that 
happens:.



great, thanks for that.
The issue I have is that the V4L2 driver I am using to test - yes this 
needs to be detected by the decoder- has not implemented the feature 
yet. Will keep it in mind before sending the last series of the patchset.


also, I have rewritten the patchset to add clarity and clean some of the 
sedimentation from testing/hacking. is it ok with you if I post v4 now? 
it will simplify the discussion - the driver now much easier to follow.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Mark Thompson 
On 04/08/17 14:06, Jorge Ramirez wrote:
> On 08/04/2017 01:51 PM, Mark Thompson wrote:
>>> +if (ret)
>>> +return 0;
>>> +
>>> +return avpriv_v4l_dequeue(cap_pool, frame, NULL, timeout);
>>> +}
>> What happens to the decoder if some parameters like resolution change 
>> mid-stream?  (Try the stream h264/reinit-large_420_8-to-small_420_8.h264 in 
>> the fate samples.)
> 
> yes, it breaks - the player still shows and image but it is corrupted.
> I am not sure how to handle this situation to be honest...so need to look 
> into this.

There should be some way to detect it from the decoder, I think?  (I don't 
actually know how this works in V4L2.)

To compare, if you look at MMAL (perhaps the most similar API in current lavc) 
there is a special FORMAT_CHANGED return when dequeuing a frame, and the 
decoder is reconfigured with new buffers when that happens: 
.

>>
>> Also, are the frames returned here actually still used by the decoder 
>> internally as reference frames, or is the reference frame copied somewhere 
>> else? 
> 
> I think once the buffers are dequeued, the v4l2 decoder doesnt keep any 
> references to it.
>>  (Seems slightly unlikely that they would be copied given the additional 
>> memory requirements, but I know discrete desktop GPUs (which admittedly have 
>> 9001x the memory bandwidth and power budget) do do this to hide the possible 
>> issues.)  I think, though I'm not entirely sure because I haven't run it, 
>> that the returned buffers will only have the user reference, so will appear 
>> to be writable to following filters - if that ends up writing on reference 
>> frames then the decoder will break.
> 
> ack

Ok.  I'm a bit surprised because of the performance implications of writing 
everything twice, but if we don't have to think about it at all then whatever.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Jorge Ramirez 

On 08/04/2017 01:51 PM, Mark Thompson wrote:

+if (ret)
+return 0;
+
+return avpriv_v4l_dequeue(cap_pool, frame, NULL, timeout);
+}

What happens to the decoder if some parameters like resolution change 
mid-stream?  (Try the stream h264/reinit-large_420_8-to-small_420_8.h264 in the 
fate samples.)


yes, it breaks - the player still shows and image but it is corrupted.
I am not sure how to handle this situation to be honest...so need to 
look into this.




Also, are the frames returned here actually still used by the decoder 
internally as reference frames, or is the reference frame copied somewhere else?


I think once the buffers are dequeued, the v4l2 decoder doesnt keep any 
references to it.

  (Seems slightly unlikely that they would be copied given the additional 
memory requirements, but I know discrete desktop GPUs (which admittedly have 
9001x the memory bandwidth and power budget) do do this to hide the possible 
issues.)  I think, though I'm not entirely sure because I haven't run it, that 
the returned buffers will only have the user reference, so will appear to be 
writable to following filters - if that ends up writing on reference frames 
then the decoder will break.


ack

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Jorge Ramirez 

On 08/04/2017 01:51 PM, Mark Thompson wrote:

+REGISTER_ENCDEC (VP8_V4L2M2M,   vp8_v4l2m2m);
  REGISTER_DECODER(VP9,   vp9);
+REGISTER_DECODER(VP9_V4L2M2M,   vp9_v4l2m2m);
  REGISTER_DECODER(VQA,   vqa);
  REGISTER_DECODER(BITPACKED, bitpacked);
  REGISTER_DECODER(WEBP,  webp);

MPEG-2 is never mentioned at all in this configure change, so it will never 
even be compiled.  It might be best if you only add the decoders you can 
actually verify.




thanks. yes fixed in v4
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Mark Thompson 
On 02/08/17 08:32, Jorge Ramirez-Ortiz wrote:
> diff --git a/configure b/configure
> index ed94de0..650c8fb 100755
> --- a/configure
> +++ b/configure
> @@ -10,7 +10,6 @@
>  # Prevent locale nonsense from breaking basic text processing.
>  LC_ALL=C
>  export LC_ALL
> -
>  # make sure we are running under a compatible shell
>  # try to make this part work with most shells
>  
> @@ -149,6 +148,7 @@ Component options:
>--disable-pixelutils disable pixel utils in libavutil
>  
>  Individual component options:
> +  --disable-v4l2_m2m   disable V4L2 mem2mem code [autodetect]
>--disable-everything disable all components listed below
>--disable-encoder=NAME   disable encoder NAME
>--enable-encoder=NAMEenable encoder NAME
> @@ -1432,6 +1432,7 @@ AVCODEC_COMPONENTS="
>  
>  AVDEVICE_COMPONENTS="
>  indevs
> +v4l2_m2m
>  outdevs
>  "
>  AVFILTER_COMPONENTS="
> @@ -2269,11 +2270,12 @@ map 'eval ${v}_inline_deps=inline_asm' 
> $ARCH_EXT_LIST_ARM
>  
>  loongson2_deps="mips"
>  loongson3_deps="mips"
> -v4l2_deps_any="linux_videodev2_h sys_videoio_h"
> +v4l2_m2m_select="v4l2"
>  mipsfpu_deps="mips"
>  mipsdsp_deps="mips"
>  mipsdspr2_deps="mips"
>  mips32r2_deps="mips"
> +vc1_v4l2m2m_decoder_deps="v4l2_m2m"
>  mips32r5_deps="mips"
>  mips32r6_deps="mips"
>  mips64r2_deps="mips"
> @@ -2284,6 +2286,9 @@ mmi_deps="mips"
>  altivec_deps="ppc"
>  dcbzl_deps="ppc"
>  ldbrx_deps="ppc"
> +vp8_v4l2m2m_decoder_deps="v4l2_m2m"
> +vp8_v4l2m2m_encoder_deps="v4l2_m2m"
> +vp9_v4l2m2m_decoder_deps="v4l2_m2m"
>  ppc4xx_deps="ppc"
>  vsx_deps="altivec"
>  power8_deps="vsx"
> @@ -2437,15 +2442,22 @@ h261_decoder_select="mpegvideo"
>  h261_encoder_select="aandcttables mpegvideoenc"
>  h263_decoder_select="h263_parser h263dsp mpegvideo qpeldsp"
>  h263_encoder_select="aandcttables h263dsp mpegvideoenc"
> +h263_v4l2m2m_decoder_deps="v4l2_m2m"
> +h263_v4l2m2m_encoder_deps="v4l2_m2m"
>  h263i_decoder_select="h263_decoder"
>  h263p_decoder_select="h263_decoder"
>  h263p_encoder_select="h263_encoder"
>  h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred 
> h264qpel videodsp"
>  h264_decoder_suggest="error_resilience"
> +h264_v4l2m2m_decoder_deps="v4l2_m2m"
> +h264_v4l2m2m_encoder_deps="v4l2_m2m"
>  hap_decoder_select="snappy texturedsp"
>  hap_encoder_deps="libsnappy"
>  hap_encoder_select="texturedspenc"
>  hevc_decoder_select="bswapdsp cabac golomb hevcparse videodsp"
> +hevc_encoder_select="hevc_v4l2m2m"
> +hevc_v4l2m2m_decoder_deps="v4l2_m2m"
> +hevc_v4l2m2m_encoder_deps="v4l2_m2m"
>  huffyuv_decoder_select="bswapdsp huffyuvdsp llviddsp"
>  huffyuv_encoder_select="bswapdsp huffman huffyuvencdsp llvidencdsp"
>  iac_decoder_select="imc_decoder"
> @@ -2482,6 +2494,7 @@ mpc7_decoder_select="bswapdsp mpegaudiodsp"
>  mpc8_decoder_select="mpegaudiodsp"
>  mpeg_xvmc_decoder_deps="X11_extensions_XvMClib_h"
>  mpeg_xvmc_decoder_select="mpeg2video_decoder"
> +mpeg1_v4l2m2m_decoder_deps="v4l2_m2m"
>  mpegvideo_decoder_select="mpegvideo"
>  mpeg1video_decoder_select="mpegvideo"
>  mpeg1video_encoder_select="aandcttables mpegvideoenc h263dsp"
> @@ -2489,6 +2502,8 @@ mpeg2video_decoder_select="mpegvideo"
>  mpeg2video_encoder_select="aandcttables mpegvideoenc h263dsp"
>  mpeg4_decoder_select="h263_decoder mpeg4video_parser"
>  mpeg4_encoder_select="h263_encoder"
> +mpeg4_v4l2m2m_decoder_deps="v4l2_m2m"
> +mpeg4_v4l2m2m_encoder_deps="v4l2_m2m"
>  msa1_decoder_select="mss34dsp"
>  mscc_decoder_select="zlib"
>  msmpeg4v1_decoder_select="h263_decoder"
> @@ -3042,7 +3057,6 @@ qtkit_indev_select="qtkit"
>  sdl2_outdev_deps="sdl2"
>  sndio_indev_deps="sndio"
>  sndio_outdev_deps="sndio"
> -v4l_indev_deps="linux_videodev_h"
>  v4l2_indev_select="v4l2"
>  v4l2_outdev_select="v4l2"
>  vfwcap_indev_deps="vfw32 vfwcap_defines"
> @@ -3592,7 +3606,7 @@ done
>  enable_weak audiotoolbox
>  
>  # Enable hwaccels by default.
> -enable_weak d3d11va dxva2 vaapi vda vdpau videotoolbox_hwaccel xvmc
> +enable_weak d3d11va dxva2 vaapi v4l2_m2m vda vdpau videotoolbox_hwaccel xvmc
>  enable_weak xlib
>  
>  enable_weak cuda cuvid nvenc vda_framework videotoolbox videotoolbox_encoder
> @@ -6058,12 +6072,10 @@ pod2man --help > /dev/null 2>&1 && enable pod2man 
>   || disable pod2man
>  rsync --help 2> /dev/null | grep -q 'contimeout' && enable rsync_contimeout 
> || disable rsync_contimeout
>  
>  check_header linux/fb.h
> -check_header linux/videodev.h
> -check_header linux/videodev2.h
> -check_code cc linux/videodev2.h "struct v4l2_frmsizeenum vfse; 
> vfse.discrete.width = 0;" && enable_safe struct_v4l2_frmivalenum_discrete
>  
>  check_header sys/videoio.h
>  check_code cc sys/videoio.h "struct v4l2_frmsizeenum vfse; 
> vfse.discrete.width = 0;" && enable_safe struct_v4l2_frmivalenum_discrete
> +add_cflags -I$source_path/compat/v4l2
>  
>  check_lib user32 "windows.h winuser.h" GetShellWindow -luser32
>  check_lib vfw32 "windows.h vfw.h" capCreateCaptureWindow -lvfw32
> diff --git a/libavcodec/Makefile b/libavco

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Mark Thompson 
On 04/08/17 09:46, Jorge Ramirez wrote:
> On 08/03/2017 01:53 AM, Mark Thompson wrote:
>>> +}
>>> +
>>> +ret = poll(&pfd, 1, timeout);
>> Can this poll be interrupted?
> yes. when that happens, we indicate no buffer available.
> I think the main program should set the necessary masks.

I was thinking this might violate the send/receive semantics without EINTR 
handling - consider a situation where send returns EAGAIN (e.g. by trying to 
send 9001 packets to a decoder all at once) so the caller does a receive but 
that then races with a signal such that poll returns EINTR, then receive would 
return EAGAIN as well which isn't allowed.

If you want to impose signal handling requirements on the caller then that 
would need to be documented somewhere.

>>> +if (ret <=  0)
>>> +return NULL;
>>> +
>>> +memset(&buf, 0, sizeof(buf));
>>> +buf.memory = bp->memory;
>>> +buf.type = bp->type;
>>> +if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) {
>>> +memset(planes, 0, sizeof(planes));
>>> +buf.length = VIDEO_MAX_PLANES;
>>> +buf.m.planes = planes;
>>> +}
>>> +ret = ioctl(bp->fd, VIDIOC_DQBUF, &buf);
>>> +if (ret) {
>>> +if (errno != EAGAIN && errno != EINVAL) {
>> EINVAL is considered ok?
> 
> no any ioctl that returns non zero is a fault condition.
> EINVAL (just as EAGAIN) is just not an unrecoverable error (we will just 
> return null indicating that there is no buffer available)

EINVAL usually indicates that the arguments were invalid somehow, but I think 
with this setup it's just going to call it again with pretty much arguments.  
In what case can EINVAL be recoverable?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-04 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

agree with all your previous comments up to here (will fix).


+case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+pfd.events = POLLOUT | POLLERR | POLLWRNORM;
+break;
+default:
+pfd.events = POLLIN | POLLERR | POLLRDNORM;

What hits this default case?


nothing, it can't happen will remove.



+}
+
+ret = poll(&pfd, 1, timeout);

Can this poll be interrupted?

yes. when that happens, we indicate no buffer available.
I think the main program should set the necessary masks.


+if (ret <=  0)
+return NULL;
+
+memset(&buf, 0, sizeof(buf));
+buf.memory = bp->memory;
+buf.type = bp->type;
+if (V4L2_TYPE_IS_MULTIPLANAR(bp->type)) {
+memset(planes, 0, sizeof(planes));
+buf.length = VIDEO_MAX_PLANES;
+buf.m.planes = planes;
+}
+ret = ioctl(bp->fd, VIDIOC_DQBUF, &buf);
+if (ret) {
+if (errno != EAGAIN && errno != EINVAL) {

EINVAL is considered ok?


no any ioctl that returns non zero is a fault condition.
EINVAL (just as EAGAIN) is just not an unrecoverable error (we will just 
return null indicating that there is no buffer available)



+av_log(bp->log_ctx, AV_LOG_DEBUG, "%s: VIDIOC_DQBUF, errno (%d)\n", 
bp->name, errno);
+bp->broken = errno;

errno can be overwritten by the av_log() call.


OK.




+}
+return NULL;
+}
+


___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-03 Thread Jorge Ramirez 

On 08/03/2017 01:53 AM, Mark Thompson wrote:

On 02/08/17 08:32, Jorge Ramirez-Ortiz wrote:

...
diff --git a/compat/v4l2/v4l2-common.h b/compat/v4l2/v4l2-common.h
diff --git a/compat/v4l2/videodev2.h b/compat/v4l2/videodev2.h

These are discussed in other threads.  I don't really have any comment either 
way.


ok. will be ammended in v4.




diff --git a/configure b/configure
index ed94de0..650c8fb 100755
--- a/configure
+++ b/configure
@@ -10,7 +10,6 @@
  # Prevent locale nonsense from breaking basic text processing.
  LC_ALL=C
  export LC_ALL
-

Spurious change?

ack



  # make sure we are running under a compatible shell
  # try to make this part work with most shells
  
@@ -149,6 +148,7 @@ Component options:

--disable-pixelutils disable pixel utils in libavutil
  
  Individual component options:

+  --disable-v4l2_m2m   disable V4L2 mem2mem code [autodetect]

s/_/-/ here.

ack




--disable-everything disable all components listed below
--disable-encoder=NAME   disable encoder NAME
--enable-encoder=NAMEenable encoder NAME
@@ -1432,6 +1432,7 @@ AVCODEC_COMPONENTS="
  
  AVDEVICE_COMPONENTS="

  indevs
+v4l2_m2m
  outdevs
  "
  AVFILTER_COMPONENTS="
@@ -2269,11 +2270,12 @@ map 'eval ${v}_inline_deps=inline_asm' 
$ARCH_EXT_LIST_ARM
  
  loongson2_deps="mips"

  loongson3_deps="mips"
-v4l2_deps_any="linux_videodev2_h sys_videoio_h"
+v4l2_m2m_select="v4l2"
  mipsfpu_deps="mips"
  mipsdsp_deps="mips"
  mipsdspr2_deps="mips"
  mips32r2_deps="mips"
+vc1_v4l2m2m_decoder_deps="v4l2_m2m"
  mips32r5_deps="mips"
  mips32r6_deps="mips"
  mips64r2_deps="mips"
@@ -2284,6 +2286,9 @@ mmi_deps="mips"
  altivec_deps="ppc"
  dcbzl_deps="ppc"
  ldbrx_deps="ppc"
+vp8_v4l2m2m_decoder_deps="v4l2_m2m"
+vp8_v4l2m2m_encoder_deps="v4l2_m2m"
+vp9_v4l2m2m_decoder_deps="v4l2_m2m"

These seem to be placed randomly in the file.  There is a hardware codec 
section at around line 2800, put all of these declarations there.


thanks for the guideline. will do.



  ppc4xx_deps="ppc"
  vsx_deps="altivec"
  power8_deps="vsx"
@@ -2437,15 +2442,22 @@ h261_decoder_select="mpegvideo"
  h261_encoder_select="aandcttables mpegvideoenc"
  h263_decoder_select="h263_parser h263dsp mpegvideo qpeldsp"
  h263_encoder_select="aandcttables h263dsp mpegvideoenc"
+h263_v4l2m2m_decoder_deps="v4l2_m2m"
+h263_v4l2m2m_encoder_deps="v4l2_m2m"
  h263i_decoder_select="h263_decoder"
  h263p_decoder_select="h263_decoder"
  h263p_encoder_select="h263_encoder"
  h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel 
videodsp"
  h264_decoder_suggest="error_resilience"
+h264_v4l2m2m_decoder_deps="v4l2_m2m"
+h264_v4l2m2m_encoder_deps="v4l2_m2m"
  hap_decoder_select="snappy texturedsp"
  hap_encoder_deps="libsnappy"
  hap_encoder_select="texturedspenc"
  hevc_decoder_select="bswapdsp cabac golomb hevcparse videodsp"
+hevc_encoder_select="hevc_v4l2m2m"
+hevc_v4l2m2m_decoder_deps="v4l2_m2m"
+hevc_v4l2m2m_encoder_deps="v4l2_m2m"
  huffyuv_decoder_select="bswapdsp huffyuvdsp llviddsp"
  huffyuv_encoder_select="bswapdsp huffman huffyuvencdsp llvidencdsp"
  iac_decoder_select="imc_decoder"
@@ -2482,6 +2494,7 @@ mpc7_decoder_select="bswapdsp mpegaudiodsp"
  mpc8_decoder_select="mpegaudiodsp"
  mpeg_xvmc_decoder_deps="X11_extensions_XvMClib_h"
  mpeg_xvmc_decoder_select="mpeg2video_decoder"
+mpeg1_v4l2m2m_decoder_deps="v4l2_m2m"
  mpegvideo_decoder_select="mpegvideo"
  mpeg1video_decoder_select="mpegvideo"
  mpeg1video_encoder_select="aandcttables mpegvideoenc h263dsp"
@@ -2489,6 +2502,8 @@ mpeg2video_decoder_select="mpegvideo"
  mpeg2video_encoder_select="aandcttables mpegvideoenc h263dsp"
  mpeg4_decoder_select="h263_decoder mpeg4video_parser"
  mpeg4_encoder_select="h263_encoder"
+mpeg4_v4l2m2m_decoder_deps="v4l2_m2m"
+mpeg4_v4l2m2m_encoder_deps="v4l2_m2m"
  msa1_decoder_select="mss34dsp"
  mscc_decoder_select="zlib"
  msmpeg4v1_decoder_select="h263_decoder"
@@ -3042,7 +3057,6 @@ qtkit_indev_select="qtkit"
  sdl2_outdev_deps="sdl2"
  sndio_indev_deps="sndio"
  sndio_outdev_deps="sndio"
-v4l_indev_deps="linux_videodev_h"
  v4l2_indev_select="v4l2"
  v4l2_outdev_select="v4l2"
  vfwcap_indev_deps="vfw32 vfwcap_defines"
@@ -3592,7 +3606,7 @@ done
  enable_weak audiotoolbox
  
  # Enable hwaccels by default.

-enable_weak d3d11va dxva2 vaapi vda vdpau videotoolbox_hwaccel xvmc
+enable_weak d3d11va dxva2 vaapi v4l2_m2m vda vdpau videotoolbox_hwaccel xvmc
  enable_weak xlib
  
  enable_weak cuda cuvid nvenc vda_framework videotoolbox videotoolbox_encoder

@@ -6058,12 +6072,10 @@ pod2man --help > /dev/null 2>&1 && enable pod2man   
|| disable pod2man
  rsync --help 2> /dev/null | grep -q 'contimeout' && enable rsync_contimeout 
|| disable rsync_contimeout
  
  check_header linux/fb.h

-check_header linux/videodev.h
-check_header linux/videodev2.h
-check_code cc linux/videodev2.h "struct v4l2_frmsizeenum vfse; vfse.discrete.width = 
0;" && enable_safe struct_v4l2_frmivalenum_discrete
  
  check_header sys/vi

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-02 Thread Mark Thompson 
On 02/08/17 08:32, Jorge Ramirez-Ortiz wrote:
> ...
> diff --git a/compat/v4l2/v4l2-common.h b/compat/v4l2/v4l2-common.h
> diff --git a/compat/v4l2/videodev2.h b/compat/v4l2/videodev2.h

These are discussed in other threads.  I don't really have any comment either 
way.

> diff --git a/configure b/configure
> index ed94de0..650c8fb 100755
> --- a/configure
> +++ b/configure
> @@ -10,7 +10,6 @@
>  # Prevent locale nonsense from breaking basic text processing.
>  LC_ALL=C
>  export LC_ALL
> -

Spurious change?

>  # make sure we are running under a compatible shell
>  # try to make this part work with most shells
>  
> @@ -149,6 +148,7 @@ Component options:
>--disable-pixelutils disable pixel utils in libavutil
>  
>  Individual component options:
> +  --disable-v4l2_m2m   disable V4L2 mem2mem code [autodetect]

s/_/-/ here.

>--disable-everything disable all components listed below
>--disable-encoder=NAME   disable encoder NAME
>--enable-encoder=NAMEenable encoder NAME
> @@ -1432,6 +1432,7 @@ AVCODEC_COMPONENTS="
>  
>  AVDEVICE_COMPONENTS="
>  indevs
> +v4l2_m2m
>  outdevs
>  "
>  AVFILTER_COMPONENTS="
> @@ -2269,11 +2270,12 @@ map 'eval ${v}_inline_deps=inline_asm' 
> $ARCH_EXT_LIST_ARM
>  
>  loongson2_deps="mips"
>  loongson3_deps="mips"
> -v4l2_deps_any="linux_videodev2_h sys_videoio_h"
> +v4l2_m2m_select="v4l2"
>  mipsfpu_deps="mips"
>  mipsdsp_deps="mips"
>  mipsdspr2_deps="mips"
>  mips32r2_deps="mips"
> +vc1_v4l2m2m_decoder_deps="v4l2_m2m"
>  mips32r5_deps="mips"
>  mips32r6_deps="mips"
>  mips64r2_deps="mips"
> @@ -2284,6 +2286,9 @@ mmi_deps="mips"
>  altivec_deps="ppc"
>  dcbzl_deps="ppc"
>  ldbrx_deps="ppc"
> +vp8_v4l2m2m_decoder_deps="v4l2_m2m"
> +vp8_v4l2m2m_encoder_deps="v4l2_m2m"
> +vp9_v4l2m2m_decoder_deps="v4l2_m2m"

These seem to be placed randomly in the file.  There is a hardware codec 
section at around line 2800, put all of these declarations there.

>  ppc4xx_deps="ppc"
>  vsx_deps="altivec"
>  power8_deps="vsx"
> @@ -2437,15 +2442,22 @@ h261_decoder_select="mpegvideo"
>  h261_encoder_select="aandcttables mpegvideoenc"
>  h263_decoder_select="h263_parser h263dsp mpegvideo qpeldsp"
>  h263_encoder_select="aandcttables h263dsp mpegvideoenc"
> +h263_v4l2m2m_decoder_deps="v4l2_m2m"
> +h263_v4l2m2m_encoder_deps="v4l2_m2m"
>  h263i_decoder_select="h263_decoder"
>  h263p_decoder_select="h263_decoder"
>  h263p_encoder_select="h263_encoder"
>  h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred 
> h264qpel videodsp"
>  h264_decoder_suggest="error_resilience"
> +h264_v4l2m2m_decoder_deps="v4l2_m2m"
> +h264_v4l2m2m_encoder_deps="v4l2_m2m"
>  hap_decoder_select="snappy texturedsp"
>  hap_encoder_deps="libsnappy"
>  hap_encoder_select="texturedspenc"
>  hevc_decoder_select="bswapdsp cabac golomb hevcparse videodsp"
> +hevc_encoder_select="hevc_v4l2m2m"
> +hevc_v4l2m2m_decoder_deps="v4l2_m2m"
> +hevc_v4l2m2m_encoder_deps="v4l2_m2m"
>  huffyuv_decoder_select="bswapdsp huffyuvdsp llviddsp"
>  huffyuv_encoder_select="bswapdsp huffman huffyuvencdsp llvidencdsp"
>  iac_decoder_select="imc_decoder"
> @@ -2482,6 +2494,7 @@ mpc7_decoder_select="bswapdsp mpegaudiodsp"
>  mpc8_decoder_select="mpegaudiodsp"
>  mpeg_xvmc_decoder_deps="X11_extensions_XvMClib_h"
>  mpeg_xvmc_decoder_select="mpeg2video_decoder"
> +mpeg1_v4l2m2m_decoder_deps="v4l2_m2m"
>  mpegvideo_decoder_select="mpegvideo"
>  mpeg1video_decoder_select="mpegvideo"
>  mpeg1video_encoder_select="aandcttables mpegvideoenc h263dsp"
> @@ -2489,6 +2502,8 @@ mpeg2video_decoder_select="mpegvideo"
>  mpeg2video_encoder_select="aandcttables mpegvideoenc h263dsp"
>  mpeg4_decoder_select="h263_decoder mpeg4video_parser"
>  mpeg4_encoder_select="h263_encoder"
> +mpeg4_v4l2m2m_decoder_deps="v4l2_m2m"
> +mpeg4_v4l2m2m_encoder_deps="v4l2_m2m"
>  msa1_decoder_select="mss34dsp"
>  mscc_decoder_select="zlib"
>  msmpeg4v1_decoder_select="h263_decoder"
> @@ -3042,7 +3057,6 @@ qtkit_indev_select="qtkit"
>  sdl2_outdev_deps="sdl2"
>  sndio_indev_deps="sndio"
>  sndio_outdev_deps="sndio"
> -v4l_indev_deps="linux_videodev_h"
>  v4l2_indev_select="v4l2"
>  v4l2_outdev_select="v4l2"
>  vfwcap_indev_deps="vfw32 vfwcap_defines"
> @@ -3592,7 +3606,7 @@ done
>  enable_weak audiotoolbox
>  
>  # Enable hwaccels by default.
> -enable_weak d3d11va dxva2 vaapi vda vdpau videotoolbox_hwaccel xvmc
> +enable_weak d3d11va dxva2 vaapi v4l2_m2m vda vdpau videotoolbox_hwaccel xvmc
>  enable_weak xlib
>  
>  enable_weak cuda cuvid nvenc vda_framework videotoolbox videotoolbox_encoder
> @@ -6058,12 +6072,10 @@ pod2man --help > /dev/null 2>&1 && enable pod2man 
>   || disable pod2man
>  rsync --help 2> /dev/null | grep -q 'contimeout' && enable rsync_contimeout 
> || disable rsync_contimeout
>  
>  check_header linux/fb.h
> -check_header linux/videodev.h
> -check_header linux/videodev2.h
> -check_code cc linux/videodev2.h "struct v4l2_frmsizeenum vfse; 
> vfse.discrete.width = 0;" && enable_safe struct_v4l2_frmivalenum_dis

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-02 Thread Jorge Ramirez 

On 08/02/2017 12:16 PM, Mark Thompson wrote:

On 02/08/17 08:32, Jorge Ramirez-Ortiz wrote:

From: Alexis Ballier 

This patchset enhances Alexis Ballier's original patch and validates
it using Qualcomm's Venus hardware (driver recently landed upstream
[1]).

This has been tested on Qualcomm's DragonBoard 410c and 820c

Tested decoders:
- h264
- mpeg4
- vp8
- vp9
- hevc

Tested encoders:
-h264
-h263
-mpeg4

Tested transcoding (concurrent encoding/decoding)

Some of the changes introduced:
- v4l2: code cleanup.
- v4l2: follow the decode api.
- v4l2: fix display size for NV12 output pool.
- v4l2: handle EOS.
- v4l2: vp8 and mpeg4 decoding.
- v4l2: hevc and vp9 support.
- v4l2: generate EOF on dequeue errors.
- v4l2: h264_mp4toannexb filtering.
- v4l2: import compat/v4l2 header files.
- v4l2: fixed make install and fate issues.

[1] https://lwn.net/Articles/697956/

Reviewed-by: Jorge Ramirez 
Reviewed-by: Alexis Ballier 
Tested-by: Jorge Ramirez 
---
  Changelog |3 +-
  compat/v4l2/v4l2-common.h |  107 ++
  compat/v4l2/v4l2-controls.h   |  987 +
  compat/v4l2/videodev2.h   | 2402 +
  configure |   26 +-
  libavcodec/Makefile   |   18 +-
  libavcodec/allcodecs.c|9 +
  libavcodec/v4l2-common.c  |  136 ---
  libavcodec/v4l2-common.h  |   62 --
  libavcodec/v4l2_buffers.c |  614 +++
  libavcodec/v4l2_buffers.h |  226 
  libavcodec/v4l2_fmt.c |  142 +++
  libavcodec/v4l2_fmt.h |   62 ++
  libavcodec/v4l2_m2m.c |  356 ++
  libavcodec/v4l2_m2m.h |   69 ++
  libavcodec/v4l2_m2m_avcodec.h |   32 +
  libavcodec/v4l2_m2m_dec.c |  229 
  libavcodec/v4l2_m2m_enc.c |  270 +
  libavdevice/v4l2.c|2 +-
  libavdevice/v4l2enc.c |2 +-
  tests/ref/fate/source |3 +
  21 files changed, 5547 insertions(+), 210 deletions(-)
  create mode 100644 compat/v4l2/v4l2-common.h
  create mode 100644 compat/v4l2/v4l2-controls.h
  create mode 100644 compat/v4l2/videodev2.h
  delete mode 100644 libavcodec/v4l2-common.c
  delete mode 100644 libavcodec/v4l2-common.h
  create mode 100644 libavcodec/v4l2_buffers.c
  create mode 100644 libavcodec/v4l2_buffers.h
  create mode 100644 libavcodec/v4l2_fmt.c
  create mode 100644 libavcodec/v4l2_fmt.h
  create mode 100644 libavcodec/v4l2_m2m.c
  create mode 100644 libavcodec/v4l2_m2m.h
  create mode 100644 libavcodec/v4l2_m2m_avcodec.h
  create mode 100644 libavcodec/v4l2_m2m_dec.c
  create mode 100644 libavcodec/v4l2_m2m_enc.c

Hi,

Some general questions first; I'll try to do some detailed review later.


thanks! very much appreciated.

Alexis feel free to add your comments as well.



Is the intent here only to really support software memory cases?  That is, to 
only consider things on the user side and to copy with the CPU between 
different components.


This commit adds a generic way of supporting v4l2 in ffmpeg 
(V4L2_MEMORY_MMAP)


enum v4l2_memory {
V4L2_MEMORY_MMAP = 1,
V4L2_MEMORY_USERPTR  = 2,
V4L2_MEMORY_OVERLAY  = 3,
V4L2_MEMORY_DMABUF   = 4,
};

You are right, this patch uses the CPU to copy the preprocessed/demuxed 
bitstream to the v4l2 buffers and then as you said uses the pointers.


Extending it to support V4L2_MEMORY_DMABUF for performance is in the 
todo list.
however I think (this is just my opinion) there is value today in 
enabling the available v4l2 codecs present in many SoCs and bringing 
them under ffmpeg.

And right after work on performance and incremental optimizations.



If not, I don't think the method with only user virtual pointers being visible 
externally is really the right approach.


if by "right" you mean with the best performance possible yes I don't 
disagree with you.



For example, mapping to DRM PRIME fds is going to be wanted by a number of 
use-cases (for import with EGL or other APIs, or display directly via 
modesetting), and needs the V4L2 buffer metadata for VIDIOC_EXPBUF to work.  To 
that end, I think it might be better to have an opaque AV_PIX_FMT_V4L2 type 
carrying the buffer information and managed by the hwcontext API, which can 
then be mapped to user virtual memory as needed.


yes I am looking at this WIP 
https://github.com/LongChair/FFmpeg/commit/f852e3d6d89be3fcee90482971c3f769bdaa8e9c. 

I think it is work like this that can be used later on with v4l2 for 
optimal performance...thought I have to admit I still need to understand 
how I would use it.

but I don't see why the v4l2 patchset can not be extended afterwards?

Yes we need to add VIDIOC_EXPBUF and then on dequeing cacpture we coud 
either set the DRM plane or just pass the pointer.




Such a setup would support sharing between components inside lav* (e.g. 
transcode with copy) as well.  Alternatively, it might be possibl

Re: [FFmpeg-devel] [PATCHv3 4/4] libavcodec: v4l2: add support for v4l2 mem2mem codecs

2017-08-02 Thread Mark Thompson 
On 02/08/17 08:32, Jorge Ramirez-Ortiz wrote:
> From: Alexis Ballier 
> 
> This patchset enhances Alexis Ballier's original patch and validates
> it using Qualcomm's Venus hardware (driver recently landed upstream
> [1]).
> 
> This has been tested on Qualcomm's DragonBoard 410c and 820c
> 
> Tested decoders:
>- h264
>- mpeg4
>- vp8
>- vp9
>- hevc
> 
> Tested encoders:
>-h264
>-h263
>-mpeg4
> 
> Tested transcoding (concurrent encoding/decoding)
> 
> Some of the changes introduced:
> - v4l2: code cleanup.
> - v4l2: follow the decode api.
> - v4l2: fix display size for NV12 output pool.
> - v4l2: handle EOS.
> - v4l2: vp8 and mpeg4 decoding.
> - v4l2: hevc and vp9 support.
> - v4l2: generate EOF on dequeue errors.
> - v4l2: h264_mp4toannexb filtering.
> - v4l2: import compat/v4l2 header files.
> - v4l2: fixed make install and fate issues.
> 
> [1] https://lwn.net/Articles/697956/
> 
> Reviewed-by: Jorge Ramirez 
> Reviewed-by: Alexis Ballier 
> Tested-by: Jorge Ramirez 
> ---
>  Changelog |3 +-
>  compat/v4l2/v4l2-common.h |  107 ++
>  compat/v4l2/v4l2-controls.h   |  987 +
>  compat/v4l2/videodev2.h   | 2402 
> +
>  configure |   26 +-
>  libavcodec/Makefile   |   18 +-
>  libavcodec/allcodecs.c|9 +
>  libavcodec/v4l2-common.c  |  136 ---
>  libavcodec/v4l2-common.h  |   62 --
>  libavcodec/v4l2_buffers.c |  614 +++
>  libavcodec/v4l2_buffers.h |  226 
>  libavcodec/v4l2_fmt.c |  142 +++
>  libavcodec/v4l2_fmt.h |   62 ++
>  libavcodec/v4l2_m2m.c |  356 ++
>  libavcodec/v4l2_m2m.h |   69 ++
>  libavcodec/v4l2_m2m_avcodec.h |   32 +
>  libavcodec/v4l2_m2m_dec.c |  229 
>  libavcodec/v4l2_m2m_enc.c |  270 +
>  libavdevice/v4l2.c|2 +-
>  libavdevice/v4l2enc.c |2 +-
>  tests/ref/fate/source |3 +
>  21 files changed, 5547 insertions(+), 210 deletions(-)
>  create mode 100644 compat/v4l2/v4l2-common.h
>  create mode 100644 compat/v4l2/v4l2-controls.h
>  create mode 100644 compat/v4l2/videodev2.h
>  delete mode 100644 libavcodec/v4l2-common.c
>  delete mode 100644 libavcodec/v4l2-common.h
>  create mode 100644 libavcodec/v4l2_buffers.c
>  create mode 100644 libavcodec/v4l2_buffers.h
>  create mode 100644 libavcodec/v4l2_fmt.c
>  create mode 100644 libavcodec/v4l2_fmt.h
>  create mode 100644 libavcodec/v4l2_m2m.c
>  create mode 100644 libavcodec/v4l2_m2m.h
>  create mode 100644 libavcodec/v4l2_m2m_avcodec.h
>  create mode 100644 libavcodec/v4l2_m2m_dec.c
>  create mode 100644 libavcodec/v4l2_m2m_enc.c

Hi,

Some general questions first; I'll try to do some detailed review later.

Is the intent here only to really support software memory cases?  That is, to 
only consider things on the user side and to copy with the CPU between 
different components.

If not, I don't think the method with only user virtual pointers being visible 
externally is really the right approach.  For example, mapping to DRM PRIME fds 
is going to be wanted by a number of use-cases (for import with EGL or other 
APIs, or display directly via modesetting), and needs the V4L2 buffer metadata 
for VIDIOC_EXPBUF to work.  To that end, I think it might be better to have an 
opaque AV_PIX_FMT_V4L2 type carrying the buffer information and managed by the 
hwcontext API, which can then be mapped to user virtual memory as needed.  Such 
a setup would support sharing between components inside lav* (e.g. transcode 
with copy) as well.  Alternatively, it might be possible to use 
AV_PIX_FMT_DRM_PRIME directly (see the Rockchip codec patchset), though that 
would exclude drivers not using dma_buf inside the kernel which you suggested 
previously was a needed case.

Are there more patches to follow?  (Filtering?)  I note that all of the buffer 
functions in the current version are tagged avpriv, but aren't used outside 
lavc.

On devices, you mention it's tested on Qualcomm's DragonBoard 410c and 820c; is 
it expected to work with cores from other vendors?  If I wanted to test this, 
what hardware would be easiest to use?

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel