On Mon, 27 Mar 2017 21:38:12 +0100 Mark Thompson <s...@jkqxz.net> wrote:
> This also adds a new field to AVHWAccel to set supported hardware device > types, so that we can query the right hwaccel. > --- Btw., this won't work for d3d11va and dxva2. There are the following problems: - In DXVA2, there is a 1:1 correspondence between decoder and frame pool. (This seems to be mostly an artifact of bad drivers, but nevertheless we must do it.) This means AVDXVA2FramesContext has a decoder_to_release field. The libavcodec API requires that you create hw_frames_ctx before the decoder is created. So you either need to change this field on an initialized hwframes context, or there needs to be a way to create the decoder before the hwframes context. - The API user might want to cache/reuse the hwframes ctx if it's possible, e.g. on seeks, where the codec is flushed and get_format will be called again. This is probably not possible with DXVA2 and D3D11VA when the decoder is recreated, so the decoder needs to be cached. There is no mechanism for this. (I'm an API user who absolutely wants this.) - Like the DXVA2 hwframes ctx API, the proposed D3D11VA hwframes ctx API patches take a decoder on init. This might fulfill a similar role as DXVA2 (releasing decoder and surfaces at the same time to deal with bad drivers). But in addition, the surfaces are represented by ID3D11VideoDecoderOutputViews, which in turn depend on the decoder config GUID. This is retrieved from the video decoder. This means that even if the API is somehow changed not to require a decoder up-front, it needs the decoder config GUID. The API user can't provide this GUID without duplicating all the decoder selection logic this API is supposed to provide/hide. Possible solutions: - Add native hwframes_ctx and decoder instance caching to the libavcodec API, make it allocate the hwframex ctx internally (so it can be done after the decoder is created). Add a callback that allows the user to customize the hwframes_ctx configuration before initialization. This would basically short-cut these issues by implementing the wanted features in libavcodec - but I think it's reasonable. This would probably also cover half of the use-cases targetted by the patch I'm replying too, but other parts are orthogonal. It solves the problem, but had to be implemented in other hwaccels too (I guess). At least for vaapi/vdpau it could actually be handled in generic code, since the surfaces don't depend on the decoder. - Somehow change the D3D11VA libavcodec API, and manage the ID3D11VideoDecoderOutputViews internally. For example, the hwframes_ctx and libavcodec could somehow allocate them internally once the decoder is created. The array texture can still be allocated in a way that's similar to vaapi. This sounds messy and intrusive, especially because output views are the canonical type for AV_PIX_FMT_D3D11VA_VLD. Also doesn't take care of some other issues. (Although it might have been a nice if it had been like this in the first place.) - Somehow change how the hwframes contexts depend on the decoder, or how they free it. Like replacing the decoder refs with an AVBuffer. This could point to something we could freely change, without having to worry about interfering with hw_frames_ctx internals. So we could for example put the decoder reference into there and change it in libavcodec. Maybe this is pointless, and changing the AVDXVA2FramesContext decoder_to_release field directly would be allowed even after hw_frames_ctx init. Also this doesn't solve the part where D3D11VA needs the decoder config up-front. - Introduce an API that returns the raw decoder config before returning from get_format. It could be included in the API that determines the other hwframes ctx parameters. This might be an OK solution, though doesn't take care of the other hwframes/decoder dependencies. AVD3D11VAFramesContext would be extended so the user can provide the decoder config directly. The config would need to be returned in a portable way, and the user would somehow have to compare the decoder config to check whether an existing hw_frames_ctx can be reused. (These issues make me not like it much.) - Introduce an API that creates a hw decoder. This would mean representing the hw decoder as its own API object. Then the user would have full flexibility how to manage decoders/frames on reinits of any kind. Thoughts? I'm hoping this frames caching/reusing is acknowledged as important use-case. I'm interested in porting dxva2/d3d11va to the new hwaccel API, without losing that caching/reusing capability, and without having to implement it in a way that would duplicate too much of libavcodec's decoder setup. Personally I'd probably my first proposed solution (letting libavcodec handle this frames reusing). _______________________________________________ libav-devel mailing list libav-devel@libav.org https://lists.libav.org/mailman/listinfo/libav-devel
