Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Based on this discussion, I say that the need for an event loop is
confirmed.

I will now start discussing actual plans for going forward. Since the
coding work will be significant and not incremental, I want the outcome
of this discussion to be binding: if we agree to do it a certain way,
then the code for doing it will be accepted, pending code quality or new
unforeseen issues. If anybody objects to this, please explain your
reasons.

First, again: Why we need an event loop.

We need an event loop because we already have several ones. We already
have five protocols and two devices implementing their own event loops.
Factoring this duplicated code and implementing it properly is just the
obvious way forward.

Plus, it makes implementing new protocols simpler, especially when they
are somewhat complex. And it can make applications simpler.

So, my plan. Note that a lot of it will need to be ready before anything
can be applied.


1. Add an event loop to libavutil.

1.1. Add a simple event loop implementation.

It needs to be powerful enough to replace all our current needs:

- watching file descriptors;
- timeouts;
- low-latency I/O threads (for the UDP thread).

And I will add:

- threads for bulk tasks

because it is needed for something later (see below).

1.2. Add an event loop API to libavutil.

It will be mostly made with callbacks, so that applications can use
the event loop of their choice instead of the more limited one we
provide. But by default, it will use our implementation, of course.

I propose AVScheduler for the root structure of this API, and
av_scheduler as function prefix.

The API would consist of just a few entry points:

- allocate, init, free an AVScheduler;
- allocate, add, alter, remove, free events;
- start, run, stop the loop.

1.3. Implement a libev wrapper.

Implement the callbacks of the event loop API with libev as
back-end.

Make it either an example or a build option. A build option would
have the benefit of more extensive testing.


2. Add a callback-based API for protocols.

2.1. Implement the new API.

To use this, an application will:

- create an AVScheduler;
- attach its protocols to the AVScheduler;
- set the callbacks;
- if useful, set the buffers;
- run the AVScheduler.

As soon as the AVScheduler.

2.2. Implement a compatibility layer for new protocols.

Protocols using the new design need to be callable through the old
API. The compatibility layer will need to allocate an AVScheduler
just for this protocol, and start, run, stop it for each read or
write operation.

2.3. Port the low-level network protocols.

To get any benefit from this, at least TCP and UDP need to be
ported.

2.4. Implement a compatibility layer for old protocols.

We need to be able to still use the protocols that have not been
ported to the new design. It will involve reading or writing on them
with a short timeout. It is inefficient, but the same kind as
inefficient as we have now.

2.5. Port complex protocols.

To check that it works as expected, porting at least one of the
protocols that use several sockets is necessary.


3. Use the even loop for running libavfilter.

The reason I want threads for bulk tasks is that we can then use
them for inter-filter threading. Having a well-designed event loop
in libavutil will give us a better API and parallelism for
libavfilter for very little effort.


4. Use the new API in the fftools.

This is necessary to prove the API works.


This is it, please share your comments.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Paul B Mahol]

On Mon, Feb 1, 2021 at 8:14 PM Nicolas George  wrote:

> Based on this discussion, I say that the need for an event loop is
> confirmed.
>
> I will now start discussing actual plans for going forward. Since the
> coding work will be significant and not incremental, I want the outcome
> of this discussion to be binding: if we agree to do it a certain way,
> then the code for doing it will be accepted, pending code quality or new
> unforeseen issues. If anybody objects to this, please explain your
> reasons.
>
> First, again: Why we need an event loop.
>
> We need an event loop because we already have several ones. We already
> have five protocols and two devices implementing their own event loops.
> Factoring this duplicated code and implementing it properly is just the
> obvious way forward.
>
> Plus, it makes implementing new protocols simpler, especially when they
> are somewhat complex. And it can make applications simpler.
>
> So, my plan. Note that a lot of it will need to be ready before anything
> can be applied.
>
>
> 1. Add an event loop to libavutil.
>
> 1.1. Add a simple event loop implementation.
>
> It needs to be powerful enough to replace all our current needs:
>
> - watching file descriptors;
> - timeouts;
> - low-latency I/O threads (for the UDP thread).
>
> And I will add:
>
> - threads for bulk tasks
>
> because it is needed for something later (see below).
>
> 1.2. Add an event loop API to libavutil.
>
> It will be mostly made with callbacks, so that applications can use
> the event loop of their choice instead of the more limited one we
> provide. But by default, it will use our implementation, of course.
>
> I propose AVScheduler for the root structure of this API, and
> av_scheduler as function prefix.
>
> The API would consist of just a few entry points:
>
> - allocate, init, free an AVScheduler;
> - allocate, add, alter, remove, free events;
> - start, run, stop the loop.
>
> 1.3. Implement a libev wrapper.
>
> Implement the callbacks of the event loop API with libev as
> back-end.
>
> Make it either an example or a build option. A build option would
> have the benefit of more extensive testing.
>
>
> 2. Add a callback-based API for protocols.
>
> 2.1. Implement the new API.
>
> To use this, an application will:
>
> - create an AVScheduler;
> - attach its protocols to the AVScheduler;
> - set the callbacks;
> - if useful, set the buffers;
> - run the AVScheduler.
>
> As soon as the AVScheduler.
>
> 2.2. Implement a compatibility layer for new protocols.
>
> Protocols using the new design need to be callable through the old
> API. The compatibility layer will need to allocate an AVScheduler
> just for this protocol, and start, run, stop it for each read or
> write operation.
>
> 2.3. Port the low-level network protocols.
>
> To get any benefit from this, at least TCP and UDP need to be
> ported.
>
> 2.4. Implement a compatibility layer for old protocols.
>
> We need to be able to still use the protocols that have not been
> ported to the new design. It will involve reading or writing on them
> with a short timeout. It is inefficient, but the same kind as
> inefficient as we have now.
>
> 2.5. Port complex protocols.
>
> To check that it works as expected, porting at least one of the
> protocols that use several sockets is necessary.
>
>
> 3. Use the even loop for running libavfilter.
>
> The reason I want threads for bulk tasks is that we can then use
> them for inter-filter threading. Having a well-designed event loop
> in libavutil will give us a better API and parallelism for
> libavfilter for very little effort.
>

Why event loop is needed for filters?

libavcodec have frame threads and it does not need it.



>
>
> 4. Use the new API in the fftools.
>
> This is necessary to prove the API works.
>
>
> This is it, please share your comments.
>
> Regards,
>
> --
>   Nicolas George
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Paul B Mahol (12021-02-01):
> Why event loop is needed for filters?
> 
> libavcodec have frame threads and it does not need it.

libavcodec has its own scheduler. Duplicated code everywhere.

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[James Almer]

On 2/1/2021 4:14 PM, Nicolas George wrote:

1.3. Implement a libev wrapper.

 Implement the callbacks of the event loop API with libev as
 back-end.

 Make it either an example or a build option. A build option would
 have the benefit of more extensive testing.


I support the idea of it being a build option (Perhaps simply 
--enable-libev, like any other external dep). There's a precedent of 
external libraries being used as backend for certain features, with the 
most prominent example being libsoxr as the resample engine in 
libswresample.


This of course as long as it's not autodetected, so FATE clients don't 
use it. The last thing we want is the internal implementation being 
neglected because all devs use libev.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

James Almer (12021-02-01):
> I support the idea of it being a build option (Perhaps simply
> --enable-libev, like any other external dep). There's a precedent of
> external libraries being used as backend for certain features, with the most
> prominent example being libsoxr as the resample engine in libswresample.
> 
> This of course as long as it's not autodetected, so FATE clients don't use
> it. The last thing we want is the internal implementation being neglected
> because all devs use libev.

Thanks for the feedback. I agree with all this, although I suspect
the "certain features" brought by libev would prove to be just a slight
performance enhancement in extreme cases. But interacting with any event
loop is a necessary feature, and as such it needs to be properly
implemented and tested.

Are there other remarks on this? Otherwise I shall start the code
proper.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 17, 2021, 12:47 by geo...@nsup.org:

> James Almer (12021-02-01):
>
>> I support the idea of it being a build option (Perhaps simply
>> --enable-libev, like any other external dep). There's a precedent of
>> external libraries being used as backend for certain features, with the most
>> prominent example being libsoxr as the resample engine in libswresample.
>>
>> This of course as long as it's not autodetected, so FATE clients don't use
>> it. The last thing we want is the internal implementation being neglected
>> because all devs use libev.
>>
>
> Thanks for the feedback. I agree with all this, although I suspect
> the "certain features" brought by libev would prove to be just a slight
> performance enhancement in extreme cases. But interacting with any event
> loop is a necessary feature, and as such it needs to be properly
> implemented and tested.
>
> Are there other remarks on this? Otherwise I shall start the code
> proper.
>

I think I'd prefer an asynchronous library rather than libev.
So libuv?

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-17):
> I think I'd prefer an asynchronous library rather than libev.
> So libuv?

I have on occasion straced node to debug things, and what I have
observed is: I don't want to touch it even with a very long stick.

Wan you explain more precisely why you think libuv would be a better
choice than libev?

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 01/02/2021 19:14, Nicolas George wrote:

Based on this discussion, I say that the need for an event loop is
confirmed.

I will now start discussing actual plans for going forward. Since the
coding work will be significant and not incremental, I want the outcome
of this discussion to be binding: if we agree to do it a certain way,
then the code for doing it will be accepted, pending code quality or new
unforeseen issues. If anybody objects to this, please explain your
reasons.

First, again: Why we need an event loop.

We need an event loop because we already have several ones. We already
have five protocols and two devices implementing their own event loops.
Factoring this duplicated code and implementing it properly is just the
obvious way forward.

Plus, it makes implementing new protocols simpler, especially when they
are somewhat complex. And it can make applications simpler.

So, my plan. Note that a lot of it will need to be ready before anything
can be applied.


...


I can see that this is all sensible stuff on Unix, but can you explain a bit 
more of what your Windows implementation is going to look like?

The current file descriptor model with select() only admits the things which 
are associated with file descriptors on Windows, which means BSD sockets only 
(no eventfds or self-pipes to signal things to your event loop).

If you want the send()/recv() calls to feed back into an I/O completion port 
for a Windows-style event loop then a lot more needs to change, because 
everything will have to be able to instead call WSASend()/WSARecv() with 
additional options.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-17):
> I can see that this is all sensible stuff on Unix, but can you explain
> a bit more of what your Windows implementation is going to look like?

I have no intention to write Windows code for this. Our current
protocols use poll() internally, this event loop will do the same, and
be exactly as portable as things are now.

But it will be modular: if somebody later wants to write specific code
for Windows, it will be possible and I will make sure it is as easy as
possible.

Does that sound acceptable?

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 17, 2021, 15:38 by geo...@nsup.org:

> Lynne (12021-02-17):
>
>> I think I'd prefer an asynchronous library rather than libev.
>> So libuv?
>>
>
> I have on occasion straced node to debug things, and what I have
> observed is: I don't want to touch it even with a very long stick.
>
> Wan you explain more precisely why you think libuv would be a better
> choice than libev?
>

It's more actively developed by a large margin, and I generally
prefer its API and how it works internally.
If no one else prefers libuv I'm okay with libev.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 17/02/2021 20:57, Nicolas George wrote:

Mark Thompson (12021-02-17):

I can see that this is all sensible stuff on Unix, but can you explain
a bit more of what your Windows implementation is going to look like?


I have no intention to write Windows code for this. Our current
protocols use poll() internally, this event loop will do the same, and
be exactly as portable as things are now.

But it will be modular: if somebody later wants to write specific code
for Windows, it will be possible and I will make sure it is as easy as
possible.

Does that sound acceptable?


I'm not sure you can avoid writing Windows code for this to work at all, 
because I don't think poll() as we have now is sufficient to get the 
functionality you want.

I also worry that codifying something in this form is going to exclude the 
possibility of getting a better result on Windows later, because that wouldn't 
involve file descriptors or internals calling default BSD send()/recv().

For example:

On 01/02/2021 19:14, Nicolas George wrote:
>  The API would consist of just a few entry points:
>
>  - allocate, init, free an AVScheduler;
>  - allocate, add, alter, remove, free events;
>  - start, run, stop the loop.

How are you going to implement altering/removing an event or stopping the loop?

On Unix you of course have an eventfd/self-pipe sitting in the poll() set on 
the event loop to interrupt it.  On Windows you can't, because those things are 
not file descriptors.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-17):
> I'm not sure you can avoid writing Windows code for this to work at
> all, because I don't think poll() as we have now is sufficient to get
> the functionality you want.

I'm sure I can avoid writing Windows code, because, as I have already
pointed, our current implementation already work using poll(). If it
currently works, I have no intention of breaking it, just reorganizing
it to make it work better and make easier to extend.

> I also worry that codifying something in this form is going to exclude
> the possibility of getting a better result on Windows later, because
> that wouldn't involve file descriptors or internals calling default
> BSD send()/recv().

You have got it backwards: it will make the possibility of getting a
better result on Windows later easier, because it will make everything
modular.

In fact, libev and libuv already implement code specific to Windows, so
we get them for free as soon as we implement support for one of these
libraries.

> How are you going to implement altering/removing an event or stopping
> the loop?
> 
> On Unix you of course have an eventfd/self-pipe sitting in the poll()
> set on the event loop to interrupt it.  On Windows you can't, because
> those things are not file descriptors.

These are needed for stopping the loop from the outside, from another
thread. They are not needed to stop the loop from the inside.

It is possible that our threading capabilities for this will be more
limited on Windows than on Unix until specific code is implemented (not
by me) and unless the external library is used. Just as our threading
capabilities are currently more limited on Windows.

I insist: it is the opposite as what you are saying, this project makes
it possible to have code that works on Windows, unlike now. It just will
not be me who write specific code.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-17):
> It's more actively developed by a large margin, and I generally

This is an interesting argument.

> prefer its API and how it works internally.

Can you give more details? Which differences make you prefer it?

One of the things that I like about libev is that it makes effort to be
efficient and simple in terms of allocating data structures. No such
thing as always dynamically allocating stuff like the Gnome projects, so
much that it looks more like Java than C. We can have the libev
structures directly in our own data structures.

Is libuv similar in that respect?

The fact that it is designed to run the interpreter of a language with a
garbage collector worries me.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 18/02/2021 15:39, Nicolas George wrote:

Mark Thompson (12021-02-17):

I'm not sure you can avoid writing Windows code for this to work at
all, because I don't think poll() as we have now is sufficient to get
the functionality you want.


I'm sure I can avoid writing Windows code, because, as I have already
pointed, our current implementation already work using poll(). If it
currently works, I have no intention of breaking it, just reorganizing
it to make it work better and make easier to extend.


If the existing remote-socket-only poll is sufficent (so no local events), then 
I think I have misunderstood your original definition of the event loop because 
I was expecting that other threads would be able to interact with it.

Would it be possible to add an example what you intend API use to actually look 
like to your design summary?  (In loose pseudocode.)


I also worry that codifying something in this form is going to exclude
the possibility of getting a better result on Windows later, because
that wouldn't involve file descriptors or internals calling default
BSD send()/recv().


You have got it backwards: it will make the possibility of getting a
better result on Windows later easier, because it will make everything
modular.

In fact, libev and libuv already implement code specific to Windows, so
we get them for free as soon as we implement support for one of these
libraries.


libev does not implement enough Windows support to be used generally, though it 
is sufficient if you know the event loop will never want to interact with 
anything which is not a socket (it hacks up self-pipes by connecting TCP 
sockets to random ports on localhost).

I believe that libuv and libevent both implement it properly (I have no 
experience with libuv, but Lynne seems to be advocating for it), so I would 
strongly recommend starting your implementation using one of them in order to 
ensure that you don't end up tied to the Unix model in an unhelpful way.  (Note 
in particular that this means essentially everything dealing with sockets has 
to be delegated to the event library so that it can deal with asynchronous 
calls internally, especially send()/recv() calls.)

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-18):
> If the existing remote-socket-only poll is sufficent (so no local
> events), then I think I have misunderstood your original definition of
> the event loop because I was expecting that other threads would be
> able to interact with it.

Threading is one of the goals, but not the main one. If it is available
on Unix only until somebody writes a little Windows code or unless one
use the external library, that is not an issue.

> Would it be possible to add an example what you intend API use to
> actually look like to your design summary?  (In loose pseudocode.)

loop = create_event_loop()
vid_st = create_rtp_stream(...)
aud_st = create_rtp_stream(...)
attach_to_loop(loop, vid_st)
attach_to_loop(loop, aud_st)
connect_callback(vid_st, video_callback)
connect_callback(aud_st, video_callback)
run_loop(loop)

> libev does not implement enough Windows support to be used generally,
> though it is sufficient if you know the event loop will never want to
> interact with anything which is not a socket (it hacks up self-pipes
> by connecting TCP sockets to random ports on localhost).

If it works...

> I believe that libuv and libevent both implement it properly

That would be an argument for it. I am waiting for Lynne answer to my
request for details.

> (Note in particular that this means essentially everything dealing
> with sockets has to be delegated to the event library so that it can
> deal with asynchronous calls internally, especially send()/recv()
> calls.)

Using non-standard IO code and tying to the library is completely out of
the question.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 18/02/2021 20:55, Nicolas George wrote:

Mark Thompson (12021-02-18):

If the existing remote-socket-only poll is sufficent (so no local
events), then I think I have misunderstood your original definition of
the event loop because I was expecting that other threads would be
able to interact with it.


Threading is one of the goals, but not the main one. If it is available
on Unix only until somebody writes a little Windows code or unless one
use the external library, that is not an issue.


Would it be possible to add an example what you intend API use to
actually look like to your design summary?  (In loose pseudocode.)


loop = create_event_loop()
vid_st = create_rtp_stream(...)
aud_st = create_rtp_stream(...)
attach_to_loop(loop, vid_st)
attach_to_loop(loop, aud_st)
connect_callback(vid_st, video_callback)
connect_callback(aud_st, video_callback)
run_loop(loop)


The callback for an input stream is called when you receive something, sure.

When is the callback for an output stream called?  What if it doesn't have a 
packet available to write right now, but will do later?


libev does not implement enough Windows support to be used generally,
though it is sufficient if you know the event loop will never want to
interact with anything which is not a socket (it hacks up self-pipes
by connecting TCP sockets to random ports on localhost).


If it works...


I believe that libuv and libevent both implement it properly


That would be an argument for it. I am waiting for Lynne answer to my
request for details.


(Note in particular that this means essentially everything dealing
with sockets has to be delegated to the event library so that it can
deal with asynchronous calls internally, especially send()/recv()
calls.)


Using non-standard IO code and tying to the library is completely out of
the question.

I think this means that integration with most external libraries (and 
non-network things) will not be possible, then.

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-18):
> When is the callback for an output stream called?  What if it doesn't
> have a packet available to write right now, but will do later?

It will be called when data has been written, of course. If there is no
data to write, the stream is just idle.

> I think this means that integration with most external libraries (and
> non-network things) will not be possible, then.

I am sure this is entirely the opposite.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 18/02/2021 21:44, Nicolas George wrote:

Mark Thompson (12021-02-18):

When is the callback for an output stream called?  What if it doesn't
have a packet available to write right now, but will do later?


It will be called when data has been written, of course. If there is no
data to write, the stream is just idle.


I wrote the last packet a second ago and got a callback for writeability from 
the loop, but didn't have anything to write back then so I did nothing.

Now I have a packet to write, but the event loop is sitting idle in its poll() 
call.

You've said I'm not allowed to interact with the loop from another thread, so 
how do I make the loop generate a callback so I can write something on the 
thread which is allowed to write?

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-18):
> I wrote the last packet a second ago and got a callback for
> writeability from the loop, but didn't have anything to write back
> then so I did nothing.
> 
> Now I have a packet to write, but the event loop is sitting idle in its 
> poll() call.
> 
> You've said I'm not allowed to interact with the loop from another
> thread, so how do I make the loop generate a callback so I can write
> something on the thread which is allowed to write?

Where does your packet come from? If it comes from inside the loop, then
there is no problem.

If it comes from another thread, that means your application either
requires a libavutil with a thread-capable event loop backend or must
provide one.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 18, 2021, 16:43 by geo...@nsup.org:

> Lynne (12021-02-17):
>
>> It's more actively developed by a large margin, and I generally
>>
>
> This is an interesting argument.
>
>> prefer its API and how it works internally.
>>
>
> Can you give more details? Which differences make you prefer it?
>
> One of the things that I like about libev is that it makes effort to be
> efficient and simple in terms of allocating data structures. No such
> thing as always dynamically allocating stuff like the Gnome projects, so
> much that it looks more like Java than C. We can have the libev
> structures directly in our own data structures.
>
> Is libuv similar in that respect?
>
> The fact that it is designed to run the interpreter of a language with a
> garbage collector worries me.
>

I found this document useful:
https://gist.github.com/andreybolonin/2413da76f088e2c5ab04df53f07659ea

Although it's main user was node.js, it's now used in dozens upon dozens
of different non-language projects, some of which get featured here:
https://github.com/libuv/libuv/wiki/Projects-that-use-libuv

It also has plenty of bindings, so projects don't have to invent their
own C bindings.
Most high-level users have long since moved to asynchronous event
processing on threads, so having a synchronous single-threaded
implementation seems like somewhat behind with the times.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-19):
> I found this document useful:
> https://gist.github.com/andreybolonin/2413da76f088e2c5ab04df53f07659ea

Thanks. But I think this document makes it clear that what we, FFmpeg,
need, is not libuv.

In fact, after looking in the docs and in the source, I could not even
find how we can use it: it does not seem to have an API to integrate
foreign protocols.

Unless somebody points me that I am mistaken, it seems libuv is out and
it will be libev.

> Most high-level users have long since moved to asynchronous event
> processing on threads, so having a synchronous single-threaded
> implementation seems like somewhat behind with the times.

Yes, that is a terrible trend. Throw memory and CPU at I/O performance
instead of writing optimized code. POSIX threads never were a solution
for parallel I/O.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 19, 2021, 12:41 by geo...@nsup.org:

> Lynne (12021-02-19):
>
>> I found this document useful:
>> https://gist.github.com/andreybolonin/2413da76f088e2c5ab04df53f07659ea
>>
>
> Thanks. But I think this document makes it clear that what we, FFmpeg,
> need, is not libuv.
>
> In fact, after looking in the docs and in the source, I could not even
> find how we can use it: it does not seem to have an API to integrate
> foreign protocols.
>

https://github.com/libuv/libuv/tree/v1.x/docs/code
Pipes, file descriptors, sockets, examples.
The docs are even searchable: http://docs.libuv.org/en/v1.x/index.html


> Unless somebody points me that I am mistaken, it seems libuv is out and
> it will be libev.
>
>> Most high-level users have long since moved to asynchronous event
>> processing on threads, so having a synchronous single-threaded
>> implementation seems like somewhat behind with the times.
>>
>
> Yes, that is a terrible trend. Throw memory and CPU at I/O performance
> instead of writing optimized code. POSIX threads never were a solution
> for parallel I/O.
>

They kind of are, since I/O is blocking on most platforms, and you don't
want your event loop to grind to a halt if a disk read from an asleep HDD
is issued and libev's  event dispatch locks up everything else.
io_uring is still extremely new, no one wants to break API compatibility
with older kernels, and for some applications it's still unusable as new
additions make it into newer kernels that aren't even released yet.

You seem really prejudiced towards anything 'infected' by node, big data,
or http servers, but they've been solving the issues we have now since
before we knew we even had them.

While it's up to you to understand, the libuv community are definitely
going to help you if you just ask them a few questions about whether
it's suitable or not if you give them your requirements. They may even
do some work if there's something missing.

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-19):
> https://github.com/libuv/libuv/tree/v1.x/docs/code
> Pipes, file descriptors, sockets, examples.
> The docs are even searchable: http://docs.libuv.org/en/v1.x/index.html

I have found all this. Nothing allows to handle a foreign protocol
handled by existing code. It is even stated explicitly in the docs:
everything assumes libuv owns the fd. That means no ALSA, no libX11/xcb,
etc.

> > Yes, that is a terrible trend. Throw memory and CPU at I/O performance
> > instead of writing optimized code. POSIX threads never were a solution
> > for parallel I/O.
> They kind of are, since I/O is blocking on most platforms

No, they are not. We had non-blocking I/O working perfectly way before
POSIX threads were even remotely stable.

I have come to understands that "threads as a solution for I/O" is a
Windows solution.

> You seem really prejudiced towards anything 'infected' by node, big data,
> or http servers, but they've been solving the issues we have now since
> before we knew we even had them.

Unfortunately, their solutions are industrial ones, i.e. throw money at
the problem. I want a hacker solution: throw efficient programming at
the problem.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 19, 2021, 14:52 by geo...@nsup.org:

> Lynne (12021-02-19):
>
>> https://github.com/libuv/libuv/tree/v1.x/docs/code
>> Pipes, file descriptors, sockets, examples.
>> The docs are even searchable: http://docs.libuv.org/en/v1.x/index.html
>>
>
> I have found all this. Nothing allows to handle a foreign protocol
> handled by existing code. It is even stated explicitly in the docs:
> everything assumes libuv owns the fd. That means no ALSA, no libX11/xcb,
> etc.
>

File descriptors are reference counted. So if something takes ownership
of an FD (completely reasonable, since it may want to clean up, seek, and
flush+close it afterwards), then you can just dup() it before giving it to 
libuv.


>> > Yes, that is a terrible trend. Throw memory and CPU at I/O performance
>> > instead of writing optimized code. POSIX threads never were a solution
>> > for parallel I/O.
>> They kind of are, since I/O is blocking on most platforms
>>
>
> No, they are not. We had non-blocking I/O working perfectly way before
> POSIX threads were even remotely stable.
>
> I have come to understands that "threads as a solution for I/O" is a
> Windows solution.
>

Windows is something we explicitly support. So we need something
which works on Windows as well, otherwise we still lose a very very
large majority of users.


>> You seem really prejudiced towards anything 'infected' by node, big data,
>> or http servers, but they've been solving the issues we have now since
>> before we knew we even had them.
>>
>
> Unfortunately, their solutions are industrial ones, i.e. throw money at
> the problem. I want a hacker solution: throw efficient programming at
> the problem.
>

That's some very prejudiced thinking. Their solutions are ugly because they
got paid for it? Maybe they got money because their hacker solutions failed
after someone put in much more effort than what's reasonable?
By that logic a lot of our code is also "throw money at the problem". Frame
threading for decoders is definitely up there. What about reference counted
frames?
There's a time and a place for hacker solutions, but this area's definitely not 
it,
unless TempleOS integration is more important than having Windows support.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-19):
> File descriptors are reference counted. So if something takes ownership
> of an FD (completely reasonable, since it may want to clean up, seek, and
> flush+close it afterwards), then you can just dup() it before giving
> it to libuv.

You are completely misunderstanding the problem. A foreign protocol is a
protocol for which we have a read function in a library; if the library
is not bad, we can have it non-blocking; if the library is good, we can
have the file descriptor behind for polling.

When we have that, we can integrate it an event loop: add the file
descriptor to our watch list; when poll() tells us it is readable, call
the read function of the library.

Many libraries can work like that, including libX11 and libxcb, ALSA,
and actually, to some extent, libavformat.

Many event loops can work like that, including libev, and Gtk+/GLib.

It seems that libuv cannot work like that: it insists for doing the
reading itself.

> Windows is something we explicitly support. So we need something
> which works on Windows as well, otherwise we still lose a very very
> large majority of users.

But we are primarily a Unix project, with a Unix culture and Unix-style
code. If we forget this, we get something that half-way each and all the
way bad.

> That's some very prejudiced thinking. Their solutions are ugly because they
> got paid for it?

No, "throw money at the problem" means bigger hardware because the code
is needlessly slow and not optimized, because they chose the easy
solution.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 19, 2021, 15:52 by geo...@nsup.org:

> Lynne (12021-02-19):
>
>> File descriptors are reference counted. So if something takes ownership
>> of an FD (completely reasonable, since it may want to clean up, seek, and
>> flush+close it afterwards), then you can just dup() it before giving
>> it to libuv.
>>
>
> You are completely misunderstanding the problem. A foreign protocol is a
> protocol for which we have a read function in a library; if the library
> is not bad, we can have it non-blocking; if the library is good, we can
> have the file descriptor behind for polling.
>
> When we have that, we can integrate it an event loop: add the file
> descriptor to our watch list; when poll() tells us it is readable, call
> the read function of the library.
>
> Many libraries can work like that, including libX11 and libxcb, ALSA,
> and actually, to some extent, libavformat.
>
> Many event loops can work like that, including libev, and Gtk+/GLib.
>
> It seems that libuv cannot work like that: it insists for doing the
> reading itself.
>

That's okay, I'd let it read and call events as they happen on the fds.
I've written several pollin/pollout loops for Wayland FDs before and
if possible I'd rather not have written them at all.


>> Windows is something we explicitly support. So we need something
>> which works on Windows as well, otherwise we still lose a very very
>> large majority of users.
>>
>
> But we are primarily a Unix project, with a Unix culture and Unix-style
> code. If we forget this, we get something that half-way each and all the
> way bad.
>

I'm not even going to respond to this...


>> That's some very prejudiced thinking. Their solutions are ugly because they
>> got paid for it?
>>
>
> No, "throw money at the problem" means bigger hardware because the code
> is needlessly slow and not optimized, because they chose the easy
> solution.
>

Not going to respond to this either.

But I'll say this: maybe it's better that you don't actually get in touch with 
anyone
from libuv or otherwise, or if you do please don't mention anything about 
FFmpeg.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-19):
> That's okay, I'd let it read and call events as they happen on the fds.

Pray tell me, how do you do that with a /dev/snd/pcmC0D0p in mmap mode?

Sorry, but as promising as libuv looked, it is not generic enough for
our use case, unless I missed something big.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Lynne]

Feb 19, 2021, 16:21 by geo...@nsup.org:

> Lynne (12021-02-19):
>
>> That's okay, I'd let it read and call events as they happen on the fds.
>>
>
> Pray tell me, how do you do that with a /dev/snd/pcmC0D0p in mmap mode?
>

You poll it in a busy loop in a thread in an event. Or you find a solution
to use libuv's polling instead via a "hacker solution".
Or you simply don't use its mmap API, because polling-only sound systems
only made sense in the 90's.

This is as far as I'm willing to take this discussion. You're stubbornly 
unwilling to
see any reason and you openly cite your biases as a reason to hate libuv,
Windows or threads.
Next time if you have absolutely no interest in any other solution apart from it
being a opportunity to express your opinions, don't ask.

You can proceed on your own, but you don't have my support. If the patch ends up
locking us up with libev or its unmaintained Windows forks I'll voice my 
disapproval
and we'll end up with exactly the type of situation this RFC was meant to 
resolve.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Lynne (12021-02-19):
> You poll it in a busy loop in a thread in an event.

Yeah. Events loop are supposed to avoid busy loops. So no.

> Or you find a solution to use libuv's polling instead via a "hacker
> solution".

This is what I am asking: can you give a good solution with libuv? It
seems you cannot.

> Or you simply don't use its mmap API, because polling-only sound systems
> only made sense in the 90's.

Yeah, let us not use the more efficient API because the library does not
support it is not a great argument. And it was only an example.
Sometimes, it's an ioctl that needs to be done. Sometimes it's just that
the library handling the protocol expects the data on the file
descriptor and not already read by another library.

All this discussion has proven that libuv is not suited for our use. it
may be great for the kind of things it is meant for, but it is just not
what we, FFmpeg, need.

> This is as far as I'm willing to take this discussion. You're stubbornly 
> unwilling to
> see any reason and you openly cite your biases as a reason to hate libuv,
> Windows or threads.
> Next time if you have absolutely no interest in any other solution apart from 
> it
> being a opportunity to express your opinions, don't ask.

This is untrue. I have expressed my biases as replies to your own biases
("only made sense in the 90's" and such), but the arguments I base my
decision on are technical.

And in this case it is not just a matter of preference. I was convinced
to use libuv, I looked how to do it, and only then found it seemed it
was not possible.

> You can proceed on your own, but you don't have my support. If the patch ends 
> up
> locking us up with libev or its unmaintained Windows forks I'll voice my 
> disapproval
> and we'll end up with exactly the type of situation this RFC was meant to 
> resolve.

It seems you have missed the part where I mentioned the design will be
modular. That's not too bad, I only repeated it half a dozen times.

And that is the core of the problem. We may be able to use libuv if we
sacrifice devices, but we would be able to use ONLY libuv. Nor our own
lightweight implementation, nor a completely different event loop
provided by the application.

This is not what we want.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Kieran Kunhya]

On Fri, 19 Feb 2021 at 17:41, Nicolas George  wrote:

> Lynne (12021-02-19):
> > You poll it in a busy loop in a thread in an event.
>
> Yeah. Events loop are supposed to avoid busy loops. So no.
>
> > Or you find a solution to use libuv's polling instead via a "hacker
> > solution".
>
> This is what I am asking: can you give a good solution with libuv? It
> seems you cannot.
>

I don't have a strong opinion on either. But I think you can use
container_of on the fd.

Kieran
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Kieran Kunhya (12021-02-19):
> I don't have a strong opinion on either. But I think you can use
> container_of on the fd.

Thanks. I find no trace of it in the docs:

http://docs.libuv.org/en/v1.x/search.html?q=container_of&check_keywords=yes&area=default

Can you be a little more precise?

The portability arguments have been compelling for libuv over libev, so
if we find a way of getting libuv to work for our needs, it would be
best.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Kieran Kunhya]

On Fri, 19 Feb 2021 at 19:04, Nicolas George  wrote:

> Kieran Kunhya (12021-02-19):
> > I don't have a strong opinion on either. But I think you can use
> > container_of on the fd.
>
> Thanks. I find no trace of it in the docs:
>
>
> http://docs.libuv.org/en/v1.x/search.html?q=container_of&check_keywords=yes&area=default
>
> Can you be a little more precise?
>
> The portability arguments have been compelling for libuv over libev, so
> if we find a way of getting libuv to work for our needs, it would be
> best.
>

Actually maybe you can't use container_of and mmap since the fd is not a
pointer...
Sorry for noise.

Kieran
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Mark Thompson]

On 19/02/2021 19:34, Kieran Kunhya wrote:

On Fri, 19 Feb 2021 at 19:04, Nicolas George  wrote:


Kieran Kunhya (12021-02-19):

I don't have a strong opinion on either. But I think you can use
container_of on the fd.


Thanks. I find no trace of it in the docs:


http://docs.libuv.org/en/v1.x/search.html?q=container_of&check_keywords=yes&area=default

Can you be a little more precise?

The portability arguments have been compelling for libuv over libev, so
if we find a way of getting libuv to work for our needs, it would be
best.



Actually maybe you can't use container_of and mmap since the fd is not a
pointer...


I'm not entirely sure where this discussion was going, but I believe the 
original questions would have been simple to answer if anyone had bothered to 
read the documentation.

Adding a file descriptor referring to something independent of libuv (such as an alsa 
or xcb fd) is handled by uv_poll_init(), see 
.

A user context pointer is carried into callbacks via the "data" member of uv_handle_t 
(and hence the same member in all subtypes of it, like uv_poll_t), see 
.


I think we need to be clear here that libuv and libev are solving some of the 
same problems but not really in the same way.

libev is essentially super-poll(): it gives you a uniform interface to whatever 
the preferred synchronous multiplexing method is on your platform (epoll on 
Linux, kqueue on BSD and derivatives, traditional BSD select on Windows, etc.). 
 For Windows, that of course only supports things to which the synchronous 
multiplexing interface applies, which is network sockets only.

libev isn't really giving you anything which we don't have already, though the 
common interface may be nice for other applications which also use libev.

libuv is trying to solve event loop problem in a uniform way across all 
platforms, with no requirement that the underlying method is synchronous.

That lets it provide better support on systems which want to use asynchrous I/O 
(like Windows with I/O completion ports), but has the consequence that it has 
to own the sockets internally because the details of the multiplexer affect all 
socket I/O.  If the underlying method is asynchrous, then every socket call 
will need additional detail that the user doesn't want to deal with (e.g. 
send() becomes WSASend() with an OVERLAPPED argument on Windows so the result 
can be delivered to an I/O completion port later).


In my opinion, we would be best off ensuring that we support the more general 
solution in libuv (or, if libuv has separate problems which rule it out, 
another library with the same approach such as libevent), since FFmpeg provides 
cross-platform libraries which we would like to work properly on all systems we 
support.  This has future benefits for the systems which in current form would 
be covered by libev, as asynchronous I/O becomes is becoming more common on 
Unix platforms as well (e.g. io_uring on Linux) and we will be able to handle 
it without another redesign.

Thanks,

- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Mark Thompson (12021-02-20):
> I'm not entirely sure where this discussion was going, but I believe
> the original questions would have been simple to answer if anyone had
> bothered to read the documentation.
> 
> Adding a file descriptor referring to something independent of libuv
> (such as an alsa or xcb fd) is handled by uv_poll_init(), see
> .

I had looked in the documentation, but not read it entirely, and I had
missed this part. I was precisely to get help finding the relevant
tidbits that I asked here.

I want to take the occasion to explain how this discussion is going, in
my opinion.

Lynne, this is in great part for you.

When I exposed my prejudices against Node, it was not to close the
discussion, it was not a rejection. Quite the contrary, I was opening
the discussion, I was inviting precisions: I pointed that my prejudices
were weak, and that I knew they were, and I expected somebody to correct
me with more accurate information relevant to the discussion.

Somebody could have pointed to me that the horrible strace output of
Node is not the fault of libuv, that libuv does all the necessary things
to use efficiently the available system features.

(About my opinions on industrial development, I believe they are
founded, but they are not relevant, as they relate to practices that are
commonplace there, but are absolutely not what libuv does.)

Now that I have looked into it more carefully, I can say that libuv
seems pretty good. Apparently, it does the right thing wherever I
looked.

In fact, what libuv does looks a lot like what I want to do with
libavformat.

If we were looking for a library to found our network protocols upon,
then libuv would be probably an excellent candidate.

But that is not what we are doing.

The policy of this project has always been: no hard dependencies for
core features.

Network protocols are a core feature, at least some of them. We cannot
depend on libuv for them.

There is another policy: we do not chose a side. We do not decide if
OpenSSL is better than GnuTLS, we support both.

For these reasons, we are not looking for a single library to base the
protocols. We will do that ourselves, and we are looking for libraries
to bring extra features and most importantly to prove that the new
design is usable with them.

The last point is very important. Even if we finally decide to go with
libuv, we must make sure that other applications will be able to use
libev, and GLib/Gtk+, and Qt.

And that means we cannot use the advanced features of the libraries, we
need to stay with the basic features that are available, one way or
another, in all of them.

I know that GLib/Gtk+, libev and Qt are compatible with each other and
with the design that is immediately obvious for an experienced Unix
developer. This is what I intended to do.

For a time, it seemed that libuv was not compatible, not compatible with
GLib/Gtk+, libev and Qt and not compatible with the obvious design. That
would have meant it was too high-level for our needs. That would have
removed a promising option from our choices, but that would not have
been terrible.

Fortunately, Mark found the bit that was missing, so we can go ahead.

But there was another option, and there still is.

We can decide to change the policy. We can decide that the way libuv
does things is good, and we want to use it, and FFmpeg will depend on it
to become better.

This is not what we were discussing, but we can discuss it.

I have no intention of undertaking this discussion, but I think it could
be a good idea, and I would not oppose it, quite the contrary.

So, Lynne, if you want to defend this change in policy, please go ahead,
you have my support.

But otherwise, it seems we can use libuv, but we will still have to
restrict to its basic features.

I hope this makes things clearer and removes any aggression that may
have seeped into the discussion.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Kieran Kunhya]

>
> I want to replace it with an event loop. That means that instead of reading
> on a protocol context, we would register a callback for when data is
> available, and then let the loop run.
>

This would be a good idea in general and bring FFmpeg into the early 21st
century.
As I understand it the hard part is cross-platform event loops (does libev
support windows now?)

Happy new year!
Kieran
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Anton Khirnov]

Quoting Nicolas George (2020-12-31 14:37:21)
> This mail is about a project I have to make FFmpeg's API and
> infrastructure more convenient. For a common introduction, see this thread:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274167.html
> 
> The API to access our protocols, in particular network protocols, is copied
> from the Unix file descriptor API, basically read() and write(). While this
> API is very simple and convenient when it works, it works mostly only with
> simple applications that do with only one input or output at a time. Extra
> features like timeouts or handling several streams simultaneously are
> complex to add.
> 
> Furthermore, timeouts and non-blocking mode are currently done with a
> periodic polling, which makes the application always active and can prevent
> embedded devices from going into deep sleep, draining the battery.
> 
> I want to replace it with an event loop. That means that instead of reading
> on a protocol context, we would register a callback for when data is
> available, and then let the loop run.

I do not think this is a good idea. There are enough event loops in the
world and making yet another one is not in scope for us. What we should
do instead is allow clean integration with existing event loops, mainly
by exporting the underlying file descriptors so they can be poll()ed on.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Anton Khirnov (12021-01-01):
> I do not think this is a good idea. There are enough event loops in the
> world and making yet another one is not in scope for us. What we should
> do instead is allow clean integration with existing event loops, mainly
> by exporting the underlying file descriptors so they can be poll()ed on.

We must do that indeed, but that is not enough, you are missing part of
the problem: FFmpeg has complex protocols that rely on several sockets,
and that is 100% part of the scope. Currently, these protocols are
implemented with an unsavory mix of polling and short timeouts. To fix
them and make them properly usable, we need an event loop.

So, we have only two choices (I consider "leaving everything in the mess
it is" is not a choice): implement our own event loop or choose one as a
mandatory requirement. AFAIK, adding new mandatory requirements is not
the preferred policy.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Kieran Kunhya (12020-12-31):
> This would be a good idea in general and bring FFmpeg into the early 21st
> century.

Thanks for the feedback.

> As I understand it the hard part is cross-platform event loops (does libev
> support windows now?)

AFAIK, it is hard to make something really optimized, but what we
already do for cross-platforms timeouts should be enough, properly used,
for our needs.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Anton Khirnov]

Quoting Nicolas George (2021-01-02 12:28:30)
> Anton Khirnov (12021-01-01):
> > I do not think this is a good idea. There are enough event loops in the
> > world and making yet another one is not in scope for us. What we should
> > do instead is allow clean integration with existing event loops, mainly
> > by exporting the underlying file descriptors so they can be poll()ed on.
> 
> We must do that indeed, but that is not enough, you are missing part of
> the problem: FFmpeg has complex protocols that rely on several sockets,
> and that is 100% part of the scope. Currently, these protocols are
> implemented with an unsavory mix of polling and short timeouts. To fix
> them and make them properly usable, we need an event loop.

I don't see how that conclusion follows. If a protocol has multiple FDs,
we design the API to allow exporting multiple FDs. Why would we
absolutely need our own event loop?

> 
> So, we have only two choices (I consider "leaving everything in the mess
> it is" is not a choice): implement our own event loop or choose one as a
> mandatory requirement. AFAIK, adding new mandatory requirements is not
> the preferred policy.

It seems massively preferable to leave the choice of an event loop (if
any - polling is still a viable alternative for many simple cases) to
the user.

-- 
Anton Khirnov
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Anton Khirnov (12021-01-02):
> I don't see how that conclusion follows. If a protocol has multiple FDs,
> we design the API to allow exporting multiple FDs. Why would we
> absolutely need our own event loop?

The protocol exports multiple FDs... ok... and then what? How do you
make the protocol work from ffmpeg.c? Or from any application that does
not have its own event loop? How do you make the protocol's url_read()
callback work?

I do not know how extensive your experience with network programming
is¹, but you need to realize that a protocol with several sockets needs
an event loop, it cannot be implemented without. And since FFmpeg does
not have a global, clean event loop, all protocols with multiple sockets
end up re-implementing their own half-assed event loop in the url_read()
callback.

I think you do not realize that there are already at least four crappy
event loops (rtpproto.c, rtsp.c, rtspenc.c, sapdec.c).

What I am proposing is to have just one, not crappy. Written like this,

I will add this: what made FFmpeg's success? What allowed FFmpeg to get
the fastest VP9 decoder out there, for example? The fact that it already
has all the infrastructure required by codecs: good FFTs, frame
management, bistream readers, etc.

If we want FFmpeg to have good protocols as well as good codecs, it
needs to have the infrastructure to make them possible. And for
protocols, the infrastructure is mostly two things: crypto protocols and
en event loop.


1: Please consider enlightening us on this. If I can say so myself, I
consider I am quite proficient at Unix network and system programming,
except for fringe topics like multicast.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Kieran Kunhya]

On Sat, 2 Jan 2021 at 16:52, Nicolas George  wrote:

> If we want FFmpeg to have good protocols as well as good codecs, it
> needs to have the infrastructure to make them possible. And for
> protocols, the infrastructure is mostly two things: crypto protocols and
> en event loop.
>

In my opinion we should use libev as the default and make it a submodule
(possibly as part of some sort of plugin system for event loops). This
allows us to use modern things such as io_uring as they come.

For crypto there is a valid argument to avoid the complexity of OpenSSL
etc, but libev is relatively elegant and clean.

Kieran
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Peter Ross]

On Thu, Dec 31, 2020 at 02:37:21PM +0100, Nicolas George wrote:
> This mail is about a project I have to make FFmpeg's API and
> infrastructure more convenient. For a common introduction, see this thread:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274167.html
> 
> The API to access our protocols, in particular network protocols, is copied
> from the Unix file descriptor API, basically read() and write(). While this
> API is very simple and convenient when it works, it works mostly only with
> simple applications that do with only one input or output at a time. Extra
> features like timeouts or handling several streams simultaneously are
> complex to add.
> 
> Furthermore, timeouts and non-blocking mode are currently done with a
> periodic polling, which makes the application always active and can prevent
> embedded devices from going into deep sleep, draining the battery.
> 
> I want to replace it with an event loop. That means that instead of reading
> on a protocol context, we would register a callback for when data is
> available, and then let the loop run.
> 
> Of course, a compatibility layer will still allow to read directly.

Majority of our demuxers and muxers assume a blocking model. Will they need to 
be
rewritten (eventually) to use event-based i/o?

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Peter Ross (12021-01-04):
> Majority of our demuxers and muxers assume a blocking model. Will they
> need to be rewritten (eventually) to use event-based i/o?

The way I see it, compatibility is ensured: the current API that reads
on protocol should still be supported, by creating a local loop and
running it until a packet is obtained. Exactly what we are doing now,
but with cleaner code and less duplication.

Therefore, muxers and demuxers written for it would continue working.

But muxers and demuxers would also have the option of being rewritten to
use events, and in turn gain better non-blocking operation in the
process.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Kieran Kunhya (12021-01-02):
> In my opinion we should use libev as the default and make it a submodule
> (possibly as part of some sort of plugin system for event loops). This
> allows us to use modern things such as io_uring as they come.

I can be on board with that, since I like libev very much, and I find it
matches well the philosophy of FFmpeg's coding practices.

But I am afraid I am not the one who will need convincing on this.

Making sure any event loop can be used with a pluggable system is, IMHO,
an absolute requisite. Based on this, providing a basic event loop,
enough to run most of our protocols, would be compatible with FFmpeg's
habits, and not that much work. And libev can be used if available for
more complex protocols or better performance.

But as I said, I would be fine with libev the main, official event loop
for the project.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Xiang Xiao]

Nicolas, thank you for bringing up this topic.
My team is working on similar functionality: the event loop is the
central point to integrate protocol, demux/mux, decoder/encoder, avfilter
and avdevice to form a complete multimedia subsystem(like DirectShow or
GStreamer). Without the builtin event loop, it is very hard to support it
with a clean and elegant way outside FFmpeg:

   1. Multiple streaming(either local or network)
   2. Multiple devices(either capture, playback even Bluetooth A2DP/SCO)
   3. Interact and coordinate between the components
   4. Provide a system wide and easy used API to application

We are very excited that the community is considering to provide the
generic event loop support and want to make the contribution.

Thanks
Xiang

On Mon, Jan 4, 2021 at 9:45 PM Nicolas George  wrote:

> Kieran Kunhya (12021-01-02):
> > In my opinion we should use libev as the default and make it a submodule
> > (possibly as part of some sort of plugin system for event loops). This
> > allows us to use modern things such as io_uring as they come.
>
> I can be on board with that, since I like libev very much, and I find it
> matches well the philosophy of FFmpeg's coding practices.
>
> But I am afraid I am not the one who will need convincing on this.
>
> Making sure any event loop can be used with a pluggable system is, IMHO,
> an absolute requisite. Based on this, providing a basic event loop,
> enough to run most of our protocols, would be compatible with FFmpeg's
> habits, and not that much work. And libev can be used if available for
> more complex protocols or better performance.
>
> But as I said, I would be fine with libev the main, official event loop
> for the project.
>
> Regards,
>
> --
>   Nicolas George
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Steven Liu]

> 2021年1月4日 下午9:44，Nicolas George  写道：
> 
> Kieran Kunhya (12021-01-02):
>> In my opinion we should use libev as the default and make it a submodule
>> (possibly as part of some sort of plugin system for event loops). This
>> allows us to use modern things such as io_uring as they come.
> 
> I can be on board with that, since I like libev very much, and I find it
> matches well the philosophy of FFmpeg's coding practices.
> 
> But I am afraid I am not the one who will need convincing on this.
> 
> Making sure any event loop can be used with a pluggable system is, IMHO,
> an absolute requisite. Based on this, providing a basic event loop,
> enough to run most of our protocols, would be compatible with FFmpeg's
> habits, and not that much work. And libev can be used if available for
> more complex protocols or better performance.
> 
> But as I said, I would be fine with libev the main,
Is that mean I must enable libev if I use ffmpeg, or must enable libev if I use 
libav* API?
Or do you mean I can configure by myself if I don’t want use third part lib in 
ffmpeg?

> official event loop
> for the project.
> 
> Regards,
> 
> -- 
>  Nicolas George
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Thanks

Steven Liu



___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Steven Liu (12021-01-05):
> > But as I said, I would be fine with libev the main,

> Is that mean I must enable libev if I use ffmpeg, or must enable libev
> if I use libav* API? Or do you mean I can configure by myself if I
> don’t want use third part lib in ffmpeg?

If you are referring to this last sentence, it would mean you need the
development files of libev to build ffmpeg.

But the point of this discussion is precisely to decide what is
necessary and what we want exactly.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [RFC] Event loop

[Nicolas George]

Xiang Xiao (12021-01-04):
> Nicolas, thank you for bringing up this topic.
> My team is working on similar functionality: the event loop is the
> central point to integrate protocol, demux/mux, decoder/encoder, avfilter
> and avdevice to form a complete multimedia subsystem(like DirectShow or
> GStreamer). Without the builtin event loop, it is very hard to support it
> with a clean and elegant way outside FFmpeg:
> 
>1. Multiple streaming(either local or network)
>2. Multiple devices(either capture, playback even Bluetooth A2DP/SCO)
>3. Interact and coordinate between the components
>4. Provide a system wide and easy used API to application
> 
> We are very excited that the community is considering to provide the
> generic event loop support and want to make the contribution.

Thank you for the feedback. It confirms that we need this also to make
applications easier.

Note that GStreamer is considered a swear word around here. But I think
the flaw of GStreamer is to be over-engineered, the GNOME way, with more
abstraction layers than actual features, and not the base idea to have a
graph of processing units. The trick is to keep things as simple as
possible while as powerful as needed; I think we can pull it off.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".