[1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Austin Group Bug Tracker via austin-group-l at The Open Group]

The following issue has been CLOSED. 
== 
https://austingroupbugs.net/view.php?id=1165 
== 
Reported By:EdSchouten
Assigned To:ajosey
== 
Project:1003.1(2008)/Issue 7
Issue ID:   1165
Category:   Base Definitions and Headers
Type:   Enhancement Request
Severity:   Editorial
Priority:   normal
Status: Closed
Name:   Ed Schouten 
Organization:
User Reference:  
Section:aio.h, sys/select.h, poll.h 
Page Number:n/a 
Line Number:n/a 
Interp Status:  --- 
Final Accepted Text: 
Resolution: Rejected
Fixed in Version:   
== 
Date Submitted: 2017-10-22 10:35 UTC
Last Modified:  2022-07-18 16:02 UTC
== 
Summary:What is the working group's plan for stateful
polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?
== 

-- 
 (0005896) Don Cragun (manager) - 2022-07-18 16:02
 https://austingroupbugs.net/view.php?id=1165#c5896 
-- 
Since no full proposal has been submitted, this bug is rejected.  If a full
proposal is available at a later date, please file a new bug with the
details. 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2017-10-22 10:35 EdSchouten New Issue
2017-10-22 10:35 EdSchouten Status   New => Under Review 
2017-10-22 10:35 EdSchouten Assigned To   => ajosey  
2017-10-22 10:35 EdSchouten Name  => Ed Schouten 
2017-10-22 10:35 EdSchouten Section   => aio.h,
sys/select.h, poll.h
2017-10-22 10:35 EdSchouten Page Number   => n/a 
2017-10-22 10:35 EdSchouten Line Number   => n/a 
2017-10-22 10:37 EdSchouten Description Updated  
2018-02-25 22:23 tn Issue Monitored: tn  
2019-01-28 16:59 nick   Note Added: 0004233  
2019-01-28 18:26 EdSchouten Note Added: 0004234  
2022-07-18 16:02 Don Cragun Interp Status => --- 
2022-07-18 16:02 Don Cragun Note Added: 0005896  
2022-07-18 16:02 Don Cragun Status   Under Review => Closed
2022-07-18 16:02 Don Cragun Resolution   Open => Rejected
==

[1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Austin Group Bug Tracker]

A NOTE has been added to this issue. 
== 
http://austingroupbugs.net/view.php?id=1165 
== 
Reported By:EdSchouten
Assigned To:ajosey
== 
Project:1003.1(2008)/Issue 7
Issue ID:   1165
Category:   Base Definitions and Headers
Type:   Enhancement Request
Severity:   Editorial
Priority:   normal
Status: Under Review
Name:   Ed Schouten 
Organization:
User Reference:  
Section:aio.h, sys/select.h, poll.h 
Page Number:n/a 
Line Number:n/a 
Interp Status:  --- 
Final Accepted Text: 
== 
Date Submitted: 2017-10-22 10:35 UTC
Last Modified:  2019-01-28 18:26 UTC
== 
Summary:What is the working group's plan for stateful
polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?
== 

-- 
 (0004234) EdSchouten (updater) - 2019-01-28 18:26
 http://austingroupbugs.net/view.php?id=1165#c4234 
-- 
Considering that libuv is the most popular implementation, counting GitHub
watchers/stargazers, I filed this issue:
https://github.com/libuv/libuv/issues/2164

Will contact the libev/libevent folks in case libuv declines. 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2017-10-22 10:35 EdSchouten New Issue
2017-10-22 10:35 EdSchouten Status   New => Under Review 
2017-10-22 10:35 EdSchouten Assigned To   => ajosey  
2017-10-22 10:35 EdSchouten Name  => Ed Schouten 
2017-10-22 10:35 EdSchouten Section   => aio.h,
sys/select.h, poll.h
2017-10-22 10:35 EdSchouten Page Number   => n/a 
2017-10-22 10:35 EdSchouten Line Number   => n/a 
2017-10-22 10:37 EdSchouten Description Updated  
2018-02-25 22:23 tn Issue Monitored: tn  
2019-01-28 16:59 nick   Note Added: 0004233  
2019-01-28 18:26 EdSchouten Note Added: 0004234  
==

[1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Austin Group Bug Tracker]

A NOTE has been added to this issue. 
== 
http://austingroupbugs.net/view.php?id=1165 
== 
Reported By:EdSchouten
Assigned To:ajosey
== 
Project:1003.1(2008)/Issue 7
Issue ID:   1165
Category:   Base Definitions and Headers
Type:   Enhancement Request
Severity:   Editorial
Priority:   normal
Status: Under Review
Name:   Ed Schouten 
Organization:
User Reference:  
Section:aio.h, sys/select.h, poll.h 
Page Number:n/a 
Line Number:n/a 
Interp Status:  --- 
Final Accepted Text: 
== 
Date Submitted: 2017-10-22 10:35 UTC
Last Modified:  2019-01-28 16:59 UTC
== 
Summary:What is the working group's plan for stateful
polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?
== 

-- 
 (0004233) nick (manager) - 2019-01-28 16:59
 http://austingroupbugs.net/view.php?id=1165#c4233 
-- 
The Austin Group would welcome a fully formed proposal along these lines,
including support from the current maintainers of whatever package is
selected. See also https://www.opengroup.org/austin/docs/austin_sd6.txt for
guidelines on submitting new material. 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2017-10-22 10:35 EdSchouten New Issue
2017-10-22 10:35 EdSchouten Status   New => Under Review 
2017-10-22 10:35 EdSchouten Assigned To   => ajosey  
2017-10-22 10:35 EdSchouten Name  => Ed Schouten 
2017-10-22 10:35 EdSchouten Section   => aio.h,
sys/select.h, poll.h
2017-10-22 10:35 EdSchouten Page Number   => n/a 
2017-10-22 10:35 EdSchouten Line Number   => n/a 
2017-10-22 10:37 EdSchouten Description Updated  
2018-02-25 22:23 tn Issue Monitored: tn  
2019-01-28 16:59 nick   Note Added: 0004233  
==

Re: [1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Joerg Schilling]

Stephane Chazelas  wrote:

> 2017-10-24 11:31:08 +0200, Joerg Schilling:
> [...]
> > Given that there is no standard for a VFS interface and the fact that the 
> > Linux 
> > VFS implementation is more than sub-optimal (see the discussions about 
> > reiserfs 
> > and Linux) it would be a bad idea to standardize something that is based on 
> > a 
> > Linux kernel implementation detail.
> [...]
>
> Could you please clarify what discussion you're referring to and
> in which way it is relevant to this discussion?

Well, there was a discussion because reiserfs bypasses the Linux VFS

The main argument is that such an interface would depend on the VFS interface 
lyer amd the VFS interface on Linux is Linux specific and limited.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Stephane Chazelas]

2017-10-24 11:31:08 +0200, Joerg Schilling:
[...]
> Given that there is no standard for a VFS interface and the fact that the 
> Linux 
> VFS implementation is more than sub-optimal (see the discussions about 
> reiserfs 
> and Linux) it would be a bad idea to standardize something that is based on a 
> Linux kernel implementation detail.
[...]

Could you please clarify what discussion you're referring to and
in which way it is relevant to this discussion?

I don't remember seeing a reference to reiserfs on the austin
group ml recently
(https://www.mail-archive.com/search?q=reiserfs=austin-group-l%40opengroup.org
returns nothing at all), and I can't find anything at
https://www.mail-archive.com/search?a=1=all=reiserfs+linux=10=18===1m=2017-10-24==newest
that stands out from a cursory look as being the discussion
you're referring to.

AFAIK, reiserfs is only available on Linux, so all reiserfs
discussions will be about Linux as well.

IMO, such unsubstantiated claims are not useful to the
discussion however true they may be.

Thanks,
Stephane

Re: [1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Joerg Schilling]

Steffen Nurpmeso  wrote:

>  |This is why I propose that POSIX goes into another direction: standardise
>  |an event loop. By doing this, we're not only avoiding the entire discussion
>
> What a terrible idea!  You even want to take away the possibility
> to implement the event loop that i need, if any!
>
>  |about polling frameworks, we're also creating an ecosystem where people can
>  |write portable libraries that can easily be scheduled within the same event
>  |loop, which is awesome. It also gives operating systems the ability to
>
> No it is not.  How long does it take to write a shallow layer that
> works on epoll(2) or kevent(2) the way you need it.  But
> standardizing a fixed event requires myriads of possibilities for
> configuration possibilities of this intransparent black box of
> a thing.  Just look how many people actually use the terrible
> black box posix_spawn(): how many, i am asking you?

Given that there is no standard for a VFS interface and the fact that the Linux 
VFS implementation is more than sub-optimal (see the discussions about reiserfs 
and Linux) it would be a bad idea to standardize something that is based on a 
Linux kernel implementation detail.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

[1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Austin Group Bug Tracker]

The following issue has been UPDATED. 
== 
http://austingroupbugs.net/view.php?id=1165 
== 
Reported By:EdSchouten
Assigned To:ajosey
== 
Project:1003.1(2008)/Issue 7
Issue ID:   1165
Category:   Base Definitions and Headers
Type:   Enhancement Request
Severity:   Editorial
Priority:   normal
Status: Under Review
Name:   Ed Schouten 
Organization:
User Reference:  
Section:aio.h, sys/select.h, poll.h 
Page Number:n/a 
Line Number:n/a 
Interp Status:  --- 
Final Accepted Text: 
== 
Date Submitted: 2017-10-22 10:35 UTC
Last Modified:  2017-10-22 10:37 UTC
== 
Summary:What is the working group's plan for stateful
polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?
== 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2017-10-22 10:35 EdSchouten New Issue
2017-10-22 10:35 EdSchouten Status   New => Under Review 
2017-10-22 10:35 EdSchouten Assigned To   => ajosey  
2017-10-22 10:35 EdSchouten Name  => Ed Schouten 
2017-10-22 10:35 EdSchouten Section   => aio.h,
sys/select.h, poll.h
2017-10-22 10:35 EdSchouten Page Number   => n/a 
2017-10-22 10:35 EdSchouten Line Number   => n/a 
2017-10-22 10:37 EdSchouten Description Updated  
==

[1003.1(2008)/Issue 7 0001165]: What is the working group's plan for stateful polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?

[Austin Group Bug Tracker]

The following issue has been SUBMITTED. 
== 
http://austingroupbugs.net/view.php?id=1165 
== 
Reported By:EdSchouten
Assigned To:ajosey
== 
Project:1003.1(2008)/Issue 7
Issue ID:   1165
Category:   Base Definitions and Headers
Type:   Enhancement Request
Severity:   Editorial
Priority:   normal
Status: Under Review
Name:   Ed Schouten 
Organization:
User Reference:  
Section:aio.h, sys/select.h, poll.h 
Page Number:n/a 
Line Number:n/a 
Interp Status:  --- 
Final Accepted Text: 
== 
Date Submitted: 2017-10-22 10:35 UTC
Last Modified:  2017-10-22 10:35 UTC
== 
Summary:What is the working group's plan for stateful
polling (e.g., kqueue, epoll), event loops (libuv) and coroutines (C++2a)?
Description: 
Over the last couple of years we've seen the use of event-driven
programming on UNIX-like systems increase, especially when looking outside
of the C ecosystem. For example, the Node.js framework allows you to design
fully event-driven applications in Javascript. The upcoming revision of the
C++ standard, currently referred to as C++2a, will likely contain support
for coroutines. As the Coroutines TS has already been implemented in MSVC
and Clang, people are already building awesome things on top of it.

Though these frameworks are already out there and work well, practice has
shown that the relevant APIs we have in POSIX (aio, poll, select, etc) are
insufficient to implement them. They scale badly as the number of
registered events increases (see
https://en.wikipedia.org/wiki/C10k_problem). This is why most POSIX-like
operating systems provide additional polling interfaces: BSD kqueue, Linux
epoll, Solaris event ports, etc. I think this clearly demonstrates a
disconnect between what's in the standard and what people want.

The most naïve thing we could do at this point is standardise one of the
polling interfaces mentioned above. Let the Linux folks implement kqueue,
or let the fine people at Oracle implement epoll, we all know that this
approach simply won't get any traction. Apart from technical
(dis)advantages of any of these interfaces over the other, there is likely
too much pride and history involved. Another issue with this approach is
that the polling interfaces are still fairly low-level. They are a building
block, but don't facilitate event-driven programming directly.

This is why I propose that POSIX goes into another direction: standardise
an event loop. By doing this, we're not only avoiding the entire discussion
about polling frameworks, we're also creating an ecosystem where people can
write portable libraries that can easily be scheduled within the same event
loop, which is awesome. It also gives operating systems the ability to
redesign, simplify and optimise their polling frameworks without breaking
existing applications.

Right now there are (at least) three mature event loops written in C in
use, in chronological order: libevent, libev and libuv. The latter has the
advantage that its API has been designed in such a way that it can also be
implemented on non-UNIX systems (e.g., Windows), which is why it's used by
many modern projects like Node.js, GRPC, etc. It also has a relatively
compact, well documented API and a very healthy development community.
Desired Action: 
Initially, I would like to use this bug report to explore the options.
First of all, I would love to hear the working group's opinion on this
matter. Do they see things the same way?

After that, there are two different things we could do: design our very own
event loop or standardise an existing one. I think the latter makes most
sense, personally preferring libuv. Once an event loop is chosen, we should
try to open a dialog with the maintainers of that respective event loop.
== 

Issue History 
Date ModifiedUsername   FieldChange   
== 
2017-10-22 10:35 EdSchouten New Issue
2017-10-22 10:35 EdSchouten Status   New => Under Review 
2017-10-22 10:35 EdSchouten Assigned To   => ajosey  
2017-10-22 10:35 EdSchouten Name  => Ed Schouten 
2017-10-22 10:35 EdSchouten Section   =>