Re: [Taps] User-Space Networking in iOS 11

[Tommy Pauly]

Hello Zhen,

Wi-Fi was just the example interface in the slides. We are using the user-space 
networking stack on Wi-Fi, Cellular, and some VPN interfaces. You’re correct 
that there are different driver requirements for these, but they look the same 
to user-space, which is only forming IP datagrams (not down to link layer 
headers).

With regards to performance, some of the main benefits are seen around batching 
of packets through the interface to a process, and in being able to process 
those batches through the entire stack at once. That means that certain traffic 
patterns do benefit more than others. In terms of user effect, this can 
translate into improved throughput for CPU-bound loads, or CPU usage reduction 
for network-bound loads. Of course, we will continue to tune and optimize this 
technology going forward, so stay tuned in the future.

Thanks,
Tommy

> On Jul 3, 2017, at 4:30 PM, Zhen Cao  wrote:
> 
> Hello Tommy,
> 
> Thank you for sharing this.  Going through the slides of WWDC17, I
> have some clarification questions: a) the slides only plot WiFi when
> talking about the 'user-space', does that also apply to the cellular
> link?  I ask this because the user-space packet io needs a different
> driver for different links as far as i know.  b) does the user space
> networking always performs better than the kernel counterpart?
> 
> Many thanks
> Zhen
> 
> On Fri, Jun 9, 2017 at 3:50 AM, Tommy Pauly  wrote:
>> Hello,
>> 
>> I wanted to point the TAPS group to some of the work that we announced this
>> week at WWDC that relates to the Post-Sockets API effort. You can see a
>> video of the session here (relevant section at ~13:50), along with the
>> slides:
>> 
>> https://developer.apple.com/videos/play/wwdc2017/707
>> 
>> In the current betas of iOS 11, we have introduced “User-Space Networking”
>> beneath our networking APIs. The transport and IP protocols are now being
>> co-located with the security and application protocols in the process,
>> meaning that we are no longer using sockets within the implementation of
>> these APIs. This shift allows us to reduce the context switches between
>> protocol layers, and could potentially open opportunities for the kind of
>> stack flexibility and customization that the TAPS group is looking at. We’re
>> excited to be making some first steps into a truly “Post-Sockets” world!
>> 
>> Thanks,
>> Tommy
>> 
>> ___
>> Taps mailing list
>> Taps@ietf.org
>> https://www.ietf.org/mailman/listinfo/taps
>> 
> 
> ___
> Taps mailing list
> Taps@ietf.org
> https://www.ietf.org/mailman/listinfo/taps

___
Taps mailing list
Taps@ietf.org
https://www.ietf.org/mailman/listinfo/taps

Re: [Taps] User-Space Networking in iOS 11

[Zhen Cao]

Hello Tommy,

Thank you for sharing this.  Going through the slides of WWDC17, I
have some clarification questions: a) the slides only plot WiFi when
talking about the 'user-space', does that also apply to the cellular
link?  I ask this because the user-space packet io needs a different
driver for different links as far as i know.  b) does the user space
networking always performs better than the kernel counterpart?

Many thanks
Zhen

On Fri, Jun 9, 2017 at 3:50 AM, Tommy Pauly  wrote:
> Hello,
>
> I wanted to point the TAPS group to some of the work that we announced this
> week at WWDC that relates to the Post-Sockets API effort. You can see a
> video of the session here (relevant section at ~13:50), along with the
> slides:
>
> https://developer.apple.com/videos/play/wwdc2017/707
>
> In the current betas of iOS 11, we have introduced “User-Space Networking”
> beneath our networking APIs. The transport and IP protocols are now being
> co-located with the security and application protocols in the process,
> meaning that we are no longer using sockets within the implementation of
> these APIs. This shift allows us to reduce the context switches between
> protocol layers, and could potentially open opportunities for the kind of
> stack flexibility and customization that the TAPS group is looking at. We’re
> excited to be making some first steps into a truly “Post-Sockets” world!
>
> Thanks,
> Tommy
>
> ___
> Taps mailing list
> Taps@ietf.org
> https://www.ietf.org/mailman/listinfo/taps
>

___
Taps mailing list
Taps@ietf.org
https://www.ietf.org/mailman/listinfo/taps

Re: [Taps] User-Space Networking in iOS 11

[Tommy Pauly]

> On Jun 9, 2017, at 10:04 AM, Mikael Abrahamsson  wrote:
> 
> On Fri, 9 Jun 2017, Tommy Pauly wrote:
> 
>> The kernel is still responsible for managing the port namespace between 
>> applications, and demultiplexing incoming packets to the correct processes. 
>> Since that is part of the kernel, the port demultiplexing still needs to 
>> understand where to look for port numbers in known protocols. It seems like 
>> for the use-case you’re mentioning for new protocols, it could be useful to 
>> allow that logic to be extended, or ‘learn’ new protocols. This isn’t 
>> something we’re doing now, but definitely an interesting idea!
> 
> Also sounds like a use-case for the work on "device should be handed multiple 
> addresses". If it were, you could multiple in a way that each process that 
> needed to have its own port space or run special protocols, could get unique 
> usage of a newly allocated IPv6 address.
> 
> If this is something that would be useful, it would be good if this use-case 
> was brought to 6man and v6ops. It would probably be more credible if you 
> brought it than if I did.

Hi Mikael,

A per-process IPv6 address is not something we’re doing at this time, but it’s 
certainly a natural extension of an in-process networking stack. If the device 
is being allocated multiple IPv6 address, and ideally a /64, as recommended by 
RFC 7934, then the host system could assign an address to each process, or at 
least each process that has beyond-ordinary networking needs.

This could also have interesting implications for a device that’s dealing with 
multiple Provisioning Domains (PvDs). Processes could be grouped to share the 
address/port namespaces of a given PvD; for example, enterprise applications 
could be using a common address and set of ports, while personal applications 
could be sharing another set.

The main open question I would have would be around how an application would 
express that it should be using a separate address-space, or how that would be 
indicated as some sort of TAPS parameters. Any ideas are welcome =)

Thanks,
Tommy

> 
> -- 
> Mikael Abrahamssonemail: swm...@swm.pp.se

___
Taps mailing list
Taps@ietf.org
https://www.ietf.org/mailman/listinfo/taps

Re: [Taps] User-Space Networking in iOS 11

[Tommy Pauly]

> On Jun 9, 2017, at 4:55 PM, Michael Tuexen  
> wrote:
> 
>> On 9. Jun 2017, at 12:16, Tommy Pauly  wrote:
>> 
>> HI Michael,
>> 
>>> On Jun 8, 2017, at 7:13 PM, Michael Tuexen 
>>>  wrote:
>>> 
>>> On 8. Jun 2017, at 15:50, Tommy Pauly  wrote:
 
 Hello,
 
 I wanted to point the TAPS group to some of the work that we announced 
 this week at WWDC that relates to the Post-Sockets API effort. You can see 
 a video of the session here (relevant section at ~13:50), along with the 
 slides:
 
 https://developer.apple.com/videos/play/wwdc2017/707
 
 In the current betas of iOS 11, we have introduced “User-Space Networking” 
 beneath our networking APIs. The transport and IP protocols are now being 
 co-located with the security and application protocols in the process, 
 meaning that we are no longer using sockets within the implementation of 
 these APIs. This shift allows us to reduce the context switches between 
 protocol layers, and could potentially open opportunities for the kind of 
 stack flexibility and customization that the TAPS group is looking at. 
 We’re excited to be making some first steps into a truly “Post-Sockets” 
 world!
>>> Hi Tommy,
>>> 
>>> so this means I can run my own TCP stack and my own IPv6 stack in my 
>>> application, if I understood that correctly.
>>> How is the port number space managed between the different applications? 
>>> I'm wondering how the demultiplexing is
>>> done?
>>> Can I also use the API for a transport protocol supporting the port number 
>>> concept, but not being UDP or TCP,
>>> like SCTP or UDP-Lite or DCCP or whatever might come up in the future?
>> 
>> The ability to run your own stack is not currently exposed; the user-space 
>> stack for TCP, etc, is part of the system library. However, this 
>> architecture does open up the possibility to extend or modify this in the 
>> future. If people have specific ways that they think this would be most 
>> useful to extend and modify, please let me know.
>> 
>> The kernel is still responsible for managing the port namespace between 
>> applications, and demultiplexing incoming packets to the correct processes. 
>> Since that is part of the kernel, the port demultiplexing still needs to 
>> understand where to look for port numbers in known protocols. It seems like 
>> for the use-case you’re mentioning for new protocols, it could be useful to 
>> allow that logic to be extended, or ‘learn’ new protocols. This isn’t 
>> something we’re doing now, but definitely an interesting idea!
> Leaving the port multiplexing / demultiplexing at the kernel makes a lot of 
> sense. Are you doing something
> like:
> * open a TCP socket.
> * bind it.
> * set a socket option that you want to send/recv raw TCP packets
> That way you could pass the packets up and down to a userland stack and could
> coexist with a kernel stack. (Assuming that the kernel/userland boundary 
> still uses sockets).
> However, that doesn't fit the figure shown a WWDC, where you had the IP stack 
> and the TCP
> as part of the userland stack. So does the application the route selection, 
> interface
> selection?

The figure shown on the slides shows the TCP/IP stack in the process-space 
since the actual formation of the TCP and IP headers is done there, along with 
all of the TCP state machine logic and IP fragment reassembly, etc. The 
kernel/user boundary is not using sockets at all for these connections, but 
instead is using a new mechanism for memory-mapping to the kernel. The 
application can influence the route and interface selection with preferences 
(no-cellular, for example), which are combined with the kernel policies and 
routing tables, as in the Policy Context in the Post-Sockets draft. This 
provides a binding for ports and interface, which is used for subsequent 
traffic flows and is validated by the kernel on egress. Effectively, this means 
that the IP part of the stack in user space can be thinner, since the routing 
information is more fixed and doesn’t require more routing lookups as is 
typically done in a kernel implementation.

> 
> Is there any documentation available for the interface between kernel land 
> and userland?

No, not yet at this time. This is the initial roll-out of the architecture, so 
it’s still early days.

> Is the same architecture available on Mac OS X?

We do not currently expose the same functionality on macOS. As mentioned in the 
talk, this is in part due to the fact that it is not compatible with network 
kernel extensions, which are common on macOS, but will be deprecated at some 
point in the future.

Thanks,
Tommy

> 
> Best regards
> Michael
>> 
>> Thanks,
>> Tommy
>> 
>>> 
>>> Best regards
>>> Michael
 
 Thanks,
 Tommy
 ___
 Taps mailing list
 

Re: [Taps] User-Space Networking in iOS 11

[Michael Tuexen]

> On 9. Jun 2017, at 12:16, Tommy Pauly  wrote:
> 
> HI Michael,
> 
>> On Jun 8, 2017, at 7:13 PM, Michael Tuexen 
>>  wrote:
>> 
>> On 8. Jun 2017, at 15:50, Tommy Pauly  wrote:
>>> 
>>> Hello,
>>> 
>>> I wanted to point the TAPS group to some of the work that we announced this 
>>> week at WWDC that relates to the Post-Sockets API effort. You can see a 
>>> video of the session here (relevant section at ~13:50), along with the 
>>> slides:
>>> 
>>> https://developer.apple.com/videos/play/wwdc2017/707
>>> 
>>> In the current betas of iOS 11, we have introduced “User-Space Networking” 
>>> beneath our networking APIs. The transport and IP protocols are now being 
>>> co-located with the security and application protocols in the process, 
>>> meaning that we are no longer using sockets within the implementation of 
>>> these APIs. This shift allows us to reduce the context switches between 
>>> protocol layers, and could potentially open opportunities for the kind of 
>>> stack flexibility and customization that the TAPS group is looking at. 
>>> We’re excited to be making some first steps into a truly “Post-Sockets” 
>>> world!
>> Hi Tommy,
>> 
>> so this means I can run my own TCP stack and my own IPv6 stack in my 
>> application, if I understood that correctly.
>> How is the port number space managed between the different applications? I'm 
>> wondering how the demultiplexing is
>> done?
>> Can I also use the API for a transport protocol supporting the port number 
>> concept, but not being UDP or TCP,
>> like SCTP or UDP-Lite or DCCP or whatever might come up in the future?
> 
> The ability to run your own stack is not currently exposed; the user-space 
> stack for TCP, etc, is part of the system library. However, this architecture 
> does open up the possibility to extend or modify this in the future. If 
> people have specific ways that they think this would be most useful to extend 
> and modify, please let me know.
> 
> The kernel is still responsible for managing the port namespace between 
> applications, and demultiplexing incoming packets to the correct processes. 
> Since that is part of the kernel, the port demultiplexing still needs to 
> understand where to look for port numbers in known protocols. It seems like 
> for the use-case you’re mentioning for new protocols, it could be useful to 
> allow that logic to be extended, or ‘learn’ new protocols. This isn’t 
> something we’re doing now, but definitely an interesting idea!
Leaving the port multiplexing / demultiplexing at the kernel makes a lot of 
sense. Are you doing something
like:
* open a TCP socket.
* bind it.
* set a socket option that you want to send/recv raw TCP packets
That way you could pass the packets up and down to a userland stack and could
coexist with a kernel stack. (Assuming that the kernel/userland boundary still 
uses sockets).
However, that doesn't fit the figure shown a WWDC, where you had the IP stack 
and the TCP
as part of the userland stack. So does the application the route selection, 
interface
selection?

Is there any documentation available for the interface between kernel land and 
userland?
Is the same architecture available on Mac OS X?

Best regards
Michael
> 
> Thanks,
> Tommy
> 
>> 
>> Best regards
>> Michael
>>> 
>>> Thanks,
>>> Tommy
>>> ___
>>> Taps mailing list
>>> Taps@ietf.org
>>> https://www.ietf.org/mailman/listinfo/taps
>> 
>> ___
>> Taps mailing list
>> Taps@ietf.org
>> https://www.ietf.org/mailman/listinfo/taps
> 

___
Taps mailing list
Taps@ietf.org
https://www.ietf.org/mailman/listinfo/taps

Re: [Taps] User-Space Networking in iOS 11

[Mikael Abrahamsson]

On Fri, 9 Jun 2017, Tommy Pauly wrote:

The kernel is still responsible for managing the port namespace between 
applications, and demultiplexing incoming packets to the correct 
processes. Since that is part of the kernel, the port demultiplexing 
still needs to understand where to look for port numbers in known 
protocols. It seems like for the use-case you’re mentioning for new 
protocols, it could be useful to allow that logic to be extended, or 
‘learn’ new protocols. This isn’t something we’re doing now, but 
definitely an interesting idea!


Also sounds like a use-case for the work on "device should be handed 
multiple addresses". If it were, you could multiple in a way that each 
process that needed to have its own port space or run special protocols, 
could get unique usage of a newly allocated IPv6 address.


If this is something that would be useful, it would be good if this 
use-case was brought to 6man and v6ops. It would probably be more credible 
if you brought it than if I did.


--
Mikael Abrahamssonemail: swm...@swm.pp.se___
Taps mailing list
Taps@ietf.org
https://www.ietf.org/mailman/listinfo/taps