Re: New Address Family: Inter Process Networking (IPN)

2007-12-10 Thread Chris Friesen
David Miller wrote: The kernel supports much more than 32 groups, see nlk->groups which is a bitmap which can be sized to arbitrary sizes. nlk->nl_groups is for backwards compatability only. netlink_change_ngroups() does the bitmap resizing when necessary. Thanks for the explanation. Given

Re: New Address Family: Inter Process Networking (IPN)

2007-12-10 Thread Chris Friesen
David Miller wrote: The kernel supports much more than 32 groups, see nlk-groups which is a bitmap which can be sized to arbitrary sizes. nlk-nl_groups is for backwards compatability only. netlink_change_ngroups() does the bitmap resizing when necessary. Thanks for the explanation. Given

Re: New Address Family: Inter Process Networking (IPN)

2007-12-07 Thread Andi Kleen
> Stop making excuses, with minor adjustments we have the facilities to > meet your needs. There is no need for yet-another-protocol to do what I suspect they would be better of just using IP multicast. But the localhost latency penalty vs Unix Chris was talking about probably needs to be

Re: New Address Family: Inter Process Networking (IPN)

2007-12-07 Thread Andi Kleen
Stop making excuses, with minor adjustments we have the facilities to meet your needs. There is no need for yet-another-protocol to do what I suspect they would be better of just using IP multicast. But the localhost latency penalty vs Unix Chris was talking about probably needs to be

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Miller
From: "Chris Friesen" <[EMAIL PROTECTED]> Date: Thu, 06 Dec 2007 22:21:39 -0600 > David Miller wrote: > > From: "Chris Friesen" <[EMAIL PROTECTED]> > > Date: Thu, 06 Dec 2007 14:36:54 -0600 > > > > > >>One problem we ran into was that there are only 32 multicast groups per > >>netlink protocol

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Ben Pfaff
"Chris Friesen" <[EMAIL PROTECTED]> writes: > David Miller wrote: >> From: "Chris Friesen" <[EMAIL PROTECTED]> >>> One problem we ran into was that there are only 32 multicast groups >>> per netlink protocol family. >> I'm pretty sure we've removed this limitation. > As of 2.6.23 nl_groups is a

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
David Miller wrote: From: "Chris Friesen" <[EMAIL PROTECTED]> Date: Thu, 06 Dec 2007 14:36:54 -0600 One problem we ran into was that there are only 32 multicast groups per netlink protocol family. I'm pretty sure we've removed this limitation. As of 2.6.23 nl_groups is a 32-bit bitmask

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Miller
From: "Chris Friesen" <[EMAIL PROTECTED]> Date: Thu, 06 Dec 2007 14:36:54 -0600 > One problem we ran into was that there are only 32 multicast groups per > netlink protocol family. I'm pretty sure we've removed this limitation. -- To unsubscribe from this list: send the line "unsubscribe

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Renzo Davoli
I have done some raw tests. (you can read the code here: http://www.cs.unibo.it/~renzo/rawperftest/) The programs are quite simple. The sender sends "Hello World" as fast as it can, while the receiver prints time() for each 1 million message received. On my laptop, tests on 2000 "Hello

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: On Thu, Dec 06, 2007 at 05:02:40PM -0600, Chris Friesen wrote: I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency increase. Sounds like something that should be looked into. I know of no principal reasons for that. For stream sockets, unix gives

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 05:02:40PM -0600, Chris Friesen wrote: > Andi Kleen wrote: > > >>On a 1.4GHz P4 I measured a 44% increase in latency between a unix > >>datagram and a UDP datagram. > > >That's weird. > > I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency >

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: On a 1.4GHz P4 I measured a 44% increase in latency between a unix datagram and a UDP datagram. That's weird. I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency increase. For stream sockets, unix gives approximately a 62% bandwidth increase over

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
> Renzo's IPN is a local protocol--you can't multicast to localhost. You don't need to. All local clients can join the same group without using localhost. -Andi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 11:18:37PM +0100, Renzo Davoli wrote: > * IPN is for inter-process communication. It is *not* directly related > to TCP-IP or Ethernet. > > If you want you can call it Inter Process Bus Communication. It is an > extension of AF_UNIX. Comments saying that some services

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Newall
Andi Kleen wrote: Renzo also described something new (in the socket() arena): the multi-reader, multi-writer is just not available in IP. How is that different from a multicast group? Good question. I don't know much about multicast IP. It's a bit new for me. I knew it uses

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Renzo Davoli
Some more explanations trying to describe what IPN is and what it is useful for. We are writing the complete patch Summary: * IPN is for inter-process communication. It is *not* directly related to TCP-IP or Ethernet. * IPN itself is a *level 1* virtual physical network. IPN services *

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 03:49:51PM -0600, Chris Friesen wrote: > Andi Kleen wrote: > >>Latency was very > >>important, so we ended up doing essentially a multicast unix socket > >>rather than taking the extra penalty for UDP multicast. > > > > > >What extra penalty? Local UDP shouldn't be much

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: Latency was very important, so we ended up doing essentially a multicast unix socket rather than taking the extra penalty for UDP multicast. What extra penalty? Local UDP shouldn't be much more expensive than Unix. On a 1.4GHz P4 I measured a 44% increase in latency

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
> Latency was very > important, so we ended up doing essentially a multicast unix socket > rather than taking the extra penalty for UDP multicast. What extra penalty? Local UDP shouldn't be much more expensive than Unix. -Andi -- To unsubscribe from this list: send the line "unsubscribe

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: "This document describes Linux Netlink, which is used in Linux both as an intra-kernel messaging system as well as between kernel and user space." It can be used between user space daemons as well. In fact it is. e.g. they often listen to each other's messages. One

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
> Renzo also described something new (in the socket() arena): the > multi-reader, multi-writer is just not available in IP. How is that different from a multicast group? -Andi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
> "This document describes Linux Netlink, which is used in Linux both as >an intra-kernel messaging system as well as between kernel and user >space." It can be used between user space daemons as well. In fact it is. e.g. they often listen to each other's messages. -Andi -- To

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
Renzo also described something new (in the socket() arena): the multi-reader, multi-writer is just not available in IP. How is that different from a multicast group? -Andi -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED]

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
This document describes Linux Netlink, which is used in Linux both as an intra-kernel messaging system as well as between kernel and user space. It can be used between user space daemons as well. In fact it is. e.g. they often listen to each other's messages. -Andi -- To unsubscribe

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: This document describes Linux Netlink, which is used in Linux both as an intra-kernel messaging system as well as between kernel and user space. It can be used between user space daemons as well. In fact it is. e.g. they often listen to each other's messages. One

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
Latency was very important, so we ended up doing essentially a multicast unix socket rather than taking the extra penalty for UDP multicast. What extra penalty? Local UDP shouldn't be much more expensive than Unix. -Andi -- To unsubscribe from this list: send the line unsubscribe

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: Latency was very important, so we ended up doing essentially a multicast unix socket rather than taking the extra penalty for UDP multicast. What extra penalty? Local UDP shouldn't be much more expensive than Unix. On a 1.4GHz P4 I measured a 44% increase in latency

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Renzo Davoli
Some more explanations trying to describe what IPN is and what it is useful for. We are writing the complete patch Summary: * IPN is for inter-process communication. It is *not* directly related to TCP-IP or Ethernet. * IPN itself is a *level 1* virtual physical network. IPN services *

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Newall
Andi Kleen wrote: Renzo also described something new (in the socket() arena): the multi-reader, multi-writer is just not available in IP. How is that different from a multicast group? Good question. I don't know much about multicast IP. It's a bit new for me. I knew it uses

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 03:49:51PM -0600, Chris Friesen wrote: Andi Kleen wrote: Latency was very important, so we ended up doing essentially a multicast unix socket rather than taking the extra penalty for UDP multicast. What extra penalty? Local UDP shouldn't be much more expensive

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 11:18:37PM +0100, Renzo Davoli wrote: * IPN is for inter-process communication. It is *not* directly related to TCP-IP or Ethernet. If you want you can call it Inter Process Bus Communication. It is an extension of AF_UNIX. Comments saying that some services can be

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
Renzo's IPN is a local protocol--you can't multicast to localhost. You don't need to. All local clients can join the same group without using localhost. -Andi -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: On a 1.4GHz P4 I measured a 44% increase in latency between a unix datagram and a UDP datagram. That's weird. I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency increase. For stream sockets, unix gives approximately a 62% bandwidth increase over

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Andi Kleen
On Thu, Dec 06, 2007 at 05:02:40PM -0600, Chris Friesen wrote: Andi Kleen wrote: On a 1.4GHz P4 I measured a 44% increase in latency between a unix datagram and a UDP datagram. That's weird. I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency increase. Sounds

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
Andi Kleen wrote: On Thu, Dec 06, 2007 at 05:02:40PM -0600, Chris Friesen wrote: I just reran on a 3.2GHZ P4 running 2.6.11 (Fedora Core 4). 42% latency increase. Sounds like something that should be looked into. I know of no principal reasons for that. For stream sockets, unix gives

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Renzo Davoli
I have done some raw tests. (you can read the code here: http://www.cs.unibo.it/~renzo/rawperftest/) The programs are quite simple. The sender sends Hello World as fast as it can, while the receiver prints time() for each 1 million message received. On my laptop, tests on 2000 Hello World

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Miller
From: Chris Friesen [EMAIL PROTECTED] Date: Thu, 06 Dec 2007 14:36:54 -0600 One problem we ran into was that there are only 32 multicast groups per netlink protocol family. I'm pretty sure we've removed this limitation. -- To unsubscribe from this list: send the line unsubscribe linux-kernel

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Chris Friesen
David Miller wrote: From: Chris Friesen [EMAIL PROTECTED] Date: Thu, 06 Dec 2007 14:36:54 -0600 One problem we ran into was that there are only 32 multicast groups per netlink protocol family. I'm pretty sure we've removed this limitation. As of 2.6.23 nl_groups is a 32-bit bitmask with

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread Ben Pfaff
Chris Friesen [EMAIL PROTECTED] writes: David Miller wrote: From: Chris Friesen [EMAIL PROTECTED] One problem we ran into was that there are only 32 multicast groups per netlink protocol family. I'm pretty sure we've removed this limitation. As of 2.6.23 nl_groups is a 32-bit bitmask with

Re: New Address Family: Inter Process Networking (IPN)

2007-12-06 Thread David Miller
From: Chris Friesen [EMAIL PROTECTED] Date: Thu, 06 Dec 2007 22:21:39 -0600 David Miller wrote: From: Chris Friesen [EMAIL PROTECTED] Date: Thu, 06 Dec 2007 14:36:54 -0600 One problem we ran into was that there are only 32 multicast groups per netlink protocol family. I'm

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread David Newall
Kyle Moffett wrote: On Dec 06, 2007, at 00:30:16, Renzo Davoli wrote: AF_IPN is different. AF_IPN is the broadcast and peer-to-peer extension of AF_UNIX. It supports communication among *user* processes. Ok, you say it's different, but then you describe how IP unicast and broadcast work.

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Kyle Moffett
On Dec 06, 2007, at 00:30:16, Renzo Davoli wrote: AF_IPN is different. AF_IPN is the broadcast and peer-to-peer extension of AF_UNIX. It supports communication among *user* processes. Ok, you say it's different, but then you describe how IP unicast and broadcast work. Both are

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Stephen Hemminger
On Thu, 6 Dec 2007 06:38:21 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: > On Wed, Dec 05, 2007 at 04:55:52PM -0500, Stephen Hemminger wrote: > > On Wed, 5 Dec 2007 17:40:55 +0100 > > [EMAIL PROTECTED] (Renzo Davoli) wrote: > > > 0- (Constructive) comments. > > > 1- The "official" assignment of

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
> In the meanwhile we would be grateful if the community could kindly ask to the > questions above. Obviously I meant: In the meanwhile we would be grateful if the community could kindly *answer* to the questions above sorry (it is early morning here, it happens ;-) renzo -- To

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
On Wed, Dec 05, 2007 at 04:55:52PM -0500, Stephen Hemminger wrote: > On Wed, 5 Dec 2007 17:40:55 +0100 > [EMAIL PROTECTED] (Renzo Davoli) wrote: > > 0- (Constructive) comments. > > 1- The "official" assignment of an Address Family. > > 2- Another "grabbing hook" for interfaces (like the ones

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
On Thu, Dec 06, 2007 at 12:39:22AM +0100, Andi Kleen wrote: > [EMAIL PROTECTED] (Renzo Davoli) writes: > > > Berkeley socket have been designed for client server or point to point > > communication. All existing Address Families implement this idea. > Netlink is multicast/broadcast by default for

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Andi Kleen
[EMAIL PROTECTED] (Renzo Davoli) writes: > Berkeley socket have been designed for client server or point to point > communication. All existing Address Families implement this idea. Netlink is multicast/broadcast by default for once. And BC/MC certainly works for IPv[46] and a couple of other

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Stephen Hemminger
On Wed, 5 Dec 2007 17:40:55 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: > > WHAT WE NEED FROM THE LINUX KERNEL COMMUNITY > > 0- (Constructive) comments. > > 1- The "official" assignment of an Address Family. > (It is enough for everything but

New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
Inter Process Networking: a kernel module (and some simple kernel patches) to provide AF_IPN: a new address family for process networking, i.e. multipoint, multicast/broadcast communication among processes (and networks). WHAT IS IT? --- Berkeley socket have been designed for client

New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
Inter Process Networking: a kernel module (and some simple kernel patches) to provide AF_IPN: a new address family for process networking, i.e. multipoint, multicast/broadcast communication among processes (and networks). WHAT IS IT? --- Berkeley socket have been designed for client

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Andi Kleen
[EMAIL PROTECTED] (Renzo Davoli) writes: Berkeley socket have been designed for client server or point to point communication. All existing Address Families implement this idea. Netlink is multicast/broadcast by default for once. And BC/MC certainly works for IPv[46] and a couple of other

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Stephen Hemminger
On Wed, 5 Dec 2007 17:40:55 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: WHAT WE NEED FROM THE LINUX KERNEL COMMUNITY 0- (Constructive) comments. 1- The official assignment of an Address Family. (It is enough for everything but interface

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
On Thu, Dec 06, 2007 at 12:39:22AM +0100, Andi Kleen wrote: [EMAIL PROTECTED] (Renzo Davoli) writes: Berkeley socket have been designed for client server or point to point communication. All existing Address Families implement this idea. Netlink is multicast/broadcast by default for once.

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
On Wed, Dec 05, 2007 at 04:55:52PM -0500, Stephen Hemminger wrote: On Wed, 5 Dec 2007 17:40:55 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: 0- (Constructive) comments. 1- The official assignment of an Address Family. 2- Another grabbing hook for interfaces (like the ones already We are

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Renzo Davoli
In the meanwhile we would be grateful if the community could kindly ask to the questions above. Obviously I meant: In the meanwhile we would be grateful if the community could kindly *answer* to the questions above sorry (it is early morning here, it happens ;-) renzo -- To unsubscribe

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Stephen Hemminger
On Thu, 6 Dec 2007 06:38:21 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: On Wed, Dec 05, 2007 at 04:55:52PM -0500, Stephen Hemminger wrote: On Wed, 5 Dec 2007 17:40:55 +0100 [EMAIL PROTECTED] (Renzo Davoli) wrote: 0- (Constructive) comments. 1- The official assignment of an Address

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread Kyle Moffett
On Dec 06, 2007, at 00:30:16, Renzo Davoli wrote: AF_IPN is different. AF_IPN is the broadcast and peer-to-peer extension of AF_UNIX. It supports communication among *user* processes. Ok, you say it's different, but then you describe how IP unicast and broadcast work. Both are

Re: New Address Family: Inter Process Networking (IPN)

2007-12-05 Thread David Newall
Kyle Moffett wrote: On Dec 06, 2007, at 00:30:16, Renzo Davoli wrote: AF_IPN is different. AF_IPN is the broadcast and peer-to-peer extension of AF_UNIX. It supports communication among *user* processes. Ok, you say it's different, but then you describe how IP unicast and broadcast work.