[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
On 25/07/14 10:43, Thomas Monjalon wrote: >> On 24/07/14 07:59, Matthew Hall wrote: >>> I ran into some weird symbol conflicts between system netinet/in.h and DPDK >>> rte_ip.h. They have a lot of duplicated definitions for stuff like >>> IPPROTO_IP >>> and so on. This breaks when you want to use inet_pton from arpa/inet.h, >>> because it includes netinet/in.h to define struct in_addr. > [...] >> Again, I recommend steering away from any tightrope approaches that >> "know" which types are non-conflicting, or pick out half-and-half from >> the host and IP stack. "Do, or do not, there is no half-and-half" > > The general problem here is that DPDK is conflicting with libc. > So the obvious question would be: "why DPDK needs to redefine libc stuff"? > I don't see any obvious answer since bare metal is planned to be removed. > (see http://dpdk.org/ml/archives/dev/2014-June/003868.html) One reason is if you want DPDK to be a portable network programming environment. Especially in that case you do not want definitions based on hackish assumptions of some particular version of some particular host implementation. However, I'm not trying to argue if DPDK should or shouldn't be that, just that you should either dramatically improve the current implementation or nuke it.
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
Hi all, 2014-07-24 22:55, Antti Kantee: > On 24/07/14 07:59, Matthew Hall wrote: > > I ran into some weird symbol conflicts between system netinet/in.h and DPDK > > rte_ip.h. They have a lot of duplicated definitions for stuff like > > IPPROTO_IP > > and so on. This breaks when you want to use inet_pton from arpa/inet.h, > > because it includes netinet/in.h to define struct in_addr. [...] > Again, I recommend steering away from any tightrope approaches that > "know" which types are non-conflicting, or pick out half-and-half from > the host and IP stack. "Do, or do not, there is no half-and-half" The general problem here is that DPDK is conflicting with libc. So the obvious question would be: "why DPDK needs to redefine libc stuff"? I don't see any obvious answer since bare metal is planned to be removed. (see http://dpdk.org/ml/archives/dev/2014-June/003868.html) Objections? -- Thomas
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
> > I don't know if it will work right on both Linux and BSD and/or if they also > 100% agree with the libc / glibc values compiled into the > system's .so files. That's the risk that you run if you don't have more > complete support in the DPDK itself for these features. Looking at linux and freebsd netinet/in.h files - I think it should work. But I suppose we can test it on both freebsd and linux before submitting a patch. > -- > Sent from my mobile device. > > On July 24, 2014 6:12:18 PM PDT, "Wu, Jingjing" > wrote: > >Hello, > > > >We also notice these conflicts, we just planned to fix it in our new > >feature development. The proposal is like: > > > >#ifndef _NETINET_IN_H > >#ifndef _NETINET_IN_H_ > > > >#define IPPROTO_IP 0 > > ... ... > >#define IPPROTO_MAX 256 > > > >#endif > >#endif > > > >Do you think it is a good idea? > > > >> -Original Message- > >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Antti Kantee > >> Sent: Friday, July 25, 2014 6:56 AM > >> To: Matthew Hall; dev at dpdk.org > >> Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h, > >arpa/inet.h, and rte_ip.h > >> > >> On 24/07/14 07:59, Matthew Hall wrote: > >> > Hello, > >> > > >> > I ran into some weird symbol conflicts between system netinet/in.h > >and DPDK > >> > rte_ip.h. They have a lot of duplicated definitions for stuff like > >IPPROTO_IP > >> > and so on. This breaks when you want to use inet_pton from > >arpa/inet.h, > >> > because it includes netinet/in.h to define struct in_addr. > >> > >> I would namespace the definitions in DPDK, i.e. make them > >> DPDK_IPPROTO_FOO etc. > >> > >> > Thus with all the conflicts it's impossible to use a DPDK IP struct > >instead of > >> > all the system's sockaddr stuff, to store a value from the system > >copy of > >> > inet_pton. This would be a common operation if, for example, you > >want to > >> > configure all the IP addresses on your box from a JSON file, which > >is what I > >> > was doing. > >> > > >> > The DPDK kludged around it internally by using a file called > >> > cmdline_parse_ipaddr.c with private copies of these functions. But > >it in my > >> > opinion very unwisely marked all of the functions as static except > >for > >> > cmdline_parse_ipaddr, which only works on the DPDK's proprietary > >argument > >> > handling, and not with anything the user might have which is a > >different > >> > format. > >> > >> In my experience from years of fighting with more or less this exact > >> same problem -- the fight is now thankfully over but the scars remain > >-- > >> you either want to expose a complete set of types and provide support > >> for everything, or you want to expose nothing. Approaches where you > >use > >> cute definitions and reuse some host routines is like asking for an > >> audience with Tyranthraxus when armed with a kitten. It's that > >doubly > >> so if you don't have to and do it anyway. > >> > >> > So, it would be a big help for users if the macros in librte_net > >files would > >> > check if the symbols already existed, or if they had subheader > >files available > >> > to grab only non conflicting symbols, or if they would make a > >proper .h and > >> > factor all the inet_pton and inet_ntop inside the cmdline lib into > >a place > >> > where users can access them. It would also be a help if they had a > >less ugly > >> > equivalent to struct sockaddr, which let you work with IP addresses > >a bit more > >> > easily, such as something like this: > >> > >> Again, I recommend steering away from any tightrope approaches that > >> "know" which types are non-conflicting, or pick out half-and-half > >from > >> the host and IP stack. "Do, or do not, there is no half-and-half"
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
If the bare metal mode is getting yanked, then I think we can go with Antti's advice and just yank the conflicting symbols and use the system versions. -- Sent from my mobile device. On July 25, 2014 7:40:02 AM PDT, Antti Kantee wrote: >On 25/07/14 10:43, Thomas Monjalon wrote: >>> On 24/07/14 07:59, Matthew Hall wrote: I ran into some weird symbol conflicts between system netinet/in.h >and DPDK rte_ip.h. They have a lot of duplicated definitions for stuff like >IPPROTO_IP and so on. This breaks when you want to use inet_pton from >arpa/inet.h, because it includes netinet/in.h to define struct in_addr. >> [...] >>> Again, I recommend steering away from any tightrope approaches that >>> "know" which types are non-conflicting, or pick out half-and-half >from >>> the host and IP stack. "Do, or do not, there is no half-and-half" >> >> The general problem here is that DPDK is conflicting with libc. >> So the obvious question would be: "why DPDK needs to redefine libc >stuff"? >> I don't see any obvious answer since bare metal is planned to be >removed. >> (see http://dpdk.org/ml/archives/dev/2014-June/003868.html) > >One reason is if you want DPDK to be a portable network programming >environment. Especially in that case you do not want definitions based > >on hackish assumptions of some particular version of some particular >host implementation. However, I'm not trying to argue if DPDK should >or >shouldn't be that, just that you should either dramatically improve the > >current implementation or nuke it.
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
Hello, We also notice these conflicts, we just planned to fix it in our new feature development. The proposal is like: #ifndef _NETINET_IN_H #ifndef _NETINET_IN_H_ #define IPPROTO_IP 0 ... ... #define IPPROTO_MAX 256 #endif #endif Do you think it is a good idea? > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Antti Kantee > Sent: Friday, July 25, 2014 6:56 AM > To: Matthew Hall; dev at dpdk.org > Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, > and rte_ip.h > > On 24/07/14 07:59, Matthew Hall wrote: > > Hello, > > > > I ran into some weird symbol conflicts between system netinet/in.h and DPDK > > rte_ip.h. They have a lot of duplicated definitions for stuff like > > IPPROTO_IP > > and so on. This breaks when you want to use inet_pton from arpa/inet.h, > > because it includes netinet/in.h to define struct in_addr. > > I would namespace the definitions in DPDK, i.e. make them > DPDK_IPPROTO_FOO etc. > > > Thus with all the conflicts it's impossible to use a DPDK IP struct instead > > of > > all the system's sockaddr stuff, to store a value from the system copy of > > inet_pton. This would be a common operation if, for example, you want to > > configure all the IP addresses on your box from a JSON file, which is what I > > was doing. > > > > The DPDK kludged around it internally by using a file called > > cmdline_parse_ipaddr.c with private copies of these functions. But it in my > > opinion very unwisely marked all of the functions as static except for > > cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument > > handling, and not with anything the user might have which is a different > > format. > > In my experience from years of fighting with more or less this exact > same problem -- the fight is now thankfully over but the scars remain -- > you either want to expose a complete set of types and provide support > for everything, or you want to expose nothing. Approaches where you use > cute definitions and reuse some host routines is like asking for an > audience with Tyranthraxus when armed with a kitten. It's that doubly > so if you don't have to and do it anyway. > > > So, it would be a big help for users if the macros in librte_net files would > > check if the symbols already existed, or if they had subheader files > > available > > to grab only non conflicting symbols, or if they would make a proper .h and > > factor all the inet_pton and inet_ntop inside the cmdline lib into a place > > where users can access them. It would also be a help if they had a less ugly > > equivalent to struct sockaddr, which let you work with IP addresses a bit > > more > > easily, such as something like this: > > Again, I recommend steering away from any tightrope approaches that > "know" which types are non-conflicting, or pick out half-and-half from > the host and IP stack. "Do, or do not, there is no half-and-half"
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
On 24/07/14 07:59, Matthew Hall wrote: > Hello, > > I ran into some weird symbol conflicts between system netinet/in.h and DPDK > rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP > and so on. This breaks when you want to use inet_pton from arpa/inet.h, > because it includes netinet/in.h to define struct in_addr. I would namespace the definitions in DPDK, i.e. make them DPDK_IPPROTO_FOO etc. > Thus with all the conflicts it's impossible to use a DPDK IP struct instead of > all the system's sockaddr stuff, to store a value from the system copy of > inet_pton. This would be a common operation if, for example, you want to > configure all the IP addresses on your box from a JSON file, which is what I > was doing. > > The DPDK kludged around it internally by using a file called > cmdline_parse_ipaddr.c with private copies of these functions. But it in my > opinion very unwisely marked all of the functions as static except for > cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument > handling, and not with anything the user might have which is a different > format. In my experience from years of fighting with more or less this exact same problem -- the fight is now thankfully over but the scars remain -- you either want to expose a complete set of types and provide support for everything, or you want to expose nothing. Approaches where you use cute definitions and reuse some host routines is like asking for an audience with Tyranthraxus when armed with a kitten. It's that doubly so if you don't have to and do it anyway. > So, it would be a big help for users if the macros in librte_net files would > check if the symbols already existed, or if they had subheader files available > to grab only non conflicting symbols, or if they would make a proper .h and > factor all the inet_pton and inet_ntop inside the cmdline lib into a place > where users can access them. It would also be a help if they had a less ugly > equivalent to struct sockaddr, which let you work with IP addresses a bit more > easily, such as something like this: Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half from the host and IP stack. "Do, or do not, there is no half-and-half"
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
I don't know if it will work right on both Linux and BSD and/or if they also 100% agree with the libc / glibc values compiled into the system's .so files. That's the risk that you run if you don't have more complete support in the DPDK itself for these features. -- Sent from my mobile device. On July 24, 2014 6:12:18 PM PDT, "Wu, Jingjing" wrote: >Hello, > >We also notice these conflicts, we just planned to fix it in our new >feature development. The proposal is like: > >#ifndef _NETINET_IN_H >#ifndef _NETINET_IN_H_ > >#define IPPROTO_IP 0 > ... ... >#define IPPROTO_MAX 256 > >#endif >#endif > >Do you think it is a good idea? > >> -Original Message- >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Antti Kantee >> Sent: Friday, July 25, 2014 6:56 AM >> To: Matthew Hall; dev at dpdk.org >> Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h, >arpa/inet.h, and rte_ip.h >> >> On 24/07/14 07:59, Matthew Hall wrote: >> > Hello, >> > >> > I ran into some weird symbol conflicts between system netinet/in.h >and DPDK >> > rte_ip.h. They have a lot of duplicated definitions for stuff like >IPPROTO_IP >> > and so on. This breaks when you want to use inet_pton from >arpa/inet.h, >> > because it includes netinet/in.h to define struct in_addr. >> >> I would namespace the definitions in DPDK, i.e. make them >> DPDK_IPPROTO_FOO etc. >> >> > Thus with all the conflicts it's impossible to use a DPDK IP struct >instead of >> > all the system's sockaddr stuff, to store a value from the system >copy of >> > inet_pton. This would be a common operation if, for example, you >want to >> > configure all the IP addresses on your box from a JSON file, which >is what I >> > was doing. >> > >> > The DPDK kludged around it internally by using a file called >> > cmdline_parse_ipaddr.c with private copies of these functions. But >it in my >> > opinion very unwisely marked all of the functions as static except >for >> > cmdline_parse_ipaddr, which only works on the DPDK's proprietary >argument >> > handling, and not with anything the user might have which is a >different >> > format. >> >> In my experience from years of fighting with more or less this exact >> same problem -- the fight is now thankfully over but the scars remain >-- >> you either want to expose a complete set of types and provide support >> for everything, or you want to expose nothing. Approaches where you >use >> cute definitions and reuse some host routines is like asking for an >> audience with Tyranthraxus when armed with a kitten. It's that >doubly >> so if you don't have to and do it anyway. >> >> > So, it would be a big help for users if the macros in librte_net >files would >> > check if the symbols already existed, or if they had subheader >files available >> > to grab only non conflicting symbols, or if they would make a >proper .h and >> > factor all the inet_pton and inet_ntop inside the cmdline lib into >a place >> > where users can access them. It would also be a help if they had a >less ugly >> > equivalent to struct sockaddr, which let you work with IP addresses >a bit more >> > easily, such as something like this: >> >> Again, I recommend steering away from any tightrope approaches that >> "know" which types are non-conflicting, or pick out half-and-half >from >> the host and IP stack. "Do, or do not, there is no half-and-half"
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
On Thu, Jul 24, 2014 at 10:55:59PM +, Antti Kantee wrote: > In my experience from years of fighting with more or less this exact same > problem -- the fight is now thankfully over but the scars remain -- you > either want to expose a complete set of types and provide support for > everything, or you want to expose nothing. I don't have a problem with this approach. Implementing it would require the DPDK to create user accessible functions for creating MAC addresses, IPs, CIDRs, and TCP / UDP port numbers. Many of the things required are hiding inside of *cmdline* code where it's impossible for anybody to access them. Matthew.
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
I also noticed this problem. It is a serious one. I would like it solved.
-- Niraj Sharma
Principal Eng.
Cisco System, Inc.
On 7/24/14 12:59 AM, "Matthew Hall" wrote:
>Hello,
>
>I ran into some weird symbol conflicts between system netinet/in.h and
>DPDK
>rte_ip.h. They have a lot of duplicated definitions for stuff like
>IPPROTO_IP
>and so on. This breaks when you want to use inet_pton from arpa/inet.h,
>because it includes netinet/in.h to define struct in_addr.
>
>Thus with all the conflicts it's impossible to use a DPDK IP struct
>instead of
>all the system's sockaddr stuff, to store a value from the system copy of
>inet_pton. This would be a common operation if, for example, you want to
>configure all the IP addresses on your box from a JSON file, which is
>what I
>was doing.
>
>The DPDK kludged around it internally by using a file called
>cmdline_parse_ipaddr.c with private copies of these functions. But it in
>my
>opinion very unwisely marked all of the functions as static except for
>cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument
>handling, and not with anything the user might have which is a different
>format.
>
>So, it would be a big help for users if the macros in librte_net files
>would
>check if the symbols already existed, or if they had subheader files
>available
>to grab only non conflicting symbols, or if they would make a proper .h
>and
>factor all the inet_pton and inet_ntop inside the cmdline lib into a
>place
>where users can access them. It would also be a help if they had a less
>ugly
>equivalent to struct sockaddr, which let you work with IP addresses a bit
>more
>easily, such as something like this:
>
>struct ip4_addr {
>uint32_t addr;
>};
>
>typedef struct ip4_addr ip4_addr;
>
>struct ip6_addr {
>uint8_t addr[16];
>};
>
>typedef struct ip6_addr ip6_addr;
>
>struct ip_addr {
>uint8_t family;
>uint8_t prefix;
>union {
>struct ip4_addr ipv4;
>struct ip6_addr ipv6;
>};
>};
>
>I had to create a bunch of duplicate code to handle it in my project,
>since
>the DPDK marked its copies of all these functions as "secret" and didn't
>make
>a .h for them. If any of it is useful I am happy to donate it, although I
>don't think I've got quite enough experience with this specifc part of
>the
>DPDK to code it up all by myself.
>
>Thanks,
>Matthew.
[dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h
Hello,
I ran into some weird symbol conflicts between system netinet/in.h and DPDK
rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP
and so on. This breaks when you want to use inet_pton from arpa/inet.h,
because it includes netinet/in.h to define struct in_addr.
Thus with all the conflicts it's impossible to use a DPDK IP struct instead of
all the system's sockaddr stuff, to store a value from the system copy of
inet_pton. This would be a common operation if, for example, you want to
configure all the IP addresses on your box from a JSON file, which is what I
was doing.
The DPDK kludged around it internally by using a file called
cmdline_parse_ipaddr.c with private copies of these functions. But it in my
opinion very unwisely marked all of the functions as static except for
cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument
handling, and not with anything the user might have which is a different
format.
So, it would be a big help for users if the macros in librte_net files would
check if the symbols already existed, or if they had subheader files available
to grab only non conflicting symbols, or if they would make a proper .h and
factor all the inet_pton and inet_ntop inside the cmdline lib into a place
where users can access them. It would also be a help if they had a less ugly
equivalent to struct sockaddr, which let you work with IP addresses a bit more
easily, such as something like this:
struct ip4_addr {
uint32_t addr;
};
typedef struct ip4_addr ip4_addr;
struct ip6_addr {
uint8_t addr[16];
};
typedef struct ip6_addr ip6_addr;
struct ip_addr {
uint8_t family;
uint8_t prefix;
union {
struct ip4_addr ipv4;
struct ip6_addr ipv6;
};
};
I had to create a bunch of duplicate code to handle it in my project, since
the DPDK marked its copies of all these functions as "secret" and didn't make
a .h for them. If any of it is useful I am happy to donate it, although I
don't think I've got quite enough experience with this specifc part of the
DPDK to code it up all by myself.
Thanks,
Matthew.
