Re: [tcpdump-workers] loopback interface and byte order
On Dec 1, 2004, at 3:31 PM, Robert Lowe wrote:
In testing a small app using libpcap, I noticed differences in
behaviour when
using the loopback interface vs. using a hardware interface. In
particular,
it seems the packets coming in over the loopback interface are still
in host
byte order (little endian, in this case).
Packets aren't in any byte order; particular fields in the packet might
be in a particular byte order.
In the link-layer header used with BSD-style loopback interfaces, the
4-byte packet type is, in most BSDs, in host byte order - it's
obviously known that the machines on both side of the connection have
the same byte order :-) - but, at least in newer versions of OpenBSD,
it's in network byte order.
On Linux, the link-layer header is a fake Ethernet header, and the type
field *should* be in network byte order (big-endian).
The IP, ICMP, TCP, and UDP header fields should be in network byte
order.
On what OS are you doing this?
Is that typical, or just an
OS-dependent artifact? In trying to find the answer, I came across a
function
in the pcap library, pcap_datalink(). If I really wanted my app to
work using
the loopback interface, I assume I would use this function to
determine whether
or not to invoke byte order altering functions.
If you really want your app to work on anything other than Ethernet
interfaces, you would use that function to determine what to do with
the link-layer header.
If it returns DLT_EN10MB, it's Ethernet (don't worry about the "10MB",
that's a historical artifact from the days when there was the
experimental 3Mb/s Ethernet and the standard 10Mb/s Ethernet, and they
had different link-layer headers; all Ethernets 10Mb and up have the
same header, and use DLT_EN10MB).
If it returns DLT_NULL, it's BSD-style loopback, with a link-layer
header containing a 4-byte AF_ value in *host* byte order.
If it returns DLT_LOOP, it's BSD-style loopback, with a link-layer
header containing a 4-byte AF_ value in *network* byte order.
Beware - the AF_ value is the value for the OS on which the capture is
being done, and AF_INET6 has different values on {Net,Open}BSD,
FreeBSD, and OS X! If you want to handle save files as well as live
captures, make sure you check for all the AF_INET6 values.
-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.
Re: [tcpdump-workers] loopback interface and byte order
Guy Harris wrote:
On Dec 1, 2004, at 3:31 PM, Robert Lowe wrote:
In testing a small app using libpcap, I noticed differences in
behaviour when
using the loopback interface vs. using a hardware interface. In
particular,
it seems the packets coming in over the loopback interface are still
in host
byte order (little endian, in this case).
Packets aren't in any byte order; particular fields in the packet might
be in a particular byte order.
Right, sorry for that.
In the link-layer header used with BSD-style loopback interfaces, the
4-byte packet type is, in most BSDs, in host byte order - it's obviously
known that the machines on both side of the connection have the same
byte order :-) - but, at least in newer versions of OpenBSD, it's in
network byte order.
On Linux, the link-layer header is a fake Ethernet header, and the type
field *should* be in network byte order (big-endian).
The IP, ICMP, TCP, and UDP header fields should be in network byte order.
Well, I was reporting this from memory. Let me back up a bit. When I first
looked at pcap, I went through Tim Carsten's tutorial, referenced from the
tcpdump.org website. Using that code (sniffer.c) on Linux with a downed eth0
i/f (forcing the dev to any) results in very weird ip src/dst addrs and ports
that look partly like byte swapped information, and partly like wrong offsets.
For example, source might look like 253.103.127.0:1 and dest like
0.1.127.0:32792. It works fine for packets off the wire.
This leads me to another question. I've seen recommendations to roll your
own structs for IP/TCP headers, hinting that there might be slight differences
in the definitions between platforms. Is this really true? I decided to
use the definitions in the provided header files, and when I tested with my
app, it showed correct behaviour in both cases (off the wire and loopback).
But, if portability is an issue, what is the recommended course of action?
On what OS are you doing this?
Linux, FC2 (2.6.5-1.358) and FC3 (can't retrieve right now), in particular.
But, I'd like this app to build and run on BSD-derived systems and Solaris.
Is that typical, or just an
OS-dependent artifact? In trying to find the answer, I came across a
function
in the pcap library, pcap_datalink(). If I really wanted my app to
work using
the loopback interface, I assume I would use this function to
determine whether
or not to invoke byte order altering functions.
If you really want your app to work on anything other than Ethernet
interfaces, you would use that function to determine what to do with the
link-layer header.
If it returns DLT_EN10MB, it's Ethernet (don't worry about the "10MB",
that's a historical artifact from the days when there was the
experimental 3Mb/s Ethernet and the standard 10Mb/s Ethernet, and they
had different link-layer headers; all Ethernets 10Mb and up have the
same header, and use DLT_EN10MB).
If it returns DLT_NULL, it's BSD-style loopback, with a link-layer
header containing a 4-byte AF_ value in *host* byte order.
If it returns DLT_LOOP, it's BSD-style loopback, with a link-layer
header containing a 4-byte AF_ value in *network* byte order.
Beware - the AF_ value is the value for the OS on which the capture is
being done, and AF_INET6 has different values on {Net,Open}BSD, FreeBSD,
and OS X! If you want to handle save files as well as live captures,
make sure you check for all the AF_INET6 values.
Thank you for the information! I don't think I'll bother trying this
now, but I certainly will at some point.
-Robert
-
This is the tcpdump-workers list.
Visit https://lists.sandelman.ca/ to unsubscribe.
Re: [tcpdump-workers] loopback interface and byte order
Robert Lowe wrote: Well, I was reporting this from memory. Let me back up a bit. When I first looked at pcap, I went through Tim Carsten's tutorial, referenced from the tcpdump.org website. Using that code (sniffer.c) on Linux with a downed eth0 i/f (forcing the dev to any) results in very weird ip src/dst addrs and ports that look partly like byte swapped information, and partly like wrong offsets. For example, source might look like 253.103.127.0:1 and dest like 0.1.127.0:32792. Is "sniffer.c" calling "pcap_datalink()", checking for DLT_LINUX_SLL, and processing a link-layer header that looks like what's described in recent versions of the libpcap man page? I suspect it's not, which means it won't properly handle captures on the "any" device - the link-layer header you get in that case doesn't look like an Ethernet link-layer header, and isn't 14 bytes long, so if your program is assuming a 14-byte link-layer header, it will *not* work correctly - what it's reporting as the source and destination IP addresses are other parts of the packet. I.e., to quote my previous message: If you really want your app to work on anything other than Ethernet interfaces, you would use that function to determine what to do with the link-layer header. where "that function" is "pcap_datalink()". Yes, that could be a bit messy, but you're stuck with it. See tcpdump for an example of how to handle this. This leads me to another question. I've seen recommendations to roll your own structs for IP/TCP headers, hinting that there might be slight differences in the definitions between platforms. Is this really true? Yes. For example, if I remember correctly, not all platforms' "struct ip" represent the "header length and version" field in the same way - some have bitfields and some have a single field for the header length and version and macros to extract the header length and version from it. That's why tcpdump has its own headers. I decided to use the definitions in the provided header files, and when I tested with my app, it showed correct behaviour in both cases (off the wire and loopback). But, if portability is an issue, what is the recommended course of action? Use your own definitions for all packet data structures. On what OS are you doing this? Linux, FC2 (2.6.5-1.358) and FC3 (can't retrieve right now), in particular. But, I'd like this app to build and run on BSD-derived systems and Solaris. Then you'd better use "pcap_datalink()" to determine the link-layer header - and you'd better not depend on the "any" device working, as that's Linux-specific, and you'd better not depend on being able to capture on the loopback device, because Solaris doesn't support that. If you really want your app to work on anything other than Ethernet interfaces, you would use that function to determine what to do with the link-layer header. ... Thank you for the information! I don't think I'll bother trying this now, Then you'd better not expect your program to work on anything other than a device that produces Ethernet headers - which means it won't work on the "any" device. Note also that the "any" device doesn't use DLT_LOOP or DLT_NULL headers, so you won't need to handle them if you're capturing on the "any" device - but you will need to handle them if you're going to capture on loopback devices on BSD-derived systems. but I certainly will at some point. Only when you do that stuff will it work on the "any" device. Note that your program should print an error message if "pcap_datalink()" returns a DLT_ value that it can't handle. - This is the tcpdump-workers list. Visit https://lists.sandelman.ca/ to unsubscribe.
Re: [tcpdump-workers] loopback interface and byte order
Guy Harris wrote: This leads me to another question. I've seen recommendations to roll your own structs for IP/TCP headers, hinting that there might be slight differences in the definitions between platforms. Is this really true? Yes. For example, if I remember correctly, not all platforms' "struct ip" represent the "header length and version" field in the same way - some have bitfields and some have a single field for the header length and version and macros to extract the header length and version from it. That's why tcpdump has its own headers. Thanks again for all the terrific information. I looked over the headers included with tcpdump. I think I'll use somewhat modified versions of those. In particular, I didn't traceback how u_int8_t gets defined, which seems to appear naturally on some systems, but not on others. I think instead I'll rely on an autoconf macro, ac_create_stdint_h, to make a header file for me and use uint8_t, for example, instead. Reasonable? Is there a better approach? -Robert - This is the tcpdump-workers list. Visit https://lists.sandelman.ca/ to unsubscribe.
