Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
From: Guy Harris > On Jan 9, 2015, at 8:30 AM, Michael Richardson wrote: > > > Guy Harris wrote: > >> The longer timeout can reduce capturing overhead, and if you're > >> capturing a high volume of traffic to a file, it's probably the right > >> timeout to have. If you are capturing a high volume of traffic even a short (10ms) timeout won't expire. > >> If, however, you're printing packets to the console, > >> you're probably doomed if it's a high volume of traffic, and may want > >> less of a delay if it's a low volume of traffic. > > > >> Should we reduce the timeout if -w isn't specified - or do so if -w > >> isn't specified *and* if we're outputting to a terminal (isatty(1) > >> returns a non-zero value)? Should we use immediate mode if libpcap > > > > Yes, I think that -w not specified, and isatty()==1. What about piping through 'tee', 'grep' or into a pager? In all those cases you want immediate output (as if directly writing the tty). This also means you need an fflush(stdout) before waiting for more data. Even with -w you can have problems - it is silly to have to wait a significant time between running a test that generates a small number of packets and typing ^C to stop tcpdump. > > OK, I've implemented that for immediate mode, i.e. immediate mode if -w isn't > specified and isatty(1) > is true, and added a --immediate-mode flag so the nerds in the audience have > a knob to tweak. :-) > > If pcap_set_immediate_mode() isn't available, should it set the timeout to a > lower value instead, in > those cases? > > Should we reduce the default timeout? Should we have a command-line flag to > set the timeout? I don't see any point in delaying more than 100ms. Returning to user every 50ms shouldn't be a problem either. David ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
On Jan 9, 2015, at 8:30 AM, Michael Richardson wrote: > Guy Harris wrote: >> The longer timeout can reduce capturing overhead, and if you're >> capturing a high volume of traffic to a file, it's probably the right >> timeout to have. If, however, you're printing packets to the console, >> you're probably doomed if it's a high volume of traffic, and may want >> less of a delay if it's a low volume of traffic. > >> Should we reduce the timeout if -w isn't specified - or do so if -w >> isn't specified *and* if we're outputting to a terminal (isatty(1) >> returns a non-zero value)? Should we use immediate mode if libpcap > > Yes, I think that -w not specified, and isatty()==1. OK, I've implemented that for immediate mode, i.e. immediate mode if -w isn't specified and isatty(1) is true, and added a --immediate-mode flag so the nerds in the audience have a knob to tweak. :-) If pcap_set_immediate_mode() isn't available, should it set the timeout to a lower value instead, in those cases? Should we reduce the default timeout? Should we have a command-line flag to set the timeout? ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
Hello, I am new to this community. And i would like to know from where i should start. Please guide me. Thanking you Bipul kumar On 1/12/15, David Laight wrote: > From: Guy Harris >> On Jan 9, 2015, at 2:09 AM, Michal Sekletar wrote: >> >> > Can't we use new default timeout value (lower) if we detect TPACKET_V3, >> >> The first sentence of my original mail was "With TPACKET_V3 support, Linux >> users are discovering what >> those of us using BSD-flavored OSes have known for quite a while:" >> >> This is not a TPACKET_V3 issue, it's a buffering issue. I notice it when >> testing tcpdump on Macs, >> which don't have PF_PACKET sockets of any sort, they have BPF; if, for >> example, I test on the >> generally-low-traffic loopback interface by pinging 127.0.0.1, the packets >> don't show up continuously, >> they show up in batches, with a 1-second delay. > > Is there any real reason for a delay as long as 1 second? > If the traffic is light (which might mean 100/sec) then processing > every packet separately isn't going to be a problem. > Cleary at very high rates you do want to defer the wakuep. > > So reducing the delay from 1 sec to 100ms (or even 50ms) will have > little effect on the ability to process the received data. > > David > > ___ > tcpdump-workers mailing list > tcpdump-workers@lists.tcpdump.org > https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers > ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
From: Guy Harris > On Jan 9, 2015, at 2:09 AM, Michal Sekletar wrote: > > > Can't we use new default timeout value (lower) if we detect TPACKET_V3, > > The first sentence of my original mail was "With TPACKET_V3 support, Linux > users are discovering what > those of us using BSD-flavored OSes have known for quite a while:" > > This is not a TPACKET_V3 issue, it's a buffering issue. I notice it when > testing tcpdump on Macs, > which don't have PF_PACKET sockets of any sort, they have BPF; if, for > example, I test on the > generally-low-traffic loopback interface by pinging 127.0.0.1, the packets > don't show up continuously, > they show up in batches, with a 1-second delay. Is there any real reason for a delay as long as 1 second? If the traffic is light (which might mean 100/sec) then processing every packet separately isn't going to be a problem. Cleary at very high rates you do want to defer the wakuep. So reducing the delay from 1 sec to 100ms (or even 50ms) will have little effect on the ability to process the received data. David ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
On Jan 9, 2015, at 2:09 AM, Michal Sekletar wrote: > Can't we use new default timeout value (lower) if we detect TPACKET_V3, The first sentence of my original mail was "With TPACKET_V3 support, Linux users are discovering what those of us using BSD-flavored OSes have known for quite a while:" This is not a TPACKET_V3 issue, it's a buffering issue. I notice it when testing tcpdump on Macs, which don't have PF_PACKET sockets of any sort, they have BPF; if, for example, I test on the generally-low-traffic loopback interface by pinging 127.0.0.1, the packets don't show up continuously, they show up in batches, with a 1-second delay. > so we don't > defeat the purpose of buffering entirely but delay is short enough that output > looks realtime-ish to human observer looking at the console. If we are dumping > to savefile we should use higher value. Yes, that's what I was suggesting - have tcpdump only use the 1-second timeout when writing to a file, and use a lower timeout, such as 1/10 second or less, when writing to a terminal, *regardless of what capture mechanism is being used*. That will remove the display delay for BPF-like capture mechanisms (BPF itself - except on AIX, where we use immediate mode due to bugs, TPACKET_V3, possibly the Tru64 UNIX mechanism) and have no effect on capture mechanisms that don't do timeout-based buffering (all other Linux mechanisms, IRIX, BPF on AIX, DLPI on HP-UX). DLPI+bufmod on Solaris, as I remember, does the timeout differently - it's an inter-packet timeout - but I think it might help there. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
Guy Harris wrote: > The longer timeout can reduce capturing overhead, and if you're > capturing a high volume of traffic to a file, it's probably the right > timeout to have. If, however, you're printing packets to the console, > you're probably doomed if it's a high volume of traffic, and may want > less of a delay if it's a low volume of traffic. > Should we reduce the timeout if -w isn't specified - or do so if -w > isn't specified *and* if we're outputting to a terminal (isatty(1) > returns a non-zero value)? Should we use immediate mode if libpcap Yes, I think that -w not specified, and isatty()==1. > supports it? Should we offer a command-line option to override the > default? (I'd prefer to have the default be appropriate for most > cases, which may mean a different default if printing packets or if I think that we should be able to override things. I had hoped to spend time on tcpdump over the holidays, but IETF nomcom work ate my brain. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works| network architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
On Thu, Jan 08, 2015 at 06:57:08PM -0800, Guy Harris wrote: > With TPACKET_V3 support, Linux users are discovering what those of us using > BSD-flavored OSes have known for quite a while: > > > http://askubuntu.com/questions/570885/can-tcpdump-on-ubuntu-14-04-show-packets-in-real-time > > Tcpdump uses a timeout of 1 second when opening a capture device; this can, on > platforms that do BPF-style buffering, cause a 1-second delay before packets > show up. TPACKET_V3 is the first Linux capture mechanism to do BPF-style > buffering, so when a Linux user switches to a distribution version that > introduces a kernel that supports TPACKET_V3 and a libpcap that uses > TPACKET_V3, they'll start seeing delays that they haven't seen before. Is there any rational behind 1s value. Not saying it is wrong, just curious. > > The longer timeout can reduce capturing overhead, and if you're capturing a > high volume of traffic to a file, it's probably the right timeout to have. > If, however, you're printing packets to the console, you're probably doomed if > it's a high volume of traffic, and may want less of a delay if it's a low > volume of traffic. Always there will be cases in which default setting are just not good enough. I think that users doing high volume captures are minority so I'd not optimize defaults for them. > > Should we reduce the timeout if -w isn't specified - or do so if -w isn't > specified *and* if we're outputting to a terminal (isatty(1) returns a > non-zero value)? Should we use immediate mode if libpcap supports it? Should > we offer a command-line option to override the default? (I'd prefer to have > the default be appropriate for most cases, which may mean a different default > if printing packets or if printing them to the console rather than a file, > but, in addition to that, a command-line option to allow more sophisticated > users to tweak the timeout might be useful.) I don't think using immediate mode is the right thing to do, because it defeats the purpose of buffering done by kernel. Can't we use new default timeout value (lower) if we detect TPACKET_V3, so we don't defeat the purpose of buffering entirely but delay is short enough that output looks realtime-ish to human observer looking at the console. If we are dumping to savefile we should use higher value. Michal > ___ > tcpdump-workers mailing list > tcpdump-workers@lists.tcpdump.org > https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
[tcpdump-workers] Libpcap timeout settings in tcpdump - too long when printing to a terminal?
With TPACKET_V3 support, Linux users are discovering what those of us using BSD-flavored OSes have known for quite a while: http://askubuntu.com/questions/570885/can-tcpdump-on-ubuntu-14-04-show-packets-in-real-time Tcpdump uses a timeout of 1 second when opening a capture device; this can, on platforms that do BPF-style buffering, cause a 1-second delay before packets show up. TPACKET_V3 is the first Linux capture mechanism to do BPF-style buffering, so when a Linux user switches to a distribution version that introduces a kernel that supports TPACKET_V3 and a libpcap that uses TPACKET_V3, they'll start seeing delays that they haven't seen before. The longer timeout can reduce capturing overhead, and if you're capturing a high volume of traffic to a file, it's probably the right timeout to have. If, however, you're printing packets to the console, you're probably doomed if it's a high volume of traffic, and may want less of a delay if it's a low volume of traffic. Should we reduce the timeout if -w isn't specified - or do so if -w isn't specified *and* if we're outputting to a terminal (isatty(1) returns a non-zero value)? Should we use immediate mode if libpcap supports it? Should we offer a command-line option to override the default? (I'd prefer to have the default be appropriate for most cases, which may mean a different default if printing packets or if printing them to the console rather than a file, but, in addition to that, a command-line option to allow more sophisticated users to tweak the timeout might be useful.) ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
