[Wireshark-dev] Delays in real-time packet capture
Has anybody else noticed, that when capturing in realtime with a filter
that means that packets aren't captured very often, the last couple of
packets to be captured get 'stuck' until some more data comes along?
Sorry for the terrible explanation. It might be easier if I point to the
offending code:
In capture_loop.c, we have the following (simplified slightly):
> inpkts = capture_loop_dispatch(capture_opts, &ld, errmsg, sizeof(errmsg));
>
> /* Only update once a second (Win32: 500ms) so as not to overload slow
> displays */
> cur_time = TIME_GET();
> if (cur_time - upd_time > 0) {
> upd_time = cur_time;
>
> /* Let the parent process know. */
> if (inpkts_to_sync_pipe) {
> /* do sync here */
> libpcap_dump_flush(ld.pdh, NULL);
So the problem is that that if the packet count isn't flushed out just
after a packet is read, it then isn't written until another packet
arrives - which may not be for some time!
One workaround for this is to #define MUST_DO_SELECT, so that
capture_loop_dispatch uses a select() to wait for packets to turn up,
and put a second's timeout on the select() - but i'm not sure how
portable this is likely to be. Anyone got any better ideas?
Thanks,
Richard
--
Richard van der Hoff <[EMAIL PROTECTED]>
Project Manager
Tel: +44 (0) 845 666 7778
http://www.mxtelecom.com
___
Wireshark-dev mailing list
Wireshark-dev@wireshark.org
http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Richard van der Hoff wrote: > One workaround for this is to #define MUST_DO_SELECT, so that > capture_loop_dispatch uses a select() to wait for packets to turn up, > and put a second's timeout on the select() - but i'm not sure how > portable this is likely to be. It's likely to be quite non-portable, given that 1) select() on Windows can't be used in that fashion and 2) select() on at least some BSD-derived OSes doesn't work on BPF devices. > Anyone got any better ideas? I'm not sure the display update rate limiting belongs in dumpcap; it might be better to do it in Wireshark. I.e., have dumpcap send packet count updates immediately to Wireshark, and have Wireshark decide when to read the new packets. ___ Wireshark-dev mailing list Wireshark-dev@wireshark.org http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Guy Harris wrote: > Richard van der Hoff wrote: > >> One workaround for this is to #define MUST_DO_SELECT, so that >> capture_loop_dispatch uses a select() to wait for packets to turn up, >> and put a second's timeout on the select() - but i'm not sure how >> portable this is likely to be. > > It's likely to be quite non-portable... Yes. I suspected as much. >> Anyone got any better ideas? > > I'm not sure the display update rate limiting belongs in dumpcap; it > might be better to do it in Wireshark. I.e., have dumpcap send packet > count updates immediately to Wireshark, and have Wireshark decide when > to read the new packets. The only concern with this would be that dumpcap would presumably then send a packet count after every packet - which might mean a significant quantity of data. -- Richard van der Hoff <[EMAIL PROTECTED]> Project Manager Tel: +44 (0) 845 666 7778 http://www.mxtelecom.com ___ Wireshark-dev mailing list Wireshark-dev@wireshark.org http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Richard van der Hoff wrote: > The only concern with this would be that dumpcap would presumably then > send a packet count after every packet - which might mean a significant > quantity of data. Every batch of packets, at least. Some OSes support packet batching, so that a single read from the capture device can supply multiple packets; those OSes support a timeout, so that the read either waits for a full batch or for a timer to expire, so you don't wait indefinitely for a full batch. (Note that this timer is *NOT* necessarily started when the read starts; the timer on SunOS 5.x, for example, starts when the first packet arrives. I.e., this timer does *NOT* guarantee that a read will wait no longer than the timeout, and will return at that point if no packets have arrived; that does *NOT* happen on SunOS 5.x.) BSDs (including OS X) support that in BPF (AIX's BPF doesn't work correctly, so the timeout doesn't work and isn't used). As indicated, SunOS 3.x/4.x/5.x support it, too, as do Digital/Tru64 Unix and Windows with WinPcap. Linux, however, doesn't support batching, at least not in PF_PACKET sockets. ___ Wireshark-dev mailing list Wireshark-dev@wireshark.org http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Sorry for taking a while to get back to this... Guy Harris wrote: > Richard van der Hoff wrote: > >> The only concern with this would be that dumpcap would presumably then >> send a packet count after every packet - which might mean a significant >> quantity of data. > > Every batch of packets, at least. > > Some OSes support packet batching, so that a single read from the > capture device can supply multiple packets; those OSes support a > timeout, so that the read either waits for a full batch or for a timer > to expire, so you don't wait indefinitely for a full batch. > > ... > Linux, however, doesn't support batching, at least not in PF_PACKET sockets. So perhaps the ideal would be to use select() to timeout on OSes which support it (ie, Linux); batching on OSes which support that but not select() on a socket (SunOS, Digital/Tru64, WinPcap, etc?), and send an update for every packet on those that support neither (are there any?). Or is this overcomplicated and we should just send an update every packet if the OS doesn't support batching? Regards, Richard -- Richard van der Hoff <[EMAIL PROTECTED]> Project Manager Tel: +44 (0) 845 666 7778 http://www.mxtelecom.com ___ Wireshark-dev mailing list Wireshark-dev@wireshark.org http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Maybe the problems lies here (capture_opts.c) :
static gboolean capture_opts_output_to_pipe(const char *save_file, gboolean
*is_pipe)
{
int err;
if (save_file != NULL) {
/* We're writing to a capture file. */
if (strcmp(save_file, "-") == 0) {
/* Writing to stdout. */
/* XXX - should we check whether it's a pipe? It's arguably
silly to do "-w - >output_file" rather than "-w output_file",
but by not checking we might be violating the Principle Of
Least Astonishment. */
*is_pipe = TRUE;
} else {
/* not a capture file, test for a FIFO (aka named pipe) */
err = capture_opts_test_for_fifo(save_file);
switch (err) {
case ENOENT: /* it doesn't exist, so we'll be creating it,
and it won't be a FIFO */
case 0: /* found it, but it's not a FIFO */
break;
case ESPIPE: /* it is a FIFO */
*is_pipe = TRUE;
break;
default: /* couldn't stat it */
cmdarg_err("Error testing whether capture file is a pipe: %s",
strerror(errno));
return 2;
}
}
}
*is_pipe = FALSE;
return 0;
}
If you read this function, you can see that it can only set is_pipe to FALSE,
or raise an error.
The line " *is_pipe = FALSE;" should be at the beginning of the function,
not at the end.
At least for me, it fixed the problem described in bugs 1181 et 1220.
I created bug #1220 because, sorry, the keywords I used wehre not the right
ones.
I attached a patch to bug #1220, but nobody seems to have taken care of.
Hope it will close the issue for you also !
Regards,
Pierre
___
Wireshark-dev mailing list
Wireshark-dev@wireshark.org
http://www.wireshark.org/mailman/listinfo/wireshark-dev
Re: [Wireshark-dev] Delays in real-time packet capture
Hello Pierre,
>>> Pierre JUHEN <[EMAIL PROTECTED]> 11/30/06 6:33 AM >>>
>Maybe the problems lies here (capture_opts.c) :
[snip]
> At least for me, it fixed the problem described in bugs 1181 et
1220.
>
> I created bug #1220 because, sorry, the keywords I used wehre not the
right ones.
>
> I attached a patch to bug #1220, but nobody seems to have taken care
of.
>
> Hope it will close the issue for you also !
The patch for bug #1220 ("Wireshark is unduly buffered when reading
from a
pipe") does not directly fix bug #1181 ("Delays in real-time packet
capture")
when dumpcap itself is capturing from the NIC interface (which is what
happens
when one initiates a capture from within Wireshark (e.g. "wireshark -i
eth0 -k -l -S")).
Interestingly the patch for bug #1220 does affect dumpcap when dumpcap
is used in a pipeline (e.g. "tshark -i eth0 -w - | wireshark -i - -k
-l -S").
In bug #1220 you appear to be using tshark as the 1st process in a
pipeline
as a workaround for the buffering problem inherent in running dumpcap
on
linux (bug #1181).
While your proposed fix for bug #1220 doesn't directly resolve bug
#1181
it can help suppress bug #1181 in certain scenarios. I'd vote for
applying
your patch.
I hope someone finds this info helpful.
Jim Young
___
Wireshark-dev mailing list
Wireshark-dev@wireshark.org
http://www.wireshark.org/mailman/listinfo/wireshark-dev
