Re: [netsniff-ng] Synchronize file creation between netsniff-ng processes possible?

[e . bengtsson]

Den onsdag 27 januari 2016 kl. 16:17:46 UTC+1 skrev Tobias Klauser:
>
> Hi again 
>
> On 2016-01-27 at 16:10:30 +0100, Tobias Klauser  > wrote: 
> > On 2016-01-27 at 14:40:55 +0100, Erik Bengtsson  > wrote: 
> > > By "start time" I mean file creation time. It is totally possible to 
> post 
> > > process all log files and synchronize them using pcap timestamps, as 
> you 
> > > suggested, but I'm afraid that it will be less effective for us since 
> there 
> > > will be a huge amount of data to move around and process. 
> > 
> > Ah ok, I see. Thanks for clarifying. 
> > 
> > In that case, I'm afraid there currently is no method to synchronize the 
> file 
> > creation among multiple instances of netsniff-ng. As Daniel suggested, 
> > timerfd might be an option to implement a feature along these lines. If 
> > you want, feel free to have a look into it - patches are gladly accepted 
> > :) 
>
> Something which I completely forgot about... 
>
> There might be an option (though a bit curde) to solve this using the 
> permature rotation caused by SIGHUP. You could set up a separate task 
> simultaneously sending a SIGHUP every minute to your netsniff-ng 
> processes. See commit 46289df6bc8f573bc01be4fb4aa93343ecc6d50a 
> ("netsniff-ng: Rotate pcap files prematurely on SIGHUP") for details. 
>
> Tobias 
>

That sounds really interesting! :-)

I ended up using signal USR2 since i didn't want to change how SIGHUP was 
used. When USR2 is received, the current time is saved and used when naming 
the next file (which is created when the next packet is received).

A patch is included if you want to have a look.

/ Erik
 

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
commit 20ffe5b2b3734d61da727fc6c982a83847df5b8f
Author: Erik Bengtsson 
Date:   Wed Jan 27 17:25:02 2016 +0100

Added support for rotating dump files by sending signal USR2 to netsniff-ng.

diff --git a/netsniff-ng.c b/netsniff-ng.c
index 9bc4985..2831d90 100644
--- a/netsniff-ng.c
+++ b/netsniff-ng.c
@@ -48,6 +48,8 @@
 #include "dissector.h"
 #include "xmalloc.h"
 
+#define SIGNEXTDUMP SIGUSR2
+
 enum dump_mode {
DUMP_INTERVAL_TIME,
DUMP_INTERVAL_SIZE,
@@ -69,7 +71,7 @@ struct ctx {
uint64_t pkts_recvd_last, pkts_drops_last;
 };
 
-static volatile sig_atomic_t sigint = 0, sighup = 0;
+static volatile sig_atomic_t sigint = 0, sighup = 0, signextdump = 0;
 static volatile bool next_dump = false;
 
 static const char *short_options =
@@ -129,11 +131,11 @@ static const char *copyright = "Please report bugs to 
pcap]
 
 static void signal_handler(int number)
-{
+{  
switch (number) {
case SIGINT:
case SIGQUIT:
@@ -143,6 +145,10 @@ static void signal_handler(int number)
case SIGHUP:
sighup = 1;
break;
+   case SIGNEXTDUMP:
+   signextdump = 1;
+   nextdump_time = time(NULL);
+   break;
default:
break;
}
@@ -461,6 +467,7 @@ static void receive_to_xmit(struct ctx *ctx)
fflush(stdout);
 
while (likely(sigint == 0)) {
+
while (user_may_pull_from_rx(rx_ring.frames[it_in].iov_base)) {
hdr_in = rx_ring.frames[it_in].iov_base;
in = ((uint8_t *) hdr_in) + hdr_in->tp_h.tp_mac;
@@ -776,8 +783,11 @@ static int next_multi_pcap_file(struct ctx *ctx, int fd)
 
close(fd);
 
+   printf("next multi pcap file %lu\n", nextdump_time);
+   time_t ftime = nextdump_time ? nextdump_time : time(NULL);
+   nextdump_time = 0;
slprintf(fname, sizeof(fname), "%s/%s%lu.pcap", ctx->device_out,
-ctx->prefix ? : "dump-", time(NULL));
+ctx->prefix ? : "dump-", ftime);
 
fd = open_or_die_m(fname, O_RDWR | O_CREAT | O_TRUNC |
   O_LARGEFILE, DEFFILEMODE);
@@ -895,10 +905,17 @@ static void update_pcap_next_dump(struct ctx *ctx, 
unsigned long snaplen,
}
}
 
-   if (sighup) {
-   if (ctx->verbose)
-   printf("SIGHUP received, prematurely rotating pcap\n");
-   sighup = 0;
+   if (sighup || signextdump) {
+   if (sighup) {
+   if (ctx->verbose)
+   printf("SIGHUP received, prematurely rotating 
pcap\n");
+   sighup = 0;
+   }
+   if (signextdump) {
+   

[netsniff-ng] Replay pcap file on Xenomai kernel in real time

[Umair Ali]

Hello there,

I am working on the project of replaying the pcap files in the real time over 
the network. For this purpose I am using the real time Linux kernel 'Xenomai v 
3.0.1'. My idea is to write a code in C which will read the pcap file packet by 
packet and then send the packet as raw Ethernet packet over the real time 
interface. When the pcap libraries are used on the xenomai then the process of 
reading pcap files does not behave in real time any more. Moreover the xenomai 
uses the mmap for reading the files in real time. I have tried to use the mmap 
technique to read pcap file but it is not perfect and works for small files. I 
have read the netsniff.ng tool uses the same mmap technique to read the pcap 
file for replaying the pcap file. My question is that how mmap is used in 
netsniff-ng tool to read the pcap file packet by packet. Can you give me the C 
code as an example to read the pcap file using mmap packet by packet. I shall 
be highly thankful.

Thanks & regards
Ali
Tampere University of Technology.

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Replay pcap file on Xenomai kernel in real time

[Tobias Klauser]

Hi Ali

On 2016-01-27 at 13:01:55 +0100, Umair Ali  wrote:
> Hello there,
> 
> I am working on the project of replaying the pcap files in the real time over 
> the network. For this purpose I am using the real time Linux kernel 'Xenomai 
> v 3.0.1'. My idea is to write a code in C which will read the pcap file 
> packet by packet and then send the packet as raw Ethernet packet over the 
> real time interface. When the pcap libraries are used on the xenomai then the 
> process of reading pcap files does not behave in real time any more. Moreover 
> the xenomai uses the mmap for reading the files in real time. I have tried to 
> use the mmap technique to read pcap file but it is not perfect and works for 
> small files. I have read the netsniff.ng tool uses the same mmap technique to 
> read the pcap file for replaying the pcap file. My question is that how mmap 
> is used in netsniff-ng tool to read the pcap file packet by packet. Can you 
> give me the C code as an example to read the pcap file using mmap packet by 
> packet. I shall be highly thankful.

Have a look at pcap_mm.c, the mmap base pcap read/write functions are
defined there.

Hope that helps
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Synchronize file creation between netsniff-ng processes possible?

[Vadim Kochan]

Hi,

On Tue, Jan 26, 2016 at 12:12 PM,   wrote:
> Hi everyone!
>
> I've googled a bit but not found any good answer to my question. Maybe you 
> can help me out?
>
> Currently I'm running five netsniff processes capturing data from five 
> different interfaces. It works really well with the limiting factor SSD write 
> speed, not netsniff itself. To make post processing easier, it would be nice 
> if all log files were the same size and each set of files had the same start 
> time.
>
> It seems like netsniff only checks file size / time when a packet is received 
> so the start time within a set of files drift more or less depending on the 
> busload.
>
> Any ideas on how to synchronize the processes?
>
> Best regards
> Erik
>
> --
> You received this message because you are subscribed to the Google Groups 
> "netsniff-ng" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to netsniff-ng+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Really I use netsniff-ng just for simple dissecting so I might be not
aware about some usages of the tool, but
as developer may be I may help you if you provide more info with some
output's like:

1) How do you exactly run the tool ?

2) What do you see exactly with console output example ?

3) What you 'd like to expect exactly (probably with examples of output) ?

Regards,
Vadim Kochan

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [netsniff-ng] Replay pcap file on Xenomai kernel in real time

[Umair Ali]

Hi Tobias,

Thanks for the quick reply. I have read pcap_mm.c file but cannot understand 
the flow of the code. Can you explain me the flow that once the pcap file is 
open using mmap then how it is further processed to extract packet by packet 
and replay. Is it possible with netsniff-ng to send packet every 5micro secs or 
less. 

Thanks in advance

BR
Ali

-Original Message-
From: Tobias Klauser [mailto:tklau...@distanz.ch] 
Sent: January-27-16 2:36 PM
To: netsniff-ng@googlegroups.com
Cc: Umair Ali
Subject: Re: [netsniff-ng] Replay pcap file on Xenomai kernel in real time

[resending with Ali in Cc in case he's not subscribed to the list]

Hi Ali

On 2016-01-27 at 13:01:55 +0100, Umair Ali  wrote:
> Hello there,
> 
> I am working on the project of replaying the pcap files in the real time over 
> the network. For this purpose I am using the real time Linux kernel 'Xenomai 
> v 3.0.1'. My idea is to write a code in C which will read the pcap file 
> packet by packet and then send the packet as raw Ethernet packet over the 
> real time interface. When the pcap libraries are used on the xenomai then the 
> process of reading pcap files does not behave in real time any more. Moreover 
> the xenomai uses the mmap for reading the files in real time. I have tried to 
> use the mmap technique to read pcap file but it is not perfect and works for 
> small files. I have read the netsniff.ng tool uses the same mmap technique to 
> read the pcap file for replaying the pcap file. My question is that how mmap 
> is used in netsniff-ng tool to read the pcap file packet by packet. Can you 
> give me the C code as an example to read the pcap file using mmap packet by 
> packet. I shall be highly thankful.

Have a look at pcap_mm.c, the mmap base pcap read/write functions are defined 
there.

Hope that helps
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Synchronize file creation between netsniff-ng processes possible?

[Daniel Borkmann]

On 01/27/2016 02:13 PM, e.bengts...@gmail.com wrote:
[...]

Is it possible to add some kind of signal interface for controlling start
of new files instead of relying on incoming events?
I have pulled the source and had a look but don't really know how to
proceed...


If you have some cycles, it would be worth to add timerfd [1] support to
netsniff-ng, so that the timerfd is set to the dump interval we already
pass via cmdline, and we pack that fd along with the pf_packet one into
the poll() syscall, so we get a poll notification from there. We might need
to move update_pcap_next_dump() out of the loop so we can generate a new
dump file. This should reduce the latency for low/mid traffic patterns
quite a bit.

Thanks,
Daniel

  [1] http://man7.org/linux/man-pages/man2/timerfd_create.2.html

--
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Replay pcap file on Xenomai kernel in real time

[Tobias Klauser]

On 2016-01-27 at 15:15:01 +0100, Umair Ali  wrote:
> Hi Tobias,
> 
> Thanks for the quick reply. I have read pcap_mm.c file but cannot understand 
> the flow of the code. Can you explain me the flow that once the pcap file is 
> open using mmap then how it is further processed to extract packet by packet 
> and replay. Is it possible with netsniff-ng to send packet every 5micro secs 
> or less. 

The mmap pcap functions (like the scatter-gather and the standard file
i/o functions) are wired up in struct pcap_file_ops *pcap_ops and then
used by the respective functions in netsniff-ng.cx according to the pcap
access method is set in ctx->pcap (PCAP_OPS_MM in case of mmap).
read_pcap in netsniff-ng.c is probably most interesting to you.

HTH
Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [netsniff-ng] Synchronize file creation between netsniff-ng processes possible?

[Tobias Klauser]

Hi again

On 2016-01-27 at 16:10:30 +0100, Tobias Klauser  wrote:
> On 2016-01-27 at 14:40:55 +0100, Erik Bengtsson  wrote:
> > By "start time" I mean file creation time. It is totally possible to post
> > process all log files and synchronize them using pcap timestamps, as you
> > suggested, but I'm afraid that it will be less effective for us since there
> > will be a huge amount of data to move around and process.
> 
> Ah ok, I see. Thanks for clarifying.
> 
> In that case, I'm afraid there currently is no method to synchronize the file
> creation among multiple instances of netsniff-ng. As Daniel suggested,
> timerfd might be an option to implement a feature along these lines. If
> you want, feel free to have a look into it - patches are gladly accepted
> :)

Something which I completely forgot about...

There might be an option (though a bit curde) to solve this using the
permature rotation caused by SIGHUP. You could set up a separate task
simultaneously sending a SIGHUP every minute to your netsniff-ng
processes. See commit 46289df6bc8f573bc01be4fb4aa93343ecc6d50a
("netsniff-ng: Rotate pcap files prematurely on SIGHUP") for details.

Tobias

-- 
You received this message because you are subscribed to the Google Groups 
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.