Re: [netsniff-ng] Huge Packet Loss for 10Gb during output the traffic to the files
Yea you are definately limited by your disk writes. Installing addition drives should improve write performance. Use something like bonnie++ to test your write throughput. On Wednesday, May 13, 2015, Ivan Cheng wrote: > Here's the disks configuration: > > 1. Two Local 300GB SAS hard disks with RAID 0+1 for the system disk (LVM > disk) > 2. One Local 300GB SAS hard disk with RAID 0 for storing the traffic data > (Ext4 disk) > 3. Hewlett-Packard Company Smart Array G6 controllers (rev 01) > > Thanks. > > > -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] [PATCH] netsniff-ng nlmsg: Print multi-part messages
From: Vadim Kochan
Pull & print more Netlink messages from one packet
which can be sent with MULTI flag.
Signed-off-by: Vadim Kochan
---
proto_nlmsg.c | 52 +---
1 file changed, 37 insertions(+), 15 deletions(-)
diff --git a/proto_nlmsg.c b/proto_nlmsg.c
index 136cec6..290fbf6 100644
--- a/proto_nlmsg.c
+++ b/proto_nlmsg.c
@@ -133,16 +133,27 @@ static char *nlmsg_type2str(uint16_t proto, uint16_t
type, char *buf, int len)
return nl_nlmsgtype2str(type, buf, len);
}
-static void nlmsg(struct pkt_buff *pkt)
+static void nlmsg_print_each(struct pkt_buff *pkt,
+ void (*do_print)(uint16_t family, struct nlmsghdr *hdr))
+{
+ struct nlmsghdr *hdr = (struct nlmsghdr *)pkt_pull(pkt, sizeof(*hdr));
+
+ while (hdr) {
+ do_print(ntohs(pkt->proto), hdr);
+
+ if (!pkt_pull(pkt, NLMSG_PAYLOAD(hdr, 0)))
+ break;
+
+ hdr = (struct nlmsghdr *)pkt_pull(pkt, sizeof(*hdr));
+ }
+}
+
+static void nlmsg_print(uint16_t family, struct nlmsghdr *hdr)
{
- struct nlmsghdr *hdr = (struct nlmsghdr *) pkt_pull(pkt, sizeof(*hdr));
char type[32];
char flags[128];
char procname[PATH_MAX];
- if (hdr == NULL)
- return;
-
/* Look up the process name if message is not coming from the kernel.
*
* Note that the port id is not necessarily equal to the PID of the
@@ -163,13 +174,15 @@ static void nlmsg(struct pkt_buff *pkt)
snprintf(procname, sizeof(procname), "kernel");
tprintf(" [ NLMSG ");
- tprintf("Family %d (%s%s%s), ", ntohs(pkt->proto), colorize_start(bold),
- nlmsg_family2str(ntohs(pkt->proto)), colorize_end());
+ tprintf("Family %d (%s%s%s), ", family,
+ colorize_start(bold),
+ nlmsg_family2str(family),
+ colorize_end());
tprintf("Len %u, ", hdr->nlmsg_len);
tprintf("Type 0x%.4x (%s%s%s), ", hdr->nlmsg_type,
colorize_start(bold),
- nlmsg_type2str(ntohs(pkt->proto), hdr->nlmsg_type, type,
- sizeof(type)), colorize_end());
+ nlmsg_type2str(family, hdr->nlmsg_type, type, sizeof(type)),
+ colorize_end());
tprintf("Flags 0x%.4x (%s%s%s), ", hdr->nlmsg_flags,
colorize_start(bold),
nl_nlmsg_flags2str(hdr->nlmsg_flags, flags, sizeof(flags)),
@@ -182,19 +195,28 @@ static void nlmsg(struct pkt_buff *pkt)
tprintf(" ]\n");
}
+static void nlmsg(struct pkt_buff *pkt)
+{
+ nlmsg_print_each(pkt, nlmsg_print);
+}
+
static void nlmsg_less(struct pkt_buff *pkt)
{
- struct nlmsghdr *hdr = (struct nlmsghdr *) pkt_pull(pkt, sizeof(*hdr));
+ struct nlmsghdr *hdr = (struct nlmsghdr *)pkt_pull(pkt, sizeof(*hdr));
+ uint16_t family = ntohs(pkt->proto);
char type[32];
if (hdr == NULL)
return;
- tprintf(" NLMSG Family %d (%s%s%s), ", ntohs(pkt->proto),
colorize_start(bold),
- nlmsg_family2str(ntohs(pkt->proto)), colorize_end());
- tprintf("Type %u (%s%s%s)", hdr->nlmsg_type, colorize_start(bold),
- nlmsg_type2str(ntohs(pkt->proto), hdr->nlmsg_type, type,
- sizeof(type)), colorize_end());
+ tprintf(" NLMSG Family %d (%s%s%s), ", family,
+ colorize_start(bold),
+ nlmsg_family2str(family),
+ colorize_end());
+ tprintf("Type %u (%s%s%s)", hdr->nlmsg_type,
+ colorize_start(bold),
+ nlmsg_type2str(family, hdr->nlmsg_type, type, sizeof(type)),
+ colorize_end());
}
struct protocol nlmsg_ops = {
--
2.3.1
--
You received this message because you are subscribed to the Google Groups
"netsniff-ng" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to netsniff-ng+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] Huge Packet Loss for 10Gb during output the traffic to the files
What is your disk configuration? Most likely you are unable to write fast enough whic would cause your packet loss. On Wednesday, May 13, 2015, Ivan Cheng wrote: > Hi, > > I have done below 10Gb interface tests using netsniff-ng-0.5.9 under > Ubuntu 14.04 LTS x64 but found that there're more than half of the traffic > dropped during capturing the traffic to the files: > > 1. Server1: run iperf -s > 2. Server1: capture the traffic using netsniff-ng > 3. Server2: run iperf -c x.x.x.x -t 100. (Network Utilization was over > 90% or 8Gb/sec) > 4. Version: netsniff-ng-0.5.9 > > Is there anyway to resolve it? If the traffic was lower than 1Gb, this > problem doesn't occur. > > Thanks in advance. > > Best Regards, > Ivan Cheng > > -- > Test Result: > > Server1: > # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S > 2GiB -s -V -H -J -F 500MiB > pcap file I/O method: scatter-gather > RX,V3: 2048.00 MiB, 32768 Blocks, each 65536 Byte allocated > Running! Hang up with ^C! > > [ 4] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48511 > > .(+189550/-0).(+311899/-229105).(+94141/-650675).(+94434/-655971).(+94520/-661770).(+94957/-656108).(+94294/-655801).(+93884/-652061).(+94467/-652528).(+94308/-657827).(+94541/-678354).(+94247/-672099).(+94307/-678535).(+93657/-682063).(+94537/-677862).(+93673/-678498).(+93940/-672579).(+94652/-678580).(+94376/-674936).(+94673/-675870).(+94017/-678660).(+94617/-682085).(+94629/-672379).(+94177/-680516).(+94175/-671915).(+94143/-682812).(+94555/-678232) > [ 4] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec > .(+40865/-4864).(+52/-0).(+34/-0) > > # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S > 500MiB -s -V -H -J -F 500MiB > pcap file I/O method: scatter-gather > RX,V3: 500.00 MiB, 8000 Blocks, each 65536 Byte allocated > Running! Hang up with ^C! > > [ 5] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48512 > .(+173863/-28474).(+97421/-456830).(+96685/-684605).(+97628/-687301).(+97473/-686187).(+97513/-685249)[ > 5] 0.0-20.0 sec 21.8 GBytes 9.37 Gbits/sec > > > Server2: > > # iperf -c 192.168.200.2 -t 100 > > Client connecting to 192.168.200.2, TCP port 5001 > TCP window size: 85.0 KByte (default) > > [ 3] local 192.168.200.1 port 48511 connected with 192.168.200.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec > > > # iperf -c 192.168.200.2 -t 20 > > Client connecting to 192.168.200.2, TCP port 5001 > TCP window size: 85.0 KByte (default) > > [ 3] local 192.168.200.1 port 48512 connected with 192.168.200.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-20.0 sec 21.8 GBytes 9.38 Gbits/sec > > > --- > > HW & OS config: > > - 2 x Prolant 380 G7 servers with 16G RAM > - 1 10Gb NIC for each server > - Ubuntu v14.04 LTS x64 > > # uname -a > Linux BL380G7-01 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC > 2014 x86_64 x86_64 x86_64 GNU/Linux > > # lspci -v -s 0b:00.0 > 0b:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction > 1/10-Gigabit Server Adapter (rev 42) > Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server > Adapter > Physical Slot: 2 > Flags: bus master, fast devsel, latency 0, IRQ 31 > Memory at fbe0 (64-bit, non-prefetchable) [size=2M] > Memory at f800 (64-bit, non-prefetchable) [size=32M] > [virtual] Expansion ROM at df10 [disabled] [size=64K] > Capabilities: [40] MSI-X: Enable+ Count=64 Masked- > Capabilities: [80] Power Management version 3 > Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ > Capabilities: [c0] Express Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 > Kernel driver in use: netxen_nic > > # modinfo netxen_nic > filename: > /lib/modules/3.13.0-24-generic/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko > firmware: phanfw.bin > version:4.0.82 > license:GPL > description:QLogic/NetXen (1/10) GbE Intelligent Ethernet Driver > srcversion: 74E26B6913A817FEB3498A9 > alias: pci:v4040d0100sv*sd*bc02sc00i00* > alias: pci:v4040d0025sv*sd*bc02sc00i00* > alias: pci:v4040d0024sv*sd*bc02sc00i00* > alias: pci:v4040d0005sv*sd*bc02sc00i00* > alias: pci:v4040d0004sv*sd*bc02sc00i00* > alias: pci:v4040d0003sv*sd*bc02sc00i00* > alias: pci:v4040d0002sv*sd*bc02sc00i00* > alias:
Re: [netsniff-ng] Huge Packet Loss for 10Gb during output the traffic to the files
What is your disk configuration? Most likely you are unable to write fast enough whic would cause your packet loss. On Wednesday, May 13, 2015, Ivan Cheng wrote: > Hi, > > I have done below 10Gb interface tests using netsniff-ng-0.5.9 under > Ubuntu 14.04 LTS x64 but found that there're more than half of the traffic > dropped during capturing the traffic to the files: > > 1. Server1: run iperf -s > 2. Server1: capture the traffic using netsniff-ng > 3. Server2: run iperf -c x.x.x.x -t 100. (Network Utilization was over > 90% or 8Gb/sec) > 4. Version: netsniff-ng-0.5.9 > > Is there anyway to resolve it? If the traffic was lower than 1Gb, this > problem doesn't occur. > > Thanks in advance. > > Best Regards, > Ivan Cheng > > -- > Test Result: > > Server1: > # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S > 2GiB -s -V -H -J -F 500MiB > pcap file I/O method: scatter-gather > RX,V3: 2048.00 MiB, 32768 Blocks, each 65536 Byte allocated > Running! Hang up with ^C! > > [ 4] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48511 > > .(+189550/-0).(+311899/-229105).(+94141/-650675).(+94434/-655971).(+94520/-661770).(+94957/-656108).(+94294/-655801).(+93884/-652061).(+94467/-652528).(+94308/-657827).(+94541/-678354).(+94247/-672099).(+94307/-678535).(+93657/-682063).(+94537/-677862).(+93673/-678498).(+93940/-672579).(+94652/-678580).(+94376/-674936).(+94673/-675870).(+94017/-678660).(+94617/-682085).(+94629/-672379).(+94177/-680516).(+94175/-671915).(+94143/-682812).(+94555/-678232) > [ 4] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec > .(+40865/-4864).(+52/-0).(+34/-0) > > # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S > 500MiB -s -V -H -J -F 500MiB > pcap file I/O method: scatter-gather > RX,V3: 500.00 MiB, 8000 Blocks, each 65536 Byte allocated > Running! Hang up with ^C! > > [ 5] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48512 > .(+173863/-28474).(+97421/-456830).(+96685/-684605).(+97628/-687301).(+97473/-686187).(+97513/-685249)[ > 5] 0.0-20.0 sec 21.8 GBytes 9.37 Gbits/sec > > > Server2: > > # iperf -c 192.168.200.2 -t 100 > > Client connecting to 192.168.200.2, TCP port 5001 > TCP window size: 85.0 KByte (default) > > [ 3] local 192.168.200.1 port 48511 connected with 192.168.200.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec > > > # iperf -c 192.168.200.2 -t 20 > > Client connecting to 192.168.200.2, TCP port 5001 > TCP window size: 85.0 KByte (default) > > [ 3] local 192.168.200.1 port 48512 connected with 192.168.200.2 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-20.0 sec 21.8 GBytes 9.38 Gbits/sec > > > --- > > HW & OS config: > > - 2 x Prolant 380 G7 servers with 16G RAM > - 1 10Gb NIC for each server > - Ubuntu v14.04 LTS x64 > > # uname -a > Linux BL380G7-01 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC > 2014 x86_64 x86_64 x86_64 GNU/Linux > > # lspci -v -s 0b:00.0 > 0b:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction > 1/10-Gigabit Server Adapter (rev 42) > Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server > Adapter > Physical Slot: 2 > Flags: bus master, fast devsel, latency 0, IRQ 31 > Memory at fbe0 (64-bit, non-prefetchable) [size=2M] > Memory at f800 (64-bit, non-prefetchable) [size=32M] > [virtual] Expansion ROM at df10 [disabled] [size=64K] > Capabilities: [40] MSI-X: Enable+ Count=64 Masked- > Capabilities: [80] Power Management version 3 > Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ > Capabilities: [c0] Express Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 > Kernel driver in use: netxen_nic > > # modinfo netxen_nic > filename: > /lib/modules/3.13.0-24-generic/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko > firmware: phanfw.bin > version:4.0.82 > license:GPL > description:QLogic/NetXen (1/10) GbE Intelligent Ethernet Driver > srcversion: 74E26B6913A817FEB3498A9 > alias: pci:v4040d0100sv*sd*bc02sc00i00* > alias: pci:v4040d0025sv*sd*bc02sc00i00* > alias: pci:v4040d0024sv*sd*bc02sc00i00* > alias: pci:v4040d0005sv*sd*bc02sc00i00* > alias: pci:v4040d0004sv*sd*bc02sc00i00* > alias: pci:v4040d0003sv*sd*bc02sc00i00* > alias: pci:v4040d0002sv*sd*bc02sc00i00* > alias:
[netsniff-ng] Huge Packet Loss for 10Gb during output the traffic to the files
Hi, I have done below 10Gb interface tests using netsniff-ng-0.5.9 under Ubuntu 14.04 LTS x64 but found that there're more than half of the traffic dropped during capturing the traffic to the files: 1. Server1: run iperf -s 2. Server1: capture the traffic using netsniff-ng 3. Server2: run iperf -c x.x.x.x -t 100. (Network Utilization was over 90% or 8Gb/sec) 4. Version: netsniff-ng-0.5.9 Is there anyway to resolve it? If the traffic was lower than 1Gb, this problem doesn't occur. Thanks in advance. Best Regards, Ivan Cheng -- Test Result: Server1: # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S 2GiB -s -V -H -J -F 500MiB pcap file I/O method: scatter-gather RX,V3: 2048.00 MiB, 32768 Blocks, each 65536 Byte allocated Running! Hang up with ^C! [ 4] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48511 .(+189550/-0).(+311899/-229105).(+94141/-650675).(+94434/-655971).(+94520/-661770).(+94957/-656108).(+94294/-655801).(+93884/-652061).(+94467/-652528).(+94308/-657827).(+94541/-678354).(+94247/-672099).(+94307/-678535).(+93657/-682063).(+94537/-677862).(+93673/-678498).(+93940/-672579).(+94652/-678580).(+94376/-674936).(+94673/-675870).(+94017/-678660).(+94617/-682085).(+94629/-672379).(+94177/-680516).(+94175/-671915).(+94143/-682812).(+94555/-678232) [ 4] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec .(+40865/-4864).(+52/-0).(+34/-0) # /usr/local/sbin/netsniff-ng -b 16 -i any -o /data/pcap/ -P "test-" -S 500MiB -s -V -H -J -F 500MiB pcap file I/O method: scatter-gather RX,V3: 500.00 MiB, 8000 Blocks, each 65536 Byte allocated Running! Hang up with ^C! [ 5] local 192.168.200.2 port 5001 connected with 192.168.200.1 port 48512 .(+173863/-28474).(+97421/-456830).(+96685/-684605).(+97628/-687301).(+97473/-686187).(+97513/-685249)[ 5] 0.0-20.0 sec 21.8 GBytes 9.37 Gbits/sec Server2: # iperf -c 192.168.200.2 -t 100 Client connecting to 192.168.200.2, TCP port 5001 TCP window size: 85.0 KByte (default) [ 3] local 192.168.200.1 port 48511 connected with 192.168.200.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-100.0 sec 109 GBytes 9.38 Gbits/sec # iperf -c 192.168.200.2 -t 20 Client connecting to 192.168.200.2, TCP port 5001 TCP window size: 85.0 KByte (default) [ 3] local 192.168.200.1 port 48512 connected with 192.168.200.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-20.0 sec 21.8 GBytes 9.38 Gbits/sec --- HW & OS config: - 2 x Prolant 380 G7 servers with 16G RAM - 1 10Gb NIC for each server - Ubuntu v14.04 LTS x64 # uname -a Linux BL380G7-01 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # lspci -v -s 0b:00.0 0b:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10-Gigabit Server Adapter (rev 42) Subsystem: Hewlett-Packard Company NC522SFP Dual Port 10GbE Server Adapter Physical Slot: 2 Flags: bus master, fast devsel, latency 0, IRQ 31 Memory at fbe0 (64-bit, non-prefetchable) [size=2M] Memory at f800 (64-bit, non-prefetchable) [size=32M] [virtual] Expansion ROM at df10 [disabled] [size=64K] Capabilities: [40] MSI-X: Enable+ Count=64 Masked- Capabilities: [80] Power Management version 3 Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+ Capabilities: [c0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number 59-69-46-61-6e-48-73-75 Kernel driver in use: netxen_nic # modinfo netxen_nic filename: /lib/modules/3.13.0-24-generic/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko firmware: phanfw.bin version:4.0.82 license:GPL description:QLogic/NetXen (1/10) GbE Intelligent Ethernet Driver srcversion: 74E26B6913A817FEB3498A9 alias: pci:v4040d0100sv*sd*bc02sc00i00* alias: pci:v4040d0025sv*sd*bc02sc00i00* alias: pci:v4040d0024sv*sd*bc02sc00i00* alias: pci:v4040d0005sv*sd*bc02sc00i00* alias: pci:v4040d0004sv*sd*bc02sc00i00* alias: pci:v4040d0003sv*sd*bc02sc00i00* alias: pci:v4040d0002sv*sd*bc02sc00i00* alias: pci:v4040d0001sv*sd*bc02sc00i00* depends: intree: Y vermagic: 3.13.0-24-generic SMP mod_unload modversions signer: Magrathea: Glacier signing key sig_key:00:A5:A6:57:59:DE:47:4B:C5:C4:31:20:88:0C:1B:94:A5:39:F4:31 sig_hashalgo: sha512 parm: auto_fw_reset:Auto firmware reset (0=disabled, 1=enabled
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On Wed, May 13, 2015 at 11:45:16AM +0200, Daniel Borkmann wrote: > On 05/13/2015 11:31 AM, Vadim Kochan wrote: > >On Wed, May 13, 2015 at 11:24:11AM +0200, Daniel Borkmann wrote: > >>On 05/13/2015 11:21 AM, Vadim Kochan wrote: > >>... > >>>Or in case of Netlink link type enforce to use netsniff-ng pcap format > >>>... > >> > >>I think that would be rather unexpected as other tools like Wireshark > >>cannot deal with that afaik. In that case, I rather prefer your patch > >>here. > > > >Then the patch should be re-worked a little to use cooked header only > >for default & nsec header formats, which I forgot to add in > > > > pcap_pkthdr_has_ll func ... > > > > Ok, so changing defaults, I would like to avoid actually as it could > potentially break things like Security Onion and others that rely on Well currently Netlink capture pcap file is broken by default as it does not make sense while identifying Netlink family protocol :-) > the current capturing format (if they use the default one). It would > also make the pcaps bigger for that case. > > Here would be my suggestion on how to move forward: 1) add the cooked > header which is understood by tcpdump/wireshark as a new pcap type, 2) New magic number ? If yes then tcpdump/w-shark will not eat this .. > if we detect we're capturing on a netlink(/nlmon) device, use that as In the patch I already check this ... > a default there (I don't think there are many people relying on this > in netlink case), and 3) add a command-line option f.e. --cooked that > would make it easy to use as a shortcut for --magic XYZ. > > What do you think? > > Thanks, > Daniel -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On 2015-05-13 at 11:45:16 +0200, Daniel Borkmann wrote: > On 05/13/2015 11:31 AM, Vadim Kochan wrote: > >On Wed, May 13, 2015 at 11:24:11AM +0200, Daniel Borkmann wrote: > >>On 05/13/2015 11:21 AM, Vadim Kochan wrote: > >>... > >>>Or in case of Netlink link type enforce to use netsniff-ng pcap format > >>>... > >> > >>I think that would be rather unexpected as other tools like Wireshark > >>cannot deal with that afaik. In that case, I rather prefer your patch > >>here. > > > >Then the patch should be re-worked a little to use cooked header only > >for default & nsec header formats, which I forgot to add in > > > > pcap_pkthdr_has_ll func ... > > > > Ok, so changing defaults, I would like to avoid actually as it could > potentially break things like Security Onion and others that rely on > the current capturing format (if they use the default one). It would > also make the pcaps bigger for that case. > > Here would be my suggestion on how to move forward: 1) add the cooked > header which is understood by tcpdump/wireshark as a new pcap type, 2) > if we detect we're capturing on a netlink(/nlmon) device, use that as > a default there (I don't think there are many people relying on this > in netlink case), and 3) add a command-line option f.e. --cooked that > would make it easy to use as a shortcut for --magic XYZ. Full ACK. And as an addition may I suggest 4) document the new pcap type and the corresponding new command line option/magic in the manpage. Best Tobias -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On 05/13/2015 11:31 AM, Vadim Kochan wrote: On Wed, May 13, 2015 at 11:24:11AM +0200, Daniel Borkmann wrote: On 05/13/2015 11:21 AM, Vadim Kochan wrote: ... Or in case of Netlink link type enforce to use netsniff-ng pcap format ... I think that would be rather unexpected as other tools like Wireshark cannot deal with that afaik. In that case, I rather prefer your patch here. Then the patch should be re-worked a little to use cooked header only for default & nsec header formats, which I forgot to add in pcap_pkthdr_has_ll func ... Ok, so changing defaults, I would like to avoid actually as it could potentially break things like Security Onion and others that rely on the current capturing format (if they use the default one). It would also make the pcaps bigger for that case. Here would be my suggestion on how to move forward: 1) add the cooked header which is understood by tcpdump/wireshark as a new pcap type, 2) if we detect we're capturing on a netlink(/nlmon) device, use that as a default there (I don't think there are many people relying on this in netlink case), and 3) add a command-line option f.e. --cooked that would make it easy to use as a shortcut for --magic XYZ. What do you think? Thanks, Daniel -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On Wed, May 13, 2015 at 11:24:11AM +0200, Daniel Borkmann wrote: > On 05/13/2015 11:21 AM, Vadim Kochan wrote: > ... > >Or in case of Netlink link type enforce to use netsniff-ng pcap format > >... > > I think that would be rather unexpected as other tools like Wireshark > cannot deal with that afaik. In that case, I rather prefer your patch > here. Then the patch should be re-worked a little to use cooked header only for default & nsec header formats, which I forgot to add in pcap_pkthdr_has_ll func ... -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On 05/13/2015 11:21 AM, Vadim Kochan wrote: ... Or in case of Netlink link type enforce to use netsniff-ng pcap format ... I think that would be rather unexpected as other tools like Wireshark cannot deal with that afaik. In that case, I rather prefer your patch here. -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On Wed, May 13, 2015 at 12:18:37PM +0300, Vadim Kochan wrote: > On Wed, May 13, 2015 at 11:08:01AM +0200, Daniel Borkmann wrote: > > On 05/10/2015 02:37 PM, Vadim Kochan wrote: > > >From: Vadim Kochan > > > > > >Since Netlink messages are written in default pcap format there is no > > >way to identify Netlink family by socket protocol number, so > > >use pcap cooked header for Netlink messages as described here: > > > > > > http://www.tcpdump.org/linktypes/LINKTYPE_NETLINK.html > > > > > >Signed-off-by: Vadim Kochan > > > > Thanks for looking into this, Vadim! > > > > One preliminary question: you would only set that for netlink or > > also for other protocols? > Meanwhile it is useful only for Netlink link type. > > > > > The basic issue is that normal pcap hdr in front of every captured > > packet looses this information for the dissector, but alternative > > formats such as kuznet/netsniff-ng format still have it preserved, > > but are not supported by tools like wireshark. > Well, the main motivation was: > > 1) support saving Netlink protocol info by default in pcap file w/o > setting the magic type. > 2) compatibility with Wireshark (not so important but ...) > > Of course using borkmann's/kuznetsov's pcap headers solve this issue. > So the options which I see are: > >If you think that setting the magic type is enough then ignore the >patch (at least it was funny to achieve the goal :-) ), >but at least the manual page should have a comment that for >Netlink messages it is better to use borkmann's/kuznetsov's format, >and thats is :-) > > > > Ok. Thinking out loud, what if we add another pcap type as an option > > to choose? > > Regards, > Vadim Kochan Or in case of Netlink link type enforce to use netsniff-ng pcap format ... -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On Wed, May 13, 2015 at 11:08:01AM +0200, Daniel Borkmann wrote: > On 05/10/2015 02:37 PM, Vadim Kochan wrote: > >From: Vadim Kochan > > > >Since Netlink messages are written in default pcap format there is no > >way to identify Netlink family by socket protocol number, so > >use pcap cooked header for Netlink messages as described here: > > > > http://www.tcpdump.org/linktypes/LINKTYPE_NETLINK.html > > > >Signed-off-by: Vadim Kochan > > Thanks for looking into this, Vadim! > > One preliminary question: you would only set that for netlink or > also for other protocols? Meanwhile it is useful only for Netlink link type. > > The basic issue is that normal pcap hdr in front of every captured > packet looses this information for the dissector, but alternative > formats such as kuznet/netsniff-ng format still have it preserved, > but are not supported by tools like wireshark. Well, the main motivation was: 1) support saving Netlink protocol info by default in pcap file w/o setting the magic type. 2) compatibility with Wireshark (not so important but ...) Of course using borkmann's/kuznetsov's pcap headers solve this issue. So the options which I see are: If you think that setting the magic type is enough then ignore the patch (at least it was funny to achieve the goal :-) ), but at least the manual page should have a comment that for Netlink messages it is better to use borkmann's/kuznetsov's format, and thats is :-) > > Ok. Thinking out loud, what if we add another pcap type as an option > to choose? Regards, Vadim Kochan -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] [RFC] netsniff-ng: Add support of pcap cooked header
On 05/10/2015 02:37 PM, Vadim Kochan wrote: From: Vadim Kochan Since Netlink messages are written in default pcap format there is no way to identify Netlink family by socket protocol number, so use pcap cooked header for Netlink messages as described here: http://www.tcpdump.org/linktypes/LINKTYPE_NETLINK.html Signed-off-by: Vadim Kochan Thanks for looking into this, Vadim! One preliminary question: you would only set that for netlink or also for other protocols? The basic issue is that normal pcap hdr in front of every captured packet looses this information for the dissector, but alternative formats such as kuznet/netsniff-ng format still have it preserved, but are not supported by tools like wireshark. Ok. Thinking out loud, what if we add another pcap type as an option to choose? -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [netsniff-ng] Shell variables support in trafgen
On 05/13/2015 06:17 AM, Kenneth Aaron wrote: Hi, Is it possible to use shell variables in the configuration file? The use case I have is a script that sets variables such as IP address ( $SrcIP_Byte1 , $SrcIP_Byte2 , $SrcIP_Byte3 , $SrcIP_Byte4 ) then trafgen is called using a standard configuration file, but the variables $SrcIP_Byte1 etc are referenced from within the config file itself The alternative is that the shell script use 'sed' or other means of manipulating the config file, but variables are so much easier I think it should be possible to add. We could probe from the environment and at setup time read those out (once). The only issue I currently see is that there's no type information available. But you would basically see it as textual replacement, right? Cheers, Daniel -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[netsniff-ng] Shell variables support in trafgen
Hi, Is it possible to use shell variables in the configuration file? The use case I have is a script that sets variables such as IP address ( $SrcIP_Byte1 , $SrcIP_Byte2 , $SrcIP_Byte3 , $SrcIP_Byte4 ) then trafgen is called using a standard configuration file, but the variables $SrcIP_Byte1 etc are referenced from within the config file itself The alternative is that the shell script use 'sed' or other means of manipulating the config file, but variables are so much easier Thanks -- You received this message because you are subscribed to the Google Groups "netsniff-ng" group. To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
