Re: gettimeofday() in hping
Stefan Lambrev wrote: FreeBSD - ACPI em1 in 13.157 MB/s 13.162 MB/s 23.697 GB out13.150 MB/s 13.153 MB/s 17.976 GB FreeBSD - TSC em1 in 18.624 MB/s 18.832 MB/s 25.507 GB out17.008 MB/s 17.041 MB/s 19.681 GB FreeBSD - i8254 em1 in 6.763 MB/s 6.763 MB/s 26.005 GB out 6.756 MB/s 6.758 MB/s 20.151 GB FreeBSD - dummy em1 in 18.705 MB/s 18.796 MB/s 27.014 GB out17.217 MB/s 17.225 MB/s 21.082 GB I forgot to mention I was using -a to spoof the return address so time is not wasted receiving packets (since your goal was to syn flood the server). Probably there are other (possibly also bogus) gettimeofday calls that are still present in hping when receiving packets. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On 04/02/2008, Alexander Leidinger <[EMAIL PROTECTED]> wrote: > Ivan just change the page today. Feel free to suggest more specific things. Yes, I've added notes on syscall caching on Linux and socket buffer semantics. I've also linked to the page from Wikipedia article on FreeBSD - maybe it will help it to be noticed. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Stefan Lambrev wrote: Greetings, Kris Kennaway wrote: Kris Kennaway wrote: Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so that is the next place to look. [hit send too soon] Actually 15MB/sec once I disable all kernel debugging. This is identical to Linux 2.6.24 on the same hardware. The patch I use to fix hping brain-damage is attached. Kris Indeed this patch highly improve results (x2) under FreeBSD. On my hardware the max speeds under FreeBSD i still little slower compared to Linux (something like 19 vs 19.5 MB/s) but the speed is quite more stable under FreeBSD (under linux the average speed seems slower) I didn't tested the patch under linux but will do soon. Thanks for the help :) BTW 262144 is little high as this match the default value in FreeBSD, I tested with twice smaller buffer and do not see performance lost. Kris if you do not mind I'll write to hping developers to adopt this patch, and if no response from them I can try to reach the port maintainer, so we have this patched in ports? I have results from hping + patches with different counters in freebsd and linux: Linux - ACPI em1 in 22.257 MB/s 22.479 MB/s 15.384 GB out13.735 MB/s 14.247 MB/s 12.432 GB Linux -- TSC em1 in 22.334 MB/s 22.445 MB/s 18.599 GB out13.567 MB/s 14.176 MB/s 14.379 GB Linux - jiffies em1 in 22.119 MB/s 22.268 MB/s 20.078 GB out13.962 MB/s 14.058 MB/s 15.270 GB FreeBSD - ACPI em1 in 13.157 MB/s 13.162 MB/s 23.697 GB out13.150 MB/s 13.153 MB/s 17.976 GB FreeBSD - TSC em1 in 18.624 MB/s 18.832 MB/s 25.507 GB out17.008 MB/s 17.041 MB/s 19.681 GB FreeBSD - i8254 em1 in 6.763 MB/s 6.763 MB/s 26.005 GB out 6.756 MB/s 6.758 MB/s 20.151 GB FreeBSD - dummy em1 in 18.705 MB/s 18.796 MB/s 27.014 GB out17.217 MB/s 17.225 MB/s 21.082 GB It is weird that changing time counter in linux does not affect performance which may mean, that time counter is not changed at all (always TSC used?) ... But here are few others interesting findings 1) patch increase performance in linux too. 2) when flooding from linux the numbers of returned packets are less. 3) on both linux and freebsd the max combined speed is 35MB/s which is something like 630,000 packets/s Does anyone ever succeed to transfer without error more then 630kpps (300kpps incoming) with em driver and intel gigabit card? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Ivan Voras wrote: On 04/02/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote: Kris if you do not mind I'll write to hping developers to adopt this patch, and if no response from them I can try to reach the port maintainer, so we have this patched in ports? As we're in a "the whole world is a Linux" situation, would it be helpful to open a page on our wiki to enumerate differences like this, that would help people write and port applications to FreeBSD? I know the sort of advice given by Kris should be obvious to professional developers, but history shows that it is not and that people rather read blogs than manuals. That's the reason I started to dislike Linux last few years - because it's "defacto" standart in open source/unix like OS world, and their dev team act very much like MS dev team :( In the wiki there is a page about how to avoid "linuxism". I think we have to mention this there. It was the first place where I looked how to speed up hping. I think the info about gettimeofday (and replacements) and the ENOBUFS situation have to be explained little more there. I do not expect developers, that write programs for linux to read the FreeBSD wiki, but it will help people like me - not very experienced in coding, to produce patches/ports without bothering too much advanced developers :) It will be good if there are some examples too. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On 04/02/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote: > Kris if you do not mind I'll write to hping developers to adopt this > patch, and if no response from them I can try to reach the port > maintainer, so we have this patched in ports? As we're in a "the whole world is a Linux" situation, would it be helpful to open a page on our wiki to enumerate differences like this, that would help people write and port applications to FreeBSD? I know the sort of advice given by Kris should be obvious to professional developers, but history shows that it is not and that people rather read blogs than manuals. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: Greetings, Kris Kennaway wrote: Kris Kennaway wrote: Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so that is the next place to look. [hit send too soon] Actually 15MB/sec once I disable all kernel debugging. This is identical to Linux 2.6.24 on the same hardware. The patch I use to fix hping brain-damage is attached. Kris Indeed this patch highly improve results (x2) under FreeBSD. On my hardware the max speeds under FreeBSD i still little slower compared to Linux (something like 19 vs 19.5 MB/s) but the speed is quite more stable under FreeBSD (under linux the average speed seems slower) I didn't tested the patch under linux but will do soon. Thanks for the help :) BTW 262144 is little high as this match the default value in FreeBSD, I tested with twice smaller buffer and do not see performance lost. Kris if you do not mind I'll write to hping developers to adopt this patch, and if no response from them I can try to reach the port maintainer, so we have this patched in ports? My patch is a bit crude, because I didnt have the courage to digest the entire structure of the code, so it may be slightly wrong with respect to other operating modes (the #if 0 part is wrong, but that code needs to be rewritten). Some version of it should be included in the software, but perhaps after renaming the patch file ;-) Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Kris Kennaway wrote: Kris Kennaway wrote: Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so that is the next place to look. [hit send too soon] Actually 15MB/sec once I disable all kernel debugging. This is identical to Linux 2.6.24 on the same hardware. The patch I use to fix hping brain-damage is attached. Kris Indeed this patch highly improve results (x2) under FreeBSD. On my hardware the max speeds under FreeBSD i still little slower compared to Linux (something like 19 vs 19.5 MB/s) but the speed is quite more stable under FreeBSD (under linux the average speed seems slower) I didn't tested the patch under linux but will do soon. Thanks for the help :) BTW 262144 is little high as this match the default value in FreeBSD, I tested with twice smaller buffer and do not see performance lost. Kris if you do not mind I'll write to hping developers to adopt this patch, and if no response from them I can try to reach the port maintainer, so we have this patched in ports? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Kris Kennaway wrote: Stefan Lambrev wrote: I run from host A : hping --flood -p 22 -S 10.3.3.2 and systat -ifstat on host B to see the traffic that is generated (I do not want to run this monitoring on the flooder host as it will effect his performance) OK, I finally got time to look at this. Firstly, this is quite an inefficient program. It performs 5 syscalls for each packet that it sends: 2391 initial thread CALL sendto(0x3,0x61b050,0x28,0,0x5232c0,0x10) 2391 initial thread GIO fd 3 wrote 40 bytes 0x 4500 2800 7491 4006 0a00 0004 0a00 0001 3a96 0016 1865 a781 39d8 12aa 5002 0200 52c9 |E.([EMAIL PROTECTED]:e..9...P...R.| 0x0026 |..| 2391 initial thread RET sendto 40/0x28 2391 initial thread CALL sigaction(SIGALRM,0x7fffe6b0,0x7fffe690) 2391 initial thread RET sigaction 0 2391 initial thread CALL setitimer(0,0x7fffe6c0,0x7fffe6a0) 2391 initial thread RET setitimer 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 Here is a further litany of some of the ways in which this software is terrible: * It does not attempt to increase the socket buffer size (as we have already discussed), but * It also doesn't cope with the possibility that the packet may not be sent because the send buffer is full. * With every packet sent in flood mode it sets a timer for 1 second in the future even though we have told it not to send packets once a second but as fast as possible * We also set the signal handler with each packet sent, instead of setting it once and leaving it. * We call gettimeofday twice for each packet, once to retrieve the second timestamp and once to retrieve the microseconds. This is only for the purpose of computing the RTT. However, we can only keep track of 400 in-flight packets, which means this is also useless in flood mode. * The suspend handler does not work * This does not strike me as quality software :) Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so The "doesn't cope with the possibility that ... the send buffer is full" issue is classic linux-specific mis-behaviour. On linux the process will block when the default qdisc finds the device q is stopped (due to being full). I remember cursing iperf for this. Sam ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Kris Kennaway wrote:
Fixing all of the above I can send at about 13MB/sec (timecounter is
not relevant any more). The CPU is spending about 75% of the time in
the kernel, so
that is the next place to look. [hit send too soon]
Actually 15MB/sec once I disable all kernel debugging. This is
identical to Linux 2.6.24 on the same hardware. The patch I use to fix
hping brain-damage is attached.
Kris
diff -ru work/hping3-20051105/opensockraw.c work~/hping3-20051105/opensockraw.c
--- opensockraw.c.orig 2003-09-01 00:22:06.0 +
+++ opensockraw.c 2008-02-03 19:45:28.0 +
@@ -17,7 +17,7 @@
int open_sockraw()
{
- int s;
+ int s, t, val;
s = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);
if (s == -1) {
@@ -25,5 +25,12 @@
return -1;
}
+ val = 262144;
+ t = setsockopt(s, SOL_SOCKET, SO_SNDBUF, &val, sizeof(val));
+ if (t == -1) {
+ perror("[open_sockraw] setsockopt()");
+ return -1;
+ }
+
return s;
}
diff -ru work/hping3-20051105/sendip.c work~/hping3-20051105/sendip.c
--- sendip.c.orig 2004-04-09 23:38:56.0 +
+++ sendip.c2008-02-03 19:50:35.0 +
@@ -110,7 +110,7 @@
result = sendto(sockraw, packet, packetsize, 0,
(struct sockaddr*)&remote, sizeof(remote));
- if (result == -1 && errno != EINTR && !opt_rand_dest &&
!opt_rand_source) {
+ if (result == -1 && errno != ENOBUFS && errno != EINTR &&
!opt_rand_dest && !opt_rand_source) {
perror("[send_ip] sendto");
if (close(sockraw) == -1)
perror("[ipsender] close(sockraw)");
diff -ru work/hping3-20051105/sendtcp.c work~/hping3-20051105/sendtcp.c
--- sendtcp.c.orig 2003-09-01 00:22:06.0 +
+++ sendtcp.c 2008-02-03 20:30:51.0 +
@@ -85,8 +85,10 @@
packet_size);
#endif
+#if 0
/* adds this pkt in delaytable */
delaytable_add(sequence, src_port, time(NULL), get_usec(), S_SENT);
+#endif
/* send packet */
send_ip_handler(packet+PSEUDOHDR_SIZE, packet_size);
--- send.c.orig 2003-08-31 17:23:53.0 +
+++ send.c 2008-02-03 21:58:59.0 +
@@ -63,6 +63,8 @@
}
}
+static int sigalarm_handler = 0;
+
/* The signal handler for SIGALRM will send the packets */
void send_packet (int signal_id)
{
@@ -79,12 +81,15 @@
elsesend_tcp();
sent_pkt++;
- Signal(SIGALRM, send_packet);
+ if (!opt_flood && !sigalarm_handler) {
+ Signal(SIGALRM, send_packet);
+ sigalarm_handler = 1;
+ }
if (count != -1 && count == sent_pkt) { /* count reached? */
Signal(SIGALRM, print_statistics);
alarm(COUNTREACHED_TIMEOUT);
- } else if (!opt_listenmode) {
+ } else if (!opt_listenmode && !opt_flood) {
if (opt_waitinusec == FALSE)
alarm(sending_wait);
else
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Kris Kennaway wrote: Stefan Lambrev wrote: I run from host A : hping --flood -p 22 -S 10.3.3.2 and systat -ifstat on host B to see the traffic that is generated (I do not want to run this monitoring on the flooder host as it will effect his performance) OK, I finally got time to look at this. Firstly, this is quite an inefficient program. It performs 5 syscalls for each packet that it sends: 2391 initial thread CALL sendto(0x3,0x61b050,0x28,0,0x5232c0,0x10) 2391 initial thread GIO fd 3 wrote 40 bytes 0x 4500 2800 7491 4006 0a00 0004 0a00 0001 3a96 0016 1865 a781 39d8 12aa 5002 0200 52c9 |E.([EMAIL PROTECTED]:e..9...P...R.| 0x0026 |..| 2391 initial thread RET sendto 40/0x28 2391 initial thread CALL sigaction(SIGALRM,0x7fffe6b0,0x7fffe690) 2391 initial thread RET sigaction 0 2391 initial thread CALL setitimer(0,0x7fffe6c0,0x7fffe6a0) 2391 initial thread RET setitimer 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 Here is a further litany of some of the ways in which this software is terrible: * It does not attempt to increase the socket buffer size (as we have already discussed), but * It also doesn't cope with the possibility that the packet may not be sent because the send buffer is full. * With every packet sent in flood mode it sets a timer for 1 second in the future even though we have told it not to send packets once a second but as fast as possible * We also set the signal handler with each packet sent, instead of setting it once and leaving it. * We call gettimeofday twice for each packet, once to retrieve the second timestamp and once to retrieve the microseconds. This is only for the purpose of computing the RTT. However, we can only keep track of 400 in-flight packets, which means this is also useless in flood mode. * The suspend handler does not work * This does not strike me as quality software :) Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so that is the next place to look. [hit send too soon] Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: I run from host A : hping --flood -p 22 -S 10.3.3.2 and systat -ifstat on host B to see the traffic that is generated (I do not want to run this monitoring on the flooder host as it will effect his performance) OK, I finally got time to look at this. Firstly, this is quite an inefficient program. It performs 5 syscalls for each packet that it sends: 2391 initial thread CALL sendto(0x3,0x61b050,0x28,0,0x5232c0,0x10) 2391 initial thread GIO fd 3 wrote 40 bytes 0x 4500 2800 7491 4006 0a00 0004 0a00 0001 3a96 0016 1865 a781 39d8 12aa 5002 0200 52c9 |E.([EMAIL PROTECTED]:e..9...P...R.| 0x0026 |..| 2391 initial thread RET sendto 40/0x28 2391 initial thread CALL sigaction(SIGALRM,0x7fffe6b0,0x7fffe690) 2391 initial thread RET sigaction 0 2391 initial thread CALL setitimer(0,0x7fffe6c0,0x7fffe6a0) 2391 initial thread RET setitimer 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 2391 initial thread CALL gettimeofday(0x7fffe680,0) 2391 initial thread RET gettimeofday 0 Here is a further litany of some of the ways in which this software is terrible: * It does not attempt to increase the socket buffer size (as we have already discussed), but * It also doesn't cope with the possibility that the packet may not be sent because the send buffer is full. * With every packet sent in flood mode it sets a timer for 1 second in the future even though we have told it not to send packets once a second but as fast as possible * We also set the signal handler with each packet sent, instead of setting it once and leaving it. * We call gettimeofday twice for each packet, once to retrieve the second timestamp and once to retrieve the microseconds. This is only for the purpose of computing the RTT. However, we can only keep track of 400 in-flight packets, which means this is also useless in flood mode. * The suspend handler does not work * This does not strike me as quality software :) Fixing all of the above I can send at about 13MB/sec (timecounter is not relevant any more). The CPU is spending about 75% of the time in the kernel, so Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings,
Stefan Lambrev wrote:
Greetings,
Kris Kennaway wrote:
Stefan Lambrev wrote:
It is the socket buffer that is filling up. Either the application
is not increasing it to large enough size or the default maximum is
too low (Linux may set a larger default). Try increasing
kern.ipc.maxsockbuf and confirming with the source and/or ktrace
that it is doing the right setsockopt() call.
Increasing kern.ipc.maxsockbuf doesn't help.
Actually this is the code that failed and print this error:
result = sendto(sockraw, packet, packetsize, 0,
(struct sockaddr*)&remote, sizeof(remote));
if (result == -1 && errno != EINTR && !opt_rand_dest &&
!opt_rand_source) {
perror("[send_ip] sendto");
Those are the only references for setsockopt when ktracing:
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0x28305180,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
3385 hpingRET socket 3
3385 hpingCALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe884,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL connect(0x3,0x8067da0,0x10)
3385 hpingRET connect 0
3385 hpingCALL getsockname(0x3,0xbfbfe874,0xbfbfe888)
3385 hpingRET getsockname 0
3385 hpingCALL close(0x3)
3385 hpingRET close 0
3385 hpingCALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
3385 hpingRET socket 3
3385 hpingCALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL setsockopt(0x3,0,0x2,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf0"
3385 hpingRET open -1 errno 16 Device busy
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf1"
3385 hpingRET open 4
OK, try adding the setsockopt(...SO_SNDBUF...) call.
Will something like this do the trick?
void socket_sndbuf(int sd)
{
long int bufsize;
bufsize = 65536;
if (setsockopt(sd, SOL_SOCKET, SO_SNDBUF,
(char *)&bufsize, sizeof(int)) == -1)
{
printf("[socket_sndbuf] can't set SO_SNDBUF option\n");
}
}
I'm not a C developer so pardon me if I made something stupid :)
Also how can I make bufsize = default settings*2 for example?
I tried this code and here is what ktrace show now:
65372 hping3 CALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
65372 hping3 RET socket 3
65372 hping3 CALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe844,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL connect(0x3,0x8067e20,0x10)
65372 hping3 RET connect 0
65372 hping3 CALL getsockname(0x3,0xbfbfe834,0xbfbfe848)
65372 hping3 RET getsockname 0
65372 hping3 CALL close(0x3)
65372 hping3 RET close 0
65372 hping3 CALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
65372 hping3 RET socket 3
65372 hping3 CALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL setsockopt(0x3,0,0x2,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL setsockopt(0x3,SOL_SOCKET,SO_SNDBUF,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
Kris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to
"[EMAIL PROTECTED]"
I finally managed to get ktrace of falling hping
2250 hping3 RET sendto 40/0x28
2250 hping3 CALL sigaction(SIGALRM,0x7fffe7b0,0x7fffe790)
2250 hping3 RET sigaction 0
2250 hping3 CALL setitimer(0,0x7fffe7c0,0x7fffe7a0)
2250 hping3 RET setitimer 0
2250 hping3 CALL gettimeofday(0x7fffe780,0)
2250 hping3 RET gettimeofday 0
2250 hping3 CALL gettimeofday(0x7fffe780,0)
2250 hping3 RET gettimeofday 0
2250 hping3 CALL sendto(0x3,0x800e1b050,0x28,0,0x522600,0x10)
2250 hping3 GIO fd 3 wrote 40 bytes
0x 4500 2800 c3f0 4006 0a03 0303 0a03 0301 9570
0050 6b32 4398 30f3 e723 5002 0200 3737
|E.([EMAIL PROTECTED]|
2250 hping3 RET sendto 40/0x28
2250 hping3 CALL sigaction(SIGALRM,0x7fffe7b0,0x7fffe790)
2250 hping3 RET sigaction 0
2250 hping3 CALL setitimer(0,0x7fffe7c0,0x7fffe7a0)
2250 hping3 RET setitimer 0
2250 hping3 CALL gettimeofday(0x7fffe780,0)
2250 hping3 RET gettimeofday 0
2250 hping3 CALL gettimeofday(0x7fffe780,0)
2250 hping3 RET gettimeofday 0
2250 hping3 CALL sendto(0x3,0x800e1b050,0x28,0,0x522600,0x10)
2250 hping3 RET sendto -1 errno 55 No buffer space available
2250 hping3 CALL writev(0x2,0x7fffe6a0,0x4)
2250 hping3 GIO fd 2 wrote 44 bytes
"[send_ip] sendto: No buffer space available
"
2250 hping
Re: gettimeofday() in hping
Greetings, Kris Kennaway wrote: Joseph Koshy wrote: OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 "Famous problem" indeed :). I declined the patch because it is incorrect and incomplete. I will accept a patch that demonstrates clue about the workings of the overall system---the changes in the patch should be safe, complete, should demonstrate that the submitter has read and understood vendor documentation, should preserve user experience for naming events, and each supported PMC event needs to be documented in pmc.3. I am aware of these issues but repeat my statement that the lack of working pmc on modern CPUs is causing serious difficulties for our developer and user base, as witnessed again in this thread. Kris Kris do you think the information, that I send you (private email) is useful ? :) Also using hwpmc with your patch shows a serious problem with pf and dynamic rules. I have desktop PC at home where hwpmc work out of the box, but I'm running 6.3 on it. If the information that I send you is not useful, I can spend my weekend upgrading my desktop PC to RELENG_7_0 and providing new pmc stats, but I have to be sure that developers are interested in fixing those issues. I'm willing to invest my time in this, but my skills are not enough to solve the issues alone, and without help I'll just waste my time. How hping works is not my biggest problem - for me as I said on few other mail list the real showstopper is pf and it's keep-state feature. The interesting part is that both problems point to kernel spending most of it's time in _mtx_lock_sleep(): pmcstat - pf - during syn flood: % cumulative self self total time seconds secondscalls ms/call ms/call name 24.0 268416.00 268416.000 100.00% _mtx_lock_sleep [1] 6.7 343572.50 75156.500 100.00% pf_state_compare_ext_gwy [2] 6.7 418405.50 74833.000 100.00% pf_src_compare [3] 3.9 462298.50 43893.000 100.00% pf_state_compare_lan_ext [4] 3.6 503019.50 40721.000 100.00% pf_test [5] pmcstat - hping: % cumulative self self total time seconds secondscalls ms/call ms/call name 8.6 116120.00 116120.000 100.00% _mtx_lock_sleep [1] 5.5 190764.00 74644.000 100.00% syscall [2] 3.0 231390.00 40626.000 100.00% bpf_mtap [3] 2.9 270334.00 38944.000 100.00% Xfast_syscall [4] 2.5 304458.00 34124.000 100.00% bus_dmamap_load_mbuf_sg [5] 2.3 335825.00 31367.000 100.00% uma_zalloc_arg [6] P.S. my desktop PC i single core, but I think I'll find old server with 2x intel p4 CPUs. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Joseph Koshy wrote: OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 "Famous problem" indeed :). I declined the patch because it is incorrect and incomplete. I will accept a patch that demonstrates clue about the workings of the overall system---the changes in the patch should be safe, complete, should demonstrate that the submitter has read and understood vendor documentation, should preserve user experience for naming events, and each supported PMC event needs to be documented in pmc.3. I am aware of these issues but repeat my statement that the lack of working pmc on modern CPUs is causing serious difficulties for our developer and user base, as witnessed again in this thread. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
> OK, this is the famous problem with modern CPUs that jkoshy has declined > to work around :( There are patches for this in perforce, see > > http://perforce.freebsd.org/changeView.cgi?CH=126189 "Famous problem" indeed :). I declined the patch because it is incorrect and incomplete. First, you cannot safely treat the Core and Core 2 PMCs as P6 PMCs. Doing so would give userland a way to program CPU MSRs with bit patterns that have been expressly documented by the manufacturer as "reserved" and forbidden. Second, the change ignores the sometimes changed semantics between P6 and Core PMCs for events sharing the same numeric selector value. Not handling the changed semantics could easily trip up users; they would think that they are measuring hardware behaviour X while the measurements would actually be for behaviour Y. Third, the change ignores the user interface. libpmc's event naming conventions are to follow vendor names for events and event modifiers as closely as possible (within the constraints of the overall generic syntax). This is so as to make it easy (easier?) for users to read vendor documentation and map the information there to the necessary event selection syntax. Core PMCs support tons of new measurement events. Additionally, naming conventions in vendor documentation for Core event names and modifiers are different from the older P6 names, even for events with similar semantics. The submitted patch does not address these aspects. I will accept a patch that demonstrates clue about the workings of the overall system---the changes in the patch should be safe, complete, should demonstrate that the submitter has read and understood vendor documentation, should preserve user experience for naming events, and each supported PMC event needs to be documented in pmc.3. Regards, Koshy ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, binto wrote: Hi, Sorry if a little bit insist & curious. what is result from: sysctl -a kern.ipc.maxsockbuf sysctl -a net.inet.tcp.recvspace net.inet.tcp.sendspace: 32768 net.inet.tcp.recvspace: 65536 kern.ipc.maxsockbuf: 262144 kern.ipc.nmbclusters: 262144 -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Hi,
Sorry if a little bit insist & curious.
what is result from:
sysctl -a kern.ipc.maxsockbuf
sysctl -a net.inet.tcp.recvspace
sysctl -a net.inet.tcp.sendspace
??
binto
> Greetings,
>
> Kris Kennaway wrote:
>> Stefan Lambrev wrote:
>>
It is the socket buffer that is filling up. Either the application
is not increasing it to large enough size or the default maximum is
too low (Linux may set a larger default). Try increasing
kern.ipc.maxsockbuf and confirming with the source and/or ktrace
that it is doing the right setsockopt() call.
>>> Increasing kern.ipc.maxsockbuf doesn't help.
>>>
>>> Actually this is the code that failed and print this error:
>>>
>>>result = sendto(sockraw, packet, packetsize, 0,
>>>(struct sockaddr*)&remote, sizeof(remote));
>>>
>>>if (result == -1 && errno != EINTR && !opt_rand_dest &&
>>> !opt_rand_source) {
>>>perror("[send_ip] sendto");
>>>
>>> Those are the only references for setsockopt when ktracing:
>>> 3385 hpingCALL __sysctl(0xbfbfe870,0x6,0,0xbfbfe888,0,0)
>>> 3385 hpingRET __sysctl 0
>>> 3385 hpingCALL __sysctl(0xbfbfe870,0x6,0x28305180,0xbfbfe888,0,0)
>>> 3385 hpingRET __sysctl 0
>>> 3385 hpingCALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
>>> 3385 hpingRET socket 3
>>> 3385 hpingCALL
>>> setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe884,0x4)
>>> 3385 hpingRET setsockopt 0
>>> 3385 hpingCALL connect(0x3,0x8067da0,0x10)
>>> 3385 hpingRET connect 0
>>> 3385 hpingCALL getsockname(0x3,0xbfbfe874,0xbfbfe888)
>>> 3385 hpingRET getsockname 0
>>> 3385 hpingCALL close(0x3)
>>> 3385 hpingRET close 0
>>> 3385 hpingCALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
>>> 3385 hpingRET socket 3
>>> 3385 hpingCALL
>>> setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe914,0x4)
>>> 3385 hpingRET setsockopt 0
>>> 3385 hpingCALL setsockopt(0x3,0,0x2,0xbfbfe914,0x4)
>>> 3385 hpingRET setsockopt 0
>>> 3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
>>> 3385 hpingNAMI "/dev/bpf0"
>>> 3385 hpingRET open -1 errno 16 Device busy
>>> 3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
>>> 3385 hpingNAMI "/dev/bpf1"
>>> 3385 hpingRET open 4
>>
>> OK, try adding the setsockopt(...SO_SNDBUF...) call.
> Will something like this do the trick?
>
> void socket_sndbuf(int sd)
> {
> long int bufsize;
> bufsize = 65536;
> if (setsockopt(sd, SOL_SOCKET, SO_SNDBUF,
> (char *)&bufsize, sizeof(int)) == -1)
> {
> printf("[socket_sndbuf] can't set SO_SNDBUF option\n");
> }
> }
>
> I'm not a C developer so pardon me if I made something stupid :)
> Also how can I make bufsize = default settings*2 for example?
>
> I tried this code and here is what ktrace show now:
> 65372 hping3 CALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
> 65372 hping3 RET socket 3
> 65372 hping3 CALL
> setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe844,0x4)
> 65372 hping3 RET setsockopt 0
> 65372 hping3 CALL connect(0x3,0x8067e20,0x10)
> 65372 hping3 RET connect 0
> 65372 hping3 CALL getsockname(0x3,0xbfbfe834,0xbfbfe848)
> 65372 hping3 RET getsockname 0
> 65372 hping3 CALL close(0x3)
> 65372 hping3 RET close 0
> 65372 hping3 CALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
> 65372 hping3 RET socket 3
> 65372 hping3 CALL
> setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe8d4,0x4)
> 65372 hping3 RET setsockopt 0
> 65372 hping3 CALL setsockopt(0x3,0,0x2,0xbfbfe8d4,0x4)
> 65372 hping3 RET setsockopt 0
> 65372 hping3 CALL setsockopt(0x3,SOL_SOCKET,SO_SNDBUF,0xbfbfe8d4,0x4)
> 65372 hping3 RET setsockopt 0
>
>>
>> Kris
>> ___
>> freebsd-hackers@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to
>> "[EMAIL PROTECTED]"
>
> --
>
> Best Wishes,
> Stefan Lambrev
> ICQ# 24134177
>
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "[EMAIL PROTECTED]"
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings,
Kris Kennaway wrote:
Stefan Lambrev wrote:
It is the socket buffer that is filling up. Either the application
is not increasing it to large enough size or the default maximum is
too low (Linux may set a larger default). Try increasing
kern.ipc.maxsockbuf and confirming with the source and/or ktrace
that it is doing the right setsockopt() call.
Increasing kern.ipc.maxsockbuf doesn't help.
Actually this is the code that failed and print this error:
result = sendto(sockraw, packet, packetsize, 0,
(struct sockaddr*)&remote, sizeof(remote));
if (result == -1 && errno != EINTR && !opt_rand_dest &&
!opt_rand_source) {
perror("[send_ip] sendto");
Those are the only references for setsockopt when ktracing:
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0x28305180,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
3385 hpingRET socket 3
3385 hpingCALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe884,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL connect(0x3,0x8067da0,0x10)
3385 hpingRET connect 0
3385 hpingCALL getsockname(0x3,0xbfbfe874,0xbfbfe888)
3385 hpingRET getsockname 0
3385 hpingCALL close(0x3)
3385 hpingRET close 0
3385 hpingCALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
3385 hpingRET socket 3
3385 hpingCALL
setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL setsockopt(0x3,0,0x2,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf0"
3385 hpingRET open -1 errno 16 Device busy
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf1"
3385 hpingRET open 4
OK, try adding the setsockopt(...SO_SNDBUF...) call.
Will something like this do the trick?
void socket_sndbuf(int sd)
{
long int bufsize;
bufsize = 65536;
if (setsockopt(sd, SOL_SOCKET, SO_SNDBUF,
(char *)&bufsize, sizeof(int)) == -1)
{
printf("[socket_sndbuf] can't set SO_SNDBUF option\n");
}
}
I'm not a C developer so pardon me if I made something stupid :)
Also how can I make bufsize = default settings*2 for example?
I tried this code and here is what ktrace show now:
65372 hping3 CALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
65372 hping3 RET socket 3
65372 hping3 CALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe844,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL connect(0x3,0x8067e20,0x10)
65372 hping3 RET connect 0
65372 hping3 CALL getsockname(0x3,0xbfbfe834,0xbfbfe848)
65372 hping3 RET getsockname 0
65372 hping3 CALL close(0x3)
65372 hping3 RET close 0
65372 hping3 CALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
65372 hping3 RET socket 3
65372 hping3 CALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL setsockopt(0x3,0,0x2,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
65372 hping3 CALL setsockopt(0x3,SOL_SOCKET,SO_SNDBUF,0xbfbfe8d4,0x4)
65372 hping3 RET setsockopt 0
Kris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to
"[EMAIL PROTECTED]"
--
Best Wishes,
Stefan Lambrev
ICQ# 24134177
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote:
It is the socket buffer that is filling up. Either the application is
not increasing it to large enough size or the default maximum is too
low (Linux may set a larger default). Try increasing
kern.ipc.maxsockbuf and confirming with the source and/or ktrace that
it is doing the right setsockopt() call.
Increasing kern.ipc.maxsockbuf doesn't help.
Actually this is the code that failed and print this error:
result = sendto(sockraw, packet, packetsize, 0,
(struct sockaddr*)&remote, sizeof(remote));
if (result == -1 && errno != EINTR && !opt_rand_dest &&
!opt_rand_source) {
perror("[send_ip] sendto");
Those are the only references for setsockopt when ktracing:
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0x28305180,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
3385 hpingRET socket 3
3385 hpingCALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe884,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL connect(0x3,0x8067da0,0x10)
3385 hpingRET connect 0
3385 hpingCALL getsockname(0x3,0xbfbfe874,0xbfbfe888)
3385 hpingRET getsockname 0
3385 hpingCALL close(0x3)
3385 hpingRET close 0
3385 hpingCALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
3385 hpingRET socket 3
3385 hpingCALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL setsockopt(0x3,0,0x2,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf0"
3385 hpingRET open -1 errno 16 Device busy
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf1"
3385 hpingRET open 4
OK, try adding the setsockopt(...SO_SNDBUF...) call.
Kris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: You also need changes to the userland libpmc and pmcstat. They should also be in that (or related) p4 changeset though. Those are the files that I fetched from p4 /usr/src/lib/libpmc/libpmc.c - rev2 /usr/src/sys/amd64/include/pmc_mdep.h rev2 /usr/src/sys/dev/hwpmc/hwpmc_x86.c rev2 /usr/src/sys/modules/hwpmc/Makefile rev3 but I rebuild only kernel - I'll try to rebuild everything :) do you think this will help getting hwpmc working? I can only say that it mostly works for me :) Btw I have hwpmc working on other machine (it's with FreeBSD 6.3-prerelease). Can you tell me what stats exactly you need from running hping? A gprof trace with the -S instructions counter should be enough to get oriented. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Hi Kris, Kris Kennaway wrote: Stefan Lambrev wrote: Hi Kris, Kris Kennaway wrote: Stefan Lambrev wrote: Kris Kennaway wrote: Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 I got hwpmc to compile (but only as module) hwpmc: TSC/1/0x20 P6/2/0x1ff but when I try to use pmcstat I receive this error message: pmcstat: ERROR: Initialization of the pmc(3) library failed: Device not configured kldstat lists hwpmc.ko as loaded and I have options HWPMC_HOOKS in my kernel. Any idea what's go wrong? You also need changes to the userland libpmc and pmcstat. They should also be in that (or related) p4 changeset though. Those are the files that I fetched from p4 /usr/src/lib/libpmc/libpmc.c - rev2 /usr/src/sys/amd64/include/pmc_mdep.h rev2 /usr/src/sys/dev/hwpmc/hwpmc_x86.c rev2 /usr/src/sys/modules/hwpmc/Makefile rev3 but I rebuild only kernel - I'll try to rebuild everything :) do you think this will help getting hwpmc working? Btw I have hwpmc working on other machine (it's with FreeBSD 6.3-prerelease). Can you tell me what stats exactly you need from running hping? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greets,
Kris Kennaway wrote:
Ivan Voras wrote:
On 23/01/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote:
Greets,
Now I have final results with Linux and FreeBSD on the same hardware
CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz - dual core
Lan: [EMAIL PROTECTED]:3:0:0: class=0x02 card=0x10bc8086 chip=0x10bc8086
rev=0x06 hdr=0x00
vendor = 'Intel Corporation'
device = '82571EB Gigabit Ethernet Controller (Copper)'
class = network
subclass = ethernet
FreeBSD releng_7_0 from today - amd64, sched_ule.
ACPI-Fast - 6.187 MB/s
TSC - 9.455 MB/s
dummy - 9.577 MB/s
Linux rambo2 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007
x86_64 GNU/Linux - kubuntu
TSC - 19.456 MB/s
acpi_pm - 15.394 MB/s
jiffies - 19.480 MB/s
This is really not what I expected.
For once, it's something I expected :) I just hope it isn't one of
those cases where Kris absolutely cannot reproduce it and arrives at
numbers in favour of FreeBSD :)
(just joking here, absolutely no ill feelings involved).
Harumph :) The first step is that we need to understand where the
application is spending its time. Hopefully Stefan or someone else
will be able to test it under hwpmc.
It would be helpful if you post exact command line arguments from all
cases.
The other thing that bothers me is, that under freebsd is quite easy
to get:
[send_ip] sendto: No buffer space available
It happens almost always on my laptop just few seconds after I start
hping with timecounter=TSC
I'm not sure, but from what I understood of Robert Watson's
explanation in the big ZFS thread on -current, maybe increasing
kmem_size (exactly as for ZFS...) could help you with these buffers.
It is the socket buffer that is filling up. Either the application is
not increasing it to large enough size or the default maximum is too
low (Linux may set a larger default). Try increasing
kern.ipc.maxsockbuf and confirming with the source and/or ktrace that
it is doing the right setsockopt() call.
Increasing kern.ipc.maxsockbuf doesn't help.
Actually this is the code that failed and print this error:
result = sendto(sockraw, packet, packetsize, 0,
(struct sockaddr*)&remote, sizeof(remote));
if (result == -1 && errno != EINTR && !opt_rand_dest &&
!opt_rand_source) {
perror("[send_ip] sendto");
Those are the only references for setsockopt when ktracing:
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL __sysctl(0xbfbfe870,0x6,0x28305180,0xbfbfe888,0,0)
3385 hpingRET __sysctl 0
3385 hpingCALL socket(PF_INET,SOCK_DGRAM,IPPROTO_IP)
3385 hpingRET socket 3
3385 hpingCALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe884,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL connect(0x3,0x8067da0,0x10)
3385 hpingRET connect 0
3385 hpingCALL getsockname(0x3,0xbfbfe874,0xbfbfe888)
3385 hpingRET getsockname 0
3385 hpingCALL close(0x3)
3385 hpingRET close 0
3385 hpingCALL socket(PF_INET,SOCK_RAW,IPPROTO_RAW)
3385 hpingRET socket 3
3385 hpingCALL setsockopt(0x3,SOL_SOCKET,SO_BROADCAST,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL setsockopt(0x3,0,0x2,0xbfbfe914,0x4)
3385 hpingRET setsockopt 0
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf0"
3385 hpingRET open -1 errno 16 Device busy
3385 hpingCALL open(0xbfbfe8a4,O_RDWR,0)
3385 hpingNAMI "/dev/bpf1"
3385 hpingRET open 4
--
Best Wishes,
Stefan Lambrev
ICQ# 24134177
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greets, binto wrote: Greetings, Steven Hartland wrote: - Original Message - From: "Ivan Voras" <[EMAIL PROTECTED]> The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. Is this not just running out of mbufs? netstat -m will show if it is and the fix is to just increase kern.ipc.nmbclusters. 670/1520/2190 mbufs in use (current/cache/total) 462/322/784/25600 mbuf clusters in use (current/cache/total/max) 462/306 mbuf+clusters out of packet secondary zone in use (current/cache) 0/35/35/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1091K/1164K/2255K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/4/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines kern.ipc.nmbclusters: 25600 sysctl -a kern.ipc.maxsockbuf ? No, this doesn't change things. with timecounter=TSC it's easily reproduced on on 6.3. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
> Greetings, > > Steven Hartland wrote: >> - Original Message - From: "Ivan Voras" <[EMAIL PROTECTED]> The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC >>> >>> I'm not sure, but from what I understood of Robert Watson's >>> explanation in the big ZFS thread on -current, maybe increasing >>> kmem_size (exactly as for ZFS...) could help you with these buffers. >> >> Is this not just running out of mbufs? netstat -m will show if it is >> and the fix >> is to just increase kern.ipc.nmbclusters. > 670/1520/2190 mbufs in use (current/cache/total) > 462/322/784/25600 mbuf clusters in use (current/cache/total/max) > 462/306 mbuf+clusters out of packet secondary zone in use (current/cache) > 0/35/35/12800 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 1091K/1164K/2255K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/4/6656 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > kern.ipc.nmbclusters: 25600 sysctl -a kern.ipc.maxsockbuf ? if net.inet.tcp.recvspace / net.inet.tcp.sendspace greater than kern.ipc.maxsockbuf that can also make : "No buffer space available" > > I do not think I'm running out of mbufs. > And increasing nmbclusters doesn't help. > > Here is what I have for kmem_size. How can I see how much of the kmem > size is used ? vmstat -m :) > > vm.kmem_size_scale: 3 > vm.kmem_size_max: 335544320 > vm.kmem_size_min: 0 > vm.kmem_size: 335544320 > > Something suspicious that I notice is: > vmstat -m|grep devbuf >devbuf 5214 42780K - 6390 > 16,32,64,128,256,512,1024,2048,4096 > > 42MB memory allocated for devbuf ? Is this ok ? > > This is the only thing that 'eat' more then 1-2MB memory reported by > vmstat -m. > > -- > > Best Wishes, > Stefan Lambrev > ICQ# 24134177 > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Ivan Voras wrote: On 23/01/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote: Greets, Now I have final results with Linux and FreeBSD on the same hardware CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz - dual core Lan: [EMAIL PROTECTED]:3:0:0: class=0x02 card=0x10bc8086 chip=0x10bc8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet FreeBSD releng_7_0 from today - amd64, sched_ule. ACPI-Fast - 6.187 MB/s TSC - 9.455 MB/s dummy - 9.577 MB/s Linux rambo2 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007 x86_64 GNU/Linux - kubuntu TSC - 19.456 MB/s acpi_pm - 15.394 MB/s jiffies - 19.480 MB/s This is really not what I expected. For once, it's something I expected :) I just hope it isn't one of those cases where Kris absolutely cannot reproduce it and arrives at numbers in favour of FreeBSD :) (just joking here, absolutely no ill feelings involved). Harumph :) The first step is that we need to understand where the application is spending its time. Hopefully Stefan or someone else will be able to test it under hwpmc. It would be helpful if you post exact command line arguments from all cases. The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. It is the socket buffer that is filling up. Either the application is not increasing it to large enough size or the default maximum is too low (Linux may set a larger default). Try increasing kern.ipc.maxsockbuf and confirming with the source and/or ktrace that it is doing the right setsockopt() call. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: Hi Kris, Kris Kennaway wrote: Stefan Lambrev wrote: Kris Kennaway wrote: Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 I got hwpmc to compile (but only as module) hwpmc: TSC/1/0x20 P6/2/0x1ff but when I try to use pmcstat I receive this error message: pmcstat: ERROR: Initialization of the pmc(3) library failed: Device not configured kldstat lists hwpmc.ko as loaded and I have options HWPMC_HOOKS in my kernel. Any idea what's go wrong? You also need changes to the userland libpmc and pmcstat. They should also be in that (or related) p4 changeset though. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Steven Hartland wrote: - Original Message - From: "Ivan Voras" <[EMAIL PROTECTED]> The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. Is this not just running out of mbufs? netstat -m will show if it is and the fix is to just increase kern.ipc.nmbclusters. 670/1520/2190 mbufs in use (current/cache/total) 462/322/784/25600 mbuf clusters in use (current/cache/total/max) 462/306 mbuf+clusters out of packet secondary zone in use (current/cache) 0/35/35/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1091K/1164K/2255K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/4/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines kern.ipc.nmbclusters: 25600 I do not think I'm running out of mbufs. And increasing nmbclusters doesn't help. Here is what I have for kmem_size. How can I see how much of the kmem size is used ? vmstat -m :) vm.kmem_size_scale: 3 vm.kmem_size_max: 335544320 vm.kmem_size_min: 0 vm.kmem_size: 335544320 Something suspicious that I notice is: vmstat -m|grep devbuf devbuf 5214 42780K - 6390 16,32,64,128,256,512,1024,2048,4096 42MB memory allocated for devbuf ? Is this ok ? This is the only thing that 'eat' more then 1-2MB memory reported by vmstat -m. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Hi, Ivan Voras wrote: On 23/01/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote: Greets, Now I have final results with Linux and FreeBSD on the same hardware CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz - dual core Lan: [EMAIL PROTECTED]:3:0:0: class=0x02 card=0x10bc8086 chip=0x10bc8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet FreeBSD releng_7_0 from today - amd64, sched_ule. ACPI-Fast - 6.187 MB/s TSC - 9.455 MB/s dummy - 9.577 MB/s Linux rambo2 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007 x86_64 GNU/Linux - kubuntu TSC - 19.456 MB/s acpi_pm - 15.394 MB/s jiffies - 19.480 MB/s This is really not what I expected. For once, it's something I expected :) I just hope it isn't one of those cases where Kris absolutely cannot reproduce it and arrives at numbers in favour of FreeBSD :) (just joking here, absolutely no ill feelings involved). It would be helpful if you post exact command line arguments from all cases. hping is quite simple program - jsut: cd /usr/ports/net/hping-devel && make install Here are my goals, configuration and problems - http://lists.freebsd.org/pipermail/freebsd-performance/2008-January/003071.html For this test, where I benchmark freebsd and linux I just have 2 servers connected with cable (no switch) Host A (flooder) 10.3.3.1 and host B (target) 10.3.3.2 I run from host A : hping --flood -p 22 -S 10.3.3.2 and systat -ifstat on host B to see the traffic that is generated (I do not want to run this monitoring on the flooder host as it will effect his performance) After few minutes running I change the kern.timecounter.hardware to next available counter and move to next test. On linux (kubuntu) you can change the counter by executing: echo tsc > /sys/devices/system/clocksource/clocksource0/current_clocksource and cat /sys/devices/system/clocksource/clocksource0/current_clocksource is the alternative of sysctl kern.timecounter.choice Also I understand that hping is probably written with linux in mind, so is there something else, that is more bsd native and will let me accomplish my goals? :) I need small tool to flood the network and test my bridge firewall. The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
- Original Message - From: "Ivan Voras" <[EMAIL PROTECTED]> The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. Is this not just running out of mbufs? netstat -m will show if it is and the fix is to just increase kern.ipc.nmbclusters. Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On 23/01/2008, Stefan Lambrev <[EMAIL PROTECTED]> wrote: > Greets, > > Now I have final results with Linux and FreeBSD on the same hardware > CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz - dual core > Lan: [EMAIL PROTECTED]:3:0:0: class=0x02 card=0x10bc8086 chip=0x10bc8086 > rev=0x06 hdr=0x00 > vendor = 'Intel Corporation' > device = '82571EB Gigabit Ethernet Controller (Copper)' > class = network > subclass = ethernet > > FreeBSD releng_7_0 from today - amd64, sched_ule. > > ACPI-Fast - 6.187 MB/s > TSC - 9.455 MB/s > dummy - 9.577 MB/s > > Linux rambo2 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007 > x86_64 GNU/Linux - kubuntu > > TSC - 19.456 MB/s > acpi_pm - 15.394 MB/s > jiffies - 19.480 MB/s > > This is really not what I expected. For once, it's something I expected :) I just hope it isn't one of those cases where Kris absolutely cannot reproduce it and arrives at numbers in favour of FreeBSD :) (just joking here, absolutely no ill feelings involved). It would be helpful if you post exact command line arguments from all cases. > The other thing that bothers me is, that under freebsd is quite easy to get: > [send_ip] sendto: No buffer space available > It happens almost always on my laptop just few seconds after I start > hping with timecounter=TSC I'm not sure, but from what I understood of Robert Watson's explanation in the big ZFS thread on -current, maybe increasing kmem_size (exactly as for ZFS...) could help you with these buffers. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greets, Now I have final results with Linux and FreeBSD on the same hardware CPU: Intel(R) Xeon(R) CPU 3070 @ 2.66GHz - dual core Lan: [EMAIL PROTECTED]:3:0:0: class=0x02 card=0x10bc8086 chip=0x10bc8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet FreeBSD releng_7_0 from today - amd64, sched_ule. ACPI-Fast - 6.187 MB/s TSC - 9.455 MB/s dummy - 9.577 MB/s Linux rambo2 2.6.22-14-generic #1 SMP Tue Dec 18 05:28:27 UTC 2007 x86_64 GNU/Linux - kubuntu TSC - 19.456 MB/s acpi_pm - 15.394 MB/s jiffies - 19.480 MB/s This is really not what I expected. The other thing that bothers me is, that under freebsd is quite easy to get: [send_ip] sendto: No buffer space available It happens almost always on my laptop just few seconds after I start hping with timecounter=TSC and it is harder to reproduce with ACPI-fast and HPET. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: > Greetings, > > Kris Kennaway wrote: >> Stefan Lambrev wrote: >> How much can Linux handle? >>> Will install ubuntu on the same machine and let you know, but my >>> experience shows that FreeBSD + TSC >>> have the same performance as Linux >> >> With which timecounter? > On my colleague laptop which is little slower compared to mine hping > with hpet timer > reach 7.7MB/s and with TSC 8.5MB/s, and on my laptop I I do only > 5.01MB/s with hpet. All these are with Linux? signature.asc Description: OpenPGP digital signature
Re: gettimeofday() in hping
Hi Kris, Kris Kennaway wrote: Stefan Lambrev wrote: Kris Kennaway wrote: Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 I got hwpmc to compile (but only as module) hwpmc: TSC/1/0x20 P6/2/0x1ff but when I try to use pmcstat I receive this error message: pmcstat: ERROR: Initialization of the pmc(3) library failed: Device not configured kldstat lists hwpmc.ko as loaded and I have options HWPMC_HOOKS in my kernel. Any idea what's go wrong? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Kris Kennaway wrote: Stefan Lambrev wrote: How much can Linux handle? Will install ubuntu on the same machine and let you know, but my experience shows that FreeBSD + TSC have the same performance as Linux With which timecounter? On my colleague laptop which is little slower compared to mine hping with hpet timer reach 7.7MB/s and with TSC 8.5MB/s, and on my laptop I I do only 5.01MB/s with hpet. I have planned more accurate test for today. Btw can you tell me in which file pmc_initialize_p6 is defined, as searching in p4 is not very easy :) -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Kris Kennaway wrote: Stefan Lambrev wrote: How much can Linux handle? Will install ubuntu on the same machine and let you know, but my experience shows that FreeBSD + TSC have the same performance as Linux With which timecounter? I guess the default as it is not set anywhere (in linux it can be changed only during boot time i think) Do you have idea how can I see the current timecounter on linux ? :) This is what I found in dmesg: time.c: Using 3.579545 MHz PM timer. Calibrating delay using timer specific routine.. 4416.34 BogoMIPS (lpj=2208170) Calibrating delay using timer specific routine.. 4409.37 BogoMIPS (lpj=2204688) Using local APIC timer interrupts. Detected 12.528 MHz APIC timer. Disabling vsyscall due to use of PM timer I guess this means that ACPI timer or PM Timer is used? :) Here are the max speeds I can reach with different counters (on the test server): i8254 - 3.660 MB/s ACPI-fast - 7.137 MB/s TSC - 12.519 MB/s dummy - 12.366 MB/s Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: Kris Kennaway wrote: Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -march=nocona -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mcmodel=kernel -mno-red-zone -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror vers.c linking kernel.debug hwpmc_x86.o(.text+0x1a5): In function `pmc_md_initialize': /usr/src/sys/dev/hwpmc/hwpmc_x86.c:144: undefined reference to `pmc_initialize_p6' Any chance you have patches against RELENG_7_0 ? :) They should apply, but maybe you will also need other patches from that branch. What was the other way to do this profiling? No other that I have found to be useful. Can ktrace help? Not really, it only tells you what syscalls were made. But it can also display relative timestamps (time since previous entry). Can't this be useful? Not really, it doesn't give enough of a cross-section into what your machine is spending its time doing. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Kris Kennaway wrote: Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -march=nocona -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mcmodel=kernel -mno-red-zone -mfpmath=387 -mno-sse -mno-sse2 -mno-mmx -mno-3dnow -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -Werror vers.c linking kernel.debug hwpmc_x86.o(.text+0x1a5): In function `pmc_md_initialize': /usr/src/sys/dev/hwpmc/hwpmc_x86.c:144: undefined reference to `pmc_initialize_p6' Any chance you have patches against RELENG_7_0 ? :) What was the other way to do this profiling? No other that I have found to be useful. Can ktrace help? Not really, it only tells you what syscalls were made. But it can also display relative timestamps (time since previous entry). Can't this be useful? Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: How much can Linux handle? Will install ubuntu on the same machine and let you know, but my experience shows that FreeBSD + TSC have the same performance as Linux With which timecounter? Here are the max speeds I can reach with different counters (on the test server): i8254 - 3.660 MB/s ACPI-fast - 7.137 MB/s TSC - 12.519 MB/s dummy - 12.366 MB/s Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Hi, Ivan Voras wrote: Stefan Lambrev wrote: I do not have HEPT on the servers that I test, but simple test on my laptop shows that hping can generate with ACPI-fast ~4MB/s traffic, 5MB/s with HPET and 8MB/s with TSC. How much can Linux handle? Will install ubuntu on the same machine and let you know, but my experience shows that FreeBSD + TSC have the same performance as Linux. >I didn't check dummy time counter. If you do, it would give a nice indicator of how much the time counters are expensive in both absolute terms and relative to the rest of the stuff the program does (network IO). The "dummy" timecounter just increments a number when called, no time keeping is involved. Here are the max speeds I can reach with different counters (on the test server): i8254 - 3.660 MB/s ACPI-fast - 7.137 MB/s TSC - 12.519 MB/s dummy - 12.366 MB/s -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 OK, this is the famous problem with modern CPUs that jkoshy has declined to work around :( There are patches for this in perforce, see http://perforce.freebsd.org/changeView.cgi?CH=126189 What was the other way to do this profiling? No other that I have found to be useful. Can ktrace help? Not really, it only tells you what syscalls were made. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Kris Kennaway wrote: Stefan Lambrev wrote: Hi, Dag-Erling Smørgrav wrote: Stefan Lambrev <[EMAIL PROTECTED]> writes: I tested all different combination. The performance change is almost invisible (100-200KB/s), and can't be compared with the performance boost that TSC gain over ACPI-fast timecounter. Unfortunately TSC doesn't play nice with power saving modes. This will vary greatly from machine to machine, depending on the exact hardware and the ACPI BIOS. More modern machines have an HPET timer which is supposedly faster than ACPI yet more reliable than TSC. DES I do not have HEPT on the servers that I test, but simple test on my laptop shows that hping can generate with ACPI-fast ~4MB/s traffic, 5MB/s with HPET and 8MB/s with TSC. I didn't check dummy time counter. Also I noticed that there is a kern.timecounter.tc.XXX.quality (read only). Can this be used to reduce quality and speed up performance? No, they are meaningless values only used to rank the time counters and choose one at boot. You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. pmc: Unknown Intel CPU. module_register_init: MOD_LOAD (hwpmc, 0x8029906d, 0x8054c500) error 78 What was the other way to do this profiling? Can ktrace help? Here is sample of kdump -R: 7501 hping0.13 RET sendto 40/0x28 7501 hping0.19 CALL sigaction(SIGALRM,0xbfbfe8a8,0xbfbfe890) 7501 hping0.14 RET sigaction 0 7501 hping0.13 CALL setitimer(0,0xbfbfe8b4,0xbfbfe8a4) 7501 hping0.16 RET setitimer 0 7501 hping0.17 CALL gettimeofday(0xbfbfe870,0) 7501 hping0.22 RET gettimeofday 0 7501 hping0.15 CALL gettimeofday(0xbfbfe868,0) 7501 hping0.16 RET gettimeofday 0 7501 hping0.15 CALL sendto(0x3,0x28317040,0x28,0,0x8067d80,0x10) 7501 hping0.26 GIO fd 3 wrote 40 bytes -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev wrote: I do not have HEPT on the servers that I test, but simple test on my laptop shows that hping can generate with ACPI-fast ~4MB/s traffic, 5MB/s with HPET and 8MB/s with TSC. How much can Linux handle? >I didn't check dummy time counter. If you do, it would give a nice indicator of how much the time counters are expensive in both absolute terms and relative to the rest of the stuff the program does (network IO). The "dummy" timecounter just increments a number when called, no time keeping is involved. signature.asc Description: OpenPGP digital signature
Re: gettimeofday() in hping
Stefan Lambrev wrote: Hi, Dag-Erling Smørgrav wrote: Stefan Lambrev <[EMAIL PROTECTED]> writes: I tested all different combination. The performance change is almost invisible (100-200KB/s), and can't be compared with the performance boost that TSC gain over ACPI-fast timecounter. Unfortunately TSC doesn't play nice with power saving modes. This will vary greatly from machine to machine, depending on the exact hardware and the ACPI BIOS. More modern machines have an HPET timer which is supposedly faster than ACPI yet more reliable than TSC. DES I do not have HEPT on the servers that I test, but simple test on my laptop shows that hping can generate with ACPI-fast ~4MB/s traffic, 5MB/s with HPET and 8MB/s with TSC. I didn't check dummy time counter. Also I noticed that there is a kern.timecounter.tc.XXX.quality (read only). Can this be used to reduce quality and speed up performance? No, they are meaningless values only used to rank the time counters and choose one at boot. You should use hwpmc to verify where the application is really spending time, since gettimeofday doesn't seem to account for it all. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Hi, Dag-Erling Smørgrav wrote: Stefan Lambrev <[EMAIL PROTECTED]> writes: I tested all different combination. The performance change is almost invisible (100-200KB/s), and can't be compared with the performance boost that TSC gain over ACPI-fast timecounter. Unfortunately TSC doesn't play nice with power saving modes. This will vary greatly from machine to machine, depending on the exact hardware and the ACPI BIOS. More modern machines have an HPET timer which is supposedly faster than ACPI yet more reliable than TSC. DES I do not have HEPT on the servers that I test, but simple test on my laptop shows that hping can generate with ACPI-fast ~4MB/s traffic, 5MB/s with HPET and 8MB/s with TSC. I didn't check dummy time counter. Also I noticed that there is a kern.timecounter.tc.XXX.quality (read only). Can this be used to reduce quality and speed up performance? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On Tue, Jan 22, 2008 at 03:34:55PM +0100, Dag-Erling Sm?rgrav wrote: > More modern machines have an HPET timer which is supposedly faster than > ACPI yet more reliable than TSC. For NetBSD on AMD64 on a 1.2GHz Core2: ACPI ~2400 cycles HPET ~1500 cycles TSC ~800 cycles clockinterrupt ~600 cycles Those are from memory, give or take 10%. This is for the gettimeofday system call. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev <[EMAIL PROTECTED]> writes: > I tested all different combination. The performance change is almost > invisible (100-200KB/s), and can't be compared with the performance > boost that TSC gain over ACPI-fast timecounter. Unfortunately TSC > doesn't play nice with power saving modes. This will vary greatly from machine to machine, depending on the exact hardware and the ACPI BIOS. More modern machines have an HPET timer which is supposedly faster than ACPI yet more reliable than TSC. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Daniel Eischen <[EMAIL PROTECTED]> writes: > Not to discount any of your suggestions, but isn't the better > performance of gettimeofday() (and perhaps clock_gettime() also) > in Linux because they have access to the time in userland and > can implement it without a system call? I seem to recall this > being mentioned before, I could be wrong... Please don't bring this up again. Search the archives instead. There are very good reasons why this was solved the way it was in FreeBSD. Executive summary: a) gettimeofday() *is* a system call in Linux. b) gettimeofday() is slow in FreeBSD because it is correct. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Greetings, Dag-Erling Smørgrav wrote: Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: Stefan Lambrev <[EMAIL PROTECTED]> writes: I tried clock_gettime() (using CLOCK_REALTIME for clock_id), but this yield worse performance. Try CLOCK_MONOTONIC instead. I forgot - there are also the FreeBSD-specific CLOCK_REALTIME_FAST and CLOCK_MONOTONIC_FAST if you can live with a little bit of jitter. DES Thanks for your reply. I tested all different combination. The performance change is almost invisible (100-200KB/s), and can't be compared with the performance boost that TSC gain over ACPI-fast timecounter. Unfortunately TSC doesn't play nice with power saving modes. -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On Tue, 22 Jan 2008, Dag-Erling Smørgrav wrote: Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: Stefan Lambrev <[EMAIL PROTECTED]> writes: I tried clock_gettime() (using CLOCK_REALTIME for clock_id), but this yield worse performance. Try CLOCK_MONOTONIC instead. I forgot - there are also the FreeBSD-specific CLOCK_REALTIME_FAST and CLOCK_MONOTONIC_FAST if you can live with a little bit of jitter. Not to discount any of your suggestions, but isn't the better performance of gettimeofday() (and perhaps clock_gettime() also) in Linux because they have access to the time in userland and can implement it without a system call? I seem to recall this being mentioned before, I could be wrong... -- DE___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > Stefan Lambrev <[EMAIL PROTECTED]> writes: > > I tried clock_gettime() (using CLOCK_REALTIME for clock_id), but this > > yield worse performance. > Try CLOCK_MONOTONIC instead. I forgot - there are also the FreeBSD-specific CLOCK_REALTIME_FAST and CLOCK_MONOTONIC_FAST if you can live with a little bit of jitter. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
Stefan Lambrev <[EMAIL PROTECTED]> writes: > I tried clock_gettime() (using CLOCK_REALTIME for clock_id), but this > yield worse performance. Try CLOCK_MONOTONIC instead. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: gettimeofday() in hping
On 1/22/08, Stefan Lambrev <[EMAIL PROTECTED]> wrote: > Greetings, > > I noticed that hping3 (from ports) is quite slower when running on > FreeBSD compared to Linux. > Simple ktrace shows lot of gettimeofday() calls, so I'm looking for > replacement of this function. > I tried clock_gettime() (using CLOCK_REALTIME for clock_id), but this > yield worse performance. > Of course changing timecounter to TSC make hping almost twice faster, > but I'm wandering if I can optimize things more? > Also, is it possible to bypass the ACPI timer entirely without disabling the whole ACPI, in order to use TSC as the default timecounter? > -- > > Best Wishes, > Stefan Lambrev > ICQ# 24134177 > > ___ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > -- Mahnahmahnah! ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
