Re: kernel build failure
On 15.04.2012 00:01 (UTC+1), AN wrote: On Fri, 13 Apr 2012, O. Hartmann wrote: Am 04/13/12 00:21, schrieb AN: At Thu Apr 12 17:52:05 EDT 2012: [root@FBSD10 /usr/src]# svn up Updating '.': At revision 234196. Trying to build the kernel I get the following failure: time make -j8 buildkernel KERNCONF=MYKERNEL snip === zlib (all) cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/MYKERNEL/opt_global.h -I. -I@ -I@/contrib/altq -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-common -g -fno-omit-frame-pointer -I/usr/obj/usr/src/sys/MYKERNEL -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -std=iso9899:1999 -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -c /usr/src/sys/modules/zlib/../../net/zlib.c ld -d -warn-common -r -d -o zlib.ko.debug zlib.o : export_syms awk -f /usr/src/sys/conf/kmod_syms.awk zlib.ko.debug export_syms | xargs -J% objcopy % zlib.ko.debug objcopy --only-keep-debug zlib.ko.debug zlib.ko.symbols objcopy --strip-debug --add-gnu-debuglink=zlib.ko.symbols zlib.ko.debug zlib.ko 1 error *** [buildkernel] Error code 2 1 error *** [buildkernel] Error code 2 1 error real 8m20.095s user 12m37.161s sys 6m59.844s I tried this 4 times, twice with MYKERNEL and twice with GENERIC. It failed in the same spot every time. Is anyone else seeing this? Also, I tried without -j8, and it fails with the following: cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/src/sys/kern/subr_turnstile.c cc1: warnings being treated as errors /usr/src/sys/kern/subr_turnstile.c: In function 'propagate_priority': /usr/src/sys/kern/subr_turnstile.c:220: warning: implicit declaration of function 'kdb_backtrace_thread' /usr/src/sys/kern/subr_turnstile.c:220: warning: nested extern declaration of 'kdb_backtrace_thread' [-Wnested-externs] *** [subr_turnstile.o] Error code 1 Stop in /usr/obj/usr/src/sys/MYKERNEL. *** [buildkernel] Error code 1 Stop in /usr/src. *** [buildkernel] Error code 1 Stop in /usr/src. real 5m19.701s user 4m33.051s sys 0m51.466s Here is /etc/make.conf # cat /etc/make.conf OVERRIDE_LINUX_BASE_PORT=f10 QT4_OPTIONS= QGTKSTYLE WITH_OPENSSL_PORT=yes # added by use.perl 2012-04-04 01:11:13 PERL_VERSION=5.14.2 The kernel previously built without problems with this same configuration. ___ clang -c -O2 -pipe -pipe -O3 -fno-strict-aliasing -march=native -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/src/sys/kern/subr_turnstile.c /usr/src/sys/kern/subr_turnstile.c:220:4: error: implicit declaration of function 'kdb_backtrace_thread' is invalid in C99 [-Werror,-Wimplicit-function-declaration] kdb_backtrace_thread(td); ^ 1 error generated. *** [subr_turnstile.o] Error code 1 1 error *** [buildkernel] Error code 2 1 error *** [buildkernel] Error code 2 1 error I still get this error on one of my boxes. Another, with almost the same setup and config, build fine! All systems build with CLANG. They share the same /etc/src.conf and have the same /etc/make.conf. Before building kernel/world, I cleanup/delete /usr/obj. But the error still persists. Regards, Oliver I updated source with svn within the last hour, and rebuilt world and kernel. When I tried to boot to single-user mode to installworld the machine panicked and rebooted. I needed to boot kernel.old to get machine back. Is anyone else seeing a problem with the kernel? It seems it is still broken for me. Same problem here with revision 234306. I am not able to boot
Re: kernel build failure
Am 04/15/12 00:01, schrieb AN: On Fri, 13 Apr 2012, O. Hartmann wrote: Am 04/13/12 00:21, schrieb AN: At Thu Apr 12 17:52:05 EDT 2012: [root@FBSD10 /usr/src]# svn up Updating '.': At revision 234196. Trying to build the kernel I get the following failure: time make -j8 buildkernel KERNCONF=MYKERNEL snip === zlib (all) cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/MYKERNEL/opt_global.h -I. -I@ -I@/contrib/altq -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-common -g -fno-omit-frame-pointer -I/usr/obj/usr/src/sys/MYKERNEL -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -std=iso9899:1999 -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -c /usr/src/sys/modules/zlib/../../net/zlib.c ld -d -warn-common -r -d -o zlib.ko.debug zlib.o : export_syms awk -f /usr/src/sys/conf/kmod_syms.awk zlib.ko.debug export_syms | xargs -J% objcopy % zlib.ko.debug objcopy --only-keep-debug zlib.ko.debug zlib.ko.symbols objcopy --strip-debug --add-gnu-debuglink=zlib.ko.symbols zlib.ko.debug zlib.ko 1 error *** [buildkernel] Error code 2 1 error *** [buildkernel] Error code 2 1 error real8m20.095s user12m37.161s sys6m59.844s I tried this 4 times, twice with MYKERNEL and twice with GENERIC. It failed in the same spot every time. Is anyone else seeing this? Also, I tried without -j8, and it fails with the following: cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/src/sys/kern/subr_turnstile.c cc1: warnings being treated as errors /usr/src/sys/kern/subr_turnstile.c: In function 'propagate_priority': /usr/src/sys/kern/subr_turnstile.c:220: warning: implicit declaration of function 'kdb_backtrace_thread' /usr/src/sys/kern/subr_turnstile.c:220: warning: nested extern declaration of 'kdb_backtrace_thread' [-Wnested-externs] *** [subr_turnstile.o] Error code 1 Stop in /usr/obj/usr/src/sys/MYKERNEL. *** [buildkernel] Error code 1 Stop in /usr/src. *** [buildkernel] Error code 1 Stop in /usr/src. real5m19.701s user4m33.051s sys0m51.466s Here is /etc/make.conf # cat /etc/make.conf OVERRIDE_LINUX_BASE_PORT=f10 QT4_OPTIONS= QGTKSTYLE WITH_OPENSSL_PORT=yes # added by use.perl 2012-04-04 01:11:13 PERL_VERSION=5.14.2 The kernel previously built without problems with this same configuration. ___ clang -c -O2 -pipe -pipe -O3 -fno-strict-aliasing -march=native -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/src/sys/kern/subr_turnstile.c /usr/src/sys/kern/subr_turnstile.c:220:4: error: implicit declaration of function 'kdb_backtrace_thread' is invalid in C99 [-Werror,-Wimplicit-function-declaration] kdb_backtrace_thread(td); ^ 1 error generated. *** [subr_turnstile.o] Error code 1 1 error *** [buildkernel] Error code 2 1 error *** [buildkernel] Error code 2 1 error I still get this error on one of my boxes. Another, with almost the same setup and config, build fine! All systems build with CLANG. They share the same /etc/src.conf and have the same /etc/make.conf. Before building kernel/world, I cleanup/delete /usr/obj. But the error still persists. Regards, Oliver I updated source with svn within the last hour, and rebuilt world and kernel. When I tried to boot to single-user mode to installworld the machine panicked and rebooted. I
Kernel builds, but crashes at boot (amd64, Revision: 234306)
Today I'm suddenly unable to boot a newly built kernel without crashing right near the end of the device probes, just before the system is about to actually come up: Fatal trap 18: integer divide fault while in kernel mode Stopped at 0x803b2646 = g_label_ufs_taste_common+0x36 divl 0x50(%rcx),%eax Backtrace lists this chain of calls: g_label_ufs_taste_common g_label_taste g_new_provider_event g_run_events g_event_procbody fork_exit fork_trampoline Whether built with clang or gcc, CUSTOM config or GENERIC, same results on rebooting. No idea why this suddenly started happening, haven't changed anything at all in my setup. -- Conrad J. Sabatier conr...@cox.net ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kernel build failure
On Sun, 15 Apr 2012 09:47:38 +0200 Rainer Hurling rhur...@gwdg.de wrote: On 15.04.2012 00:01 (UTC+1), AN wrote: I updated source with svn within the last hour, and rebuilt world and kernel. When I tried to boot to single-user mode to installworld the machine panicked and rebooted. I needed to boot kernel.old to get machine back. Is anyone else seeing a problem with the kernel? It seems it is still broken for me. Same problem here with revision 234306. I am not able to boot the new kernel. It panics with: Fatal trap 18: integer divide fault while in kernel mode cpuid = 3; apic id = 03 instruction pointer = 0x20:0x803b9d34 stack pointer = 0x28:0xff80002a9a10 frame pointer = 0x28:0xff80002a9a60 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 13 (g_event) trap number = 18 panic: Integer divide fault cpuid = 3 Same here. Tried building with both clang and gcc. Compiles OK, but same results at startup. Crash! -- Conrad J. Sabatier conr...@cox.net ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ufs multilabel performance (fwd)
Am 04/15/12 15:59, schrieb Richard Kojedzinszky: Thank you for the reply. Unfortunately, dont know why, but on my xen virtualised environment, fbsd amd64 domU performs much slower, not only 30 times. Without multilabel, file creation speed is around 2500/s, but with multilabels enabled, it is only 15/s (!). so it is more than 100 times slower. And anyway freebsd is known to be fast as well, as functional. The power to serve. :) But in my environment, 15/s file creation is very-very slow. The hardware is a q6700 cpu with 4G ram, 2x1T sata disks in raid1, the host runs linux. I think with this hw the mentioned speed is really slow. Regards, Kojedzinszky Richard Euronet Magyarorszag Informatikai Zrt. On Sun, 15 Apr 2012, O. Hartmann wrote: Date: Sun, 15 Apr 2012 13:20:23 +0200 From: O. Hartmann ohart...@zedat.fu-berlin.de To: Richard Kojedzinszky kri...@tvnetwork.hu Cc: freebsd-secur...@freebsd.org Subject: Re: ufs multilabel performance (fwd) Am 04/14/12 21:37, schrieb Richard Kojedzinszky: Dear list, Although it is not only security-related question, I did not get any answer from freebsd-performance. The original question is below. Can someone give some advice? Thanks in advance, Kojedzinszky Richard Euronet Magyarorszag Informatikai Zrt. -- Forwarded message -- Date: Thu, 10 Nov 2011 06:16:57 +0100 (CET) From: Richard Kojedzinszky kri...@tvnetwork.hu To: freebsd-performa...@freebsd.org Subject: ufs multilabel performance Dear List, I've noticed that when I enable multilabel on an fs, a file creation gets around 20-30 times slower than without multilabel set. This one-liner can be used to test the differences: $ truss -D perl -e 'open(F, $_.file) for 1 .. 1000' Same here, creating files seems to be 10 - 30 times slower with multilabels as it is without. But as several posts and discussions reflects, FreeBSD isn't supposed to be fast although it is claimed that writing is the major than reading; FBSD should serve functionality. And one can see that the open call takes much more when multilabel is set on an fs. It seems that only file creation needs that many time, when a file exists it is opened much faster. Could someone acknowledge this, and have some suggestions how to make it faster? Regards, Kojedzinszky Richard TvNetWork Nyrt. E-mail: krichy (at) tvnetwork [dot] hu PGP: 0x54B2BF0C8F59B1B7 Fingerprint = F6D4 3FFE AF03 CACF 0DCB 46A1 54B2 BF0C 8F59 B1B7 At the moment, I'm troubled with a nasty kernel bug on all FreeBSD 10 boxes I have spare to test. I just tried to reproduce your observation and as far as I can go with my experience, I can confirm that by using your perl script. I'd like to test this again with a small C program. I can only test the issue (test is too far optimistic, it's simply a reproduction of your observation) on FreeBSD 10, the only remaining FreeBSD server at our department is running FBSD 9-STABLE/amd64 and in production, so changing multilabel support is a bit harsh at the moment. Sorry about crossposting, but I think this belongs more to CURRENT and PERFORMANCE than SECURITY. Regards, Oliver signature.asc Description: OpenPGP digital signature
Re: ufs multilabel performance (fwd)
On Apr 15, 2012, at 12:30 PM, O. Hartmann wrote:
Am 04/15/12 15:59, schrieb Richard Kojedzinszky:
Thank you for the reply.
Unfortunately, dont know why, but on my xen virtualised environment,
fbsd amd64 domU performs much slower, not only 30 times. Without
multilabel, file creation speed is around 2500/s, but with multilabels
enabled, it is only 15/s (!). so it is more than 100 times slower.
And anyway freebsd is known to be fast as well, as functional. The power
to serve. :)
But in my environment, 15/s file creation is very-very slow. The
hardware is a q6700 cpu with 4G ram, 2x1T sata disks in raid1, the host
runs linux. I think with this hw the mentioned speed is really slow.
Regards,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
On Sun, 15 Apr 2012, O. Hartmann wrote:
Date: Sun, 15 Apr 2012 13:20:23 +0200
From: O. Hartmann ohart...@zedat.fu-berlin.de
To: Richard Kojedzinszky kri...@tvnetwork.hu
Cc: freebsd-secur...@freebsd.org
Subject: Re: ufs multilabel performance (fwd)
Am 04/14/12 21:37, schrieb Richard Kojedzinszky:
Dear list,
Although it is not only security-related question, I did not get any
answer from freebsd-performance. The original question is below.
Can someone give some advice?
Thanks in advance,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
-- Forwarded message --
Date: Thu, 10 Nov 2011 06:16:57 +0100 (CET)
From: Richard Kojedzinszky kri...@tvnetwork.hu
To: freebsd-performa...@freebsd.org
Subject: ufs multilabel performance
Dear List,
I've noticed that when I enable multilabel on an fs, a file creation
gets around 20-30 times slower than without multilabel set.
This one-liner can be used to test the differences:
$ truss -D perl -e 'open(F, $_.file) for 1 .. 1000'
Same here, creating files seems to be 10 - 30 times slower with
multilabels as it is without.
But as several posts and discussions reflects, FreeBSD isn't supposed to
be fast although it is claimed that writing is the major than reading;
FBSD should serve functionality.
And one can see that the open call takes much more when multilabel is
set on an fs. It seems that only file creation needs that many time,
when a file exists it is opened much faster.
Could someone acknowledge this, and have some suggestions how to make it
faster?
Regards,
Kojedzinszky Richard
TvNetWork Nyrt.
E-mail: krichy (at) tvnetwork [dot] hu
PGP: 0x54B2BF0C8F59B1B7
Fingerprint = F6D4 3FFE AF03 CACF 0DCB 46A1 54B2 BF0C 8F59 B1B7
At the moment, I'm troubled with a nasty kernel bug on all FreeBSD 10
boxes I have spare to test.
I just tried to reproduce your observation and as far as I can go with
my experience, I can confirm that by using your perl script.
I'd like to test this again with a small C program.
I can only test the issue (test is too far optimistic, it's simply a
reproduction of your observation) on FreeBSD 10, the only remaining
FreeBSD server at our department is running FBSD 9-STABLE/amd64 and in
production, so changing multilabel support is a bit harsh at the moment.
Sorry about crossposting, but I think this belongs more to CURRENT and
PERFORMANCE than SECURITY.
My suggestion is completely take perl out of the equation because the way
you're invoking it above uses stdio and a few other things that add unnecessary
overhead.
Try the attached C program/bourne shell snippet instead.
Cheers,
-Garrett
#!/bin/sh
set -e
tmp=$(mktemp -d tmp.XX)
trap cd /; rm -Rf $tmp EXIT
cd $tmp
cat test_open.c EOF
#include fcntl.h
#include stdio.h
#include unistd.h
int
main(void)
{
char buf[20];
int i;
for (i = 0; i 1000; i++) {
sprintf(buf, %d, i);
close(open(buf, O_WRONLY|O_CREAT, 0600));
}
return (0);
}
EOF
gcc -o test_open test_open.c
time ./test_open___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ufs multilabel performance (fwd)
Am 04/15/12 22:00, schrieb Garrett Cooper:
On Apr 15, 2012, at 12:30 PM, O. Hartmann wrote:
Am 04/15/12 15:59, schrieb Richard Kojedzinszky:
Thank you for the reply.
Unfortunately, dont know why, but on my xen virtualised environment,
fbsd amd64 domU performs much slower, not only 30 times. Without
multilabel, file creation speed is around 2500/s, but with multilabels
enabled, it is only 15/s (!). so it is more than 100 times slower.
And anyway freebsd is known to be fast as well, as functional. The power
to serve. :)
But in my environment, 15/s file creation is very-very slow. The
hardware is a q6700 cpu with 4G ram, 2x1T sata disks in raid1, the host
runs linux. I think with this hw the mentioned speed is really slow.
Regards,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
On Sun, 15 Apr 2012, O. Hartmann wrote:
Date: Sun, 15 Apr 2012 13:20:23 +0200
From: O. Hartmann ohart...@zedat.fu-berlin.de
To: Richard Kojedzinszky kri...@tvnetwork.hu
Cc: freebsd-secur...@freebsd.org
Subject: Re: ufs multilabel performance (fwd)
Am 04/14/12 21:37, schrieb Richard Kojedzinszky:
Dear list,
Although it is not only security-related question, I did not get any
answer from freebsd-performance. The original question is below.
Can someone give some advice?
Thanks in advance,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
-- Forwarded message --
Date: Thu, 10 Nov 2011 06:16:57 +0100 (CET)
From: Richard Kojedzinszky kri...@tvnetwork.hu
To: freebsd-performa...@freebsd.org
Subject: ufs multilabel performance
Dear List,
I've noticed that when I enable multilabel on an fs, a file creation
gets around 20-30 times slower than without multilabel set.
This one-liner can be used to test the differences:
$ truss -D perl -e 'open(F, $_.file) for 1 .. 1000'
Same here, creating files seems to be 10 - 30 times slower with
multilabels as it is without.
But as several posts and discussions reflects, FreeBSD isn't supposed to
be fast although it is claimed that writing is the major than reading;
FBSD should serve functionality.
And one can see that the open call takes much more when multilabel is
set on an fs. It seems that only file creation needs that many time,
when a file exists it is opened much faster.
Could someone acknowledge this, and have some suggestions how to make it
faster?
Regards,
Kojedzinszky Richard
TvNetWork Nyrt.
E-mail: krichy (at) tvnetwork [dot] hu
PGP: 0x54B2BF0C8F59B1B7
Fingerprint = F6D4 3FFE AF03 CACF 0DCB 46A1 54B2 BF0C 8F59 B1B7
At the moment, I'm troubled with a nasty kernel bug on all FreeBSD 10
boxes I have spare to test.
I just tried to reproduce your observation and as far as I can go with
my experience, I can confirm that by using your perl script.
I'd like to test this again with a small C program.
I can only test the issue (test is too far optimistic, it's simply a
reproduction of your observation) on FreeBSD 10, the only remaining
FreeBSD server at our department is running FBSD 9-STABLE/amd64 and in
production, so changing multilabel support is a bit harsh at the moment.
Sorry about crossposting, but I think this belongs more to CURRENT and
PERFORMANCE than SECURITY.
My suggestion is completely take perl out of the equation because the way
you're invoking it above uses stdio and a few other things that add
unnecessary overhead.
Try the attached C program/bourne shell snippet instead.
Cheers,
-Garrett
#!/bin/sh
set -e
tmp=$(mktemp -d tmp.XX)
trap cd /; rm -Rf $tmp EXIT
cd $tmp
cat test_open.c EOF
#include fcntl.h
#include stdio.h
#include unistd.h
int
main(void)
{
char buf[20];
int i;
for (i = 0; i 1000; i++) {
sprintf(buf, %d, i);
close(open(buf, O_WRONLY|O_CREAT, 0600));
}
return (0);
}
EOF
gcc -o test_open test_open.c
time ./test_open___
This was pretty fast ;-)
signature.asc
Description: OpenPGP digital signature
Re: Changes to sbin/init/init.c (r233944) makes x11/xdm impossible to start from /etc/ttys
Hi Oliver, * O. Hartmann ohart...@zedat.fu-berlin.de, 20120413 19:09: Hope the discussion is going on and will make the problem go away. It is still present and bothering ... Sorry about taking so long to respond. I've been very busy the last week. I just got xdm working on a stock FreeBSD HEAD install just now. Is there a way for me to reproduce? -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ pgpOIEZIlv1x0.pgp Description: PGP signature
Re: ufs multilabel performance (fwd)
On Apr 15, 2012, at 1:17 PM, O. Hartmann wrote:
Am 04/15/12 22:00, schrieb Garrett Cooper:
On Apr 15, 2012, at 12:30 PM, O. Hartmann wrote:
Am 04/15/12 15:59, schrieb Richard Kojedzinszky:
Thank you for the reply.
Unfortunately, dont know why, but on my xen virtualised environment,
fbsd amd64 domU performs much slower, not only 30 times. Without
multilabel, file creation speed is around 2500/s, but with multilabels
enabled, it is only 15/s (!). so it is more than 100 times slower.
And anyway freebsd is known to be fast as well, as functional. The power
to serve. :)
But in my environment, 15/s file creation is very-very slow. The
hardware is a q6700 cpu with 4G ram, 2x1T sata disks in raid1, the host
runs linux. I think with this hw the mentioned speed is really slow.
Regards,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
On Sun, 15 Apr 2012, O. Hartmann wrote:
Date: Sun, 15 Apr 2012 13:20:23 +0200
From: O. Hartmann ohart...@zedat.fu-berlin.de
To: Richard Kojedzinszky kri...@tvnetwork.hu
Cc: freebsd-secur...@freebsd.org
Subject: Re: ufs multilabel performance (fwd)
Am 04/14/12 21:37, schrieb Richard Kojedzinszky:
Dear list,
Although it is not only security-related question, I did not get any
answer from freebsd-performance. The original question is below.
Can someone give some advice?
Thanks in advance,
Kojedzinszky Richard
Euronet Magyarorszag Informatikai Zrt.
-- Forwarded message --
Date: Thu, 10 Nov 2011 06:16:57 +0100 (CET)
From: Richard Kojedzinszky kri...@tvnetwork.hu
To: freebsd-performa...@freebsd.org
Subject: ufs multilabel performance
Dear List,
I've noticed that when I enable multilabel on an fs, a file creation
gets around 20-30 times slower than without multilabel set.
This one-liner can be used to test the differences:
$ truss -D perl -e 'open(F, $_.file) for 1 .. 1000'
Same here, creating files seems to be 10 - 30 times slower with
multilabels as it is without.
But as several posts and discussions reflects, FreeBSD isn't supposed to
be fast although it is claimed that writing is the major than reading;
FBSD should serve functionality.
And one can see that the open call takes much more when multilabel is
set on an fs. It seems that only file creation needs that many time,
when a file exists it is opened much faster.
Could someone acknowledge this, and have some suggestions how to make it
faster?
Regards,
Kojedzinszky Richard
TvNetWork Nyrt.
E-mail: krichy (at) tvnetwork [dot] hu
PGP: 0x54B2BF0C8F59B1B7
Fingerprint = F6D4 3FFE AF03 CACF 0DCB 46A1 54B2 BF0C 8F59 B1B7
At the moment, I'm troubled with a nasty kernel bug on all FreeBSD 10
boxes I have spare to test.
I just tried to reproduce your observation and as far as I can go with
my experience, I can confirm that by using your perl script.
I'd like to test this again with a small C program.
I can only test the issue (test is too far optimistic, it's simply a
reproduction of your observation) on FreeBSD 10, the only remaining
FreeBSD server at our department is running FBSD 9-STABLE/amd64 and in
production, so changing multilabel support is a bit harsh at the moment.
Sorry about crossposting, but I think this belongs more to CURRENT and
PERFORMANCE than SECURITY.
My suggestion is completely take perl out of the equation because the way
you're invoking it above uses stdio and a few other things that add
unnecessary overhead.
Try the attached C program/bourne shell snippet instead.
Cheers,
-Garrett
#!/bin/sh
set -e
tmp=$(mktemp -d tmp.XX)
trap cd /; rm -Rf $tmp EXIT
cd $tmp
cat test_open.c EOF
#include fcntl.h
#include stdio.h
#include unistd.h
int
main(void)
{
char buf[20];
int i;
for (i = 0; i 1000; i++) {
sprintf(buf, %d, i);
close(open(buf, O_WRONLY|O_CREAT, 0600));
}
return (0);
}
EOF
gcc -o test_open test_open.c
time ./test_open___
This was pretty fast ;-)
If it turns out that it wasn't that particular item that's causing a
slowdown, I can easily modify my above snippet to use stdio, etc. But unless
the version of perl and a few other items are the same, I wouldn't necessarily
blame the guest OS. Please also make sure that Xen, etc is completely
up-to-date because there were some performance degradation issues that were
fixed post-6.0.
-Garrett___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Changes to sbin/init/init.c (r233944) makes x11/xdm impossible to start from /etc/ttys
Am 04/15/12 22:25, schrieb Ed Schouten: Hi Oliver, * O. Hartmann ohart...@zedat.fu-berlin.de, 20120413 19:09: Hope the discussion is going on and will make the problem go away. It is still present and bothering ... Sorry about taking so long to respond. I've been very busy the last week. I just got xdm working on a stock FreeBSD HEAD install just now. Is there a way for me to reproduce? Hello Ed. xdm works fine for me as long as it doens't get upstarted from /etc/ttys. This morning, even with a newly make world, I had the same problems as before when starting xdm from /etc/ttys. My lab's box seems to be instable, when having xdm started - it crashes on heavy i/o. It might be possible that blaming xdm isn't correct, but the crashes came very quickly when xdm was started (remotely via console command). But since there is a integer divide fault issue with the mos recent kernels from which my oldish Intel E8500 box at home suffers with crashaes while booting the most recent kernels while the box in the lab, a brand new i7 3930K, does not but crashes under heavy i/i, this could also be a coincidence. Do you startup xdm via issuinng a command on the console or did you fire it up via /etc/ttys? Regards, Oliver signature.asc Description: OpenPGP digital signature
Re: Changes to sbin/init/init.c (r233944) makes x11/xdm impossible to start from /etc/ttys
Am 04/15/12 22:25, schrieb Ed Schouten: Hi Oliver, * O. Hartmann ohart...@zedat.fu-berlin.de, 20120413 19:09: Hope the discussion is going on and will make the problem go away. It is still present and bothering ... Sorry about taking so long to respond. I've been very busy the last week. I just got xdm working on a stock FreeBSD HEAD install just now. Is there a way for me to reproduce? Hello Ed. xdm works fine for me as long as it doens't get upstarted from /etc/ttys. This morning, even with a newly make world, I had the same problems as before when starting xdm from /etc/ttys. My lab's box seems to be instable, when having xdm started - it crashes on heavy i/o. It might be possible that blaming xdm isn't correct, but the crashes came very quickly when xdm was started (remotely via console command). But since there is a integer divide fault issue with the mos recent kernels from which my oldish Intel E8500 box at home suffers with crashaes while booting the most recent kernels while the box in the lab, a brand new i7 3930K, does not but crashes under heavy i/i, this could also be a coincidence. Do you startup xdm via issuinng a command on the console or did you fire it up via /etc/ttys? Regards, Oliver signature.asc Description: OpenPGP digital signature
Re: ufs multilabel performance (fwd)
Thanks for the replies.
I dont think so that perl counts anything in the test, just run the code
with truss -D, you will see the timing that the open() syscall takes just
more time with multilabel enabled.
Even a single touch shows that:
# truss -D touch 1
0.56431
mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) =
34366259200 (0x800638000)
0.20672
issetugid(0x800639015,0x80062e4be,0x80084a0b0,0x80084a080,0xab57,0x0) = 0
(0x0)
0.60342 open(/etc/libmap.conf,O_RDONLY,0666) ERR#2 'No such file or
directory'
0.32964 open(/var/run/ld-elf.so.hints,O_RDONLY,057) = 3 (0x3)
0.29891 read(3,Ehnt\^A\0\0\0\M^@\0\0\0-\0\0\0\0...,128) = 128 (0x80)
0.63135 lseek(3,0x80,SEEK_SET) = 128 (0x80)
0.40228 read(3,/lib:/usr/lib:/usr/lib/compat:/u...,45) = 45 (0x2d)
0.34361 close(3) = 0 (0x0)
0.31288 access(/lib/libc.so.7,0) = 0 (0x0)
0.38551 open(/lib/libc.so.7,O_RDONLY,040754040) = 3 (0x3)
0.45256 fstat(3,{ mode=-r--r--r--
,inode=4107,size=1306952,blksize=32768 }) = 0 (0x0)
0.31847
pread(0x3,0x80083c800,0x1000,0x0,0x101010101010101,0x8080808080808080) =
4096 (0x1000)
0.24025
mmap(0x0,3420160,PROT_NONE,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) =
34368434176 (0x80084b000)
0.55872
mmap(0x80084b000,1171456,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE,3,0x0)
= 34368434176 (0x80084b000)
0.46095
mmap(0x800b68000,45056,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED,3,0x11d000)
= 34371698688 (0x800b68000)
0.22628
mmap(0x800b73000,110592,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0)
= 34371743744 (0x800b73000)
0.22069 close(3) = 0 (0x0)
0.24584 munmap(0x80063f000,4096) = 0 (0x0)
0.30450
mmap(0x0,102400,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) =
34366287872 (0x80063f000)
0.36875
sysarch(0x81,0x7fffd450,0x80063c0c8,0x0,0xffad2228,0x8080808080808080)
= 0 (0x0)
0.41345
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
0.65370 sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
0.41066
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
0.32685 sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
0.51681 readlink(/etc/malloc.conf,0x7fffd5e0,1024) ERR#2 'No
such file or directory'
0.20393
issetugid(0x800945101,0x7fffd5e0,0x,0x0,0x2,0x7fffef79)
= 0 (0x0)
0.25143 break(0x80) = 0 (0x0)
0.24863
mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) =
34371854336 (0x800b8e000)
0.38273
mmap(0x800f8e000,466944,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0)
= 34376048640 (0x800f8e000)
0.25142 munmap(0x800b8e000,466944) = 0 (0x0)
0.25143 gettimeofday({1334522860.892157 },0x0) = 0 (0x0)
0.42463 stat(1,0x7fffdb10) ERR#2 'No such file or
directory'
0.116764212 open(1,O_WRONLY|O_CREAT,0666)= 3 (0x3)
0.25701 fstat(3,{ mode=-rw-r--r-- ,inode=5,size=0,blksize=32768 }) = 0
(0x0)
0.26819 close(3) = 0 (0x0)
0.21231
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
0.26818 sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
0.20952
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
0.32127 sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
-1.999864511 process exit, rval = 0
Can you see the open()'s spent time?
Without multilabel:
# truss -D touch 1
0.78779
mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) =
34366259200 (0x800638000)
0.53078
issetugid(0x800639015,0x80062e4be,0x80084a0b0,0x80084a080,0xab57,0x0) = 0
(0x0)
0.78500 open(/etc/libmap.conf,O_RDONLY,0666) ERR#2 'No such file or
directory'
0.39390 open(/var/run/ld-elf.so.hints,O_RDONLY,057) = 3 (0x3)
0.34920 read(3,Ehnt\^A\0\0\0\M^@\0\0\0-\0\0\0\0...,128) = 128 (0x80)
0.31847 lseek(3,0x80,SEEK_SET) = 128 (0x80)
0.29612 read(3,/lib:/usr/lib:/usr/lib/compat:/u...,45) = 45 (0x2d)
0.53357 close(3) = 0 (0x0)
0.40508 access(/lib/libc.so.7,0) = 0 (0x0)
0.35758 open(/lib/libc.so.7,O_RDONLY,040754040) = 3 (0x3)
0.57828 fstat(3,{ mode=-r--r--r--
,inode=4107,size=1306952,blksize=32768 })
Re: /sys/conf/kmod.mk, line 111: Malformed conditional (${MK_CLANG_IS_CC}
Hello,
I recently updated my system from 8.2-STABLE to 9.0-STABLE and I'm
seeing a similar error when trying to build the VirtualBox kernel
module:
*** Building 'vboxdrv' module ***
/usr/share/mk/bsd.own.mk, line 476: MK_CLANG_IS_CC can't be set by a user.
*** Error code 1
I tried the tips from all the previous posters with no luck. Any
suggestions would greatly be appreciated!
Chris
On Sat, Mar 3, 2012 at 10:43 AM, O. Hartmann
ohart...@zedat.fu-berlin.de wrote:
On 03/03/12 15:19, Florian Smeets wrote:
On 03.03.12 14:24, Chris Rees wrote:
On 3 March 2012 11:48, O. Hartmann ohart...@zedat.fu-berlin.de wrote:
On one of my FreeBSD 10.0-CURRENT boxes I receive this morning this
error message as shown below.
I need to add, that I compiled the shown nvidia-driver hours ago on all
FreeBSD 9.0-STABLE boxes with the same settings and I compiled the
driver just two days before the same way I tried it this morning. What's
wrong?
Some unexpected breakage? Then this is my shout to the community.
Message below.
Regards and thanks in advance,
Oliver
=== Vulnerability check disabled, database not found
=== License NVIDIA accepted by the user
=== Found saved configuration for nvidia-driver-295.20
=== Extracting for nvidia-driver-295.20
= SHA256 Checksum OK for NVIDIA-FreeBSD-x86_64-295.20.tar.gz.
=== Patching for nvidia-driver-295.20
=== nvidia-driver-295.20 depends on file:
/usr/local/libdata/pkgconfig/xorg-server.pc - found
=== nvidia-driver-295.20 depends on shared library: GL.1 - found
=== Configuring for nvidia-driver-295.20
=== Building for nvidia-driver-295.20
=== src (all)
/sys/conf/kmod.mk, line 111: Malformed conditional (${MK_CLANG_IS_CC}
== no ${CC:T:Mclang} != clang)
/sys/conf/kmod.mk, line 115: if-less endif
/sys/conf/kern.mk, line 18: Malformed conditional (${MK_CLANG_IS_CC}
!= no || ${CC:T:Mclang} == clang)
/sys/conf/kern.mk, line 31: if-less endif
/sys/conf/kern.mk, line 101: Malformed conditional (${MK_CLANG_IS_CC}
== no ${CC:T:Mclang} != clang)
/sys/conf/kern.mk, line 109: if-less endif
make: fatal errors encountered -- cannot continue
*** [all] Error code 1
Please post your make.conf and src.conf.
No need, a buildworld / installworld cycle will fix it. A make install
in src/share/mk *could* also be enough, but i haven't tested it.
Florian
The last two attempts of make -jX buildworld (X=2) or just make
buildworld failed.
But issuing make install in /usr/src/share/mk made the failure go away.
Thanks for the help.
Where can I read more about how to fine tune /etc/src.conf and
/etc/make.conf with new knobs?
My /etc/src.conf looks like this (on FBSD 9 and 10):
WITH_CLANG= YES
WITH_CLANG_EXTRAS= YES
#
#CC= clang
#CXX= clang++
#CPP= clang-cpp
#
#CFLAGS.clang+= -O3 -pipe -fno-strict-aliasing
#COPTFLAGS.clang+= -O3 -pipe
#CFLAGS.cc+= -O2 -pipe -fno-strict-aliasing
#COPTFLAGS.cc+= -O2 -pipe
#
WITH_BIND_LIBS= YES
WITH_BIND_SIGCHASE= YES
WITH_BIND_LARGE_FILE= YES
#
WITH_IDEA= YES
WITH_HESIOD= YES
#
#WITH_ICONV= YES
#WITH_BSD_GREP= YES
#
WITH_LIBCPLUSPLUS= YES
#
#WITH_OFED= YES
The CLANG relevant portion of /etc/make.conf looks like this:
# BUGFIX FreeBSD 10.0
#WITH_FBSD10_FIX= YES
#
#DISABLE_MAKE_JOBS= YES
# Set to disable assertions and statistics gathering in malloc(3)
MALLOC_PRODUCTION= YES
# Set this to use svn(1) to update your src tree with make update
SVN_UPDATE= YES
# Program to use
SVN= /usr/local/bin/svn
SVNFLAGS= -r HEAD
#
#PORTS_MODULES= x11/nvidia-driver
###
### CLANG
###
.if !defined(NO_CLANG)
#.if ${.CURDIR:M/usr/src*} || ${.CURDIR:M/usr/obj*} || ${.CURDIR:M/sys*}
.if !defined(CC) || ${CC} == cc
CC= clang
.endif
.if !defined(CXX) || ${CXX} == c++
CXX= clang++
.endif
.if !defined(CPP) || ${CPP} == cpp
CPP= clang-cpp
.endif
## Don't die on warnings
NO_WERROR=
WERROR=
### Don't forget this when using Jails!
#NO_FSCHG=
#
CFLAGS+= -pipe -O3 -fno-strict-aliasing
COPTFLAGS+= -pipe -O3
#.endif
.endif
FORCE_PKG_REGISTER= YES
# OpenLDAP
#WANT_OPENLDAP_VER= 24
WANT_OPENLDAP_SASL= yes
# BDB Version
WITH_BDB_VER= 5
WITH_BDB_HIGHEST= YES
# Rubin 1,9 wird Kaiser
RUBY_DEFAULT_VER= 1.9
# What PostgreSQL port should be set default
WANT_PGSQL_VER= 91
#
#FORCE_MAKE_JOBS= YES
# Wir wollen SAMBA36 statt des senilen SAMABA35
SAMBA_PORT= samba36
# KdF an der Optimierung
WITH_OPTIMIZED_FLAGS= YES
# Disables xf86-video-noveau driver and makes use of newer
libdrm/libGLUT possible
WITHOUT_NOUVEAU= YES
# New Xorg and Mesa
WITH_NEW_XORG=
Re: pf firewall and ftp
Fbsd8 wrote: Running 9.0 as a gateway host with pf firewall enabled. FTP is launched by inetd. Both active and passive ftp works from lan pc's to the host ftp. The lan ftp session can be initiated from the host or any lan pc and things work because there are no rules on the lan interface except single pass all rule. But I can not do host initiated or lan initiated ftp sessions to the public internet. Get operation not permitted message. Tried to setup ftp-proxy per openbsd pf manual without any joy. Looking for working rule set with nat and ftp services to study and learn from. OK I have uncovered what the problem is. The pf version running on Freebsd 9.0 matches the version running on openbsd 4.5. Found it on man pf at the end. The documentation on the Openbsd website for pf is for Openbsd 5.0 and it has warning saying NOTE: This information is for OpenBSD 4.7. NAT configuration was significantly different in earlier versions. http://pf4freebsd.love2party.net/ has more info about how back dated the 9.0 Freebsd production version of pf is. The Freebsd handbook had a detailed section on pf including rules examples matching the version of pf included with 9.0 But someone allowed it to be removed in the current version of the handbook. So here we are with an outdated version of pf in the current production 9.0 version of Freebsd and there is no documentation available on nat rule syntax in the handbook or at openbsd/pf. Going to dig through the 9.0 pf man pages for the info ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
