Re: mktemp(1) in /tmp or $PWD?
On Thu, 25 Feb 2010, Garrett Cooper wrote: So what I did was I wrote up a patch to be *I know... here it comes* more like GNU coreutils' copy of mktemp. What's the motivation for this? I'm a little confused about why we'd want to change this when the -t option already exists. Also, does POSIX say anything about what the default should be? Doug -- Improve the effectiveness of your Internet presence with a domain name makeover!http://SupersetSolutions.com/ Computers are useless. They can only give you answers. -- Pablo Picasso ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ddb and dump devices
I think a better idea would be to dump over the network to another server. We do that at $(WORK) for diskless machines and it's quite effective. Currently what we have is very rough and only implemented on 6.1, but if I get some time this weekend I'll try to clean it up and provide a patch against HEAD. Ryan Stone ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
ddb and dump devices
Hi again Hackers, Another question that popped up recently in my mind given some of the stability issues with RAID controllers is: what happens when you can't dump to a RAID or non-RAIDed PATA/SATA device? In particular, does it make sense to have the functionality where one could make the dump/panic operation fault resistant where the person could get a `freebie' and use an alternate device, like USB thumbdrives, tapedrives (not saying that'd be smart), or a network device to dump the data? I realize that not all cases are recoverable and there will be some coverage gaps still, but it'd be nice if we could fill in the gaps that do exist when stuff goes horribly south. This functionality would be helpful for my company at least (Ironport) as disk RAIDs sometimes fail and we don't have a means of writing back data, and we don't always have serial connectivity on our appliances and our customers typically don't in the field, s.t. we can use kgdb on the machine and get more useful tracebacks from panic conditions. Please let me know what you think and whether or not this is a worthwhile endeavor. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: mktemp(1) in /tmp or $PWD?
On Thu, Feb 25, 2010 at 6:50 PM, Garrett Cooper wrote:
> Hi Hackers,
> Really basic question (because I'm relatively new to the Unix
> scene -- only been using it for the last 10 years, so I don't know if
> this was done for backwards compatibility with SysV) -- is mktemp(1)
> without -t supposed to default to $PWD instead of /tmp if a template
> is specified, e.g.
>
> [r...@left4dead /usr/home/garrcoop]# mktemp fooXX
> foovE3FLt
> [r...@left4dead /usr/home/garrcoop]# ls foovE3FLt
> foovE3FLt
>
> I ask because GNU coreutils' copy of mktemp (I know... I know...)
> defaults to /tmp if $TMPDIR isn't specified.
And this is the reason why I asked...
I was getting annoyed when I ran out of space in /usr today
iteratively trying to generate mfsroots because I expected the default
temp directory to be /tmp. Turns out it wasn't, and the reason why is
that $PWD is the assumed $TMPDIR iff an argument with -t isn't
specified.
So what I did was I wrote up a patch to be *I know... here it comes*
more like GNU coreutils' copy of mktemp.
Three behavioral changes I'm proposing are:
1. If mktemp is called without a prefix or template, it will
automatically generate one file (or directory) instead of erroring
out. That way if someone wants to call mktemp, they will automatically
get a file instead of having to produce a template or prefix.
2. $TMPDIR (or /tmp if not specified) will automatically get tacked
onto each file, instead of it being implicitly $PWD; that way a bunch
of temporary files will be generated _in_ /tmp (and subsequently wiped
on each reboot if the sysadmin sets it up that way) instead of having
a boatload of temporary files spread across a machine, which the user
must clean up based on the current working directory for their
applications.
3. All files will be prefixed with `tmp' by default instead of
`mktemp' (for consistency with GNU coreutils, and also because it
allows folks to increase the size of the random string in the files).
If folks like it, I'll create a PR with this change and also update
the manpage to reflect the change.
Thanks,
-Garrett
Index: mktemp.c
===
--- mktemp.c(revision 204344)
+++ mktemp.c(working copy)
@@ -48,6 +48,12 @@
static void usage(void);
+#define MK_TEMPLATE(template) \
+ asprintf(&name, "%s%s%s", \
+ (tmpdir == NULL ? _PATH_TMP : tmpdir), \
+ (tmpdir == NULL ? "" : "/" ), \
+ template)
+
int
main(int argc, char **argv)
{
@@ -55,13 +61,13 @@
char *tmpdir;
const char *prefix;
char *name;
- int dflag, qflag, tflag, uflag;
+ int dflag, qflag, uflag;
- ret = dflag = qflag = tflag = uflag = 0;
- prefix = "mktemp";
+ ret = dflag = qflag = uflag = 0;
+ prefix = NULL;
name = NULL;
- while ((c = getopt(argc, argv, "dqt:u")) != -1)
+ while ((c = getopt(argc, argv, "dqt:u")) != -1) {
switch (c) {
case 'd':
dflag++;
@@ -73,7 +79,6 @@
case 't':
prefix = optarg;
- tflag++;
break;
case 'u':
@@ -83,16 +88,29 @@
default:
usage();
}
+ }
argc -= optind;
argv += optind;
- if (tflag) {
- tmpdir = getenv("TMPDIR");
- if (tmpdir == NULL)
- asprintf(&name, "%s%s.", _PATH_TMP, prefix);
- else
- asprintf(&name, "%s/%s.", tmpdir, prefix);
+ tmpdir = getenv("TMPDIR");
+
+ /*
+* User didn't specify anything; let's default to a file prefixed by
+* tmp*
+*/
+ if (prefix == NULL && argc == 0) {
+ prefix = "tmp.XX";
+ }
+
+ /*
+* Make sure that the user specified an option with -t (or nothing
+* at all -- which equates to a file in tmp prefixed file).
+*/
+ if (prefix != NULL) {
+
+ MK_TEMPLATE(prefix);
+
/* if this fails, the program is in big trouble already */
if (name == NULL) {
if (qflag)
@@ -100,44 +118,58 @@
else
errx(1, "cannot generate template");
}
- } else if (argc < 1) {
- usage();
+
}
-
+
/* generate all requested files */
while (name != NULL || argc > 0) {
+
if (name == NULL) {
- name = strdup(argv[0]);
+ MK_TEMPLATE(argv[0]);
argv++;
argc--;
}
- if (dflag) {
- if (mkdtemp(name) == NULL) {
- ret = 1;
-
mktemp(1) in /tmp or $PWD?
Hi Hackers, Really basic question (because I'm relatively new to the Unix scene -- only been using it for the last 10 years, so I don't know if this was done for backwards compatibility with SysV) -- is mktemp(1) without -t supposed to default to $PWD instead of /tmp if a template is specified, e.g. [r...@left4dead /usr/home/garrcoop]# mktemp fooXX foovE3FLt [r...@left4dead /usr/home/garrcoop]# ls foovE3FLt foovE3FLt I ask because GNU coreutils' copy of mktemp (I know... I know...) defaults to /tmp if $TMPDIR isn't specified. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Nasty bug in jn(3)
There's a nasty little bug lurking in jn(3).
#include
#include
int
main(void)
{
double z;
int i, n;
z = 2.4048255576957729;
for (n = 2; n < 10; n++)
printf("%d %e\n", n, jn(n,z));
return (0);
}
troutmask:kargl[446] cc -o z testjn.c -lm
troutmask:kargl[447] ./z
2 4.317548e-01
3 -inf
4 4.069027e-02
5 -inf
6 3.247664e-03
7 -inf
8 7.495602e-05
9 -inf
I can assure you that -inf is not a valid value for
an integer order Bessel function at z = 2.40482555...
A quick inspection of e_jn.c suggest a similar
problem maybe found at other zeros of j0(x).
--
Steve
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: 2 bytes allocated problems
On Thursday 25 February 2010 23:46:03 Dag-Erling Smørgrav wrote:
> "Matthias Andree" writes:
> > Dag-Erling Smørgrav writes:
> > > char a[9] = { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' };
> >
> > char a[9] = "abcdefghi";
> >
> > suffices. The compiler knows there isn't room for the terminal '\0'
> > and omits it.
>
> Some compilers (gcc at least) warn about it.
It shouldn't. C99 explicitly has this as an example (6.7.8.32). It's also of
note that
char *p = "abc"
is different from
char a[] = "abc"
in that the elements of a are modifiable, while modifying p[x] yields
undefined behavior.
But this is increasingly getting out of scope for hackers@ ... so I'll shut up
now.
--
Max
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: 2 bytes allocated problems
"Matthias Andree" writes:
> Dag-Erling Smørgrav writes:
> > char a[9] = { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' };
> char a[9] = "abcdefghi";
>
> suffices. The compiler knows there isn't room for the terminal '\0'
> and omits it.
Some compilers (gcc at least) warn about it.
DES
--
Dag-Erling Smørgrav - d...@des.no
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
RE: ntpd hangs under FBSD 8
> We'll likely go with this solution instead of downgrading Python and the
> related libraries.
In fact I came up with another solution. I realized that since the problem was
related to the process signal mask, instead of called ntpd directly, wrap it up
in a C app that resets the signal mask to something that works. I have the
following code:
sigset_t set, oset;
sigemptyset(&set);
pthread_sigmask(SIG_SETMASK, &set, &oset);
system("/usr/sbin/ntpd -g -q");
pthread_sigmask(SIG_SETMASK, &oset, NULL);
I wrapped this up into a standalone app and call this from Python instead of
calling ntpd directly. This solved the problem--no more hang. Thanks very much
to Kostik Belousov for his "wild guess" that this was related to the process
signal mask. His guess was dead on.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: 2 bytes allocated problems
Am 24.02.2010, 20:55 Uhr, schrieb Dag-Erling Smørgrav :
Why is there a 0 after the 'i'? Because when you write "abcdefghi", the
compiler actually stores "abcdefghi\0". That's the definition of
"string" in C: a sequence of characters immediately followed by a 0. If
you don't want the 0 there, you have to do something like this:
char a[9] = { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' };
char a[9] = "abcdefghi";
suffices. The compiler knows there isn't room for the terminal '\0' and
omits it.
char a[] = "abcdefghi";
would append the implicit \0 and reserve 10 bytes for a.
but then you don't have a string, just an array of 9 chars.
Not that the compiler itself could/would tell the difference after
initialization, or that it would even care. It's the library functions
that expect strings that care about the \0.
Beyond that, I recommend a C book to Andrey.
--
Matthias Andree
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
32 bit Linux lseek missing overflow check (was: Re: Linuxolator patches: stat and lseek SEEK_END for disk devices)
On Tue, Feb 23, 2010 at 10:50:10PM +0100, Juergen Lock wrote: > Hi! > > Before this gets buried on -hackers in another thead... :) > > I now have disks appear as block devices for Linux processes (there > already was commented out code for that in linux_stats.c, I hope my > version is now `correct enough' to be usable [1]), and I made a simple > patch to make lseek SEEK_END (L_XTND in the source) dtrt on disk > devices too by simply invoking the DIOCGMEDIASIZE ioctl there; [2] > both of these things are what (some) Linux processes expect. > > Patches are here: (made on stable/8, if they don't apply on head > I'll have to make extra versions for that...) > http://people.freebsd.org/~nox/linuxdisk-blk.patch [1] > http://people.freebsd.org/~nox/lseek-seek_end.patch [2] > > And yes, with these patches the Linux bsdtar mentioned on -hackers > in the `"tar tfv /dev/cd0" speedup patch' thread now also runs fast > on FreeBSD. :) I now added an vn_isdisk() check to the second patch after comments from julian, and I made a new patch that adds an overflow check to the 32 bit linux lseek: (also at http://people.freebsd.org/~nox/linux-lseek-overflow.patch ) Index: src/sys/compat/linux/linux_file.c === RCS file: /home/scvs/src/sys/compat/linux/linux_file.c,v retrieving revision 1.119.2.1 diff -u -p -u -p -r1.119.2.1 linux_file.c --- src/sys/compat/linux/linux_file.c 3 Aug 2009 08:13:06 - 1.119.2.1 +++ src/sys/compat/linux/linux_file.c 25 Feb 2010 20:08:47 - @@ -226,6 +226,7 @@ linux_lseek(struct thread *td, struct li int whence; } */ tmp_args; int error; +l_off_t l_off; #ifdef DEBUG if (ldebug(lseek)) @@ -236,6 +237,10 @@ linux_lseek(struct thread *td, struct li tmp_args.offset = (off_t)args->off; tmp_args.whence = args->whence; error = lseek(td, &tmp_args); +/* Check for overflow like Linux does. */ +l_off = *(off_t *)td->td_retval; +if (((off_t)l_off) != *(off_t *)td->td_retval) + error = EOVERFLOW; return error; } ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
debug libraries
Apologies if this is the wrong list Are there prepackaged debug versions of the system libraries available (like 'yum install *-debuginfo' in Fedora and 'apt-get install *-dbg' in Ubuntu)? Dr ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ntpd hangs under FBSD 8
On Thu, Feb 25, 2010 at 09:57:48AM -0600, Peter Steele wrote:
> >Very wild guess, check the process signal mask of the child for both methods
> >of spawning.
>
> I'm running ntpd through Python. How do I check the process signal mask? I
> did some quick searches and it seems Python does not support sigprocmask().
>
> In my searches I came across this link:
>
> http://bugs.python.org/msg61870
>
> I think you might be right that this is related to the signal mask. In my
> scenario the select call is hanging indefinitely, just like discussed in this
> article.
>
Below is the quickly made patch to add ability to show signal disposition
to the procstat(1). I am not sure about duplicating information about
catch/ignore state of the signal for all threads (this information is
process-global), but I think this is more usable for scripts.
diff --git a/usr.bin/procstat/Makefile b/usr.bin/procstat/Makefile
index 1c187b0..251fc06 100644
--- a/usr.bin/procstat/Makefile
+++ b/usr.bin/procstat/Makefile
@@ -10,6 +10,7 @@ SRCS= procstat.c \
procstat_files.c\
procstat_kstack.c \
procstat_threads.c \
+ procstat_threads_sigs.c \
procstat_vm.c
LDADD+=-lutil
diff --git a/usr.bin/procstat/procstat.c b/usr.bin/procstat/procstat.c
index bc02682..cbd4eca 100644
--- a/usr.bin/procstat/procstat.c
+++ b/usr.bin/procstat/procstat.c
@@ -38,7 +38,7 @@
#include "procstat.h"
-static int aflag, bflag, cflag, fflag, kflag, sflag, tflag, vflag;
+static int aflag, bflag, cflag, fflag, iflag, kflag, sflag, tflag, vflag;
inthflag;
static void
@@ -46,7 +46,7 @@ usage(void)
{
fprintf(stderr, "usage: procstat [-h] [-w interval] [-b | -c | -f | "
- "-k | -s | -t | -v]\n");
+ "-i | -k | -s | -t | -v]\n");
fprintf(stderr, "[-a | pid ...]\n");
exit(EX_USAGE);
}
@@ -61,6 +61,8 @@ procstat(pid_t pid, struct kinfo_proc *kipp)
procstat_args(pid, kipp);
else if (fflag)
procstat_files(pid, kipp);
+ else if (iflag)
+ procstat_threads_sigs(pid, kipp);
else if (kflag)
procstat_kstack(pid, kipp, kflag);
else if (sflag)
@@ -109,7 +111,7 @@ main(int argc, char *argv[])
char *dummy;
interval = 0;
- while ((ch = getopt(argc, argv, "abcfkhstvw:")) != -1) {
+ while ((ch = getopt(argc, argv, "abcfikhstvw:")) != -1) {
switch (ch) {
case 'a':
aflag++;
@@ -127,6 +129,10 @@ main(int argc, char *argv[])
fflag++;
break;
+ case 'i':
+ iflag++;
+ break;
+
case 'k':
kflag++;
break;
diff --git a/usr.bin/procstat/procstat.h b/usr.bin/procstat/procstat.h
index 8bacab7..10f8fce 100644
--- a/usr.bin/procstat/procstat.h
+++ b/usr.bin/procstat/procstat.h
@@ -41,6 +41,7 @@ void procstat_cred(pid_t pid, struct kinfo_proc *kipp);
void procstat_files(pid_t pid, struct kinfo_proc *kipp);
void procstat_kstack(pid_t pid, struct kinfo_proc *kipp, int kflag);
void procstat_threads(pid_t pid, struct kinfo_proc *kipp);
+void procstat_threads_sigs(pid_t pid, struct kinfo_proc *kipp);
void procstat_vm(pid_t pid, struct kinfo_proc *kipp);
#endif /* !PROCSTAT_H */
diff --git a/usr.bin/procstat/procstat_threads_sigs.c
b/usr.bin/procstat/procstat_threads_sigs.c
new file mode 100644
index 000..814f0c4
--- /dev/null
+++ b/usr.bin/procstat/procstat_threads_sigs.c
@@ -0,0 +1,111 @@
+/*-
+ * Copyright (c) 2007 Robert N. M. Watson
+ * Copyright (c) 2010 Konstantin Belousov
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARI
RE: ntpd hangs under FBSD 8
>Very wild guess, check the process signal mask of the child for both methods >of spawning. I'm running ntpd through Python. How do I check the process signal mask? I did some quick searches and it seems Python does not support sigprocmask(). In my searches I came across this link: http://bugs.python.org/msg61870 I think you might be right that this is related to the signal mask. In my scenario the select call is hanging indefinitely, just like discussed in this article. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [Proof of Concept] Stacked unionfs based 'tinderbox'
On 25 February 2010 15:53, Ulrich Spörlein wrote: > On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote: >> Hi, >> >> As some may have noticed on -current I have been working on using >> stacked unionfs to implement a 'tinderbox' type build system. I have >> successfully used the scripts to build x11/xorg (and have compared the >> results to using the traditional approach using pkg_add). The build >> system is stable except for one nasty corner case: deadlocks. > > When I did this a couple of years ago, the major problems were failing > chdir(2) calls during ports build, etc. > >> To setup a compatible test environment requires: >> - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the >> kernel will panic with a double fault. WITNESS options results in >> substantial performance degradation. >> - patch mtree (see below) [optional] >> - create the appropriate chroot environment (and reboot) [see below >> for corner case] >> >> A performance bottleneck in mtree was identified. This resulted in >> mtree (as run by port install) consuming ~20% of the build time. See >> bin/143732 for a patch and further details. > > Good work! > >> The normal tinderbox approach takes ~80% more time to install compared to the >> quick and dirty approach. The stacked unionfs approach takes ~170% more time >> (an increase of ~50% over the tinderbox approach). Some performance gains >> can >> be had if one uses memory backed storage (vs HDD in this case). > > Please explain: what is the quick and dirty approach and which one is > faster now? The quick and dirty is `make -C /usr/ports/x11/xorg install clean`. The stacked unionfs is still the slowest (even with a 20% improvement from patching mtree). > As your scripts did not make it through, perhaps you can upload them to > the wiki? What I did back then was using a clean base system as the > underlying unionfs store to avoid re-generating the clean base over and > over again. Nowadays, a ZFS clone would probably be the way to go. > > I'm not sure if a recursive approach is feasible here, as you can have > only one underlying unionfs mount. But special casing, e.g., perl may > still give a massive speedup. So for each port that has perl as > dependancy, you would not pull in the clean base + pkg_add perl, but > instead grab the clean-base+perl directory as an underlying unionfs. > > Cheers > Uli > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ntpd hangs under FBSD 8
On Thu, Feb 25, 2010 at 08:12:05AM -0600, Peter Steele wrote: > >I think problem not in ntpd, since I use ntpdate. And in 50% times, when it > >run from startup script, it hangs with kernel. > >No Ctrl+C work, kernel don`t answer for ping, just freeze. > >Problem somewhere in kernel, maybe in subsystems that set new time, maybe in > >network(UDP) parts. > >This problem don`t affect other programs, so I think this in time handling > >code. > > I think you may be describing a different problem. For one thing, we don't > use ntpdate, we use the "ntpd -g -q" alternative. Secondly, for us ntpd is > hanging 100% of the time when run via a Python thread class. The exception is > Python 2.5.1; this succeeds 100% of the time. > > >Peter, what platform You use? I use MIPS BCM5354. > > We have a variety of 1U and 3U boxes. They all hang the same way. Very wild guess, check the process signal mask of the child for both methods of spawning. pgp2TMyaOAOk4.pgp Description: PGP signature
RE: ntpd hangs under FBSD 8
>I think problem not in ntpd, since I use ntpdate. And in 50% times, when it >run from startup script, it hangs with kernel. >No Ctrl+C work, kernel don`t answer for ping, just freeze. >Problem somewhere in kernel, maybe in subsystems that set new time, maybe in >network(UDP) parts. >This problem don`t affect other programs, so I think this in time handling >code. I think you may be describing a different problem. For one thing, we don't use ntpdate, we use the "ntpd -g -q" alternative. Secondly, for us ntpd is hanging 100% of the time when run via a Python thread class. The exception is Python 2.5.1; this succeeds 100% of the time. >Peter, what platform You use? I use MIPS BCM5354. We have a variety of 1U and 3U boxes. They all hang the same way. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ntpd hangs under FBSD 8
Alexey Shuvaev writes: > The flag you should look at is '-g'. GCC supports debuggind symbols > together with -O2 optimizations. It is generally not a good idea to use -O2 for debugging versions, since gcc will optimize away many local variables. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ntpd hangs under FBSD 8
On Wed, Feb 24, 2010 at 03:56:35PM -0600, Peter Steele wrote: > >> How do I get libc built with full debug symbols? > > > >I haven't tried it by myself but think here is the way to go: put the > >following to /etc/make.conf and recompile needed libraries / ports. > >WITH_DEBUG=yes > >DEBUG_FLAGS=-g > > That didn't seem to have any effect. I still see -O2 being used > instead of -O0. > The flag you should look at is '-g'. GCC supports debuggind symbols together with -O2 optimizations. Others have posted suggenstions how to build libraries with debugging symbols which go in the same direction. However, with the above variables in make.conf you do not need to remember all the places where you have to put DEBUG_FLAGS=-g in the command line. Just normal buildworld and buildkernel targets will dtrt. That is, you will get the complete base system with debug symbols. The variable WITH_DEBUG=yes is for the software from ports. Just FYI. > >Mmm... Do other daemons (sshd, lpd, ...) also fail when started > through this script? Normal commands (ls, ps) seem not affected. > > I tried a few other things and they all seemed to run correctly. > We use this same general approach in the full version of this script > to launch lots of applications. Its role in fact is a process > launcher/monitor. I stripped it down to the bare minimum in order > to isolate the cause of the problem. It seems that only ntpd hangs, > but not if I use Python 2.5.1. > ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [Proof of Concept] Stacked unionfs based 'tinderbox'
On Thu, 25.02.2010 at 10:08:15 +0200, David Naylor wrote: > Hi, > > As some may have noticed on -current I have been working on using > stacked unionfs to implement a 'tinderbox' type build system. I have > successfully used the scripts to build x11/xorg (and have compared the > results to using the traditional approach using pkg_add). The build > system is stable except for one nasty corner case: deadlocks. When I did this a couple of years ago, the major problems were failing chdir(2) calls during ports build, etc. > To setup a compatible test environment requires: > - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the > kernel will panic with a double fault. WITNESS options results in > substantial performance degradation. > - patch mtree (see below) [optional] > - create the appropriate chroot environment (and reboot) [see below > for corner case] > > A performance bottleneck in mtree was identified. This resulted in > mtree (as run by port install) consuming ~20% of the build time. See > bin/143732 for a patch and further details. Good work! > The normal tinderbox approach takes ~80% more time to install compared to the > quick and dirty approach. The stacked unionfs approach takes ~170% more time > (an increase of ~50% over the tinderbox approach). Some performance gains can > be had if one uses memory backed storage (vs HDD in this case). Please explain: what is the quick and dirty approach and which one is faster now? As your scripts did not make it through, perhaps you can upload them to the wiki? What I did back then was using a clean base system as the underlying unionfs store to avoid re-generating the clean base over and over again. Nowadays, a ZFS clone would probably be the way to go. I'm not sure if a recursive approach is feasible here, as you can have only one underlying unionfs mount. But special casing, e.g., perl may still give a massive speedup. So for each port that has perl as dependancy, you would not pull in the clean base + pkg_add perl, but instead grab the clean-base+perl directory as an underlying unionfs. Cheers Uli ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: ntpd hangs under FBSD 8
On Wed, 24 Feb 2010 15:56:35 -0600 Peter Steele wrote: >> >> How do I get libc built with full debug symbols? >> > >> >I haven't tried it by myself but think here is the way to go: put the >> >following to /etc/make.conf and recompile needed >> >libraries / ports. WITH_DEBUG=yes >> >DEBUG_FLAGS=-g >> >> That didn't seem to have any effect. I still see -O2 being used instead of >> -O0. Try to use make DEBUG_FLAGS=-g WITH_DEBUG=yes buildkworld make DEBUG_FLAGS=-g WITH_DEBUG=yes buildkernel >> >> >Mmm... Do other daemons (sshd, lpd, ...) also fail when started through >> >this script? Normal commands (ls, ps) seem not >> >affected. >> >> I tried a few other things and they all seemed to run correctly. We use this >> same general approach in the full version of this >> script to launch lots of applications. Its role in fact is a process >> launcher/monitor. I stripped it down to the bare minimum >> in order to isolate the cause of the problem. It seems that only ntpd hangs, >> but not if I use Python 2.5.1. >> I think problem not in ntpd, since I use ntpdate. And in 50% times, when it run from startup script, it hangs with kernel. No Ctrl+C work, kernel don`t answer for ping, just freeze. Problem somewhere in kernel, maybe in subsystems that set new time, maybe in network(UDP) parts. This problem don`t affect other programs, so I think this in time handling code. Peter, what platform You use? I use MIPS BCM5354. >> >> ___ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org" -- Alexandr Rybalko aka Alex RAY ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
[Proof of Concept] Stacked unionfs based 'tinderbox'
Hi, As some may have noticed on -current I have been working on using stacked unionfs to implement a 'tinderbox' type build system. I have successfully used the scripts to build x11/xorg (and have compared the results to using the traditional approach using pkg_add). The build system is stable except for one nasty corner case: deadlocks. To setup a compatible test environment requires: - recompile the kernel with `options KSTACK_PAGES=32`, otherwise the kernel will panic with a double fault. WITNESS options results in substantial performance degradation. - patch mtree (see below) [optional] - create the appropriate chroot environment (and reboot) [see below for corner case] A performance bottleneck in mtree was identified. This resulted in mtree (as run by port install) consuming ~20% of the build time. See bin/143732 for a patch and further details. Here are my build times (in the chroot): # time make -C /usr/ports/x11/xorg install clean 2397.42 real 2228.35 user 1151.00 sys # time ./ports-union-builder.sh (old mtree) 8123.25 real 2280.53 user 6319.77 sys # time ./ports-union-builder.sh (new mtree) 6456.11 real 2272.07 user 5778.74 sys # time ./ports-tinder-builder.sh (new mtree) 4270.82 real 2961.88 user 1636.27 sys The "new mtree" is the mtree with the above mentioned patch applied. ports-tinder-builder is an approximate to how the read tinderbox works. The normal tinderbox approach takes ~80% more time to install compared to the quick and dirty approach. The stacked unionfs approach takes ~170% more time (an increase of ~50% over the tinderbox approach). Some performance gains can be had if one uses memory backed storage (vs HDD in this case). This approach should have application where wearing of the underlying storage (such as SDD) is of concern. Further investigation suggests that stacking unionfs scales exponentially, not linearly. This may be an area for further improvements. The corner case: sometimes the directories involved experience a deadlock. Any IO under those directories stall the program. I have found that deleting files in the chroot prior to running the unionfs script reliably causes the deadlock. Since WITNESS had significant performance impact the only data into the deadlocks come from procstat. Below are a variety of procstat -kk calls for deadlocked programs: 68217 100119 pkg-config -mi_switch+0x16f sleepq_wait+0x3b __lockmgr_args+0x641 ffs_lock+0x8c VOP_LOCK1_APV+0x46 unionfs_lock+0x28a VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 VOP_LOCK1_APV+0x46 unionfs_lock+0x161 6574 100201 ls -mi_switch+0x16f sleepq_wait+0x3b __lockmgr_args+0x654 ffs_lock+0x8c VOP_LOCK1_APV+0x46 _vn_lock+0x44 vget+0x67 cache_lookup+0x4f8 vfs_cache_lookup+0xad VOP_LOOKUP_APV+0x40 lookup+0x44d namei+0x4ec kern_statat_vnhook+0x82 kern_statat+0x15 lstat+0x22 syscall+0x1e7 Xfast_syscall+0xe1 Is anyone able to fix the deadlocking issue? Regards, David. P.S. This has been tested under 8 and 9. P.P.S. The scripts used are attached ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
