elf obj load: skip zero-sized sections early
Proposed patch skips zero sized sections without going into trouble of
allocating section entry (progtab), doing zero-sized memory allocs and copies.
I observe that sometimes zero-sized set_pcpu sections are produced in module
objects, maybe when a module doesn't create any per cpu data of its one, but
references external pcpu data. Not sure.
Main goal of this patch is to play nice with external debugging tools (e.g.
kgdb) which ignore zero-sized sections outright. Current code effectively
ignores but may apply their alignment to the next non-zero section if its
required alignment is smaller. So the patch should get rid of that side effect
as well as do some very tiny resource conservation.
This work is based on np@'s investigation and original patch:
http://lists.freebsd.org/pipermail/freebsd-hackers/2009-November/030093.html
diff --git a/sys/boot/common/load_elf_obj.c b/sys/boot/common/load_elf_obj.c
index 4b3aaea..6f3b349 100644
--- a/sys/boot/common/load_elf_obj.c
+++ b/sys/boot/common/load_elf_obj.c
@@ -221,6 +221,8 @@ __elfN(obj_loadimage)(struct preloaded_file *fp, elf_file_t
ef, u_int64_t off)
for (i = 0; i < hdr->e_shnum; i++)
shdr[i].sh_addr = 0;
for (i = 0; i < hdr->e_shnum; i++) {
+ if (shdr[i].sh_size == 0)
+ continue;
switch (shdr[i].sh_type) {
case SHT_PROGBITS:
case SHT_NOBITS:
diff --git a/sys/kern/link_elf_obj.c b/sys/kern/link_elf_obj.c
index a337fd0..b0df57d 100644
--- a/sys/kern/link_elf_obj.c
+++ b/sys/kern/link_elf_obj.c
@@ -555,6 +555,8 @@ link_elf_load_file(linker_class_t cls, const char *filename,
symtabindex = -1;
symstrindex = -1;
for (i = 0; i < hdr->e_shnum; i++) {
+ if (shdr[i].sh_size == 0)
+ continue;
switch (shdr[i].sh_type) {
case SHT_PROGBITS:
case SHT_NOBITS:
@@ -677,6 +679,8 @@ link_elf_load_file(linker_class_t cls, const char *filename,
/* Size up code/data(progbits) and bss(nobits). */
alignmask = 0;
for (i = 0; i < hdr->e_shnum; i++) {
+ if (shdr[i].sh_size == 0)
+ continue;
switch (shdr[i].sh_type) {
case SHT_PROGBITS:
case SHT_NOBITS:
@@ -737,6 +741,8 @@ link_elf_load_file(linker_class_t cls, const char *filename,
ra = 0;
alignmask = 0;
for (i = 0; i < hdr->e_shnum; i++) {
+ if (shdr[i].sh_size == 0)
+ continue;
switch (shdr[i].sh_type) {
case SHT_PROGBITS:
case SHT_NOBITS:
--
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: using cupsd instead of base lpr [was Re: [HEADS UP] Kernel modules don't work properly in FreeBSD 8.1-RC1 (solved)]
On Thu, 24 Jun 2010 09:54:45 -0700 Ted Faber wrote: > On Thu, Jun 24, 2010 at 08:29:57AM -0700, Ted Faber wrote: > > On Thu, Jun 24, 2010 at 09:40:00AM +0100, Tom Evans wrote: > > > I also have this in make.conf: > > > CUPS_OVERWRITE_BASE=yes > > > WITHOUT_LPR=yes > > > > > > which print/cups-base uses to do make any lpr related binaries in > > > /usr/bin non-executable, so they are skipped over and the cups > > > specific ones in /usr/loca/bin are used instead. WITHOUT_LPR just > > > stops LPR being built by buildworld. > > > > The clear winner, and one I was unaware of. > > > > Thanks, Tom. > > CUPS_OVERWRITE_BASE seems to do an odd thing. It doesn't install the > cups binaries in /usr/bin, but it does do a chmod on everything it > replaces in /usr/bin . For example > > praxis:~$ ls -l /usr/bin/lpr > -r-sr-sr-x 1 root daemon 29876 Jun 24 09:16 /usr/bin/lpr > # portupgrade -f cups-base-1.4.3 > praxis:~$ ls -l /usr/bin/lpr > -- 1 root daemon 29876 Jun 24 09:16 /usr/bin/lpr > > I'll still use it, but interesting behavior. > IMO if you're going to make the binaries in base non-executable you might just as well delete them. But CUPS_OVERWRITE_BASE does have the advantage that it works without (active) user intervention. -- Gary Jennejohn ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: amd64 kernel modules: mapping sections to addresses
Here's a patch that is supposed to do the right thing for dtrace.
Perhaps I should have put the new code under __amd64__, but I decided to go more
"generic" and check for module's ELF type (ET_REL).
Reviews and testing are welcome!
diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_impl.h
b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_impl.h
index 6bcc5bc..a712d24 100644
--- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_impl.h
+++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_impl.h
@@ -137,6 +137,7 @@ typedef struct dt_module {
dt_idhash_t *dm_extern; /* external symbol definitions */
#if !defined(sun)
caddr_t dm_reloc_offset;/* Symbol relocation offset. */
+ uintptr_t *dm_sec_offsets;
#endif
} dt_module_t;
diff --git a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c
b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c
index af17501..d33fb95 100644
--- a/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c
+++ b/cddl/contrib/opensolaris/lib/libdtrace/common/dt_module.c
@@ -83,6 +83,14 @@ dt_module_syminit32(dt_module_t *dmp)
uint_t i, n = dmp->dm_nsymelems;
uint_t asrsv = 0;
+#if defined(__FreeBSD__)
+ GElf_Ehdr ehdr;
+ int is_elf_obj;
+
+ gelf_getehdr(dmp->dm_elf, &ehdr);
+ is_elf_obj = (ehdr.e_type == ET_REL);
+#endif
+
for (i = 0; i < n; i++, sym++) {
const char *name = base + sym->st_name;
uchar_t type = ELF32_ST_TYPE(sym->st_info);
@@ -97,8 +105,12 @@ dt_module_syminit32(dt_module_t *dmp)
(ELF32_ST_BIND(sym->st_info) != STB_LOCAL || sym->st_size))
{
asrsv++; /* reserve space in the address map */
-#if !defined(sun)
+#if defined(__FreeBSD__)
sym->st_value += (Elf_Addr) dmp->dm_reloc_offset;
+ if (is_elf_obj && sym->st_shndx != SHN_UNDEF &&
+ sym->st_shndx < ehdr.e_shnum)
+ sym->st_value +=
+ dmp->dm_sec_offsets[sym->st_shndx];
#endif
}
@@ -117,6 +129,14 @@ dt_module_syminit64(dt_module_t *dmp)
uint_t i, n = dmp->dm_nsymelems;
uint_t asrsv = 0;
+#if defined(__FreeBSD__)
+ GElf_Ehdr ehdr;
+ int is_elf_obj;
+
+ gelf_getehdr(dmp->dm_elf, &ehdr);
+ is_elf_obj = (ehdr.e_type == ET_REL);
+#endif
+
for (i = 0; i < n; i++, sym++) {
const char *name = base + sym->st_name;
uchar_t type = ELF64_ST_TYPE(sym->st_info);
@@ -130,9 +150,12 @@ dt_module_syminit64(dt_module_t *dmp)
if (sym->st_value != 0 &&
(ELF64_ST_BIND(sym->st_info) != STB_LOCAL || sym->st_size))
{
asrsv++; /* reserve space in the address map */
-
-#if !defined(sun)
+#if defined(__FreeBSD__)
sym->st_value += (Elf_Addr) dmp->dm_reloc_offset;
+ if (is_elf_obj && sym->st_shndx != SHN_UNDEF &&
+ sym->st_shndx < ehdr.e_shnum)
+ sym->st_value +=
+ dmp->dm_sec_offsets[sym->st_shndx];
#endif
}
@@ -722,7 +745,12 @@ dt_module_unload(dtrace_hdl_t *dtp, dt_module_t *dmp)
free(dmp->dm_asmap);
dmp->dm_asmap = NULL;
}
-
+#if defined(__FreeBSD__)
+ if (dmp->dm_sec_offsets != NULL) {
+ free(dmp->dm_sec_offsets);
+ dmp->dm_sec_offsets = NULL;
+ }
+#endif
dmp->dm_symfree = 0;
dmp->dm_nsymbuckets = 0;
dmp->dm_nsymelems = 0;
@@ -846,9 +874,12 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat
*k_stat)
(void) snprintf(fname, sizeof (fname),
"%s/%s/object", OBJFS_ROOT, name);
#else
+ GElf_Ehdr ehdr;
GElf_Phdr ph;
char name[MAXPATHLEN];
+ uintptr_t mapbase, alignmask;
int i = 0;
+ int is_elf_obj;
(void) strlcpy(name, k_stat->name, sizeof(name));
(void) strlcpy(fname, k_stat->pathname, sizeof(fname));
@@ -893,7 +924,20 @@ dt_module_update(dtrace_hdl_t *dtp, struct kld_file_stat
*k_stat)
dt_module_destroy(dtp, dmp);
return;
}
-
+#if defined(__FreeBSD__)
+ mapbase = (uintptr_t)k_stat->address;
+ gelf_getehdr(dmp->dm_elf, &ehdr);
+ is_elf_obj = (ehdr.e_type == ET_REL);
+ if (is_elf_obj) {
+ dmp->dm_sec_offsets =
+ malloc(ehdr.e_shnum * sizeof(*dmp->dm_sec_offsets));
+ if (dmp->dm_sec_offsets == NULL) {
+ dt_dprintf("failed to allocate memory\n");
+ dt_module_destroy(dtp, dmp);
+ return;
+ }
+ }
+#endif
/*
* Iterate over the section headers locating various sections of
* interest and use their attributes to flesh out the dt
hg convert stopped working
Hi,
probably wrong place to ask, but I great minds lurk here :-)
I have been mirroing FreeBSD via svn since last summer,
svnsync sync file:///cs/svn/freebsd/base
then converting to mercurial
hg convert ... file:///cs/svn/freebsd/base ${HG_HOME}/bsd/stable/8 ...
since I can better track local changes, all worked nice till someone
- probably me, while upgrading firefox, screwed up svn - it happens :-)
So I upgraded svn, and now my svnsync worked again, but mercurial stopped
working, so upgarded mercurial, but it's still not working:
Traceback (most recent call last):
File "/usr/local/lib/python2.6/site-packages/hgext/convert/hg.py", line 223,
in __init__
self.repo = hg.repository(self.ui, path)
File "/usr/local/lib/python2.6/site-packages/mercurial/hg.py", line 82, in
repository
repo = _lookup(path).instance(ui, path, create)
File "/usr/local/lib/python2.6/site-packages/mercurial/localrepo.py", line
2223, in instance
return localrepository(ui, util.drop_scheme('file', path), create)
File "/usr/local/lib/python2.6/site-packages/mercurial/localrepo.py", line
62, in __init__
raise error.RepoError(_("repository %s not found") % path)
RepoError: repository /cs/svn/freebsd/base not found
but /cs/svn/freebsd/base is ok:
ls -lsd /cs/svn/freebsd/base
4 drwxr-xr-x 6 svn svn 4096 Jun 7 2008 /cs/svn/freebsd/base
any help will be much appreciated,
thanks,
danny
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
Dag-Erling, I am using option 3 below. I have the following in my shell: amd64-kernel.sh #!/bin/sh trap "exit 1" 1 2 3 15 export SRCROOT=`pwd -P` export MAKEOBJDIRPREFIX=$SRCROOT/../amd64/obj export SRCCONF=$SRCROOT/src.conf # Use our private copy export SECKNOB="-DPRIVATE" KERNCONF=TCONF make NO_CLEAN=yes NO_PROFILE=yes NO_MODULES=yes KERNCONF=$KERNCONF buildkernel || exit 1 ... In the top-level makefile I have the following label: src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 2>&1 | tee build_amd64_kernel.log If there is a build failure with the kernel, it can be seen in the file 'build_amd64_kernel.log'. However, the top-level make file just continues on to the next label as if no error occurs. The reason we are using shell scripts is because of the environment variables that need to be defined and some other house-keeping stuff that I did not include in the above example. Also, I wanted scripts so the developer could just build individual "groups" without resorting to building everything. Thanks, Patrick Dag-Erling Smørgrav wrote: Patrick Mahan writes: My issue is that if there is a build failure at any point, the status does not seem to be propagated upward. For example, if the kernel fails to build due to incorrect code, the script -kernel64.sh stops (verifable by examining the logfile), however, the make will continue to the next target, src-world, and continue building. How do I propagate the status up to the top-level make? Your shell script needs to exit with a non-zero status if it fails. There are several ways to do this. For instance, if your shell script looks like this: #!/bin/sh make TARGET=amd64 kernel-toolchain you can: - prefix "make" with "exec": sh will exec make instead of running it in a subshell, and the exit status will be whatever make returns. - add the following line at the bottom: exit $? which causes the shell script to exit with the same exit status as the last command it ran, in this case make. - append "|| exit 1" to the the "make" command line; this will cause the script to exit immediately with status 1 if make fails, otherwise it will continue (in case you want to do something else if make succeeds) - wrap the make command line in an if statement, which allows you do additional work depending on the outcome, and end the failure case with "exit 1" or similar - insert the following line at any point between the shebang and the make command line: set -e this will cause sh to terminate the script immediately with a non-zero exit status if any command after the "set" line fails. However, I don't see the point of using shell scripts in the first place... DES ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
Patrick Mahan writes: > In the top-level makefile I have the following label: > > src-kernel: src-kernel-tools > cd src; ./amd64-kernel.sh 2>&1 | tee build_amd64_kernel.log > > If there is a build failure with the kernel, it can be seen in the > file 'build_amd64_kernel.log'. However, the top-level make file just > continues on to the next label as if no error occurs. Make looks at tee's exit status, not the script's. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
On Tue, Jun 22, 2010 at 11:18:43PM -0700, Patrick Mahan wrote:
> src-kern-tools:
> cd src; ./-kernel-toolchain.sh 2>&1 | tee
The pipeline will return the status of 'tee' which is almost always 0.
The exit status of the build script is ignored.
A simple fix is store the status in a file and refer to that afterwards.
Another fix uses a separate file descriptor to pass through the status,
like
{ st=$(
{
{ ./kernel-toolchain.sh 2>&1 3>&- 4>&-; echo $? >&3; } | tee logfile >&4
} 3>&1)
} 4>&1
but this is fairly complicated.
The issue can be sidestepped entirely by sending the output to a file
only; a developer can use tail -f to follow the build if necessary.
--
Jilles Tjoelker
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
What is the exected behavior with the NMI button?
While trying to get a deadlock sorted out in the GPROF code, I attempted to use this fancy shmancy NMI button on my Dell server. I noted that, not unlike the goggles, it did nothing once the system was deadlocked. I noted that when the system was running normally, an NMI log message would be spewed to the console. What is supposed to happen in these two cases when we toggle the NMI button? Sean ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: What is the exected behavior with the NMI button?
On Fri, 2010-06-25 at 07:17 -0700, Sean Bruno wrote: > While trying to get a deadlock sorted out in the GPROF code, I attempted > to use this fancy shmancy NMI button on my Dell server. > > I noted that, not unlike the goggles, it did nothing once the system was > deadlocked. I noted that when the system was running normally, an NMI > log message would be spewed to the console. In the event you don't get the goggles reference. http://www.youtube.com/watch?v=OqfOxm_1BE0 Sean ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
Oy vey! Good point! Not something I had considered (but should have). Is there a way to propogate that status through the pipe? I am using 'tee' so that the master build (which invokes the top-level make) can also see the make output. But going back and reading the man page on 'sh' shows me that on a pipline, the status is the status of the *last* command. Maybe I should do this instead? src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 2>&1 > build_amd64_kernel.log; \ tail -f build_amd64_kernel.log It is not too clear if the status is the last one in a compound command. I will give that a try. Thanks, Patrick Dag-Erling Smørgrav wrote: Patrick Mahan writes: In the top-level makefile I have the following label: src-kernel: src-kernel-tools cd src; ./amd64-kernel.sh 2>&1 | tee build_amd64_kernel.log If there is a build failure with the kernel, it can be seen in the file 'build_amd64_kernel.log'. However, the top-level make file just continues on to the next label as if no error occurs. Make looks at tee's exit status, not the script's. DES ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
Jilles,
Thanks for the more complicated example. I am always interested in
shell hacks like these, especially when they involve interesting uses
of file I/O redirection.
The tee is there so that the master build package (a perl script)
that builds not only my groups sources but other sources as well
is required to provide make 'output' for the master build log.
How is the status handled on compound commands?
./amd64-kernel.sh 2>&1 > build.log; tail -f build.log
would that obscure the exit status of the shell script as well? The
man page is not really clear on this.
Thanks,
Patrick
Jilles Tjoelker wrote:
On Tue, Jun 22, 2010 at 11:18:43PM -0700, Patrick Mahan wrote:
src-kern-tools:
cd src; ./-kernel-toolchain.sh 2>&1 | tee
The pipeline will return the status of 'tee' which is almost always 0.
The exit status of the build script is ignored.
A simple fix is store the status in a file and refer to that afterwards.
Another fix uses a separate file descriptor to pass through the status,
like
{ st=$(
{
{ ./kernel-toolchain.sh 2>&1 3>&- 4>&-; echo $? >&3; } | tee logfile >&4
} 3>&1)
} 4>&1
but this is fairly complicated.
The issue can be sidestepped entirely by sending the output to a file
only; a developer can use tail -f to follow the build if necessary.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
[RFC] mtree improvements
Hi,
I've created a patch that increases the performance of mtree. This is of
particular use during a port install. In an extreme case I have experienced a
~20% increase [1].
For a full discussion see PR bin/143732. This arose out of [2] where I
experienced the increase.
For your convenience I have attached the patch.
Please review this patch and if it is acceptable, commit it.
Regards,
David
1] http://markmail.org/message/iju3l6hyv7s7emrb
2] http://markmail.org/message/gfztjpszl5dozzii
--- /usr/src/usr.sbin/mtree/verify.c 2010-02-07 15:07:28.0 +0200
+++ verify.c 2010-02-07 15:04:10.0 +0200
@@ -50,17 +50,23 @@
static NODE *root;
static char path[MAXPATHLEN];
-static void miss(NODE *, char *);
+static int miss(NODE *, char *);
+static int check(NODE *, char *);
static int vwalk(void);
int
mtree_verifyspec(FILE *fi)
{
- int rval;
+ int rval = 0;
root = mtree_readspec(fi);
- rval = vwalk();
- miss(root, path);
+ /*
+ * No need to walk entire tree if we are only updating the structure
+ * and extra files are ignored.
+ */
+ if (!(uflag && eflag))
+ rval = vwalk();
+ rval |= miss(root, path);
return (rval);
}
@@ -155,15 +161,47 @@
return (rval);
}
-static void
+static int
+check(NODE *p, char *tail)
+{
+ FTSENT fts;
+ struct stat fts_stat;
+
+ strcpy(tail, p->name);
+
+ /*
+ * It is assumed that compare() only requires fts_accpath and fts_statp
+ * fields in the FTSENT structure.
+ */
+ fts.fts_accpath = path;
+ fts.fts_statp = &fts_stat;
+
+ if (stat(path, fts.fts_statp))
+ return (0);
+
+ p->flags |= F_VISIT;
+ if ((p->flags & F_NOCHANGE) == 0 && compare(p->name, p, &fts))
+ return (MISMATCHEXIT);
+ else
+ return (0);
+
+ /*
+ * tail is not restored to '\0' as the next time tail (or path) is used
+ * is with a strcpy (thus overriding the '\0'). See +19 lines below.
+ */
+}
+
+static int
miss(NODE *p, char *tail)
{
int create;
char *tp;
const char *type, *what;
- int serr;
+ int serr, rval = 0;
for (; p; p = p->next) {
+ if (uflag && eflag)
+ rval |= check(p, tail);
if (p->flags & F_OPT && !(p->flags & F_VISIT))
continue;
if (p->type != F_DIR && (dflag || p->flags & F_VISIT))
@@ -256,4 +294,5 @@
(void)printf("%s: file flags not set: %s\n",
path, strerror(errno));
}
+ return (rval);
}
signature.asc
Description: This is a digitally signed message part.
Re: [RFC] mtree improvements
On Fri, Jun 25, 2010 at 9:52 AM, David Naylor wrote:
> Hi,
>
> I've created a patch that increases the performance of mtree. This is of
> particular use during a port install. In an extreme case I have experienced a
> ~20% increase [1].
>
> For a full discussion see PR bin/143732. This arose out of [2] where I
> experienced the increase.
>
> For your convenience I have attached the patch.
>
> Please review this patch and if it is acceptable, commit it.
>
> Regards,
>
> David
>
> 1] http://markmail.org/message/iju3l6hyv7s7emrb
> 2] http://markmail.org/message/gfztjpszl5dozzii
Hmm... this has other interesting applications other than just ports,
but unfortunately pkg_install won't really feel as much of a
performance boost (because it uses mtree -e -U when +MTREE exists in
the package).
Comments follow.
Thanks!
-Garrett
--- /usr/src/usr.sbin/mtree/verify.c2010-02-07 15:07:28.0 +0200
+++ verify.c2010-02-07 15:04:10.0 +0200
@@ -50,17 +50,23 @@
static NODE *root;
static char path[MAXPATHLEN];
-static voidmiss(NODE *, char *);
+static int miss(NODE *, char *);
+static int check(NODE *, char *);
static int vwalk(void);
int
mtree_verifyspec(FILE *fi)
{
- int rval;
+ int rval = 0;
root = mtree_readspec(fi);
- rval = vwalk();
- miss(root, path);
+ /*
+* No need to walk entire tree if we are only updating the structure
+* and extra files are ignored.
+*/
+ if (!(uflag && eflag))
+ rval = vwalk();
gcooper> This is where the performance boost is coming from as you're
not walking the directory tree, correct?
+ rval |= miss(root, path);
return (rval);
}
@@ -155,15 +161,47 @@
return (rval);
}
-static void
+static int
+check(NODE *p, char *tail)
+{
+ FTSENT fts;
+ struct stat fts_stat;
+
+ strcpy(tail, p->name);
gcooper> Dangerous. Please use strlcpy with appropriate bounds.
+ /*
+* It is assumed that compare() only requires fts_accpath and fts_statp
+* fields in the FTSENT structure.
+*/
+ fts.fts_accpath = path;
+ fts.fts_statp = &fts_stat;
+
+ if (stat(path, fts.fts_statp))
+ return (0);
gcooper> What about symlink functionality? lstat(2)?
+ p->flags |= F_VISIT;
+ if ((p->flags & F_NOCHANGE) == 0 && compare(p->name, p, &fts))
+ return (MISMATCHEXIT);
+ else
+ return (0);
+
+ /*
+* tail is not restored to '\0' as the next time tail (or path) is used
+* is with a strcpy (thus overriding the '\0'). See +19 lines below.
+*/
+}
+
+static int
miss(NODE *p, char *tail)
{
int create;
char *tp;
const char *type, *what;
- int serr;
+ int serr, rval = 0;
gcooper> This isn't correct as per-style(9). Please do:
gcooper>
gcooper> int rval = 0;
gcooper> int serr;
gcooper>
gcooper> This reduces diff churn and is more style(9)-istically correct.
for (; p; p = p->next) {
+ if (uflag && eflag)
+ rval |= check(p, tail);
if (p->flags & F_OPT && !(p->flags & F_VISIT))
continue;
if (p->type != F_DIR && (dflag || p->flags & F_VISIT))
@@ -256,4 +294,5 @@
(void)printf("%s: file flags not set: %s\n",
path, strerror(errno));
}
+ return (rval);
}
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
PCI Express and drivers
I have a Xilinx PCI Express board that has an on board PCIe interface macro. I intend to have an address space with memory and another with my devices control registers. I wish to program this board under FreeBSD. It would seem to me that the way to do this would be to write a driver that would allow me to map these two address spaces into a user process which could then directly interact with the device. In my case my board doesn't really support multiple users, and it doesn't really seem to make sense to me to put a lot of code in the kernel to create some sort of interface to allow multiple processes to interact with my device via some sort of syscall which would impose a lot of overhead in any case. By mapping the address ranges into a user process I can write user programs to drive the board without having to recompile and reboot the kernel each time I make a change, plus if I do something stupid and crash my user space process it shouldn't bring down the whole kernel. Am I thinking about this wrong? Is there some place I can go to read up on what I should be doing? If I am thinking about this correctly, then how does one map device memory into a user space process? How does one make sure that only one process has such a mapping? Thanks Christopher ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: Help with some makefile hackery
Patrick Mahan writes: > Maybe I should do this instead? > > src-kernel: src-kernel-tools > cd src; ./amd64-kernel.sh 2>&1 > build_amd64_kernel.log; \ > tail -f build_amd64_kernel.log > > It is not too clear if the status is the last one in a compound > command. This won't work, because tail won't start until the build is done. You should just pass the file name as an argument to your script and handle it there. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: [RFC] mtree improvements
On Friday 25 June 2010 20:01:42 Garrett Cooper wrote:
> On Fri, Jun 25, 2010 at 9:52 AM, David Naylor
wrote:
> > Hi,
> >
> > I've created a patch that increases the performance of mtree. This is of
> > particular use during a port install. In an extreme case I have
> > experienced a ~20% increase [1].
> >
> > For a full discussion see PR bin/143732. This arose out of [2] where I
> > experienced the increase.
> >
> > For your convenience I have attached the patch.
> >
> > Please review this patch and if it is acceptable, commit it.
> >
> > Regards,
> >
> > David
> >
> > 1] http://markmail.org/message/iju3l6hyv7s7emrb
> > 2] http://markmail.org/message/gfztjpszl5dozzii
>
> Hmm... this has other interesting applications other than just ports,
> but unfortunately pkg_install won't really feel as much of a
> performance boost (because it uses mtree -e -U when +MTREE exists in
> the package).
>
> Comments follow.
> Thanks!
> -Garrett
I've included some more changes. I do not think -[uU] needs to be set when
skipping vwalk. I also move sflag so it will get called even if eflag is set
(bug in my patch).
I'm not sure sflag will produce the same results as check() and vwalk() may
call compare() in different orders? Will this impact on crc?
> --- /usr/src/usr.sbin/mtree/verify.c 2010-02-07 15:07:28.0 +0200
> +++ verify.c 2010-02-07 15:04:10.0 +0200
> @@ -50,17 +50,23 @@
> static NODE *root;
> static char path[MAXPATHLEN];
>
> -static void miss(NODE *, char *);
> +static int miss(NODE *, char *);
> +static int check(NODE *, char *);
> static int vwalk(void);
>
> int
> mtree_verifyspec(FILE *fi)
> {
> - int rval;
> + int rval = 0;
>
> root = mtree_readspec(fi);
> - rval = vwalk();
> - miss(root, path);
> + /*
> + * No need to walk entire tree if we are only updating the structure
> + * and extra files are ignored.
> + */
> + if (!(uflag && eflag))
> + rval = vwalk();
>
> gcooper> This is where the performance boost is coming from as you're
> not walking the directory tree, correct?
Yes, if an update is requested without reporting extra files. In some cases
with a well populated /usr/local (and a slowish HDD) it could take some time.
I think it is possible to even use this optimisation if uflag is not set?
> + rval |= miss(root, path);
> return (rval);
> }
>
> @@ -155,15 +161,47 @@
> return (rval);
> }
>
> -static void
> +static int
> +check(NODE *p, char *tail)
> +{
> + FTSENT fts;
> + struct stat fts_stat;
> +
> + strcpy(tail, p->name);
>
> gcooper> Dangerous. Please use strlcpy with appropriate bounds.
This is the same code used in miss.c:171.
It appears the code assumes that a path a mtree file describes will not exceed
MAXPATHLEN and does not check to see if the buffer overflows. I could create a
patch to fix that. Perhaps a separate patch?
> + /*
> + * It is assumed that compare() only requires fts_accpath and fts_statp
> + * fields in the FTSENT structure.
> + */
> + fts.fts_accpath = path;
> + fts.fts_statp = &fts_stat;
> +
> + if (stat(path, fts.fts_statp))
> + return (0);
>
> gcooper> What about symlink functionality? lstat(2)?
Fixed. This should now be consistent with fts_read.
> + p->flags |= F_VISIT;
> + if ((p->flags & F_NOCHANGE) == 0 && compare(p->name, p, &fts))
> + return (MISMATCHEXIT);
> + else
> + return (0);
> +
> + /*
> + * tail is not restored to '\0' as the next time tail (or path) is used
> + * is with a strcpy (thus overriding the '\0'). See +19 lines below.
> + */
> +}
> +
> +static int
> miss(NODE *p, char *tail)
> {
> int create;
> char *tp;
> const char *type, *what;
> - int serr;
> + int serr, rval = 0;
>
> gcooper> This isn't correct as per-style(9). Please do:
> gcooper>
> gcooper> int rval = 0;
> gcooper> int serr;
> gcooper>
> gcooper> This reduces diff churn and is more style(9)-istically correct.
I didn't know that. Good point.
> for (; p; p = p->next) {
> + if (uflag && eflag)
> + rval |= check(p, tail);
> if (p->flags & F_OPT && !(p->flags & F_VISIT))
> continue;
> if (p->type != F_DIR && (dflag || p->flags & F_VISIT))
> @@ -256,4 +294,5 @@
> (void)printf("%s: file flags not set: %s\n",
> path, strerror(errno));
> }
> + return (rval);
> }
--- mtree/verify.c 2010-05-27 13:47:20.0 +0200
+++ verify.c 2010-06-25 22:04:40.0 +0200
@@ -50,17 +50,25 @@
static NODE *root;
static char path[MAXPATHLEN];
-static void miss(NODE *, char *);
+static int miss(NODE *, char *);
+static int check(NODE *, char *);
static int vwalk(void);
int
mtree_verifyspec(FILE *fi)
{
- int rval;
+ int rval = 0;
root = mtree_readspec(fi);
-
Re: I need reply in Embedded FreeBSD Kernel Theme
hi sorry for being late in reply but I had some problems in the last week. I hope u still remeber what I was talking about :) @chargen Thanx for ur reply. Embedded computer systems permeate all aspects of our daily lives. Alarm clocks, coffee makers, digital watches, cell phones, and automobiles are just a few of the devices that make use of embedded systems. The design and development of such systems is unique, because the design constraints are different for each system. I supposed a new design for reducing the size of the kernel to be able to work in such an environment which has some constraints make it does not have the ability to get large memory, large storage devices or others. / as far as your document goes: "We will unload all the drivers that indicated with zeros in the module metadata file. That would make the OS to be a few of Megabytes." unload? what is the logic here? I'm sorry but what is the real gain here, // The configuration file is dependent on the kernel (which is included as a compiled kernel). If I unload these unneeded drivers, I decrease the size of the compiled kernel So it will have the ability to work in more constrained environment and that is suitable to work for Operating Systems for small chips for embedded systems. As the Internet grew, the requirements of embedded systems also began to grow in complexity. In addition to solving classic embedded systems problems, system designers were required to add connectivity for sending and receiving data or providing an automated method for software upgrades. Rather than increase the development effort, system designers have moved toward using third-party hardware and evaluating open source software. Thanx Chargen @ andrew Thanx for your reply. I appreciate your effort to download the document. Regarding the uploading of the document to that site, I am sorry for that but I tried to attach it with the email but it was refused because its size was too large. I did not send it in the text part because the text format and tables will be lost. I am sorry for that. // /// After a couple of quick readings, I'm not sure I correctly understand what you plan to achieve. It sounds like you are trying to achive something like hardware specific dynamic module loading (like in Solaris, for example), but then it also sounds like you are trying to build a kernel based on a generated config which somehow involves building a tiny binary hardware profile built from POST data. Are you compiling a kernel at some point rather than just generating a loader.conf for modules in order to minimise the running size? This approach will combine the two both. It will build a kernel based on a generated config which somehow involves building a tiny binary hardware profile built from POST data. It also will use hardware specific dynamic module loading because I don't have to load all the drivers for devices connected to the machine. I will use this dynamic module loading at the start-up only not at the run time because some considerations of the embedded systems for chips not being to load modules and change how to work at run time. that would make power problems. I was most confused by We will unload all the drivers that indicated with zeros in the > module metadata file. That would make the OS to be a > few of Megabytes. > Whether you are compiling a kernel for the (chosen) hardware based on a generated kernel or loader config, I don't see a sensible case in which you would ever need to "unload" any driver. /// yeah I know it is not sensible but I wrote it as a result of what I supposed before so it has no meaning. just a result to clarify what I reduced here. Thanx for ur sympathy and I will be glad to send you my next file to get ur comments about. As much fun as it is spending hours manually tweaking and testing a kernel config for each system and deciding what modules to use, I like the idea of a one time guided process based on accurately identified hardware to build an optimal kernel, so I hope that's what you're proposing. // Actually, I am deciding many time guided process based on accurately identified hardware to build an optimal kernel because I think this is more dynamic for considerations of changing the purpose of the embedded system. I mean sometimes u may need to perform deterministic operation in this boot so u do not have to load all drivers which there will be a lot of unneeded drivers of them at this boot process. I will determine the dependencies to provide optimal kernel. Thanx Andrew @ Matt Thanx for ur reply Matt.
Re: PCI Express and drivers
On 26/06/2010, at 3:01, Christopher Bowman wrote: > I have a Xilinx PCI Express board that has an on board PCIe interface > macro. I intend to have an address space with memory and another with my > devices control registers. I wish to program this board under FreeBSD. It > would seem to me that the way to do this would be to write a driver that > would allow me to map these two address spaces into a user process which > could then directly interact with the device. In my case my board doesn't > really support multiple users, and it doesn't really seem to make sense to > me to put a lot of code in the kernel to create some sort of interface to > allow multiple processes to interact with my device via some sort of syscall > which would impose a lot of overhead in any case. By mapping the address > ranges into a user process I can write user programs to drive the board > without having to recompile and reboot the kernel each time I make a change, > plus if I do something stupid and crash my user space process it shouldn't > bring down the whole kernel. Am I thinking about this wrong? Is there some > place I can go to read up on what I should be doing? If I am thinking about > this correctly, then how does one map device memory into a user space > process? How does one make sure that only one process has such a mapping? You could use mmap() I think, For a driver I maintain I did the following -> /* Magic sauce stolen from src/sys/pci/xrpu.c via phk * http://www.mail-archive.com/freebsd-hackers%40freebsd.org/msg11729.html */ *paddr = rman_get_start(sc->g_membase.reshandle) + offset; g_membase is.. struct resource *reshandle; /* Resource handle */ bus_space_tag_t sc_st; /* bus space tag */ bus_space_handle_t sc_sh; /* bus space handle */ PS what board are you using? :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
Re: I need reply in Embedded FreeBSD Kernel Theme
On 24 June 2010 11:06, Mohammed Farrag wrote: > @ Matt > Thanx for ur reply Matt. > / > FreeBSD is already a very modular system and the traditional way (a > traditional way) to build for embedded systems is to follow the > NanoBSD build method (tools included in the source tree) with a > stripped down kernel in which you only load the modules your hardware > requires using the FreeBSD loader (or after the initial boot). > > yeah I read about that. My mentor suggested that before and my idea is very > close to NanoBSD but I don't know if the freebsd loader will load the moduls > based on the hardware requirements or user requirements. I will be glad to > reply me. Modules are loaded by the user adding entries to /boot/loader.conf. e.g. if_sis_load="YES". That example will load the "sis" driver for the Silicon Image network interfaces on my Soekris net4801 board as I have removed almost everything in my kernel and just load the modules I require. Some modules will automatically load when another driver requires them. FreeBSD does not try to discover and reconfigure your hardware at boot time like the "kudzu" utility in Linux. Instead it will attach to the hardware for which you have drivers in your kernel or for which you have told the loader to load modules for. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
32 bit DRI on amd64 - possible bug?
Hi. I've had two people tell me that this is supposed to be working these days (8.0-RELEASE-p2) but I'm not having much luck. I have a 32 bit chroot, built with "make buildworld TARGET=i386" and have built a pile of ports in that jail (i386 libGL, i386 dri, etc, etc). I've tried running the chroot as both a jail and an ordinary chroot and in both cases, any program that tries to use DRI behaves erratically and usually (but not always) segfaults immediately: i386-jail$ LIBGL_DEBUG=verbose glxinfo name of display: 10.1.3.1:0.0 libGL: XF86DRIGetClientDriverName: 5.3.0 r300 (screen 0) libGL: OpenDriver: trying /usr/local/lib/dri/r300_dri.so drmOpenDevice: node name is /dev/dri/card0 drmOpenDevice: open result is 4, (OK) DRM_IOCTL_VERSION: Bad address Segmentation fault: 11 i386-jail$ LIBGL_DEBUG=verbose glxinfo name of display: 10.1.3.1:0.0 libGL: XF86DRIGetClientDriverName: 5.3.0 r300 (screen 0) libGL: OpenDriver: trying /usr/local/lib/dri/r300_dri.so drmOpenDevice: node name is /dev/dri/card0 drmOpenDevice: open result is 4, (OK) Segmentation fault: 11 i386-jail$ LIBGL_DEBUG=verbose glxinfo name of display: 10.1.3.1:0.0 libGL: XF86DRIGetClientDriverName: 5.3.0 r300 (screen 0) libGL: OpenDriver: trying /usr/local/lib/dri/r300_dri.so drmOpenDevice: node name is /dev/dri/card0 drmOpenDevice: open result is 4, (OK) drmOpenByBusid: Searching for BusID pci::06:00.0 drmOpenDevice: node name is /dev/dri/card0 drmOpenDevice: open result is 4, (OK) drmOpenByBusid: drmOpenMinor returns 4 drmOpenByBusid: drmGetBusid reports (null) drmOpenDevice: node name is /dev/dri/card1 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card2 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card3 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card4 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card5 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card6 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card7 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card8 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card9 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card10 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card11 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card12 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card13 drmOpenByBusid: drmOpenMinor returns -1003 drmOpenDevice: node name is /dev/dri/card14 drmOpenByBusid: drmOpenMinor returns -1003 libGL error: drmOpenOnce failed (Operation not permitted) libGL error: reverting to software direct rendering libGL: OpenDriver: trying /usr/local/lib/dri/swrast_dri.so display: 10.1.3.1:0 screen: 0 direct rendering: Yes Note the lack of a segfault in the third invocation. The glxinfo program runs to completion about one in every six invocations. Programs such as glxdemo sometimes manage to open a window and process a few events before crashing (it's pretty much over before anything's visible onscreen - less than half a second): i386-chroot$ glxdemo Segmentation fault: 11 (core dumped) i386-chroot$ glxdemo Segmentation fault: 11 (core dumped) i386-chroot$ glxdemo Resize event Resize event Redraw event Segmentation fault: 11 (core dumped) It doesn't work outside of the chroot either (just using LD_32_LIBRARY_PATH to allow the binaries to find their libraries): $ LD_32_LIBRARY_PATH=/jail/i386/usr/local/lib \ LIBGL_DEBUG=verbose \ LIBGL_DRIVERS_PATH=/jail/i386/usr/local/lib/dri \ /jail/i386/usr/local/bin/glxinfo 2>&1 name of display: :0.0 libGL: XF86DRIGetClientDriverName: 5.3.0 r300 (screen 0) libGL: OpenDriver: trying /jail/wine/usr/local/lib/dri/r300_dri.so drmOpenDevice: node name is /dev/dri/card0 drmOpenDevice: open result is 6, (OK) Segmentation fault (core dumped) A ktrace shows nothing particular interesting in any case. Sometimes an ioctl() will fail with EFAULT but usually, things just crash: 75860 initial thread RET stat 0 75860 initial thread CALL open(0xd0f4,O_RDWR,0) 75860 initial thread NAMI "/dev/dri/card0" 75860 initial thread RET open 6 75860 initial thread CALL write(0x2,0xcaf0,0x26) 75860 initial thread GIO fd 2 wrote 38 bytes "drmOpenDevice: open result is 6, (OK) " 75860 initial thread RET write 38/0x26 75860 initial thread CALL ioctl(0x6,0xc0246400 ,0x28414100) 75860 initial thread RET ioctl 0 75860 initial thread CALL ioctl(0x6,0xc0246400 ,0x28414100) 75860 initial thread RET ioctl 0 75860 initial thread PSIG SIGSEGV SIG_DFL 75860 initial thread NAMI "glxinfo.core" The "operation not permitted" error is confusing too as permissions on /dev/dri/card0 are fine: i386$ id uid=11001(m0) gid=11001(m0) groups=1100
