On Sun, Jul 01, 2012 at 08:33:09PM +0200, Andreas Tobler wrote: > On 01.07.12 19:05, Konstantin Belousov wrote: > >On Sun, Jul 01, 2012 at 03:56:52PM +0200, Andreas Tobler wrote: > >>On 01.07.12 15:41, Konstantin Belousov wrote: > >>>On Sun, Jul 01, 2012 at 03:37:18PM +0200, Andreas Tobler wrote: > >>>>On 01.07.12 14:04, Konstantin Belousov wrote: > >>>>>On Sun, Jul 01, 2012 at 01:23:02PM +0200, Andreas Tobler wrote: > >>>>>>On 21.06.12 11:26, Konstantin Belousov wrote: > >>>>>>>Author: kib > >>>>>>>Date: Thu Jun 21 09:26:06 2012 > >>>>>>>New Revision: 237367 > >>>>>>>URL: http://svn.freebsd.org/changeset/base/237367 > >>>>>>> > >>>>>>>Log: > >>>>>>> Enable deadlock avoidance code for NFS client. > >>>>>> > >>>>>> > >>>>>>Hm, since this commit I fail with my nfs installworld/kernel. > >>>>>> > >>>>>>I have a builder which installs world/kernel to a nfs mounted > >>>>>>directory. > >>>>>>Namely used for cross builds. > >>>>>> > >>>>>>Now since this commit I get the following when I install kernel to the > >>>>>>nfs directory: > >>>>>> > >>>>>>.. > >>>>>>install -o root -g wheel -m 555 zfs.ko.symbols > >>>>>>/netboot/sparc64/boot/kernel > >>>>>>install: /netboot/sparc64/boot/kernel/zfs.ko.symbols: No such file or > >>>>>>directory > >>>>>>*** [_kmodinstall] Error code 71 > >>>>>>.. > >>>>>> > >>>>>>The file is there, a local install of the tree works without problems. > >>>>>>Reverting to r237366 also makes it work again. > >>>>>> > >>>>>>The server is a -CURRENT, r237880, The client, -CURRENT too. > >>>>>> > >>>>>>How can I help to track down the real issue? > >>>>> > >>>>>Is it always the same file in the install procedure which causes the > >>>>>failure ? Even more, is the failure pattern always the same ? > >>>> > >>>>I'd say so yes. When installing a kernel onto a nfs mounted fs then > >>>>always (in my cases) the zfs.ko.symbols was the failing pattern. > >>>>I tried ppc64 and sparc64 as target. With both it was the above file. > >>>> > >>>>When doing a installworld, it was, also in both cases, ppc64/sparc64, > >>>>the cc1 in libexec which failed. > >>>> > >>>>>Might be, start with ktrace-ing the whole make invocation, including > >>>>>the children processes. > >>>> > >>>>Some recipes how to start? > >>>ktrace -o <file on local fs> -i make installkernel > >>>Then kdump and cut the lines around relevant failure. > >> > >>ktrace -f, right? > >Right, but without -i it is useless. > > Ah, yes, seems clear now after reading the man page. > > >>I placed the whole kdump here: > >> > >>http://people.freebsd.org/~andreast/dumped_installkernel.log > >> > >>It is not clear to me where the failure starts :) > >Because logs do not contain tracepoints from the children. > >See above about -i. > > > >I asked about excerpt because I expect the proper log to have an order > >of magnitude bigger size. > > Ok. The dump is around 100MB, I hope I extracted as much as needed: > > http://people.freebsd.org/~andreast/dumped_installkernel-7.log > > >>>>>I used buildworld on the NFS-mounted obj/ as the test for the changes. > >>>> > >>>>Here the obj is local, only the src and the destination is on the > >>>>nfs/netboot server. > >>> > >>>I just finished build on NFS obj/ and did several rounds of installs > >>>for world and kernel into nfs-mounted destdir. It seems I cannot > >>>reproduce > >>>this locally. > >> > >>Ok. I try with an nfs obj too. > > So, I was not able to reproduce the failure with an nfs mounted obj dir. > > But I was able to reproduce the failure with three different machines > which all have the obj local and the destination mounted via nfs. > > Are you able to try with a local obj too? Below are two patches. Please follow my instructions literally to get most of your bug report.
First, please apply the usr.bin/xinstall patch only, and retry installkernel (no need to use ktrace). It should show the proper error, short write, with zero-sized result, instead of garbage ENOENT from errno. Next, please apply the sys/fs/nfsclient patch, which should fix the core cause. diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 71286e3..f7af6fb 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -897,7 +897,7 @@ ncl_write(struct vop_write_args *ap) struct nfsmount *nmp = VFSTONFS(vp->v_mount); daddr_t lbn; int bcount; - int bp_cached, n, on, error = 0; + int bp_cached, n, on, error = 0, error1; size_t orig_resid, local_resid; off_t orig_size, tmp_off; @@ -1259,9 +1259,12 @@ again: if ((ioflag & IO_SYNC)) { if (ioflag & IO_INVAL) bp->b_flags |= B_NOCACHE; - error = bwrite(bp); - if (error) + error1 = bwrite(bp); + if (error1 != 0) { + if (error == 0) + error = error1; break; + } } else if ((n + on) == biosize) { bp->b_flags |= B_ASYNC; (void) ncl_writebp(bp, 0, NULL); diff --git a/usr.bin/xinstall/xinstall.c b/usr.bin/xinstall/xinstall.c index a920f85..3eba4f7 100644 --- a/usr.bin/xinstall/xinstall.c +++ b/usr.bin/xinstall/xinstall.c @@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$"); #include <errno.h> #include <fcntl.h> #include <grp.h> +#include <inttypes.h> #include <paths.h> #include <pwd.h> #include <stdio.h> @@ -671,11 +672,18 @@ copy(int from_fd, const char *from_name, int to_fd, const char *to_name, if (size <= 8 * 1048576 && trymmap(from_fd) && (p = mmap(NULL, (size_t)size, PROT_READ, MAP_SHARED, from_fd, (off_t)0)) != (char *)MAP_FAILED) { - if ((nw = write(to_fd, p, size)) != size) { + nw = write(to_fd, p, size); + if (nw != size) { serrno = errno; (void)unlink(to_name); - errno = nw > 0 ? EIO : serrno; - err(EX_OSERR, "%s", to_name); + if (nw >= 0) { + errx(EX_OSERR, + "short write to %s: %jd bytes written, %jd bytes asked to write", + to_name, (uintmax_t)nw, (uintmax_t)size); + } else { + errno = serrno; + err(EX_OSERR, "%s", to_name); + } } done_copy = 1; } @@ -684,8 +692,15 @@ copy(int from_fd, const char *from_name, int to_fd, const char *to_name, if ((nw = write(to_fd, buf, nr)) != nr) { serrno = errno; (void)unlink(to_name); - errno = nw > 0 ? EIO : serrno; - err(EX_OSERR, "%s", to_name); + if (nw >= 0) { + errx(EX_OSERR, + "short write to %s: %jd bytes written, %jd bytes asked to write", + to_name, (uintmax_t)nw, + (uintmax_t)size); + } else { + errno = serrno; + err(EX_OSERR, "%s", to_name); + } } if (nr != 0) { serrno = errno;
pgp6GkPitjxY6.pgp
Description: PGP signature