date:20161008

Re: Your buildworld system is pulling userland libraries/headers

2016-10-08 Thread Nathan Lay

Hello FreeBSD-stable,
Here's another example:

--- begin snip ---
...
/usr/src/usr.bin/nfsstat/nfsstat.c:262:22: error: use of undeclared
identifier 'NFSSTATS_V1'
ext_nfsstats.vers = NFSSTATS_V1;
^
/usr/src/usr.bin/nfsstat/nfsstat.c:272:27: error: incomplete definition of
type 'struct nfsstatsv1'
(uintmax_t)ext_nfsstats.rpccnt[NFSPROC_GETATTR],
   ^
/usr/src/usr.bin/nfsstat/nfsstat.c:107:15: note: forward declaration of
'struct nfsstatsv1'
static struct nfsstatsv1 ext_nfsstats;
  ^
/usr/src/usr.bin/nfsstat/nfsstat.c:273:27: error: incomplete definition of
type 'struct nfsstatsv1'
(uintmax_t)ext_nfsstats.rpccnt[NFSPROC_SETATTR],
   ^
/usr/src/usr.bin/nfsstat/nfsstat.c:107:15: note: forward declaration of
'struct nfsstatsv1'
static struct nfsstatsv1 ext_nfsstats;
  ^
/usr/src/usr.bin/nfsstat/nfsstat.c:274:27: error: incomplete definition of
type 'struct nfsstatsv1'
(uintmax_t)ext_nfsstats.rpccnt[NFSPROC_LOOKUP],

...
--- end snip ---

Now why is this? Your build system is including userland headers. Look it's
correct in /usr/src/sys/fs/nfs/nfsport.h
root@RADIO:/usr/src/lib/libprocstat # grep -r NFSSTATS_V1 /usr/src/sys/
/usr/src/sys/fs/nfs/nfsport.h: * The vers field will be set to NFSSTATS_V1
by the caller.
/usr/src/sys/fs/nfs/nfsport.h:#define   NFSSTATS_V1 1
/usr/src/sys/fs/nfs/nfs_commonport.c:   if (error == 0 &&
nfsstatver.vers != NFSSTATS_V1)
^C


But nfsstats is including /usr/include/fs/nfs/nfsports.h. This is poisoning
the build.

root@RADIO:/usr/src/lib/libprocstat # diff /usr/include/fs/nfs/nfsport.h
/usr/src/sys/fs/nfs/nfsport.h | less
...
> #define   NFSSTATS_V1 1
...

So I'll copy this and build nfsstat myself:
cp /usr/src/sys/fs/nfs/nfsport.h /usr/include/fs/nfs/nfsport.h

root@RADIO:/usr/src/usr.bin/nfsstat # make
clang -O2 -pipe -DNFS -march=nocona  -g -MD  -MF.depend.nfsstat.o
-MTnfsstat.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror
-Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes
-Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual
-Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align
-Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls
-Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations
-Wthread-safety -Wno-empty-body -Wno-string-plus-int
-Wno-unused-const-variable  -Qunused-arguments  -c
/usr/src/usr.bin/nfsstat/nfsstat.c -o nfsstat.o
clang -O2 -pipe -DNFS -march=nocona -g -std=gnu99 -fstack-protector-strong
-Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type
-Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter
-Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls
-Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations
-Wthread-safety -Wno-empty-body -Wno-string-plus-int
-Wno-unused-const-variable -Qunused-arguments  -o nfsstat.full nfsstat.o
-ldevstat
objcopy --only-keep-debug nfsstat.full nfsstat.debug
objcopy --strip-debug --add-gnu-debuglink=nfsstat.debug  nfsstat.full
nfsstat
gzip -cn /usr/src/usr.bin/nfsstat/nfsstat.1 > nfsstat.1.gz

This is very very disturbing. A build system that builds a self-contained
operating system should not be pulling in the existing userland's headers
and libraries.

Best regards,
Nathan Lay



On Sat, Oct 8, 2016 at 12:26 AM, Nathan Lay  wrote:

> Hello once again FreeBSD-stable,
> While upgrading from FreeBSD 10-STABLE and later updating my FreeBSD
> 11-STABLE system, I noticed that your build system is pulling headers and
> libraries from the userland rather than the /usr/src tree. This can't
> possibly be intended. I kind of figured with recent news of delayed
> released you might have realized this. After all, _FOR EXAMPLE_ (may not
> reflect reality), who wants to accidentally include userland's older
> version of OpenSSL headers and link with userland's older version of
> OpenSSL?
>
> I can cope with this by manually copying headers from /usr/src/sys/sys and
> /usr/src/include to their appropriate locations and/or installing the newer
> library manually and then restarting the build. But others may not be so
> familiar with FreeBSD. And I did manage to get FreeBSD 11-STABLE up and
> running just fine eventually.
>
> But just a few days ago, I updated SVN to rebuild world and kernel
> following the news of your delayed release and found libprocstat failing to
> compile. And it failed to compile because it was including
> /usr/include/ufs/ufs/extattr.h instead of /usr/src/sys/sys/ufs/ufs/extattr.h.
> This is very disturbing to me. During the FreeBSD 10 updating phase, I saw
> way more examples of this with even unresolved reference errors during
> linking since

Re: fix for use-after-free problem in 10.x

2016-10-08 Thread Oliver Pinter

On 10/5/16, Julian Elischer  wrote:
> In 11 and 12 the taskqueue code has been rewritten in this area but
> under 10 this bug still occurs.
>
> On our appliances this bug stops the system from mounting the ZFS
> root, so it is quite severe.
> Basically while the thread is sleeping during the ZFS mount of root
> (in the while loop), another thread can free the 'task' item it is
> checking in that while loop and it can be reused or filled with
> 'deadcode' etc., with the waiting code unaware of the change.. The fix
> is to refetch the item at the end of the queue each time around the loop.
> I don't really want to do the bigger change of MFCing the change in
> 11, as it is more extensive, though if someone else does, that's ok by
> me. (If it's ABI compatible)
>
> Any comments or suggestions?

Yes, please commit them. This patch fixes the ZFS + GELI + INVARIANTS
problem for us.
There is the FreeBSD PR about the issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209580

>
> here's the fix in diff form:
>
>
> [robot@porridge /usr/src]$ p4 diff -du ...
> --- //depot/pbranches/jelischer/FreeBSD-PZ/10.3/sys/kern/subr_taskqueue.c
>2016-09-27 09:14:59.0 -0700
> +++ /usr/src/sys/kern/subr_taskqueue.c  2016-09-27 09:14:59.0 -0700
> @@ -441,9 +441,10 @@
>
>  TQ_LOCK(queue);
>  task = STAILQ_LAST(>tq_queue, task, ta_link);
> -   if (task != NULL)
> -   while (task->ta_pending != 0)
> -   TQ_SLEEP(queue, task, >tq_mutex, PWAIT, "-",
> 0);
> +   while (task != NULL && task->ta_pending != 0) {
> +   TQ_SLEEP(queue, task, >tq_mutex, PWAIT, "-", 0);
> +   task = STAILQ_LAST(>tq_queue, task, ta_link);
> +   }
>  taskqueue_drain_running(queue);
>  KASSERT(STAILQ_EMPTY(>tq_queue),
>  ("taskqueue queue is not empty after draining"));
>
> ___
> freebsd-hack...@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
>
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: buildworld errors at outset on fresh svn checkout

2016-10-08 Thread Scott Bennett

 I wrote:

>"Matthew D. Fuller"  wrote:
>
>> On Fri, Oct 07, 2016 at 10:13:02PM -0700 I heard the voice of
>> Kevin Oberman, and lo! it spake thus:
>> > On Thu, Oct 6, 2016 at 10:16 PM, Scott Bennett  wrote:
>> > 
>> > > "/usr/src/Makefile.inc1", line 1113: Malformed conditional
>> > > (${BUILDKERNELS:[)
>> > > Unknown modifier '['
>> > 
>> > '[' needs to be a hard link to test.
>>
>> This is make, not sh.
>>
>> That would be the line
>>
>> .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes"
>>
>> in current stable/10, and it's horking on the [...] modifier.  That's
>
> What is currently installed is
>
>FreeBSD hellas 10.3-STABLE FreeBSD 10.3-STABLE #284 r304657: Tue Aug 23 
>01:48:12 CDT 2016 bennett@hellas:/usr/obj/usr/src/sys/hellas  amd64
>
>> listed in the make manpage on a stable/10 system from last October, so
>> it's not terribly new.  Maybe an ancient make?  Or your 'make' isn't
>> the make it's expecting?
>>
>Here are the ones I have currently installed.
>
>[hellas] 26 % ls -ilgF /usr/{,local/}bin/{,?}make
> 102327 -r-xr-xr-x  1 root  wheel   466396 May  3  2014 /usr/bin/bmake*
>  82321 -r-xr-xr-x  1 root  wheel   715152 Aug 23 05:17 /usr/bin/make*
>7391987 -r-xr-xr-x  1 root  wheel   187600 Aug 23 16:24 /usr/local/bin/bmake*
>7391732 -rwxr-xr-x  1 root  wheel  3805840 Sep 18 13:36 /usr/local/bin/cmake*
>7386658 -r-xr-xr-x  1 root  wheel   112976 Sep  6  2015 /usr/local/bin/dmake*
>7384146 -r-xr-xr-x  1 root  wheel   224520 Jul 10 18:45 /usr/local/bin/gmake*
>7384478 -r-xr-xr-x  1 root  wheel24664 May 18  2015 /usr/local/bin/imake*
>7386767 -r-xr-xr-x  1 root  wheel  2541440 Dec 14  2014 /usr/local/bin/qmake*
>7392225 -rwxr-xr-x  1 root  wheel   123256 Aug 31 11:03 /usr/local/bin/smake*
>7384593 -r-xr-xr-x  1 root  wheel32941 May 19  2015 /usr/local/bin/tmake*
>
> So the worrisome one looks like the /usr/bin/bmake, which pershaps should
>have been removed by "make delete-old" during an upgrade somewhere along the
>way.  Let me try renaming it to see what happens.  I'll report back.
>
 Okay.  I tried renaming /usr/bin/bmake to /usr/bin/bmake.old and then
ran "make buildworld".  That failed the same way as before. :-(  So I guess
that wasn't the problem.


  Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
**
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."   *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: buildworld errors at outset on fresh svn checkout

2016-10-08 Thread Scott Bennett

"Matthew D. Fuller"  wrote:

> On Fri, Oct 07, 2016 at 10:13:02PM -0700 I heard the voice of
> Kevin Oberman, and lo! it spake thus:
> > On Thu, Oct 6, 2016 at 10:16 PM, Scott Bennett  wrote:
> > 
> > > "/usr/src/Makefile.inc1", line 1113: Malformed conditional
> > > (${BUILDKERNELS:[)
> > > Unknown modifier '['
> > 
> > '[' needs to be a hard link to test.
>
> This is make, not sh.
>
> That would be the line
>
> .if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes"
>
> in current stable/10, and it's horking on the [...] modifier.  That's

 What is currently installed is

FreeBSD hellas 10.3-STABLE FreeBSD 10.3-STABLE #284 r304657: Tue Aug 23 
01:48:12 CDT 2016 bennett@hellas:/usr/obj/usr/src/sys/hellas  amd64

> listed in the make manpage on a stable/10 system from last October, so
> it's not terribly new.  Maybe an ancient make?  Or your 'make' isn't
> the make it's expecting?
>
Here are the ones I have currently installed.

[hellas] 26 % ls -ilgF /usr/{,local/}bin/{,?}make
 102327 -r-xr-xr-x  1 root  wheel   466396 May  3  2014 /usr/bin/bmake*
  82321 -r-xr-xr-x  1 root  wheel   715152 Aug 23 05:17 /usr/bin/make*
7391987 -r-xr-xr-x  1 root  wheel   187600 Aug 23 16:24 /usr/local/bin/bmake*
7391732 -rwxr-xr-x  1 root  wheel  3805840 Sep 18 13:36 /usr/local/bin/cmake*
7386658 -r-xr-xr-x  1 root  wheel   112976 Sep  6  2015 /usr/local/bin/dmake*
7384146 -r-xr-xr-x  1 root  wheel   224520 Jul 10 18:45 /usr/local/bin/gmake*
7384478 -r-xr-xr-x  1 root  wheel24664 May 18  2015 /usr/local/bin/imake*
7386767 -r-xr-xr-x  1 root  wheel  2541440 Dec 14  2014 /usr/local/bin/qmake*
7392225 -rwxr-xr-x  1 root  wheel   123256 Aug 31 11:03 /usr/local/bin/smake*
7384593 -r-xr-xr-x  1 root  wheel32941 May 19  2015 /usr/local/bin/tmake*

 So the worrisome one looks like the /usr/bin/bmake, which pershaps should
have been removed by "make delete-old" during an upgrade somewhere along the
way.  Let me try renaming it to see what happens.  I'll report back.

  Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
**
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."   *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: buildworld errors at outset on fresh svn checkout

2016-10-08 Thread Matthew D. Fuller

On Fri, Oct 07, 2016 at 10:13:02PM -0700 I heard the voice of
Kevin Oberman, and lo! it spake thus:
> On Thu, Oct 6, 2016 at 10:16 PM, Scott Bennett  wrote:
> 
> > "/usr/src/Makefile.inc1", line 1113: Malformed conditional
> > (${BUILDKERNELS:[)
> > Unknown modifier '['
> 
> '[' needs to be a hard link to test.

This is make, not sh.

That would be the line

.if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes"

in current stable/10, and it's horking on the [...] modifier.  That's
listed in the make manpage on a stable/10 system from last October, so
it's not terribly new.  Maybe an ancient make?  Or your 'make' isn't
the make it's expecting?


-- 
Matthew Fuller (MF4839)   |  fulle...@over-yonder.net
Systems/Network Administrator |  http://www.over-yonder.net/~fullermd/
   On the Internet, nobody can hear you scream.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: buildworld errors at outset on fresh svn checkout

2016-10-08 Thread Scott Bennett

Kevin Oberman  wrote:

 Thanks much for replying!

> On Thu, Oct 6, 2016 at 10:16 PM, Scott Bennett  wrote:
>
> >  I'm running into a problem in updating my 10-STABLE system from
> > source.
> > A "make buildworld" quits immediately.  I tried a fresh svn checkout for
> > base/stable/10 and then tried to run buildworld again, but got the same
> > error.
> > I've been scratching my head over this for hours, but must be missing
> > something
> > simple.
> >  I have ccache installed and have been using it for a fairly long time
> > now.
> > My /etc/src.conf contains just two lines:
> >
> > PORTS_MODULES=multimedia/cuse4bsd-kmod sysutils/pefs-kmod #
> > emulators/virtualbox-ose-kmod
> > WITH_LLDB=yes
> >
> > My /etc/make.conf is rather longer, so I'll append it following .sig below.
> >
> >  Here's what happens.
> >
> > Script started on Thu Oct  6 23:31:47 2016
> > hellas# cd /usr/src
> > hellas# nice make buildworld
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1113: Malformed conditional
> > (${BUILDKERNELS:[)
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1122: if-less endif
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1144: Malformed conditional
> > (${BUILDKERNELS:[)
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1161: if-less endif
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1183: Malformed conditional
> > (${BUILDKERNELS:[)
> > Unknown modifier '['
> >
> > "/usr/src/Makefile.inc1", line 1190: if-less endif
> > bmake: fatal errors encountered -- cannot continue
> > *** Error code 1
> >
> > Stop.
> > make: stopped in /usr/src
> > hellas# exit
> > exit
> >
> > Script done on Thu Oct  6 23:37:00 2016
> >
> >  This just started happening after my machine had been down for a
> > couple
> > of days after a hang that damaged stuff in /usr/home.  I had already
> > restored
> > /usr/local from backups before narrowing down the weird behavior I was
> > seeing
> > in wmaker to /usr/home corruption.  So /usr/home has now been restored to
> > good condition, too, but perhaps I need to restore something else as well.
> > This mess was part of my justification to myself for the fresh checkout of
> > /usr/src, but that doesn't seem to have made any difference in the
> > buildworld
> > failure.
> >  If anyone else can see what's wrong and clue me in, I'd be grateful.
> > I'm subscribed to the digest for this list, so please Cc: me directly, so
> > I'll get replies right away.
> > Thanks in advance!
> >
> >  [stuff deleted --SB]
>
> Could something else have gotten corrupted?
>
> > ls -i /bin/[
> 65795 /bin/[
> > ls -i /bin/test
> 65795 /bin/test
>
> The values (inode) will not match these, but must be identical.

[hellas] 23 % ls -lgiF /bin/\[ /bin/test
131557 -r-xr-xr-x  2 root  wheel  11664 Aug 23 05:16 /bin/[*
131557 -r-xr-xr-x  2 root  wheel  11664 Aug 23 05:16 /bin/test*

So no, that's not the problem.
>
> '[' needs to be a hard link to test. I'm suspicious that something happened
> to this link. If this is the case, other corruption may have occurred, but,
> if you can re-create the hardlink (ln /bin/test /bin/[) and successfully
> make buildworld  and make buildkernel, it's likely that you can reinstall
> the system and the system will be fine. Of course, things could be damaged
> in the installed ports, too.

 Unfortunately, no, I can't re-install, or at least not without restoring
/usr (and probably / and /usr/obj) to before the damage, assuming I can figure
out exactly between which backups the damage occurred, which may not have been
at the time of the hang.  I already had a recently built /usr/obj and tried to
install its contents.  The failure of installkernel was how I discovered the
problem.
 So I'm still stuck.


  Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
**
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."   *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Your buildworld system is pulling userland libraries/headers

Re: fix for use-after-free problem in 10.x

Re: buildworld errors at outset on fresh svn checkout

Re: buildworld errors at outset on fresh svn checkout

Re: buildworld errors at outset on fresh svn checkout

Re: buildworld errors at outset on fresh svn checkout

6 matches

Site Navigation

Mail list logo

Footer information