Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 07:00:36PM -0700, Conrad Meyer wrote:
> Larry,
> 
> You just need to run netdumpd on the nearby server.  It could be a
> port (although I'm not aware that it is ported yet).

I was working on a port makefile for netdumpd before this thread
started. :)

It would be a bit more usable as a port; I prefer it that way since
Isilon's infrastructure folks would like to run netdumpd on 11.0.
For what it's worth, I've got my revision of the code here:
https://github.com/markjdb/netdumpd

This version fixes some performance problems in the original, and runs
most of the code in capability mode. The capsicum integration hasn't
yet been reviewed, however.

> 
> Best,
> Conrad
> 
> On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> > On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> > wrote:
> >
> >> Yeah, I have the following:
> > > borg.lerctr.org /home/ler $ swapctl -l
> > > Device:   1024-blocks Used:
> > > /dev/mfid0p38388608 0
> > > /dev/mfid1p38388608 0
> > > /dev/mfid2p38388608 0
> > > /dev/mfid3p38388608 0
> > > /dev/mfid4p38388608 0
> > > /dev/mfid5p38388608 0
> > > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > > hw.physmem: 137368682496
> > > borg.lerctr.org /home/ler $
> > >
> > > SO 6 8G partitions (48G), but the dump is larger than 8G.
> >
> > Larry,
> >   This is a very good concern and point given todays more
> > common huge memory foot prints and lots of spindles.  I'll
> > keep this in they back of my mind as I tromp around in the
> > dump code.  I have another solution that may work for you
> > and that is to use Netdump rather than swapdump.  This
> > basically eliminates the trip to swap space and you end
> > up going to savecore style output on the netdump server.
> >
> > --
> > Rod Grimes 
> > rgri...@freebsd.org
> >
> > What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> > since that’s what is “next to” this server?
> >
> >
> >
> 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 06:30:17PM -0700, Rodney W. Grimes wrote:
> > The patch to add compression support is here and should largely still
> > work:
> > https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
> > 
> > I've been hesitant about pushing it forward:
> > - The dump_write* APIs need some simplification after the addition of
> >   encrypted dump support and support for dumping to 4Kn drives.
> > - I'm not sure how encryption should compose with compression. It seems
> >   intuitively obvious that we should compress before encrypting if the
> >   compression is to be of any use, but I don't know enough to know
> >   whether the compression might somehow compromise the effectiveness of
> >   the encryption.
> > 
> > If anyone has some insight on the second of these two points, I'd
> > appreciate hearing it.
> 
> I have a large amount of reworking and modulization of the dump code
> incuding intergration of your (markj) compressed dumps.  Layer isnt
> implemented but is in the plan.  I should not of held off on the net
> dump code as it got smashed by encrypted dumps, then again by
> the libif'ing for all the Intel drives that had been netdump modified.
> 
> Basically now starting over :-(

Could you post your patches somewhere? I've been sitting on this (and
the netdump patches, for which I have quite a few modifications) for far
too long, and would like to finish them and get them in soon. I'll note
that the netdump code posted a few years ago had some problems that are
fixed in Isilon's version, which I'm working on rebasing on HEAD. In
particular, I simplified the driver integration a bit, changed the code
to avoid allocating mbufs from UMA after a panic, plumbed a
configuration interface through dumpon, and fixed some problems with
netdumpd. I'm working on integrating netdump with Isilon's internal
infrastructure at the moment. The conversion of em and igb to iflib does
complicate things a bit; I haven't yet looked at how hard it would be
to support netdump in iflib'ed drivers.

> Minidump is an lkm for me, and main dump is almost an lkm for me too.

Does "lkm" mean "loadable kernel module"? If so, why?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> 
> > On Apr 14, 2017, at 20:05, Rodney W. Grimes 
> >  wrote:
> > 
> >>> 
> >>> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >>>  wrote:
> >>> 
>  On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> > savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> > 
> > - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >   representable data to INT_MAX. Check the values received from
> >   strtoll(3), trimming trailing whitespace off the end to maintain
> >   POLA.
> > - Use `KiB` instead of `kB` when describing free space, total space,
> >   etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >   bytes and `kB` being the IEC standard for 1000 bytes.
>  
>  I will just rant lightly that no one actually uses this in the real 
>  world.
>  
>  Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>  kilobyte is a power of 2.  The End.
>  
>  (Next up we'll have to rename 4k displays to
>  4k)
> >>> 
> >>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> >>> a place we do this, so please, lets not start doing this here?
> >> 
> >> humanize_number(3) from libutil uses IEC units.
> > 
> > And how many things bother to use this library function?  Do the
> > ones that do call it produce the traditional output that has been
> > around for 40 years?
> > 
> >>> Yes, these are newer standards, perhaps some day we should make a global
> >>> switch to them, but lets not start mixing and matching things.
> >> 
> >> I understand and agree. I?m not 100% sold on that one way or another, but
> >> since I was going to redo the number representation in save core with
> >> humanize_number(3), because reading `KiB` is not ideal
> >^^^
> > I hope we are not already reading KiB anyplace?.
> 
> I meant it?s a lot harder for humans to read `KiB`
> instead of ``.

And I am agreeing on the scaled, and disagreeing on the KiB, another words
please revert to printing KB as the former code did as far as a Unit
goes.  Lets not use Ki Mi Gi, etc.

> >> usability wise, and I don?t want to reinvent the wheel normalizing numbers
> >> and printing out the unit.
> >> 
> >> Perhaps there should be a flag baked into humanize_number, etc for parsing 
> >> IEC vs non-IEC unit values?
> > 
> > I dont think it parses anything, but as far as producing strings from values
> > it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
> > and if we are using it anyplace lets see if we can remove that use.
> 
> I don?t see it used anywhere in the tree, based on a quick grep.

YEA!!!  Thanks for doing the search.

> > Also be careful, this function only accepts signed int 64, which means
> > we are not gona be able to use this in all places that probably need
> > this, so perhaps a larger can of paint for a bigger bike shead is needed?
> 
> I don?t necessarily follow the above statement 100%. Are you warning
> against mass-conversion to libutil (if so, I agree? this was just a case
> where it really helps readability in savecore(8))?

Yes, I am warning against mass conversion, or for that that matter much
conversion to humanize_number because it shall need to be revamped if
it is ever to become a real solutions due to it being a very narrow
minded function (no unsigned implementation for very large values
that are probably the ones that could benifit the most from this
type of function.) 

You also may want to still report full sizes in output (you maybe
I have not read the final code:)
>From kernel boot:
real memory  = 134217728 (128 MB)
avail memory = 120832000 (115 MB)

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316952 - head/share/mk

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 03:28:13 2017
New Revision: 316952
URL: https://svnweb.freebsd.org/changeset/base/316952

Log:
  Latest version, we need to realpath anyway to check if an obj
  dependency is also a src dependency.

Modified:
  head/share/mk/meta2deps.py

Modified: head/share/mk/meta2deps.py
==
--- head/share/mk/meta2deps.py  Sat Apr 15 02:24:22 2017(r316951)
+++ head/share/mk/meta2deps.py  Sat Apr 15 03:28:13 2017(r316952)
@@ -492,6 +492,21 @@ class MetaFile:
 if not file:
 f.close()
 
+def is_src(self, base, dir, rdir):
+"""is base in srctop"""
+for dir in [dir,rdir]:
+if not dir:
+continue
+path = '/'.join([dir,base])
+srctop = self.find_top(path, self.srctops)
+if srctop:
+if self.dpdeps:
+self.add(self.file_deps, path.replace(srctop,''), 'file')
+self.add(self.src_deps, dir.replace(srctop,''), 'src')
+self.seenit(dir)
+return True
+return False
+
 def parse_path(self, path, cwd, op=None, w=[]):
 """look at a path for the op specified"""
 
@@ -520,10 +535,9 @@ class MetaFile:
 # to the src dir, we may need to add dependencies for each
 rdir = dir
 dir = abspath(dir, cwd, self.last_dir, self.debug, self.debug_out)
-if rdir == dir or rdir.find('./') > 0:
-rdir = None
-if os.path.islink(dir):
 rdir = os.path.realpath(dir)
+if rdir == dir:
+rdir = None
 # now put path back together
 path = '/'.join([dir,base])
 if self.debug > 1:
@@ -544,17 +558,9 @@ class MetaFile:
 # finally, we get down to it
 if dir == self.cwd or dir == self.curdir:
 return
-srctop = self.find_top(path, self.srctops)
-if srctop:
-if self.dpdeps:
-self.add(self.file_deps, path.replace(srctop,''), 'file')
-self.add(self.src_deps, dir.replace(srctop,''), 'src')
+if self.is_src(base, dir, rdir):
 self.seenit(w[2])
-self.seenit(dir)
-if rdir and not rdir.startswith(srctop):
-dir = rdir  # for below
-rdir = None
-else:
+if not rdir:
 return
 
 objroot = None
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> 
> > On Apr 14, 2017, at 19:40, Ngie Cooper (yaneurabeya) 
> >  wrote:
> > 
> >> 
> >> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >>  wrote:
> >> 
> >>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>  Author: ngie
>  Date: Fri Apr 14 19:41:48 2017
>  New Revision: 316938
>  URL: https://svnweb.freebsd.org/changeset/base/316938
>  
>  Log:
>  savecore: fix space calculation with respect to `minfree` in 
>  check_space(..)
>  
>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>    representable data to INT_MAX. Check the values received from
>    strtoll(3), trimming trailing whitespace off the end to maintain
>    POLA.
>  - Use `KiB` instead of `kB` when describing free space, total space,
>    etc. I am now fully aware of `KiB` being the IEC standard for 1024
>    bytes and `kB` being the IEC standard for 1000 bytes.
> >>> 
> >>> I will just rant lightly that no one actually uses this in the real world.
> >>> 
> >>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> >>> kilobyte is a power of 2.  The End.
> >>> 
> >>> (Next up we'll have to rename 4k displays to
> >>> 4k)
> >> 
> >> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> >> a place we do this, so please, lets not start doing this here?
> > 
> > humanize_number(3) from libutil uses IEC units.
> > 
> >> Yes, these are newer standards, perhaps some day we should make a global
> >> switch to them, but lets not start mixing and matching things.
> > 
> > I understand and agree. I?m not 100% sold on that one way or another, but 
> > since I was going to redo the number representation in save core with 
> > humanize_number(3), because reading `KiB` is not ideal 
> > usability wise, and I don?t want to reinvent the wheel normalizing numbers 
> > and printing out the unit.
> 
> *unit. ?> *unit, KiB seemed like a logical next step after discussing it at 
> long length in the CR.

I do not see any long length discussion in the CR. I see an assertion that
KiB is the new IEC standard, I see a miss understanding of what kB and KB
means, and I see someone wanting to push the tree towards IEC.

I see here on the mailling list a push to NOT start IEC'ing things, at
least by myself and jhb.

I would not consider 2 or 3 peole have a 15 line converstaion in a CR
as reasonable grounds for a change like this.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 20:12, Peter Wemm  wrote:
> 
> On Friday, April 14, 2017 07:36:55 PM Peter Wemm wrote:
>> On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
 On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
 
 Peter, Ngie,
 
 Looks like out of that refactoring came a logical bug that is present in
 the head, which causes syslod to first to shutdown the socket for
 reading
 and then try to select/recv on it (which is somewhat stupid). And that
 issue has been masked by shutdown() on datagram socket becoming
 effectively a NOP in 11 & head 20 months ago. It only affects head
 though, 11-stable still has the old code which does not include that
 half-closed socket into the select list. Attached patch is expected to
 fix head, Peter, it would be nice if you can give it a try (restoring
 latest changes into uipc_sockets.c) and let me know if it helps.
 
 Thanks!
>>> 
>>> CCing hrs@ for input as he did the refactoring.
>>> Thanks!
>>> -Ngie
>>> 
>>> PS LGTM with the change. Will wait for feedback from wemm@.
>> 
>> This is definitely not working.  I get ENOSPC  and listen queue overflows on
>> /var/run/logpriv now.
>> 
>> Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x
>> one worked fine, aside from the include statements.
> 
> This can't be right:
> if (SecureMode || res->ai_family == AF_LOCAL) {
>/* Forbid communication in secure mode. */
>if (shutdown(s, SHUT_RD) < 0 &&
>errno != ENOTCONN) {
>logerror("shutdown");
>if (!Debug)
>die(0);
>}
>dprintf("listening on socket\n");
>sl_recv = NULL;
>   }
> 
> This appears to disable unix domain sockets like /var/run/log and
> /var/run/logpriv.

ACK. This looks like a fun bug.

-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 07:36:55 PM Peter Wemm wrote:
> On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
> > > On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> > > 
> > > Peter, Ngie,
> > > 
> > > Looks like out of that refactoring came a logical bug that is present in
> > > the head, which causes syslod to first to shutdown the socket for
> > > reading
> > > and then try to select/recv on it (which is somewhat stupid). And that
> > > issue has been masked by shutdown() on datagram socket becoming
> > > effectively a NOP in 11 & head 20 months ago. It only affects head
> > > though, 11-stable still has the old code which does not include that
> > > half-closed socket into the select list. Attached patch is expected to
> > > fix head, Peter, it would be nice if you can give it a try (restoring
> > > latest changes into uipc_sockets.c) and let me know if it helps.
> > > 
> > > Thanks!
> > 
> > CCing hrs@ for input as he did the refactoring.
> > Thanks!
> > -Ngie
> > 
> > PS LGTM with the change. Will wait for feedback from wemm@.
> 
> This is definitely not working.  I get ENOSPC  and listen queue overflows on
> /var/run/logpriv now.
> 
> Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x
> one worked fine, aside from the include statements.

This can't be right:
 if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
if (shutdown(s, SHUT_RD) < 0 &&
errno != ENOTCONN) {
logerror("shutdown");
if (!Debug)
die(0);
}
dprintf("listening on socket\n");
sl_recv = NULL;
   }

This appears to disable unix domain sockets like /var/run/log and 
/var/run/logpriv.

--
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 20:05, Rodney W. Grimes  
> wrote:
> 
>>> 
>>> On Apr 14, 2017, at 18:49, Rodney W. Grimes 
>>>  wrote:
>>> 
 On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>   representable data to INT_MAX. Check the values received from
>   strtoll(3), trimming trailing whitespace off the end to maintain
>   POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>   etc. I am now fully aware of `KiB` being the IEC standard for 1024
>   bytes and `kB` being the IEC standard for 1000 bytes.
 
 I will just rant lightly that no one actually uses this in the real world.
 
 Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
 kilobyte is a power of 2.  The End.
 
 (Next up we'll have to rename 4k displays to
 4k)
>>> 
>>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
>>> a place we do this, so please, lets not start doing this here?
>> 
>> humanize_number(3) from libutil uses IEC units.
> 
> And how many things bother to use this library function?  Do the
> ones that do call it produce the traditional output that has been
> around for 40 years?
> 
>>> Yes, these are newer standards, perhaps some day we should make a global
>>> switch to them, but lets not start mixing and matching things.
>> 
>> I understand and agree. I?m not 100% sold on that one way or another, but
>> since I was going to redo the number representation in save core with
>> humanize_number(3), because reading `KiB` is not ideal
>^^^
> I hope we are not already reading KiB anyplace….

I meant it’s a lot harder for humans to read `KiB` instead of 
``.

>> usability wise, and I don?t want to reinvent the wheel normalizing numbers
>> and printing out the unit.
>> 
>> Perhaps there should be a flag baked into humanize_number, etc for parsing 
>> IEC vs non-IEC unit values?
> 
> I dont think it parses anything, but as far as producing strings from values
> it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
> and if we are using it anyplace lets see if we can remove that use.

I don’t see it used anywhere in the tree, based on a quick grep.

> Also be careful, this function only accepts signed int 64, which means
> we are not gona be able to use this in all places that probably need
> this, so perhaps a larger can of paint for a bigger bike shead is needed?

I don’t necessarily follow the above statement 100%. Are you warning against 
mass-conversion to libutil (if so, I agree… this was just a case where it 
really helps readability in savecore(8))?

Thanks!
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> 
> > On Apr 14, 2017, at 18:49, Rodney W. Grimes 
> >  wrote:
> > 
> >> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> >>> Author: ngie
> >>> Date: Fri Apr 14 19:41:48 2017
> >>> New Revision: 316938
> >>> URL: https://svnweb.freebsd.org/changeset/base/316938
> >>> 
> >>> Log:
> >>>  savecore: fix space calculation with respect to `minfree` in 
> >>> check_space(..)
> >>> 
> >>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >>>representable data to INT_MAX. Check the values received from
> >>>strtoll(3), trimming trailing whitespace off the end to maintain
> >>>POLA.
> >>>  - Use `KiB` instead of `kB` when describing free space, total space,
> >>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >>>bytes and `kB` being the IEC standard for 1000 bytes.
> >> 
> >> I will just rant lightly that no one actually uses this in the real world.
> >> 
> >> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> >> kilobyte is a power of 2.  The End.
> >> 
> >> (Next up we'll have to rename 4k displays to
> >> 4k)
> > 
> > Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> > a place we do this, so please, lets not start doing this here?
> 
> humanize_number(3) from libutil uses IEC units.

And how many things bother to use this library function?  Do the
ones that do call it produce the traditional output that has been
around for 40 years?

> > Yes, these are newer standards, perhaps some day we should make a global
> > switch to them, but lets not start mixing and matching things.
> 
> I understand and agree. I?m not 100% sold on that one way or another, but
> since I was going to redo the number representation in save core with
> humanize_number(3), because reading `KiB` is not ideal
^^^
I hope we are not already reading KiB anyplace

> usability wise, and I don?t want to reinvent the wheel normalizing numbers
> and printing out the unit.
> 
> Perhaps there should be a flag baked into humanize_number, etc for parsing 
> IEC vs non-IEC unit values?

I dont think it parses anything, but as far as producing strings from values
it already has an IEC flag: HN_IEC_PREFIXES, please lets not use this flag,
and if we are using it anyplace lets see if we can remove that use.

Also be careful, this function only accepts signed int 64, which means
we are not gona be able to use this in all places that probably need
this, so perhaps a larger can of paint for a bigger bike shead is needed?

> Thanks for the input :)!
input++;  

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 19:40, Ngie Cooper (yaneurabeya)  
> wrote:
> 
>> 
>> On Apr 14, 2017, at 18:49, Rodney W. Grimes  
>> wrote:
>> 
>>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
 check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
>>> 
>>> I will just rant lightly that no one actually uses this in the real world.
>>> 
>>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>>> kilobyte is a power of 2.  The End.
>>> 
>>> (Next up we'll have to rename 4k displays to
>>> 4k)
>> 
>> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
>> a place we do this, so please, lets not start doing this here?
> 
> humanize_number(3) from libutil uses IEC units.
> 
>> Yes, these are newer standards, perhaps some day we should make a global
>> switch to them, but lets not start mixing and matching things.
> 
> I understand and agree. I’m not 100% sold on that one way or another, but 
> since I was going to redo the number representation in save core with 
> humanize_number(3), because reading `KiB` is not ideal 
> usability wise, and I don’t want to reinvent the wheel normalizing numbers 
> and printing out the unit.

*unit. —> *unit, KiB seemed like a logical next step after discussing it at 
long length in the CR.

-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 18:49, Rodney W. Grimes  
> wrote:
> 
>> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>> 
>> I will just rant lightly that no one actually uses this in the real world.
>> 
>> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
>> kilobyte is a power of 2.  The End.
>> 
>> (Next up we'll have to rename 4k displays to
>> 4k)
> 
> Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
> a place we do this, so please, lets not start doing this here?

humanize_number(3) from libutil uses IEC units.

> Yes, these are newer standards, perhaps some day we should make a global
> switch to them, but lets not start mixing and matching things.

I understand and agree. I’m not 100% sold on that one way or another, but since 
I was going to redo the number representation in save core with 
humanize_number(3), because reading `KiB` is not ideal 
usability wise, and I don’t want to reinvent the wheel normalizing numbers and 
printing out the unit.

Perhaps there should be a flag baked into humanize_number, etc for parsing IEC 
vs non-IEC unit values?

Thanks for the input :)!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:14:16 PM Ngie Cooper wrote:
> > On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> > 
> > Peter, Ngie,
> > 
> > Looks like out of that refactoring came a logical bug that is present in
> > the head, which causes syslod to first to shutdown the socket for reading
> > and then try to select/recv on it (which is somewhat stupid). And that
> > issue has been masked by shutdown() on datagram socket becoming
> > effectively a NOP in 11 & head 20 months ago. It only affects head
> > though, 11-stable still has the old code which does not include that
> > half-closed socket into the select list. Attached patch is expected to
> > fix head, Peter, it would be nice if you can give it a try (restoring
> > latest changes into uipc_sockets.c) and let me know if it helps.
> > 
> > Thanks!
> 
> CCing hrs@ for input as he did the refactoring.
> Thanks!
> -Ngie
> 
> PS LGTM with the change. Will wait for feedback from wemm@.

This is definitely not working.  I get ENOSPC  and listen queue overflows on 
/var/run/logpriv now.

Grabbing an old 10.3 /usr/sbin/syslogd and placing it on the top of the 12.x 
one worked fine, aside from the include statements.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:21:05 PM Maxim Sobolev wrote:
> Peter, It is actually the other way around. If you take syslogd code out of
> 11-stable and earlier that would work just fine with or without r316874.
> But since r285910 syslogd in head had been refactored a lot and I think
> that particular bug has introduced that has been masked by the shutdown()
> on datagram sockets becoming a NOP after r285910. Then r316874 restored our
> historical behavior for the shutdown(2) and bingo, bug in the new syslogd
> code is now causing it to spin when shutdown() != NOP.
> 
> -Max

Hmm, there's a new problem:

 45104 auditd   CALL  socket(PF_LOCAL,0x1002,0)
 45104 auditd   RET   socket 3
 45104 auditd   CALL  connect(0x3,0x7fffdbd8,0x6a)
 45104 auditd   STRU  struct sockaddr { AF_LOCAL, /var/run/logpriv }
 45104 auditd   NAMI  "/var/run/logpriv"
 45104 auditd   RET   connect 0
 45104 auditd   CALL  sendto(0x3,0x7fffe130,0x2f,0,0,0)
 45104 auditd   RET   sendto -1 errno 55 No buffer space available
 45104 auditd   CALL  open(0x800da5c67,0x15)
 45104 auditd   NAMI  "/dev/console"
 45104 auditd   RET   open 4
.. and it all goes to /dev/console instead.

On restarting syslogd:
Apr 15 02:17:43 repoman2 syslogd: exiting on signal 15
sonewconn: pcb 0xf80051e72680: Listen queue overflow: 16 already in queue 
already

Umm.. did the patch forget to listen to incoming connections or something?

I haven't seen this before anywhere except when syslogd is wedged.


> 
> On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > > Thanks, Peter. I will try to look into this asap.
> > 
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > whether
> > the -s code even works at all since the 2015 / r285910 change...
> > 
> > > -Max
> > > 
> > > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > > Author: sobomax
> > > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > > New Revision: 316874
> > > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > > > 
> > > > > > Log:
> > > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> > 
> > ENOTCONN to
> > 
> > > > be
> > > > 
> > > > > > returned by the shutdown(2) system call. This ability has been
> > 
> > lost as
> > 
> > > > > > part
> > > > > > of the svn revision 285910.
> > > > > > 
> > > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > > >   MFC after:2 weeks
> > > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > > > 
> > > > > This appears to have broken syslogd and had a major change in
> > 
> > behavior
> > 
> > > > with
> > > > 
> > > > > regards to select(2).
> > > > > 
> > > > > If you run syslogd with the -s flag (which is default), it now spins
> > 
> > at
> > 
> > > > 100%
> > > > 
> > > > > cpu as all the shutdown sockets now return readable from select.
> > > > > 
> > > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > > losing
> > > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > > instead, but we depend on this in the cluster and I expect others do
> > > > > too.
> > > > > 
> > > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > > can't
> > > > > hurt. I'll try reverting it on the freebsd cluster next, after
> > > > > fixing
> > > > > the
> > > > > broken auditing changes.
> > > > > 
> > > > > -Peter
> > > > 
> > > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > > compatability
> > > > with old branches.
> > > > 
> > > > --
> > > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > > KI6FJV
> > > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
> > 
> > --
> > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > KI6FJV
> > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Alan Somers]

On Fri, Apr 14, 2017 at 7:55 PM, Rodney W. Grimes
 wrote:
> [ Charset UTF-8 unsupported, converting... ]
>> Author: asomers
>> Date: Fri Apr 14 22:59:14 2017
>> New Revision: 316945
>> URL: https://svnweb.freebsd.org/changeset/base/316945
>>
>> Log:
>>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>>
>>   PR: 176049
>>   Submitted by:   docon...@gsoft.com.au
>>   Reviewed by:scottl, Larry Rosenman 
>>   MFC after:  3 weeks
>>   Relnotes:   yes
>>
>> Added:
>>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
>> Modified:
>>   head/etc/defaults/periodic.conf
>>   head/etc/periodic/daily/Makefile
>>   head/share/man/man5/periodic.conf.5
>>

>> Modified: head/etc/periodic/daily/Makefile
>> ==
>> --- head/etc/periodic/daily/Makefile  Fri Apr 14 22:02:08 2017
>> (r316944)
>> +++ head/etc/periodic/daily/Makefile  Fri Apr 14 22:59:14 2017
>> (r316945)
>> @@ -13,6 +13,7 @@ FILES=  100.clean-disks \
>>   400.status-disks \
>>   401.status-graid \
>>   406.status-gmirror \
>> + 410.status-mfi \
>>   407.status-graid3 \
>>   408.status-gstripe \
>>   409.status-gconcat \
>
> Insered into the middle of a number sorted list?

Mea culpa.  Already fixed in r316946.



> Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316951 - head/usr.sbin/syslogd

[Maxim Sobolev]

Author: sobomax
Date: Sat Apr 15 02:24:22 2017
New Revision: 316951
URL: https://svnweb.freebsd.org/changeset/base/316951

Log:
  Since shutdown(2) on datagram socket is no longer a NOP after rev 316874
  don't bother to select/recv on that socket. This prevents syslogd(8)
  from spinning endlessly when started with the -s option (default).
  
  Reported by:  peter

Modified:
  head/usr.sbin/syslogd/syslogd.c

Modified: head/usr.sbin/syslogd/syslogd.c
==
--- head/usr.sbin/syslogd/syslogd.c Sat Apr 15 01:52:49 2017
(r316950)
+++ head/usr.sbin/syslogd/syslogd.c Sat Apr 15 02:24:22 2017
(r316951)
@@ -702,7 +702,7 @@ main(int argc, char *argv[])
sizeof(fd_mask));
 
STAILQ_FOREACH(sl, , next) {
-   if (sl->sl_socket != -1)
+   if (sl->sl_socket != -1 && sl->sl_recv != NULL)
FD_SET(sl->sl_socket, fdsr);
}
i = select(fdsrmax + 1, fdsr, NULL, NULL,
@@ -2877,6 +2877,7 @@ socksetup(struct peer *pe)
struct addrinfo hints, *res, *res0;
int error;
char *cp;
+   int (*sl_recv)(struct socklist *);
/*
 * We have to handle this case for backwards compatibility:
 * If there are two (or more) colons but no '[' and ']',
@@ -3003,6 +3004,7 @@ socksetup(struct peer *pe)
}
dprintf("new socket fd is %d\n", s);
listen(s, 5);
+   sl_recv = socklist_recv_sock;
dprintf("shutdown\n");
if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
@@ -3013,13 +3015,14 @@ socksetup(struct peer *pe)
die(0);
}
dprintf("listening on socket\n");
+   sl_recv = NULL;
} else
dprintf("sending on socket\n");
addsock(res->ai_addr, res->ai_addrlen,
&(struct socklist){
.sl_socket = s,
.sl_peer = pe,
-   .sl_recv = socklist_recv_sock
+   .sl_recv = sl_recv
});
}
freeaddrinfo(res0);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

That has not actually been committed yet.  Mark Johnston has a patch for it.

On Fri, Apr 14, 2017 at 7:17 PM, Larry Rosenman  wrote:
>
> On 4/14/17, 9:16 PM, "Conrad Meyer"  wrote:
>
> On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> > It looked like the netdump code might need some stuff for current 
> –CURRENT.
>
> Not for the server (netdumpd) — it's just a simple UDP daemon that
> writes incoming UDP streams to disk.
>
> Best,
> Conrad
>
>
> I was referring to the code to actually dump to netdumpd.
>
>
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 9:16 PM, "Conrad Meyer"  wrote:

On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> It looked like the netdump code might need some stuff for current 
–CURRENT.

Not for the server (netdumpd) — it's just a simple UDP daemon that
writes incoming UDP streams to disk.

Best,
Conrad


I was referring to the code to actually dump to netdumpd.



___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

On Fri, Apr 14, 2017 at 7:02 PM, Larry Rosenman  wrote:
> It looked like the netdump code might need some stuff for current –CURRENT.

Not for the server (netdumpd) — it's just a simple UDP daemon that
writes incoming UDP streams to disk.

Best,
Conrad
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> > On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> > wrote:
> > 
> > > Yeah, I have the following:
> > > borg.lerctr.org /home/ler $ swapctl -l
> > > Device:   1024-blocks Used:
> > > /dev/mfid0p38388608 0
> > > /dev/mfid1p38388608 0
> > > /dev/mfid2p38388608 0
> > > /dev/mfid3p38388608 0
> > > /dev/mfid4p38388608 0
> > > /dev/mfid5p38388608 0
> > > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > > hw.physmem: 137368682496
> > > borg.lerctr.org /home/ler $
> > > 
> > > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> > 
> > Larry,
> >   This is a very good concern and point given todays more
> > common huge memory foot prints and lots of spindles.  I'll
> > keep this in they back of my mind as I tromp around in the
> > dump code.  I have another solution that may work for you
> > and that is to use Netdump rather than swapdump.  This
> > basically eliminates the trip to swap space and you end
> > up going to savecore style output on the netdump server.
> > 
> > -- 
> > Rod Grimes 
> > rgri...@freebsd.org
> > 
> > What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> > since that?s what is ?next to? this server?  
> 
> As a netdump server FreeNAS would be fairly trivial to compile the netdump
> client code for.I should have running 11.0 and 10.2 code around
> some place.

You know, maybe we should get the client side code pushed in, that
has been stable and not stomped on by other changes and is usable
for those of us who have kernels that can do netdump. 

It is a fairly simple daemon that listens on a socket and stores
the dumps.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

It looked like the netdump code might need some stuff for current –CURRENT. 


-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@lerctr.org
US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
 
 

On 4/14/17, 9:00 PM, "Conrad Meyer"  wrote:

Larry,

You just need to run netdumpd on the nearby server.  It could be a
port (although I'm not aware that it is ported yet).

Best,
Conrad

On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
wrote:
>
>> Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> >
> > SO 6 8G partitions (48G), but the dump is larger than 8G.
>
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
>
> --
> Rod Grimes 
rgri...@freebsd.org
>
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
since that’s what is “next to” this server?
>
>
>



___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 8:59 PM, "Rodney W. Grimes"  wrote:

> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
wrote:
> 
> > Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> > 
> > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> 
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
> 
> -- 
> Rod Grimes 
rgri...@freebsd.org
> 
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
since that?s what is ?next to? this server?  

As a netdump server FreeNAS would be fairly trivial to compile the netdump
client code for.I should have running 11.0 and 10.2 code around
some place.

-- 
Rod Grimes 
rgri...@freebsd.org

I’m also running:
borg.lerctr.org /home/ler $ uname -a
FreeBSD borg.lerctr.org 12.0-CURRENT FreeBSD 12.0-CURRENT #23 r316945: Fri Apr 
14 18:37:13 CDT 2017 r...@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64
borg.lerctr.org /home/ler $




___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

Larry,

You just need to run netdumpd on the nearby server.  It could be a
port (although I'm not aware that it is ported yet).

Best,
Conrad

On Fri, Apr 14, 2017 at 6:37 PM, Larry Rosenman  wrote:
> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> wrote:
>
>> Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> >
> > SO 6 8G partitions (48G), but the dump is larger than 8G.
>
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
>
> --
> Rod Grimes 
> rgri...@freebsd.org
>
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> since that’s what is “next to” this server?
>
>
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On 4/14/17, 8:35 PM, "Rodney W. Grimes"  
> wrote:
> 
> > Yeah, I have the following:
> > borg.lerctr.org /home/ler $ swapctl -l
> > Device:   1024-blocks Used:
> > /dev/mfid0p38388608 0
> > /dev/mfid1p38388608 0
> > /dev/mfid2p38388608 0
> > /dev/mfid3p38388608 0
> > /dev/mfid4p38388608 0
> > /dev/mfid5p38388608 0
> > borg.lerctr.org /home/ler $ sysctl hw.physmem
> > hw.physmem: 137368682496
> > borg.lerctr.org /home/ler $
> > 
> > SO 6 8G partitions (48G), but the dump is larger than 8G. 
> 
> Larry,
>   This is a very good concern and point given todays more
> common huge memory foot prints and lots of spindles.  I'll
> keep this in they back of my mind as I tromp around in the
> dump code.  I have another solution that may work for you
> and that is to use Netdump rather than swapdump.  This
> basically eliminates the trip to swap space and you end
> up going to savecore style output on the netdump server.
> 
> -- 
> Rod Grimes 
> rgri...@freebsd.org
> 
> What does it take for NetDump to work to a FreeNAS (9.10 nightly) server 
> since that?s what is ?next to? this server?  

As a netdump server FreeNAS would be fairly trivial to compile the netdump
client code for.I should have running 11.0 and 10.2 code around
some place.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> Author: asomers
> Date: Fri Apr 14 22:59:14 2017
> New Revision: 316945
> URL: https://svnweb.freebsd.org/changeset/base/316945
> 
> Log:
>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>   
>   PR: 176049
>   Submitted by:   docon...@gsoft.com.au
>   Reviewed by:scottl, Larry Rosenman 
>   MFC after:  3 weeks
>   Relnotes:   yes
> 
> Added:
>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
> Modified:
>   head/etc/defaults/periodic.conf
>   head/etc/periodic/daily/Makefile
>   head/share/man/man5/periodic.conf.5
> 
> Modified: head/etc/defaults/periodic.conf
> ==
> --- head/etc/defaults/periodic.conf   Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/defaults/periodic.conf   Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"  # Che
>  # 409.status-gconcat
>  daily_status_gconcat_enable="NO" # Check gconcat(8)
>  
> +# 410.status-mfi
> +daily_status_mfi_enable="NO" # Check mfiutil(8)
> +
>  # 420.status-network
>  daily_status_network_enable="YES"# Check network status
>  daily_status_network_usedns="YES"# DNS lookups are ok
> 
> Added: head/etc/periodic/daily/410.status-mfi
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/etc/periodic/daily/410.status-mfiFri Apr 14 22:59:14 2017
> (r316945)
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# $FreeBSD$
> +#
> +
> +# If there is a global system configuration file, suck it in.
> +#
> +if [ -r /etc/defaults/periodic.conf ]
> +then
> +. /etc/defaults/periodic.conf
> +source_periodic_confs
> +fi
> +
> +case "$daily_status_mfi_enable" in
> +[Yy][Ee][Ss])
> +   echo
> +   echo 'Checking status of mfi(4) devices:'
> +
> +   if mfiutil show volumes; then
> +   if mfiutil show volumes | grep -q DEGRADED; then
> +   rc=3
> +   else
> +   rc=0
> +   fi
> +   else
> +   rc=2
> +   fi
> +   ;;
> +
> +*)  rc=0;;
> +esac
> +
> +exit $rc
> 
> Modified: head/etc/periodic/daily/Makefile
> ==
> --- head/etc/periodic/daily/Makefile  Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/periodic/daily/Makefile  Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -13,6 +13,7 @@ FILES=  100.clean-disks \
>   400.status-disks \
>   401.status-graid \
>   406.status-gmirror \
> + 410.status-mfi \
>   407.status-graid3 \
>   408.status-gstripe \
>   409.status-gconcat \

Insered into the middle of a number sorted list?

> 
> Modified: head/share/man/man5/periodic.conf.5
> ==
> --- head/share/man/man5/periodic.conf.5   Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/share/man/man5/periodic.conf.5   Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -399,6 +399,15 @@ if you want to run
>  on your
>  .Xr gconcat 8
>  devices.
> +.It Va daily_status_mfi_enable
> +.Pq Vt bool
> +Set to
> +.Dq Li YES
> +if you want to run
> +.Nm mfiutil Cm status
> +on your
> +.Xr mfi 4
> +devices.
>  .It Va daily_status_network_enable
>  .Pq Vt bool
>  Set to
> 
> 

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316950 - head/share/mk

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 01:52:49 2017
New Revision: 316950
URL: https://svnweb.freebsd.org/changeset/base/316950

Log:
  Update to latest, handle case of __objdir=obj

Modified:
  head/share/mk/auto.obj.mk

Modified: head/share/mk/auto.obj.mk
==
--- head/share/mk/auto.obj.mk   Sat Apr 15 01:44:45 2017(r316949)
+++ head/share/mk/auto.obj.mk   Sat Apr 15 01:52:49 2017(r316950)
@@ -58,7 +58,10 @@ __objdir_made != echo ${__objdir}/; umas
 # This causes make to use the specified directory as .OBJDIR
 .OBJDIR: ${__objdir}
 .if ${.OBJDIR:tA} != ${__objdir:tA} && ${__objdir_made:Uno:M${__objdir}/*} != 
""
+# watch out for __objdir being relative path
+.if !(${__objdir:M/*} == "" && ${.OBJDIR:tA} == ${${.CURDIR}/${__objdir}:L:tA})
 .error could not use ${__objdir}: .OBJDIR=${.OBJDIR}
 .endif
 .endif
 .endif
+.endif
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >   
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> 
> I will just rant lightly that no one actually uses this in the real world.
> 
> Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
> kilobyte is a power of 2.  The End.
> 
> (Next up we'll have to rename 4k displays to
> 4k)

Do we use KiB, MiB, GiB,... any place else in the system?  I cant think of
a place we do this, so please, lets not start doing this here?

Yes, these are newer standards, perhaps some day we should make a global
switch to them, but lets not start mixing and matching things.

> -- 
> John Baldwin
-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316949 - in head: contrib/bmake contrib/bmake/mk contrib/bmake/unit-tests usr.bin/bmake

[Simon J. Gerraty]

Author: sjg
Date: Sat Apr 15 01:44:45 2017
New Revision: 316949
URL: https://svnweb.freebsd.org/changeset/base/316949

Log:
  Update to bmake 20170413

Modified:
  head/contrib/bmake/ChangeLog
  head/contrib/bmake/Makefile
  head/contrib/bmake/job.c
  head/contrib/bmake/main.c
  head/contrib/bmake/mk/ChangeLog
  head/contrib/bmake/mk/auto.obj.mk
  head/contrib/bmake/mk/dirdeps.mk
  head/contrib/bmake/mk/install-mk
  head/contrib/bmake/mk/meta.stage.mk
  head/contrib/bmake/mk/meta2deps.py
  head/contrib/bmake/mk/mkopt.sh
  head/contrib/bmake/str.c
  head/contrib/bmake/unit-tests/modmatch.exp
  head/contrib/bmake/unit-tests/modmatch.mk
  head/usr.bin/bmake/Makefile
Directory Properties:
  head/contrib/bmake/   (props changed)

Modified: head/contrib/bmake/ChangeLog
==
--- head/contrib/bmake/ChangeLogSat Apr 15 00:51:35 2017
(r316948)
+++ head/contrib/bmake/ChangeLogSat Apr 15 01:44:45 2017
(r316949)
@@ -1,8 +1,35 @@
+2017-04-13  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170413
+ Merge with NetBSD make, pick up
+ o main.c: when setting .OBJDIR ignore '$' in paths.
+
+   * job.c: use MALLOC_OPTIONS to set malloc_options.
+
+2017-04-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170411
+ Merge with NetBSD make, pick up
+ o str.c: Str_Match: allow [^a-z] to behave as expected.
+
+2017-03-26  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170326
+ Merge with NetBSD make, pick up
+ o main.c: purge relative paths from realpath cache when .OBJDIR
+   is changed.
+
+2017-03-11  Simon J. Gerraty  
+
+   * Makefile (_MAKE_VERSION): 20170311
+ Merge with NetBSD make, pick up
+  o main.c: only use -C arg "as is" if it starts with '/'.
+
 2017-03-01  Simon J. Gerraty  
 
* Makefile (_MAKE_VERSION): 20170301
  Merge with NetBSD make, pick up
- o main.c: use -C arg as is rather than getcwd()
+ o main.c: use -C arg "as is" rather than getcwd()
if they identify the same directory.
  o parse.c: ensure loadfile buffer is \n terminated in non-mmap case
 

Modified: head/contrib/bmake/Makefile
==
--- head/contrib/bmake/Makefile Sat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/Makefile Sat Apr 15 01:44:45 2017(r316949)
@@ -1,7 +1,7 @@
-#  $Id: Makefile,v 1.81 2017/03/01 17:01:23 sjg Exp $
+#  $Id: Makefile,v 1.85 2017/04/13 16:29:40 sjg Exp $
 
 # Base version on src date
-_MAKE_VERSION= 20170301
+_MAKE_VERSION= 20170413
 
 PROG=  bmake
 

Modified: head/contrib/bmake/job.c
==
--- head/contrib/bmake/job.cSat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/job.cSat Apr 15 01:44:45 2017(r316949)
@@ -389,7 +389,10 @@ static void JobSigLock(sigset_t *);
 static void JobSigUnlock(sigset_t *);
 static void JobSigReset(void);
 
-const char *malloc_options="A";
+#if !defined(MALLOC_OPTIONS)
+# define MALLOC_OPTIONS "A"
+#endif
+const char *malloc_options= MALLOC_OPTIONS;
 
 static void
 job_table_dump(const char *where)

Modified: head/contrib/bmake/main.c
==
--- head/contrib/bmake/main.c   Sat Apr 15 00:51:35 2017(r316948)
+++ head/contrib/bmake/main.c   Sat Apr 15 01:44:45 2017(r316949)
@@ -1,4 +1,4 @@
-/* $NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $  */
+/* $NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $  */
 
 /*
  * Copyright (c) 1988, 1989, 1990, 1993
@@ -69,7 +69,7 @@
  */
 
 #ifndef MAKE_NATIVE
-static char rcsid[] = "$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos 
Exp $";
+static char rcsid[] = "$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos 
Exp $";
 #else
 #include 
 #ifndef lint
@@ -81,7 +81,7 @@ __COPYRIGHT("@(#) Copyright (c) 1988, 19
 #if 0
 static char sccsid[] = "@(#)main.c 8.3 (Berkeley) 3/19/94";
 #else
-__RCSID("$NetBSD: main.c,v 1.257 2017/02/08 17:47:36 christos Exp $");
+__RCSID("$NetBSD: main.c,v 1.260 2017/04/13 13:55:23 christos Exp $");
 #endif
 #endif /* not lint */
 #endif
@@ -722,22 +722,15 @@ Boolean
 Main_SetObjdir(const char *fmt, ...)
 {
struct stat sb;
-   char *p, *path;
-   char buf[MAXPATHLEN + 1], pbuf[MAXPATHLEN + 1];
+   char *path;
+   char buf[MAXPATHLEN + 1];
Boolean rc = FALSE;
va_list ap;
 
va_start(ap, fmt);
-   vsnprintf(path = pbuf, MAXPATHLEN, fmt, ap);
+   vsnprintf(path = buf, MAXPATHLEN, fmt, ap);
va_end(ap);
 
-   /* expand variable substitutions */
-   if (strchr(path, '$') 

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:
> 
> > 
> > > On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
> > 
> > ?
> > 
> > > No, something like:
> > > % swapinfo
> > > Device  1K-blocks UsedAvail Capacity
> > > /dev/ada0p2  335544320 33554432 0%
> > > /dev/ada1p2  335544320 33554432 0%
> > > Total671088640 67108864 0%
> > 
> > Striping dumps across multiple swap devices isn?t supported. Sorry.
> 
> What about support in the future?

Added to my project list at people.freebsd.org/~rgrimes.  No promises,
but I'll look at it while I am doing the other work.

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 8:35 PM, "Rodney W. Grimes"  wrote:

> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G. 

Larry,
  This is a very good concern and point given todays more
common huge memory foot prints and lots of spindles.  I'll
keep this in they back of my mind as I tromp around in the
dump code.  I have another solution that may work for you
and that is to use Netdump rather than swapdump.  This
basically eliminates the trip to swap space and you end
up going to savecore style output on the netdump server.

-- 
Rod Grimes 
rgri...@freebsd.org

What does it take for NetDump to work to a FreeNAS (9.10 nightly) server since 
that’s what is “next to” this server?  


___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
> 
> > On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> > 
> > On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> > 
> >> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> >> 
> >> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> >> 
> >>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> >>> 
> >>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> >>> 
>  On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  
> wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> > 
> > Log:
> > savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> > 
> > - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> >  representable data to INT_MAX. Check the values received from
> >  strtoll(3), trimming trailing whitespace off the end to maintain
> >  POLA.
> > - Use `KiB` instead of `kB` when describing free space, total space,
> >  etc. I am now fully aware of `KiB` being the IEC standard for 1024
> >  bytes and `kB` being the IEC standard for 1000 bytes.
> > - Store available number of KiB in `available` so it can be more
> >  easily queried and compared to ensure that there are enough KiB to
> >  store the dump image on disk.
> > - Print out the reserved space on disk, per `minfree`, so end-users
> >  can troubleshoot why check_space(..) is reporting that there isn't
> >  enough free space.
> > 
> > MFC after:7 weeks
> > Reviewed by:  Anton Rang  (earlier diff), cem 
> (earlier diff)
> > Tested with:  positive/negative cases (see review); make tinderbox
> > Sponsored by: Dell EMC Isilon
> > Differential Revision:D10379
>  
>  The free space calculation is still uselessly conservative, because 
> it
>  doesn't account for the fact that core dumps will always be either
>  spare or compressed.  The result is that savecore will frequently
>  refuse to save corefiles even when there's plenty of space.  I
>  proposed removing the space check altogether in
>  https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>  after the compressed core dump feature was merged, because then 
> mostly
>  accurate space checks will be possible.  AFAIK the compressed core
>  dump feature still hasn't been finished.
> >>> 
> >>> Is posible (in the future) to use multiple swaps (on multiple disks)
> >>> for save core dumps?
> >> 
> >>   Multiple swap devices is already handled by savecore(8), if one uses 
> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> devices.
> >> 
> >>   As far as saving to multiple disks is concerned, I would hope that 
> one is using a redundancy capable filesystem (zfs) or RAID-like technology 
> (gmirror, graid, LSI Fusion?s RAID product line) to stripe and/or mirror the 
> data across multiple disks.
> > 
> >?
> > 
> >> How do I use multiple devices to have the system dump on all of my 
> swap?  I got a message about not enough space, but there (I think) was enough 
> between multiple drives?.
> > 
> >Something like:
> > 
> >- Create a zpool
> >- Mount zpool to /crashdumps
> >- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ?dumpdir=/crashdumps?
> > 
> >?
> >HTH,
> >-Ngie
> > 
> >PS The issue with lack of space might be the issue that Alan brought 
> up earlier with compressed dumps and overly conservative free space checks, 
> or it might be the fact that dumpdir (default: /var/crash) is full.
> > 
> > 
> > I was talking about the actual crashdump to swap by the system.  
> /var/crash has 10T of space (my root pool).
> 
> If your memory is bigger than your swap, you?re unfortunately not able to 
> save the mini dump if the size of the saved pages exceed the space on swap. I 
> think this is where markj?s compressed dumps feature will come in handy.
> Thanks!
> -Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 

Re: svn commit: r316938 - head/sbin/savecore

[Rodney W. Grimes]

> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> > On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > > Author: ngie
> > > Date: Fri Apr 14 19:41:48 2017
> > > New Revision: 316938
> > > URL: https://svnweb.freebsd.org/changeset/base/316938
> > >
> > > Log:
> > >   savecore: fix space calculation with respect to `minfree` in 
> > > check_space(..)
> > >
> > >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > > representable data to INT_MAX. Check the values received from
> > > strtoll(3), trimming trailing whitespace off the end to maintain
> > > POLA.
> > >   - Use `KiB` instead of `kB` when describing free space, total space,
> > > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > > bytes and `kB` being the IEC standard for 1000 bytes.
> > >   - Store available number of KiB in `available` so it can be more
> > > easily queried and compared to ensure that there are enough KiB to
> > > store the dump image on disk.
> > >   - Print out the reserved space on disk, per `minfree`, so end-users
> > > can troubleshoot why check_space(..) is reporting that there isn't
> > > enough free space.
> > >
> > >   MFC after:7 weeks
> > >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
> > > diff)
> > >   Tested with:  positive/negative cases (see review); make tinderbox
> > >   Sponsored by: Dell EMC Isilon
> > >   Differential Revision:D10379
> > 
> > The free space calculation is still uselessly conservative, because it
> > doesn't account for the fact that core dumps will always be either
> > spare or compressed.  The result is that savecore will frequently
> > refuse to save corefiles even when there's plenty of space.  I
> > proposed removing the space check altogether in
> > https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> > after the compressed core dump feature was merged, because then mostly
> > accurate space checks will be possible.  AFAIK the compressed core
> > dump feature still hasn't been finished.
> 
> I had held off on it for a while because it was going to conflict with
> the work to add encrypted dump support, which of course has finished.
> 
> The patch to add compression support is here and should largely still
> work:
> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
> 
> I've been hesitant about pushing it forward:
> - The dump_write* APIs need some simplification after the addition of
>   encrypted dump support and support for dumping to 4Kn drives.
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.
> 
> If anyone has some insight on the second of these two points, I'd
> appreciate hearing it.

I have a large amount of reworking and modulization of the dump code
incuding intergration of your (markj) compressed dumps.  Layer isnt
implemented but is in the plan.  I should not of held off on the net
dump code as it got smashed by encrypted dumps, then again by
the libif'ing for all the Intel drives that had been netdump modified.

Basically now starting over :-(

Minidump is an lkm for me, and main dump is almost an lkm for me too.
-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316936 - head/sys/dev/cxgbe/iw_cxgbe

[Rodney W. Grimes]

[ Charset UTF-8 unsupported, converting... ]
> Author: np
> Date: Fri Apr 14 19:15:31 2017
> New Revision: 316936
> URL: https://svnweb.freebsd.org/changeset/base/316936
> 
> Log:
>   cxgbe/iw_cxgbe: hw supports 64K (not 32K) Protection Domains.
>   
>   MFC after:  3 days
>   Sponsored by:   Chelsio Communications
> 
> Modified:
>   head/sys/dev/cxgbe/iw_cxgbe/t4.h
> 
> Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
> ==
> --- head/sys/dev/cxgbe/iw_cxgbe/t4.h  Fri Apr 14 18:56:00 2017
> (r316935)
> +++ head/sys/dev/cxgbe/iw_cxgbe/t4.h  Fri Apr 14 19:15:31 2017
> (r316936)
> @@ -61,7 +61,7 @@
>  
>  #define T4_MAX_NUM_QP (1<<16)
>  #define T4_MAX_NUM_CQ (1<<15)
> -#define T4_MAX_NUM_PD (1<<15)
> +#define T4_MAX_NUM_PD 65536

Why the change in methods of expressing powers of 2 here?
This, imho, would better match the near by code as
#define T4_MAX_NUM_PD (1<<16)

>  #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
>  #define T4_MAX_EQ_SIZE (65520 - T4_EQ_STATUS_ENTRIES)
>  #define T4_MAX_IQ_SIZE (65520 - 1)
> 
> 

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316946 - head/etc/periodic/daily

[Alan Somers]

Author: asomers
Date: Sat Apr 15 00:39:45 2017
New Revision: 316946
URL: https://svnweb.freebsd.org/changeset/base/316946

Log:
  Reorder Makefile entries from r316945
  
  PR:   176049
  Reported by:  Oliver Pinter
  MFC after:3 weeks
  X-MFC-With:   316945

Modified:
  head/etc/periodic/daily/Makefile

Modified: head/etc/periodic/daily/Makefile
==
--- head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
(r316945)
+++ head/etc/periodic/daily/MakefileSat Apr 15 00:39:45 2017
(r316946)
@@ -13,10 +13,10 @@ FILES=  100.clean-disks \
400.status-disks \
401.status-graid \
406.status-gmirror \
-   410.status-mfi \
407.status-graid3 \
408.status-gstripe \
409.status-gconcat \
+   410.status-mfi \
420.status-network \
430.status-uptime \
450.status-security \
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:10:56 PM Maxim Sobolev wrote:
> Peter, Ngie,
> 
> Looks like out of that refactoring came a logical bug that is present in
> the head, which causes syslod to first to shutdown the socket for reading
> and then try to select/recv on it (which is somewhat stupid). And that
> issue has been masked by shutdown() on datagram socket becoming effectively
> a NOP in 11 & head 20 months ago. It only affects head though, 11-stable
> still has the old code which does not include that half-closed socket into
> the select list. Attached patch is expected to fix head, Peter, it would be
> nice if you can give it a try (restoring latest changes into
> uipc_sockets.c) and let me know if it helps.

Confirmed - resting uipc_socket.c to HEAD and applying the patch to syslogd 
does solve the problem we encountered.  Thanks!

> Thanks!
> 
> On Fri, Apr 14, 2017 at 12:48 PM, Ngie Cooper (yaneurabeya) <
> 
> yaneurab...@gmail.com> wrote:
> > > On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> > > 
> > > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > >> Thanks, Peter. I will try to look into this asap.
> > > 
> > > I don't understand what is going on yet. Presumably there must be other
> > > changes in play that affect udp/select sometime between the original
> > > 2015
> > > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > 
> > whether
> > 
> > > the -s code even works at all since the 2015 / r285910 change...
> > 
> > syslogd has been refactored a lot on ^/head. I don’t think it’s safe to
> > say that the ^/head and ^/stable/11 and earlier copies will function the
> > same.
> > Thanks,
> > -Ngie

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Oliver Pinter]

On Saturday, April 15, 2017, Alan Somers  wrote:

> Author: asomers
> Date: Fri Apr 14 22:59:14 2017
> New Revision: 316945
> URL: https://svnweb.freebsd.org/changeset/base/316945
>
> Log:
>   Add 410.status-mfi, a periodic script for mfi(4) arrays
>
>   PR:   176049
>   Submitted by: docon...@gsoft.com.au 
>   Reviewed by:  scottl, Larry Rosenman >
>   MFC after:3 weeks
>   Relnotes: yes
>
> Added:
>   head/etc/periodic/daily/410.status-mfi   (contents, props changed)
> Modified:
>   head/etc/defaults/periodic.conf
>   head/etc/periodic/daily/Makefile
>   head/share/man/man5/periodic.conf.5
>
> Modified: head/etc/defaults/periodic.conf
> 
> ==
> --- head/etc/defaults/periodic.conf Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/defaults/periodic.conf Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"#
> Che
>  # 409.status-gconcat
>  daily_status_gconcat_enable="NO"   # Check gconcat(8)
>
> +# 410.status-mfi
> +daily_status_mfi_enable="NO"   # Check mfiutil(8)
> +
>  # 420.status-network
>  daily_status_network_enable="YES"  # Check network
> status
>  daily_status_network_usedns="YES"  # DNS lookups are
> ok
>
> Added: head/etc/periodic/daily/410.status-mfi
> 
> ==
> --- /dev/null   00:00:00 1970   (empty, because file is newly added)
> +++ head/etc/periodic/daily/410.status-mfi  Fri Apr 14 22:59:14 2017
>   (r316945)
> @@ -0,0 +1,33 @@
> +#!/bin/sh
> +#
> +# $FreeBSD$
> +#
> +
> +# If there is a global system configuration file, suck it in.
> +#
> +if [ -r /etc/defaults/periodic.conf ]
> +then
> +. /etc/defaults/periodic.conf
> +source_periodic_confs
> +fi
> +
> +case "$daily_status_mfi_enable" in
> +[Yy][Ee][Ss])
> +   echo
> +   echo 'Checking status of mfi(4) devices:'
> +
> +   if mfiutil show volumes; then
> +   if mfiutil show volumes | grep -q DEGRADED; then
> +   rc=3
> +   else
> +   rc=0
> +   fi
> +   else
> +   rc=2
> +   fi
> +   ;;
> +
> +*)  rc=0;;
> +esac
> +
> +exit $rc
>
> Modified: head/etc/periodic/daily/Makefile
> 
> ==
> --- head/etc/periodic/daily/MakefileFri Apr 14 22:02:08 2017
> (r316944)
> +++ head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
> (r316945)
> @@ -13,6 +13,7 @@ FILES=100.clean-disks \
> 400.status-disks \
> 401.status-graid \
> 406.status-gmirror \
> +   410.status-mfi \


This should be in different place, this list is ordered, as I see from this
context.


> 407.status-graid3 \
> 408.status-gstripe \
> 409.status-gconcat \
>
> Modified: head/share/man/man5/periodic.conf.5
> 
> ==
> --- head/share/man/man5/periodic.conf.5 Fri Apr 14 22:02:08 2017
> (r316944)
> +++ head/share/man/man5/periodic.conf.5 Fri Apr 14 22:59:14 2017
> (r316945)
> @@ -399,6 +399,15 @@ if you want to run
>  on your
>  .Xr gconcat 8
>  devices.
> +.It Va daily_status_mfi_enable
> +.Pq Vt bool
> +Set to
> +.Dq Li YES
> +if you want to run
> +.Nm mfiutil Cm status
> +on your
> +.Xr mfi 4
> +devices.
>  .It Va daily_status_network_enable
>  .Pq Vt bool
>  Set to
> ___
> svn-src-head@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/svn-src-head
> To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org
> "
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r313343 - head/sys/arm/arm

[Ronald Klop]

With this change merged to 11-CURRENT my SHEEVAPLUG/arm displays different  
information in dmesg:


messages:Apr 15 00:53:07 sheeva2 kernel: Little-endian DC enabled IC  
enabled WA disabled DC streaming enabled
messages:Apr 15 00:53:07 sheeva2 kernel: WB enabled LABT branch prediction  
enabled
messages.0.bz2:Apr 11 20:02:06 sheeva2 kernel: Little-endian DC enabled IC  
disabled WA disabled DC streaming enabled
messages.0.bz2:Apr 11 20:02:06 sheeva2 kernel: WB enabled LABT branch  
prediction disabled


IC changed from disabled to enabled and LABT branch prediction is  
displayed as enabled also.


Is this only cosmetically? Or does it really change behaviour?

Anybody willing to MFC this 11?

Regards,
Ronald.


On Mon, 06 Feb 2017 15:58:24 +0100, Stanislav Galabov  
 wrote:



Author: sgalabov
Date: Mon Feb  6 14:58:24 2017
New Revision: 313343
URL: https://svnweb.freebsd.org/changeset/base/313343

Log:
  sys/arm/arm/identcpu-v4.c: fix identify_arm_cpu()
 identify_arm_cpu() in sys/arm/arm/identcpu-v4.c incorrectly uses a
  u_int8_t variable to store the result of cpu_get_control().
  It should really use a u_int variable, the same way as done for  
cpu_ident()

  in the same function, as both cpuid and control registers are 32-bit..
  This issue causes users of identcpu-v4 to incorrectly report things  
such as

  icache status (bit 12 in cpu control register) and basically anything
  defined in bits above bit 7 :-)
 Reviewed by:   manu
  Sponsored by: Smartcom - Bulgaria AD
  Differential Revision:https://reviews.freebsd.org/D9460

Modified:
  head/sys/arm/arm/identcpu-v4.c

Modified: head/sys/arm/arm/identcpu-v4.c
==
--- head/sys/arm/arm/identcpu-v4.c  Mon Feb  6 14:41:34 2017
(r313342)
+++ head/sys/arm/arm/identcpu-v4.c  Mon Feb  6 14:58:24 2017
(r313343)
@@ -294,8 +294,7 @@ u_int cpu_pfr(int num)
 void
 identify_arm_cpu(void)
 {
-   u_int cpuid;
-   u_int8_t ctrl;
+   u_int cpuid, ctrl;
int i;
ctrl = cpu_get_control();
___
svn-src-...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316945 - in head: etc/defaults etc/periodic/daily share/man/man5

[Alan Somers]

Author: asomers
Date: Fri Apr 14 22:59:14 2017
New Revision: 316945
URL: https://svnweb.freebsd.org/changeset/base/316945

Log:
  Add 410.status-mfi, a periodic script for mfi(4) arrays
  
  PR:   176049
  Submitted by: docon...@gsoft.com.au
  Reviewed by:  scottl, Larry Rosenman 
  MFC after:3 weeks
  Relnotes: yes

Added:
  head/etc/periodic/daily/410.status-mfi   (contents, props changed)
Modified:
  head/etc/defaults/periodic.conf
  head/etc/periodic/daily/Makefile
  head/share/man/man5/periodic.conf.5

Modified: head/etc/defaults/periodic.conf
==
--- head/etc/defaults/periodic.conf Fri Apr 14 22:02:08 2017
(r316944)
+++ head/etc/defaults/periodic.conf Fri Apr 14 22:59:14 2017
(r316945)
@@ -112,6 +112,9 @@ daily_status_gstripe_enable="NO"# Che
 # 409.status-gconcat
 daily_status_gconcat_enable="NO"   # Check gconcat(8)
 
+# 410.status-mfi
+daily_status_mfi_enable="NO"   # Check mfiutil(8)
+
 # 420.status-network
 daily_status_network_enable="YES"  # Check network status
 daily_status_network_usedns="YES"  # DNS lookups are ok

Added: head/etc/periodic/daily/410.status-mfi
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/etc/periodic/daily/410.status-mfi  Fri Apr 14 22:59:14 2017
(r316945)
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# $FreeBSD$
+#
+
+# If there is a global system configuration file, suck it in.
+#
+if [ -r /etc/defaults/periodic.conf ]
+then
+. /etc/defaults/periodic.conf
+source_periodic_confs
+fi
+
+case "$daily_status_mfi_enable" in
+[Yy][Ee][Ss])
+   echo
+   echo 'Checking status of mfi(4) devices:'
+
+   if mfiutil show volumes; then
+   if mfiutil show volumes | grep -q DEGRADED; then
+   rc=3
+   else
+   rc=0
+   fi
+   else
+   rc=2
+   fi
+   ;;
+
+*)  rc=0;;
+esac
+
+exit $rc

Modified: head/etc/periodic/daily/Makefile
==
--- head/etc/periodic/daily/MakefileFri Apr 14 22:02:08 2017
(r316944)
+++ head/etc/periodic/daily/MakefileFri Apr 14 22:59:14 2017
(r316945)
@@ -13,6 +13,7 @@ FILES=100.clean-disks \
400.status-disks \
401.status-graid \
406.status-gmirror \
+   410.status-mfi \
407.status-graid3 \
408.status-gstripe \
409.status-gconcat \

Modified: head/share/man/man5/periodic.conf.5
==
--- head/share/man/man5/periodic.conf.5 Fri Apr 14 22:02:08 2017
(r316944)
+++ head/share/man/man5/periodic.conf.5 Fri Apr 14 22:59:14 2017
(r316945)
@@ -399,6 +399,15 @@ if you want to run
 on your
 .Xr gconcat 8
 devices.
+.It Va daily_status_mfi_enable
+.Pq Vt bool
+Set to
+.Dq Li YES
+if you want to run
+.Nm mfiutil Cm status
+on your
+.Xr mfi 4
+devices.
 .It Va daily_status_network_enable
 .Pq Vt bool
 Set to
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 02:28:57PM -0700, Xin LI wrote:
> On Fri, Apr 14, 2017 at 1:29 PM, Mark Johnston  wrote:
> > - I'm not sure how encryption should compose with compression. It seems
> >   intuitively obvious that we should compress before encrypting if the
> >   compression is to be of any use, but I don't know enough to know
> >   whether the compression might somehow compromise the effectiveness of
> >   the encryption.
> 
> I think the biggest concern is the added code involved in the dump
> process (which happen when the kernel is already unhealthy), which can
> jeopardize it and defeat the usefulness of having a crash dump being
> set up in the first place.  

I agree in principle but this doesn't appear to cause problems in
practice. zlib allocates memory only at initialization time, so its
requirements in panic context are quite minimal. I wrote a little bit of
zlib glue, used currently for compressed userland core dumps, which
works properly in panic context as well.

> And with textdumps available, the benefit
> of having compression is limited because we can request for minidump
> or full dumps only when the textdumps are not good enough for
> diagnosing the kernel bug.

Sure, but in this case the compression may be vital.

> 
> I don't think security (e.g. leaking information because of the use of
> compression) is a very big concern in this context because in order
> for the potential attacker to read the raw material needs a
> compromised system (unlike an attack from the network, where someone
> who controls the network would have access to the raw material); the
> dump is usually quite large, and measuring downtime would be hard at
> that scale.

Ok.

> 
> By the way (not meant to bikeshed) if I was to do this I'd prefer
> using lz4 or something that compresses faster than zlib.

I agree, but I think the existing lz4 implementation in the kernel is
not so well suited to running after a panic. It seems fixable, but
compression speed also isn't hugely important here IMO.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 02:21:05 PM Maxim Sobolev wrote:
> Peter, It is actually the other way around. If you take syslogd code out of
> 11-stable and earlier that would work just fine with or without r316874.
> But since r285910 syslogd in head had been refactored a lot and I think
> that particular bug has introduced that has been masked by the shutdown()
> on datagram sockets becoming a NOP after r285910. Then r316874 restored our
> historical behavior for the shutdown(2) and bingo, bug in the new syslogd
> code is now causing it to spin when shutdown() != NOP.

Ok, this makes sense.

Just to be sure I'm on the same page, I should apply the syslogd.diff from a 
few messages ago and restore the shutdown(2) kernel code and give it a spin.  
Correct?

-Peter

> -Max
> 
> On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > > Thanks, Peter. I will try to look into this asap.
> > 
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> > whether
> > the -s code even works at all since the 2015 / r285910 change...
> > 
> > > -Max
> > > 
> > > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > > Author: sobomax
> > > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > > New Revision: 316874
> > > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > > > 
> > > > > > Log:
> > > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> > 
> > ENOTCONN to
> > 
> > > > be
> > > > 
> > > > > > returned by the shutdown(2) system call. This ability has been
> > 
> > lost as
> > 
> > > > > > part
> > > > > > of the svn revision 285910.
> > > > > > 
> > > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > > >   MFC after:2 weeks
> > > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > > > 
> > > > > This appears to have broken syslogd and had a major change in
> > 
> > behavior
> > 
> > > > with
> > > > 
> > > > > regards to select(2).
> > > > > 
> > > > > If you run syslogd with the -s flag (which is default), it now spins
> > 
> > at
> > 
> > > > 100%
> > > > 
> > > > > cpu as all the shutdown sockets now return readable from select.
> > > > > 
> > > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > > losing
> > > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > > instead, but we depend on this in the cluster and I expect others do
> > > > > too.
> > > > > 
> > > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > > can't
> > > > > hurt. I'll try reverting it on the freebsd cluster next, after
> > > > > fixing
> > > > > the
> > > > > broken auditing changes.
> > > > > 
> > > > > -Peter
> > > > 
> > > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > > compatability
> > > > with old branches.
> > > > 
> > > > --
> > > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > > KI6FJV
> > > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
> > 
> > --
> > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > KI6FJV
> > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 2:40 PM, John Baldwin  wrote:
> 
> On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>>  savecore: fix space calculation with respect to `minfree` in check_space(..)
>> 
>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>representable data to INT_MAX. Check the values received from
>>strtoll(3), trimming trailing whitespace off the end to maintain
>>POLA.
>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>bytes and `kB` being the IEC standard for 1000 bytes.
> 
> I will just rant lightly that no one actually uses this in the real world.
> 
> Good lucking finding a "16 GiB" DIMM on crucial.com  or 
> a 4Kin drive.  A
> kilobyte is a power of 2.  The End.
> 
> (Next up we'll have to rename 4k displays to
> 4k)


https://media0.giphy.com/media/7rj2ZgttvgomY/giphy.gif 
 :P.
-Ngie


signature.asc
Description: Message signed with OpenPGP

Re: svn commit: r316938 - head/sbin/savecore

[John Baldwin]

On Friday, April 14, 2017 07:41:48 PM Ngie Cooper wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
>   savecore: fix space calculation with respect to `minfree` in check_space(..)
>   
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.

I will just rant lightly that no one actually uses this in the real world.

Good lucking finding a "16 GiB" DIMM on crucial.com or a 4Kin drive.  A
kilobyte is a power of 2.  The End.

(Next up we'll have to rename 4k displays to
4k)

-- 
John Baldwin
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316941 - head/sys/kern

[Bruce Evans]

On Fri, 14 Apr 2017, Edward Tomasz Napierala wrote:


Log:
 Don't try to write out bufs that have already failed with ENXIO.
 This fixes some panics after disconnecting mounted disks.

 Submitted by:  imp (slightly different version, which I've then lost)
 Reviewed by:   kib, imp, mckusick
 MFC after: 2 weeks
 Differential Revision: https://reviews.freebsd.org/D9674

Modified:
 head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Fri Apr 14 20:15:17 2017(r316940)
+++ head/sys/kern/vfs_bio.c Fri Apr 14 20:15:34 2017(r316941)
@@ -2290,18 +2290,28 @@ brelse(struct buf *bp)
bdirty(bp);
}
if (bp->b_iocmd == BIO_WRITE && (bp->b_ioflags & BIO_ERROR) &&
+   (bp->b_error != ENXIO || !LIST_EMPTY(>b_dep)) &&
!(bp->b_flags & B_INVAL)) {
/*
-* Failed write, redirty.  Must clear BIO_ERROR to prevent
-* pages from being scrapped.
+* Failed write, redirty.  All errors except ENXIO (which
+* means the device is gone) are expected to be potentially
+* transient - underlying media might work if tried again
+* after EIO, and memory might be available after an ENOMEM.
+*
+* Do this also for buffers that failed with ENXIO, but have
+* non-empty dependencies - the soft updates code might need
+* to access the buffer to untangle them.
+*
+* Must clear BIO_ERROR to prevent pages from being scrapped.
 */


This is hard to fix, but I have used a version that only retries after
EIO for 15-20 years.  I didn't think of ENOMEM.

The media is unlikely to come back after EIO too.  For removable media,
you might be able to get the write done to new media, but a panic reading
from the new media is just as likely.  Geom "tasting" might prevent the
new media being used.

ENXIO is actually the one error that can often be recovered from.  I
wrote a form of "tasting" in a toy OS 30-35 years ago.  It handled
removal of "mounted" disks with pending writes too well, in a way that
made recovery from non-transient I/O errors almost impossible without
turning off the system.  ENXIO was treated as a transient I/O error.
It was recovered from perfectly if the user could find the original
media and unremove it.  The "tasting" usually worked to detect different
media and disallow writing cached data to a different disk.  Media
errors were common, and when one occurred for writing the method of
replacing the disk by a garbage one did't work since it was a different
disk.  The most common one was writing to a write protected disk, and
that was recoverable by removing the write protection.  But often you
really didn't want to write to that disk, but wanted to write somewhere.
The only way to continue was to reboot to discard the write.

Bruce
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316822 - in head/sys: kern netinet netinet6

[Gleb Smirnoff]

  Andrey,

On Fri, Apr 14, 2017 at 09:00:48AM +, Andrey V. Elsukov wrote:
A> Log:
A>   Rework r316770 to make it protocol independent and general, like we
A>   do for streaming sockets.
A>   
A>   And do more cleanup in the sbappendaddr_locked_internal() to prevent
A>   leak information from existing mbuf to the one, that will be possible
A>   created later by netgraph.

Thanks!

Your comment actually made me think that this can be improved...

A> @@ -794,8 +794,20 @@ sbappendaddr_locked_internal(struct sock
A>  return (0);
A>  m->m_len = asa->sa_len;
A>  bcopy(asa, mtod(m, caddr_t), asa->sa_len);
A> -if (m0)
A> +if (m0) {
A>  m_clrprotoflags(m0);
A> +m_tag_delete_chain(m0);
A> +/*
A> + * Clear some persistent info from pkthdr.
A> + * We don't use m_demote(), because some netgraph consumers
A> + * expect M_PKTHDR presence.
A> + */

Looks like you are right and only netgraph wants to see M_PKTHDR there.
Rest of datagram socket consumers just use m_nextpkt to differentiate
records from parts of records.

I'll see if we can fix ng_ksocket and use same code on datagram sockbufs
as we do on stream ones.

-- 
Totus tuus, Glebius.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Xin LI]

On Fri, Apr 14, 2017 at 1:29 PM, Mark Johnston  wrote:
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.

I think the biggest concern is the added code involved in the dump
process (which happen when the kernel is already unhealthy), which can
jeopardize it and defeat the usefulness of having a crash dump being
set up in the first place.  And with textdumps available, the benefit
of having compression is limited because we can request for minidump
or full dumps only when the textdumps are not good enough for
diagnosing the kernel bug.

I don't think security (e.g. leaking information because of the use of
compression) is a very big concern in this context because in order
for the potential attacker to read the raw material needs a
compromised system (unlike an attack from the network, where someone
who controls the network would have access to the raw material); the
dump is usually quite large, and measuring downtime would be hard at
that scale.

By the way (not meant to bikeshed) if I was to do this I'd prefer
using lz4 or something that compresses faster than zlib.

Cheers,
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Peter, It is actually the other way around. If you take syslogd code out of
11-stable and earlier that would work just fine with or without r316874.
But since r285910 syslogd in head had been refactored a lot and I think
that particular bug has introduced that has been masked by the shutdown()
on datagram sockets becoming a NOP after r285910. Then r316874 restored our
historical behavior for the shutdown(2) and bingo, bug in the new syslogd
code is now causing it to spin when shutdown() != NOP.

-Max

On Fri, Apr 14, 2017 at 12:46 PM, Peter Wemm  wrote:

> On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> > Thanks, Peter. I will try to look into this asap.
>
> I don't understand what is going on yet. Presumably there must be other
> changes in play that affect udp/select sometime between the original 2015
> change and this. The syslogd -s code is Old(TM).  I'm also wondering
> whether
> the -s code even works at all since the 2015 / r285910 change...
>
> > -Max
> >
> > On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:
> > > On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > > > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > > > Author: sobomax
> > > > > Date: Fri Apr 14 17:23:28 2017
> > > > > New Revision: 316874
> > > > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > > > >
> > > > > Log:
> > > > >   Restore ability to shutdown DGRAM sockets, still forcing
> ENOTCONN to
> > >
> > > be
> > >
> > > > > returned by the shutdown(2) system call. This ability has been
> lost as
> > > > > part
> > > > > of the svn revision 285910.
> > > > >
> > > > >   Reviewed by:  ed, rwatson, glebius, hiren
> > > > >   MFC after:2 weeks
> > > > >   Differential Revision:https://reviews.freebsd.org/D10351
> > > >
> > > > This appears to have broken syslogd and had a major change in
> behavior
> > >
> > > with
> > >
> > > > regards to select(2).
> > > >
> > > > If you run syslogd with the -s flag (which is default), it now spins
> at
> > >
> > > 100%
> > >
> > > > cpu as all the shutdown sockets now return readable from select.
> > > >
> > > > Old releases / jails also manifest this behavor.  If it wasn't for
> > > > losing
> > > > the ability to run old branch binaries I'd suggest changing syslogd
> > > > instead, but we depend on this in the cluster and I expect others do
> > > > too.
> > > >
> > > > I'm not 100% certain that this change is the culprit but a heads-up
> > > > can't
> > > > hurt. I'll try reverting it on the freebsd cluster next, after fixing
> > > > the
> > > > broken auditing changes.
> > > >
> > > > -Peter
> > >
> > > I can confirm that reverting r316874 fixes syslogd and backwards
> > > compatability
> > > with old branches.
> > >
> > > --
> > > Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> > > KI6FJV
> > > UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
>
> --
> Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> KI6FJV
> UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 14:10, Maxim Sobolev  wrote:
> 
> Peter, Ngie,
> 
> Looks like out of that refactoring came a logical bug that is present in the 
> head, which causes syslod to first to shutdown the socket for reading and 
> then try to select/recv on it (which is somewhat stupid). And that issue has 
> been masked by shutdown() on datagram socket becoming effectively a NOP in 11 
> & head 20 months ago. It only affects head though, 11-stable still has the 
> old code which does not include that half-closed socket into the select list. 
> Attached patch is expected to fix head, Peter, it would be nice if you can 
> give it a try (restoring latest changes into uipc_sockets.c) and let me know 
> if it helps.
> 
> Thanks!

CCing hrs@ for input as he did the refactoring.
Thanks!
-Ngie

PS LGTM with the change. Will wait for feedback from wemm@.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Peter, Ngie,

Looks like out of that refactoring came a logical bug that is present in
the head, which causes syslod to first to shutdown the socket for reading
and then try to select/recv on it (which is somewhat stupid). And that
issue has been masked by shutdown() on datagram socket becoming effectively
a NOP in 11 & head 20 months ago. It only affects head though, 11-stable
still has the old code which does not include that half-closed socket into
the select list. Attached patch is expected to fix head, Peter, it would be
nice if you can give it a try (restoring latest changes into
uipc_sockets.c) and let me know if it helps.

Thanks!

On Fri, Apr 14, 2017 at 12:48 PM, Ngie Cooper (yaneurabeya) <
yaneurab...@gmail.com> wrote:

>
> > On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> >
> > On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
> >> Thanks, Peter. I will try to look into this asap.
> >
> > I don't understand what is going on yet. Presumably there must be other
> > changes in play that affect udp/select sometime between the original 2015
> > change and this. The syslogd -s code is Old(TM).  I'm also wondering
> whether
> > the -s code even works at all since the 2015 / r285910 change...
>
> syslogd has been refactored a lot on ^/head. I don’t think it’s safe to
> say that the ^/head and ^/stable/11 and earlier copies will function the
> same.
> Thanks,
> -Ngie
>



-- 
Maksym Sobolyev
Sippy Software, Inc.
Internet Telephony (VoIP) Experts
Tel (Canada): +1-778-783-0474
Tel (Toll-Free): +1-855-747-7779
Fax: +1-866-857-6942
Web: http://www.sippysoft.com
MSN: sa...@sippysoft.com
Skype: SippySoft
Index: syslogd.c
===
--- syslogd.c   (revision 316854)
+++ syslogd.c   (working copy)
@@ -702,7 +702,7 @@
sizeof(fd_mask));
 
STAILQ_FOREACH(sl, , next) {
-   if (sl->sl_socket != -1)
+   if (sl->sl_socket != -1 && sl->sl_recv != NULL)
FD_SET(sl->sl_socket, fdsr);
}
i = select(fdsrmax + 1, fdsr, NULL, NULL,
@@ -2877,6 +2877,7 @@
struct addrinfo hints, *res, *res0;
int error;
char *cp;
+   int (*sl_recv)(struct socklist *);
/*
 * We have to handle this case for backwards compatibility:
 * If there are two (or more) colons but no '[' and ']',
@@ -3003,6 +3004,7 @@
}
dprintf("new socket fd is %d\n", s);
listen(s, 5);
+   sl_recv = socklist_recv_sock;
dprintf("shutdown\n");
if (SecureMode || res->ai_family == AF_LOCAL) {
/* Forbid communication in secure mode. */
@@ -3013,6 +3015,7 @@
die(0);
}
dprintf("listening on socket\n");
+   sl_recv = NULL;
} else
dprintf("sending on socket\n");
addsock(res->ai_addr, res->ai_addrlen,
@@ -3019,7 +3022,7 @@
&(struct socklist){
.sl_socket = s,
.sl_peer = pe,
-   .sl_recv = socklist_recv_sock
+   .sl_recv = sl_recv
});
}
freeaddrinfo(res0);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Xin LI]

No, this is not the threat model for swaps.

On Fri, Apr 14, 2017 at 1:47 PM, Ronald Klop  wrote:
> On Fri, 14 Apr 2017 22:29:18 +0200, Mark Johnston  wrote:
>
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>>
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> > Author: ngie
>>> > Date: Fri Apr 14 19:41:48 2017
>>> > New Revision: 316938
>>> > URL: https://svnweb.freebsd.org/changeset/base/316938
>>> >
>>> > Log:
>>> >   savecore: fix space calculation with respect to `minfree` in
>>> > check_space(..)
>>> >
>>> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>> > representable data to INT_MAX. Check the values received from
>>> > strtoll(3), trimming trailing whitespace off the end to maintain
>>> > POLA.
>>> >   - Use `KiB` instead of `kB` when describing free space, total space,
>>> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>> > bytes and `kB` being the IEC standard for 1000 bytes.
>>> >   - Store available number of KiB in `available` so it can be more
>>> > easily queried and compared to ensure that there are enough KiB to
>>> > store the dump image on disk.
>>> >   - Print out the reserved space on disk, per `minfree`, so end-users
>>> > can troubleshoot why check_space(..) is reporting that there isn't
>>> > enough free space.
>>> >
>>> >   MFC after:7 weeks
>>> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier
>>> > diff)
>>> >   Tested with:  positive/negative cases (see review); make tinderbox
>>> >   Sponsored by: Dell EMC Isilon
>>> >   Differential Revision:D10379
>>>
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>>
>>
>> I had held off on it for a while because it was going to conflict with
>> the work to add encrypted dump support, which of course has finished.
>>
>> The patch to add compression support is here and should largely still
>> work:
>>
>> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
>>
>> I've been hesitant about pushing it forward:
>> - The dump_write* APIs need some simplification after the addition of
>>   encrypted dump support and support for dumping to 4Kn drives.
>> - I'm not sure how encryption should compose with compression. It seems
>>   intuitively obvious that we should compress before encrypting if the
>>   compression is to be of any use, but I don't know enough to know
>>   whether the compression might somehow compromise the effectiveness of
>>   the encryption.
>>
>> If anyone has some insight on the second of these two points, I'd
>> appreciate hearing it.
>
>
> About compression before encryption there is some information. For example:
> https://en.wikipedia.org/wiki/CRIME
> Openssh removed compression in pre-authentication in
> http://www.openssh.com/txt/release-7.4
>
> Regards,
> Ronald.
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Conrad Meyer]

On Fri, Apr 14, 2017 at 1:41 PM, Alan Somers  wrote:
> On Fri, Apr 14, 2017 at 2:29 PM, Mark Johnston  wrote:
>> I've been hesitant about pushing it forward:
>> - The dump_write* APIs need some simplification after the addition of
>>   encrypted dump support and support for dumping to 4Kn drives.
>> - I'm not sure how encryption should compose with compression. It seems
>>   intuitively obvious that we should compress before encrypting if the
>>   compression is to be of any use, but I don't know enough to know
>>   whether the compression might somehow compromise the effectiveness of
>>   the encryption.
>>
>> If anyone has some insight on the second of these two points, I'd
>> appreciate hearing it.
>
> I think compress then encrypt should be ok.  AFAIK all attacks against
> compress-then-encrypt systems have involved either incredibly short
> payloads that are easy to guess, or a stream of separately compressed
> blocks that can be fingerprinted.  But core dumps are very long, and
> they can't be fingerprinted in whole because they're unique.  If you
> were to encrypt each page individually then pages could be
> fingerprinted, so don't do that.  Instead, compress the entire core
> dump as a single stream and you should be ok.

+1.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:48, Larry Rosenman  wrote:

…

> No can do, the rest of the space is pool.

>_>… that sucks. This is part of the reason why I still make my swap partitions 
>greater than or equal to the amount of memory I have in systems :/..
The best recommendation I have is either: wait for compressed dump support, 
repartition/shuffle around data on your disks so the configuration fits, or try 
and get ccd to work *shrugs*.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ronald Klop]

On Fri, 14 Apr 2017 22:29:18 +0200, Mark Johnston   
wrote:



On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:

On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
>
> Log:
>   savecore: fix space calculation with respect to `minfree` in  
check_space(..)

>
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.
>   - Store available number of KiB in `available` so it can be more
> easily queried and compared to ensure that there are enough KiB to
> store the dump image on disk.
>   - Print out the reserved space on disk, per `minfree`, so end-users
> can troubleshoot why check_space(..) is reporting that there isn't
> enough free space.
>
>   MFC after:7 weeks
>   Reviewed by:  Anton Rang  (earlier diff), cem  
(earlier diff)

>   Tested with:  positive/negative cases (see review); make tinderbox
>   Sponsored by: Dell EMC Isilon
>   Differential Revision:D10379

The free space calculation is still uselessly conservative, because it
doesn't account for the fact that core dumps will always be either
spare or compressed.  The result is that savecore will frequently
refuse to save corefiles even when there's plenty of space.  I
proposed removing the space check altogether in
https://reviews.freebsd.org/D2587.  However, I agreed to wait until
after the compressed core dump feature was merged, because then mostly
accurate space checks will be possible.  AFAIK the compressed core
dump feature still hasn't been finished.


I had held off on it for a while because it was going to conflict with
the work to add encrypted dump support, which of course has finished.

The patch to add compression support is here and should largely still
work:
https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff

I've been hesitant about pushing it forward:
- The dump_write* APIs need some simplification after the addition of
  encrypted dump support and support for dumping to 4Kn drives.
- I'm not sure how encryption should compose with compression. It seems
  intuitively obvious that we should compress before encrypting if the
  compression is to be of any use, but I don't know enough to know
  whether the compression might somehow compromise the effectiveness of
  the encryption.

If anyone has some insight on the second of these two points, I'd
appreciate hearing it.


About compression before encryption there is some information. For  
example: https://en.wikipedia.org/wiki/CRIME
Openssh removed compression in pre-authentication in  
http://www.openssh.com/txt/release-7.4


Regards,
Ronald.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:46, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:
> 
>> 
>>> On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
>> 
>> …
>> 
>>> No, something like:
>>> % swapinfo
>>> Device  1K-blocks UsedAvail Capacity
>>> /dev/ada0p2  335544320 33554432 0%
>>> /dev/ada1p2  335544320 33554432 0%
>>> Total671088640 67108864 0%
>> 
>> Striping dumps across multiple swap devices isn’t supported. Sorry.
> 
> What about support in the future?

*shrugs* I don’t have a crystal ball to say yay or nay, but saving full dumps 
out to disk seems foolhardy to say the least. Shrinking down the dataset seems 
like a more sensible problem to solve than striping data across disks.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:46 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:43, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
> 
>> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
wrote:
>> 
>>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>>> 
>>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
>>> 
 On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
 
 On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>> savecore: fix space calculation with respect to `minfree` in 
check_space(..)
>> 
>> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> representable data to INT_MAX. Check the values received from
>> strtoll(3), trimming trailing whitespace off the end to maintain
>> POLA.
>> - Use `KiB` instead of `kB` when describing free space, total space,
>> etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> bytes and `kB` being the IEC standard for 1000 bytes.
>> - Store available number of KiB in `available` so it can be more
>> easily queried and compared to ensure that there are enough KiB to
>> store the dump image on disk.
>> - Print out the reserved space on disk, per `minfree`, so end-users
>> can troubleshoot why check_space(..) is reporting that there isn't
>> enough free space.
>> 
>> MFC after:7 weeks
>> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
>> Tested with:  positive/negative cases (see review); make tinderbox
>> Sponsored by: Dell EMC Isilon
>> Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.
 
 Is posible (in the future) to use multiple swaps (on multiple disks)
 for save core dumps?
>>> 
>>>  Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
>>> 
>>>  As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.
>> 
>>   …
>> 
>>> How do I use multiple devices to have the system dump on all of my 
swap?  I got a message about not enough space, but there (I think) was enough 
between multiple drives….
>> 
>>   Something like:
>> 
>>   - Create a zpool
>>   - Mount zpool to /crashdumps
>>   - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’
>> 
>>   ?
>>   HTH,
>>   -Ngie
>> 
>>   PS The issue with lack of space might be the issue that Alan brought 
up earlier with compressed dumps and overly conservative free space checks, or 
it might be the fact that dumpdir (default: /var/crash) is full.
>> 
>> 
>> I was talking about the actual crashdump to swap by the system.  
/var/crash has 10T of space (my root pool).
> 
>If your memory is bigger than your swap, you’re unfortunately not able 
to save the mini dump if the size of the saved pages exceed the space on swap. 
I think this is where markj’s compressed dumps feature will come in handy.
>Thanks!
>-Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608  

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:43, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
>> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
>> wrote:
>> 
>>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>>> 
>>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>>>  wrote:
>>> 
 On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
 
 On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>> savecore: fix space calculation with respect to `minfree` in 
>> check_space(..)
>> 
>> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> representable data to INT_MAX. Check the values received from
>> strtoll(3), trimming trailing whitespace off the end to maintain
>> POLA.
>> - Use `KiB` instead of `kB` when describing free space, total space,
>> etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> bytes and `kB` being the IEC standard for 1000 bytes.
>> - Store available number of KiB in `available` so it can be more
>> easily queried and compared to ensure that there are enough KiB to
>> store the dump image on disk.
>> - Print out the reserved space on disk, per `minfree`, so end-users
>> can troubleshoot why check_space(..) is reporting that there isn't
>> enough free space.
>> 
>> MFC after:7 weeks
>> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>> diff)
>> Tested with:  positive/negative cases (see review); make tinderbox
>> Sponsored by: Dell EMC Isilon
>> Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.
 
 Is posible (in the future) to use multiple swaps (on multiple disks)
 for save core dumps?
>>> 
>>>  Multiple swap devices is already handled by savecore(8), if one uses 
>>> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
>>> devices.
>>> 
>>>  As far as saving to multiple disks is concerned, I would hope that one is 
>>> using a redundancy capable filesystem (zfs) or RAID-like technology 
>>> (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror 
>>> the data across multiple disks.
>> 
>>   …
>> 
>>> How do I use multiple devices to have the system dump on all of my swap?  I 
>>> got a message about not enough space, but there (I think) was enough 
>>> between multiple drives….
>> 
>>   Something like:
>> 
>>   - Create a zpool
>>   - Mount zpool to /crashdumps
>>   - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
>> ‘dumpdir=/crashdumps’
>> 
>>   ?
>>   HTH,
>>   -Ngie
>> 
>>   PS The issue with lack of space might be the issue that Alan brought up 
>> earlier with compressed dumps and overly conservative free space checks, or 
>> it might be the fact that dumpdir (default: /var/crash) is full.
>> 
>> 
>> I was talking about the actual crashdump to swap by the system.  /var/crash 
>> has 10T of space (my root pool).
> 
>If your memory is bigger than your swap, you’re unfortunately not able to 
> save the mini dump if the size of the saved pages exceed the space on swap. I 
> think this is where markj’s compressed dumps feature will come in handy.
>Thanks!
>-Ngie
> 
> 
> Yeah, I have the following:
> borg.lerctr.org /home/ler $ swapctl -l
> Device:   1024-blocks Used:
> /dev/mfid0p38388608 0
> /dev/mfid1p38388608 0
> /dev/mfid2p38388608 0
> /dev/mfid3p38388608 0
> /dev/mfid4p38388608 0
> /dev/mfid5p38388608 0
> borg.lerctr.org /home/ler $ sysctl hw.physmem
> hw.physmem: 137368682496
> borg.lerctr.org /home/ler $
> 
> SO 6 8G partitions (48G), but the dump is larger than 8G.

Yup, that’s not going to work. Since you’re using mfi(4), I’d try creating 
another 48GB large RAID-0 volume to help deal with that issue.

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:45:22PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:
> 
> …
> 
> > No, something like:
> > % swapinfo
> > Device  1K-blocks UsedAvail Capacity
> > /dev/ada0p2  335544320 33554432 0%
> > /dev/ada1p2  335544320 33554432 0%
> > Total671088640 67108864 0%
> 
> Striping dumps across multiple swap devices isn’t supported. Sorry.

What about support in the future?

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:42, Slawa Olhovchenkov  wrote:

…

> No, something like:
> % swapinfo
> Device  1K-blocks UsedAvail Capacity
> /dev/ada0p2  335544320 33554432 0%
> /dev/ada1p2  335544320 33554432 0%
> Total671088640 67108864 0%

Striping dumps across multiple swap devices isn’t supported. Sorry.
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:39 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
wrote:
> 
>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
>> 
>>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>>> 
>>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>> 
 On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>  representable data to INT_MAX. Check the values received from
>  strtoll(3), trimming trailing whitespace off the end to maintain
>  POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>  etc. I am now fully aware of `KiB` being the IEC standard for 1024
>  bytes and `kB` being the IEC standard for 1000 bytes.
> - Store available number of KiB in `available` so it can be more
>  easily queried and compared to ensure that there are enough KiB to
>  store the dump image on disk.
> - Print out the reserved space on disk, per `minfree`, so end-users
>  can troubleshoot why check_space(..) is reporting that there isn't
>  enough free space.
> 
> MFC after:7 weeks
> Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
> Tested with:  positive/negative cases (see review); make tinderbox
> Sponsored by: Dell EMC Isilon
> Differential Revision:D10379
 
 The free space calculation is still uselessly conservative, because it
 doesn't account for the fact that core dumps will always be either
 spare or compressed.  The result is that savecore will frequently
 refuse to save corefiles even when there's plenty of space.  I
 proposed removing the space check altogether in
 https://reviews.freebsd.org/D2587.  However, I agreed to wait until
 after the compressed core dump feature was merged, because then mostly
 accurate space checks will be possible.  AFAIK the compressed core
 dump feature still hasn't been finished.
>>> 
>>> Is posible (in the future) to use multiple swaps (on multiple disks)
>>> for save core dumps?
>> 
>>   Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
>> 
>>   As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.
> 
>…
> 
>> How do I use multiple devices to have the system dump on all of my swap? 
 I got a message about not enough space, but there (I think) was enough between 
multiple drives….
> 
>Something like:
> 
>- Create a zpool
>- Mount zpool to /crashdumps
>- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’
> 
>?
>HTH,
>-Ngie
> 
>PS The issue with lack of space might be the issue that Alan brought 
up earlier with compressed dumps and overly conservative free space checks, or 
it might be the fact that dumpdir (default: /var/crash) is full.
> 
> 
> I was talking about the actual crashdump to swap by the system.  
/var/crash has 10T of space (my root pool).

If your memory is bigger than your swap, you’re unfortunately not able to 
save the mini dump if the size of the saved pages exceed the space on swap. I 
think this is where markj’s compressed dumps feature will come in handy.
Thanks!
-Ngie


Yeah, I have the following:
borg.lerctr.org /home/ler $ swapctl -l
Device:   1024-blocks Used:
/dev/mfid0p38388608 0
/dev/mfid1p38388608 0
/dev/mfid2p38388608 0
/dev/mfid3p38388608 0
/dev/mfid4p38388608 0
/dev/mfid5p38388608 0
borg.lerctr.org /home/ler $ sysctl hw.physmem
hw.physmem: 137368682496
borg.lerctr.org /home/ler $

SO 6 8G partitions (48G), but the dump is larger than 8G. 





___

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:33:36PM -0700, Ngie Cooper (yaneurabeya) wrote:

> 
> > On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> > 
> > On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
> >  wrote:
> > 
> >> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> >> 
> >> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> >> 
> >>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>  Author: ngie
>  Date: Fri Apr 14 19:41:48 2017
>  New Revision: 316938
>  URL: https://svnweb.freebsd.org/changeset/base/316938
>  
>  Log:
>  savecore: fix space calculation with respect to `minfree` in 
>  check_space(..)
>  
>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>    representable data to INT_MAX. Check the values received from
>    strtoll(3), trimming trailing whitespace off the end to maintain
>    POLA.
>  - Use `KiB` instead of `kB` when describing free space, total space,
>    etc. I am now fully aware of `KiB` being the IEC standard for 1024
>    bytes and `kB` being the IEC standard for 1000 bytes.
>  - Store available number of KiB in `available` so it can be more
>    easily queried and compared to ensure that there are enough KiB to
>    store the dump image on disk.
>  - Print out the reserved space on disk, per `minfree`, so end-users
>    can troubleshoot why check_space(..) is reporting that there isn't
>    enough free space.
>  
>  MFC after:7 weeks
>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>  diff)
>  Tested with:  positive/negative cases (see review); make tinderbox
>  Sponsored by: Dell EMC Isilon
>  Differential Revision:D10379
> >>> 
> >>> The free space calculation is still uselessly conservative, because it
> >>> doesn't account for the fact that core dumps will always be either
> >>> spare or compressed.  The result is that savecore will frequently
> >>> refuse to save corefiles even when there's plenty of space.  I
> >>> proposed removing the space check altogether in
> >>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> >>> after the compressed core dump feature was merged, because then mostly
> >>> accurate space checks will be possible.  AFAIK the compressed core
> >>> dump feature still hasn't been finished.
> >> 
> >> Is posible (in the future) to use multiple swaps (on multiple disks)
> >> for save core dumps?
> > 
> >Multiple swap devices is already handled by savecore(8), if one uses 
> > fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> > devices.
> > 
> >As far as saving to multiple disks is concerned, I would hope that one 
> > is using a redundancy capable filesystem (zfs) or RAID-like technology 
> > (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror 
> > the data across multiple disks.
> 
> …
> 
> > How do I use multiple devices to have the system dump on all of my swap?  I 
> > got a message about not enough space, but there (I think) was enough 
> > between multiple drives….
> 
> Something like:
> 
> - Create a zpool
> - Mount zpool to /crashdumps
> - Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ‘dumpdir=/crashdumps’
> 
> ?

No, something like:
% swapinfo
Device  1K-blocks UsedAvail Capacity
/dev/ada0p2  335544320 33554432 0%
/dev/ada1p2  335544320 33554432 0%
Total671088640 67108864 0%

> 
> PS The issue with lack of space might be the issue that Alan brought up 
> earlier with compressed dumps and overly conservative free space checks, or 
> it might be the fact that dumpdir (default: /var/crash) is full.


___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 2:29 PM, Mark Johnston  wrote:
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> > Author: ngie
>> > Date: Fri Apr 14 19:41:48 2017
>> > New Revision: 316938
>> > URL: https://svnweb.freebsd.org/changeset/base/316938
>> >
>> > Log:
>> >   savecore: fix space calculation with respect to `minfree` in 
>> > check_space(..)
>> >
>> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>> > representable data to INT_MAX. Check the values received from
>> > strtoll(3), trimming trailing whitespace off the end to maintain
>> > POLA.
>> >   - Use `KiB` instead of `kB` when describing free space, total space,
>> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
>> > bytes and `kB` being the IEC standard for 1000 bytes.
>> >   - Store available number of KiB in `available` so it can be more
>> > easily queried and compared to ensure that there are enough KiB to
>> > store the dump image on disk.
>> >   - Print out the reserved space on disk, per `minfree`, so end-users
>> > can troubleshoot why check_space(..) is reporting that there isn't
>> > enough free space.
>> >
>> >   MFC after:7 weeks
>> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
>> > diff)
>> >   Tested with:  positive/negative cases (see review); make tinderbox
>> >   Sponsored by: Dell EMC Isilon
>> >   Differential Revision:D10379
>>
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
>
> I had held off on it for a while because it was going to conflict with
> the work to add encrypted dump support, which of course has finished.
>
> The patch to add compression support is here and should largely still
> work:
> https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff
>
> I've been hesitant about pushing it forward:
> - The dump_write* APIs need some simplification after the addition of
>   encrypted dump support and support for dumping to 4Kn drives.
> - I'm not sure how encryption should compose with compression. It seems
>   intuitively obvious that we should compress before encrypting if the
>   compression is to be of any use, but I don't know enough to know
>   whether the compression might somehow compromise the effectiveness of
>   the encryption.
>
> If anyone has some insight on the second of these two points, I'd
> appreciate hearing it.

I think compress then encrypt should be ok.  AFAIK all attacks against
compress-then-encrypt systems have involved either incredibly short
payloads that are easy to guess, or a stream of separately compressed
blocks that can be fingerprinted.  But core dumps are very long, and
they can't be fingerprinted in whole because they're unique.  If you
were to encrypt each page individually then pages could be
fingerprinted, so don't do that.  Instead, compress the entire core
dump as a single stream and you should be ok.

-Alan
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:37, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  
> wrote:
> 
>> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
>> 
>> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>>  wrote:
>> 
>>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>>> 
>>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>>> 
 On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
> 
> Log:
> savecore: fix space calculation with respect to `minfree` in 
> check_space(..)
> 
> - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>  representable data to INT_MAX. Check the values received from
>  strtoll(3), trimming trailing whitespace off the end to maintain
>  POLA.
> - Use `KiB` instead of `kB` when describing free space, total space,
>  etc. I am now fully aware of `KiB` being the IEC standard for 1024
>  bytes and `kB` being the IEC standard for 1000 bytes.
> - Store available number of KiB in `available` so it can be more
>  easily queried and compared to ensure that there are enough KiB to
>  store the dump image on disk.
> - Print out the reserved space on disk, per `minfree`, so end-users
>  can troubleshoot why check_space(..) is reporting that there isn't
>  enough free space.
> 
> MFC after:7 weeks
> Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> Tested with:  positive/negative cases (see review); make tinderbox
> Sponsored by: Dell EMC Isilon
> Differential Revision:D10379
 
 The free space calculation is still uselessly conservative, because it
 doesn't account for the fact that core dumps will always be either
 spare or compressed.  The result is that savecore will frequently
 refuse to save corefiles even when there's plenty of space.  I
 proposed removing the space check altogether in
 https://reviews.freebsd.org/D2587.  However, I agreed to wait until
 after the compressed core dump feature was merged, because then mostly
 accurate space checks will be possible.  AFAIK the compressed core
 dump feature still hasn't been finished.
>>> 
>>> Is posible (in the future) to use multiple swaps (on multiple disks)
>>> for save core dumps?
>> 
>>   Multiple swap devices is already handled by savecore(8), if one uses 
>> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
>> devices.
>> 
>>   As far as saving to multiple disks is concerned, I would hope that one is 
>> using a redundancy capable filesystem (zfs) or RAID-like technology 
>> (gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
>> data across multiple disks.
> 
>…
> 
>> How do I use multiple devices to have the system dump on all of my swap?  I 
>> got a message about not enough space, but there (I think) was enough between 
>> multiple drives….
> 
>Something like:
> 
>- Create a zpool
>- Mount zpool to /crashdumps
>- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
> ‘dumpdir=/crashdumps’
> 
>?
>HTH,
>-Ngie
> 
>PS The issue with lack of space might be the issue that Alan brought up 
> earlier with compressed dumps and overly conservative free space checks, or 
> it might be the fact that dumpdir (default: /var/crash) is full.
> 
> 
> I was talking about the actual crashdump to swap by the system.  /var/crash 
> has 10T of space (my root pool).

If your memory is bigger than your swap, you’re unfortunately not able to save 
the mini dump if the size of the saved pages exceed the space on swap. I think 
this is where markj’s compressed dumps feature will come in handy.
Thanks!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:33 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
 wrote:
> 
>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>> 
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> 
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
 - Store available number of KiB in `available` so it can be more
   easily queried and compared to ensure that there are enough KiB to
   store the dump image on disk.
 - Print out the reserved space on disk, per `minfree`, so end-users
   can troubleshoot why check_space(..) is reporting that there isn't
   enough free space.
 
 MFC after:7 weeks
 Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
 Tested with:  positive/negative cases (see review); make tinderbox
 Sponsored by: Dell EMC Isilon
 Differential Revision:D10379
>>> 
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>> 
>> Is posible (in the future) to use multiple swaps (on multiple disks)
>> for save core dumps?
> 
>Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.
> 
>As far as saving to multiple disks is concerned, I would hope that one 
is using a redundancy capable filesystem (zfs) or RAID-like technology 
(gmirror, graid, LSI Fusion’s RAID product line) to stripe and/or mirror the 
data across multiple disks.

…

> How do I use multiple devices to have the system dump on all of my swap?  
I got a message about not enough space, but there (I think) was enough between 
multiple drives….

Something like:

- Create a zpool
- Mount zpool to /crashdumps
- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’

?
HTH,
-Ngie

PS The issue with lack of space might be the issue that Alan brought up 
earlier with compressed dumps and overly conservative free space checks, or it 
might be the fact that dumpdir (default: /var/crash) is full.


I was talking about the actual crashdump to swap by the system.  /var/crash has 
10T of space (my root pool).




___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:26, Larry Rosenman  wrote:
> 
> On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)" 
>  wrote:
> 
>> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
>> 
>> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
>> 
>>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
 Author: ngie
 Date: Fri Apr 14 19:41:48 2017
 New Revision: 316938
 URL: https://svnweb.freebsd.org/changeset/base/316938
 
 Log:
 savecore: fix space calculation with respect to `minfree` in 
 check_space(..)
 
 - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
   representable data to INT_MAX. Check the values received from
   strtoll(3), trimming trailing whitespace off the end to maintain
   POLA.
 - Use `KiB` instead of `kB` when describing free space, total space,
   etc. I am now fully aware of `KiB` being the IEC standard for 1024
   bytes and `kB` being the IEC standard for 1000 bytes.
 - Store available number of KiB in `available` so it can be more
   easily queried and compared to ensure that there are enough KiB to
   store the dump image on disk.
 - Print out the reserved space on disk, per `minfree`, so end-users
   can troubleshoot why check_space(..) is reporting that there isn't
   enough free space.
 
 MFC after:7 weeks
 Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
 Tested with:  positive/negative cases (see review); make tinderbox
 Sponsored by: Dell EMC Isilon
 Differential Revision:D10379
>>> 
>>> The free space calculation is still uselessly conservative, because it
>>> doesn't account for the fact that core dumps will always be either
>>> spare or compressed.  The result is that savecore will frequently
>>> refuse to save corefiles even when there's plenty of space.  I
>>> proposed removing the space check altogether in
>>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>>> after the compressed core dump feature was merged, because then mostly
>>> accurate space checks will be possible.  AFAIK the compressed core
>>> dump feature still hasn't been finished.
>> 
>> Is posible (in the future) to use multiple swaps (on multiple disks)
>> for save core dumps?
> 
>Multiple swap devices is already handled by savecore(8), if one uses 
> fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
> devices.
> 
>As far as saving to multiple disks is concerned, I would hope that one is 
> using a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, 
> graid, LSI Fusion’s RAID product line) to stripe and/or mirror the data 
> across multiple disks.

…

> How do I use multiple devices to have the system dump on all of my swap?  I 
> got a message about not enough space, but there (I think) was enough between 
> multiple drives….

Something like:

- Create a zpool
- Mount zpool to /crashdumps
- Change dumpdir in /etc/rc.conf to be /crashdumps, e.g., echo 
‘dumpdir=/crashdumps’

?
HTH,
-Ngie

PS The issue with lack of space might be the issue that Alan brought up earlier 
with compressed dumps and overly conservative free space checks, or it might be 
the fact that dumpdir (default: /var/crash) is full.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Mark Johnston]

On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> >
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> >   - Store available number of KiB in `available` so it can be more
> > easily queried and compared to ensure that there are enough KiB to
> > store the dump image on disk.
> >   - Print out the reserved space on disk, per `minfree`, so end-users
> > can troubleshoot why check_space(..) is reporting that there isn't
> > enough free space.
> >
> >   MFC after:7 weeks
> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> >   Tested with:  positive/negative cases (see review); make tinderbox
> >   Sponsored by: Dell EMC Isilon
> >   Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

I had held off on it for a while because it was going to conflict with
the work to add encrypted dump support, which of course has finished.

The patch to add compression support is here and should largely still
work:
https://people.freebsd.org/~markj/patches/core-compression/20141110-kern_dump.diff

I've been hesitant about pushing it forward:
- The dump_write* APIs need some simplification after the addition of
  encrypted dump support and support for dumping to 4Kn drives.
- I'm not sure how encryption should compose with compression. It seems
  intuitively obvious that we should compress before encrypting if the
  compression is to be of any use, but I don't know enough to know
  whether the compression might somehow compromise the effectiveness of
  the encryption.

If anyone has some insight on the second of these two points, I'd
appreciate hearing it.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Larry Rosenman]

On 4/14/17, 3:19 PM, "Ngie Cooper (yaneurabeya)"  wrote:


> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> 
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>> 
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier 
diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>> 
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
> 
> Is posible (in the future) to use multiple swaps (on multiple disks)
> for save core dumps?

Multiple swap devices is already handled by savecore(8), if one uses 
fstab(5) or dumpon(8). Otherwise, you must invoke savecore(8) on individual 
devices.

As far as saving to multiple disks is concerned, I would hope that one is 
using a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, 
graid, LSI Fusion’s RAID product line) to stripe and/or mirror the data across 
multiple disks.



Thanks!
-Ngie

How do I use multiple devices to have the system dump on all of my swap?  I got 
a message about not enough space, but there (I think) was enough between 
multiple drives….




___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:14, Slawa Olhovchenkov  wrote:
> 
> On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:
> 
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>> 
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>> 
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>> 
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>> 
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
> 
> Is posible (in the future) to use multiple swaps (on multiple disks)
> for save core dumps?

Multiple swap devices is already handled by savecore(8), if one uses fstab(5) 
or dumpon(8). Otherwise, you must invoke savecore(8) on individual devices.

As far as saving to multiple disks is concerned, I would hope that one is using 
a redundancy capable filesystem (zfs) or RAID-like technology (gmirror, graid, 
LSI Fusion’s RAID product line) to stripe and/or mirror the data across 
multiple disks.

Thanks!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

svn commit: r316941 - head/sys/kern

[Edward Tomasz Napierala]

Author: trasz
Date: Fri Apr 14 20:15:34 2017
New Revision: 316941
URL: https://svnweb.freebsd.org/changeset/base/316941

Log:
  Don't try to write out bufs that have already failed with ENXIO.
  This fixes some panics after disconnecting mounted disks.
  
  Submitted by: imp (slightly different version, which I've then lost)
  Reviewed by:  kib, imp, mckusick
  MFC after:2 weeks
  Differential Revision:https://reviews.freebsd.org/D9674

Modified:
  head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Fri Apr 14 20:15:17 2017(r316940)
+++ head/sys/kern/vfs_bio.c Fri Apr 14 20:15:34 2017(r316941)
@@ -2290,18 +2290,28 @@ brelse(struct buf *bp)
bdirty(bp);
}
if (bp->b_iocmd == BIO_WRITE && (bp->b_ioflags & BIO_ERROR) &&
+   (bp->b_error != ENXIO || !LIST_EMPTY(>b_dep)) &&
!(bp->b_flags & B_INVAL)) {
/*
-* Failed write, redirty.  Must clear BIO_ERROR to prevent
-* pages from being scrapped.
+* Failed write, redirty.  All errors except ENXIO (which
+* means the device is gone) are expected to be potentially
+* transient - underlying media might work if tried again
+* after EIO, and memory might be available after an ENOMEM.
+*
+* Do this also for buffers that failed with ENXIO, but have
+* non-empty dependencies - the soft updates code might need
+* to access the buffer to untangle them.
+*
+* Must clear BIO_ERROR to prevent pages from being scrapped.
 */
bp->b_ioflags &= ~BIO_ERROR;
bdirty(bp);
} else if ((bp->b_flags & (B_NOCACHE | B_INVAL)) ||
(bp->b_ioflags & BIO_ERROR) || (bp->b_bufsize <= 0)) {
/*
-* Either a failed read I/O or we were asked to free or not
-* cache the buffer.
+* Either a failed read I/O, or we were asked to free or not
+* cache the buffer, or we failed to write to a device that's
+* no longer present.
 */
bp->b_flags |= B_INVAL;
if (!LIST_EMPTY(>b_dep))
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316940 - head/sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 20:15:17 2017
New Revision: 316940
URL: https://svnweb.freebsd.org/changeset/base/316940

Log:
  cxgbe/iw_cxgbe: Report the actual values of various parameters as
  configured by the firmware.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/sys/dev/cxgbe/iw_cxgbe/provider.c
  head/sys/dev/cxgbe/iw_cxgbe/qp.c
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/sys/dev/cxgbe/iw_cxgbe/provider.c
==
--- head/sys/dev/cxgbe/iw_cxgbe/provider.c  Fri Apr 14 20:10:18 2017
(r316939)
+++ head/sys/dev/cxgbe/iw_cxgbe/provider.c  Fri Apr 14 20:15:17 2017
(r316940)
@@ -307,6 +307,7 @@ c4iw_query_device(struct ib_device *ibde
 {
struct c4iw_dev *dev = to_c4iw_dev(ibdev);
struct adapter *sc = dev->rdev.adap;
+   const int spg_ndesc = sc->params.sge.spg_len / EQ_ESIZE;
 
CTR3(KTR_IW_CXGBE, "%s ibdev %p, props %p", __func__, ibdev, props);
 
@@ -320,13 +321,15 @@ c4iw_query_device(struct ib_device *ibde
props->vendor_id = pci_get_vendor(sc->dev);
props->vendor_part_id = pci_get_device(sc->dev);
props->max_mr_size = T4_MAX_MR_SIZE;
-   props->max_qp = T4_MAX_NUM_QP;
-   props->max_qp_wr = T4_MAX_QP_DEPTH;
+   props->max_qp = sc->vres.qp.size / 2;
+   props->max_qp_wr = T4_MAX_QP_DEPTH(spg_ndesc);
props->max_sge = T4_MAX_RECV_SGE;
props->max_sge_rd = 1;
-   props->max_qp_rd_atom = c4iw_max_read_depth;
-   props->max_qp_init_rd_atom = c4iw_max_read_depth;
-   props->max_cq = T4_MAX_NUM_CQ;
+   props->max_res_rd_atom = sc->params.max_ird_adapter;
+   props->max_qp_rd_atom = min(sc->params.max_ordird_qp,
+   c4iw_max_read_depth);
+   props->max_qp_init_rd_atom = props->max_qp_rd_atom;
+   props->max_cq = sc->vres.qp.size;
props->max_cqe = T4_MAX_CQ_DEPTH;
props->max_mr = c4iw_num_stags(>rdev);
props->max_pd = T4_MAX_NUM_PD;

Modified: head/sys/dev/cxgbe/iw_cxgbe/qp.c
==
--- head/sys/dev/cxgbe/iw_cxgbe/qp.cFri Apr 14 20:10:18 2017
(r316939)
+++ head/sys/dev/cxgbe/iw_cxgbe/qp.cFri Apr 14 20:15:17 2017
(r316940)
@@ -133,6 +133,7 @@ static int create_qp(struct c4iw_rdev *r
int ret;
int eqsize;
struct wrqe *wr;
+   const int spg_ndesc = sc->params.sge.spg_len / EQ_ESIZE;
 
wq->sq.qid = c4iw_get_qpid(rdev, uctx);
if (!wq->sq.qid)
@@ -214,8 +215,7 @@ static int create_qp(struct c4iw_rdev *r
res->u.sqrq.op = FW_RI_RES_OP_WRITE;
 
/* eqsize is the number of 64B entries plus the status page size. */
-   eqsize = wq->sq.size * T4_SQ_NUM_SLOTS +
-   (sc->params.sge.spg_len / EQ_ESIZE);
+   eqsize = wq->sq.size * T4_SQ_NUM_SLOTS + spg_ndesc;
 
res->u.sqrq.fetchszm_to_iqid = cpu_to_be32(
V_FW_RI_RES_WR_HOSTFCMODE(0) |  /* no host cidx updates */
@@ -237,8 +237,7 @@ static int create_qp(struct c4iw_rdev *r
res->u.sqrq.op = FW_RI_RES_OP_WRITE;
 
/* eqsize is the number of 64B entries plus the status page size. */
-   eqsize = wq->rq.size * T4_RQ_NUM_SLOTS +
-   (sc->params.sge.spg_len / EQ_ESIZE);
+   eqsize = wq->rq.size * T4_RQ_NUM_SLOTS + spg_ndesc;
res->u.sqrq.fetchszm_to_iqid = cpu_to_be32(
V_FW_RI_RES_WR_HOSTFCMODE(0) |  /* no host cidx updates */
V_FW_RI_RES_WR_CPRIO(0) |   /* don't keep in chip cache */
@@ -1523,7 +1522,7 @@ c4iw_create_qp(struct ib_pd *pd, struct 
struct c4iw_create_qp_resp uresp;
int sqsize, rqsize;
struct c4iw_ucontext *ucontext;
-   int ret;
+   int ret, spg_ndesc;
struct c4iw_mm_entry *mm1, *mm2, *mm3, *mm4;
 
CTR2(KTR_IW_CXGBE, "%s ib_pd %p", __func__, pd);
@@ -1541,12 +1540,13 @@ c4iw_create_qp(struct ib_pd *pd, struct 
if (attrs->cap.max_inline_data > T4_MAX_SEND_INLINE)
return ERR_PTR(-EINVAL);
 
+   spg_ndesc = rhp->rdev.adap->params.sge.spg_len / EQ_ESIZE;
rqsize = roundup(attrs->cap.max_recv_wr + 1, 16);
-   if (rqsize > T4_MAX_RQ_SIZE)
+   if (rqsize > T4_MAX_RQ_SIZE(spg_ndesc))
return ERR_PTR(-E2BIG);
 
sqsize = roundup(attrs->cap.max_send_wr + 1, 16);
-   if (sqsize > T4_MAX_SQ_SIZE)
+   if (sqsize > T4_MAX_SQ_SIZE(spg_ndesc))
return ERR_PTR(-E2BIG);
 
ucontext = pd->uobject ? to_c4iw_ucontext(pd->uobject->context) : NULL;
@@ -1556,9 +1556,10 @@ c4iw_create_qp(struct ib_pd *pd, struct 
if (!qhp)
return ERR_PTR(-ENOMEM);
qhp->wq.sq.size = sqsize;
-   qhp->wq.sq.memsize = (sqsize + 1) * sizeof *qhp->wq.sq.queue;
+   qhp->wq.sq.memsize = (sqsize + spg_ndesc) * sizeof *qhp->wq.sq.queue +
+   16 * sizeof(__be64);
qhp->wq.rq.size = rqsize;

Re: svn commit: r316938 - head/sbin/savecore

[Slawa Olhovchenkov]

On Fri, Apr 14, 2017 at 01:49:51PM -0600, Alan Somers wrote:

> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> > Author: ngie
> > Date: Fri Apr 14 19:41:48 2017
> > New Revision: 316938
> > URL: https://svnweb.freebsd.org/changeset/base/316938
> >
> > Log:
> >   savecore: fix space calculation with respect to `minfree` in 
> > check_space(..)
> >
> >   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> > representable data to INT_MAX. Check the values received from
> > strtoll(3), trimming trailing whitespace off the end to maintain
> > POLA.
> >   - Use `KiB` instead of `kB` when describing free space, total space,
> > etc. I am now fully aware of `KiB` being the IEC standard for 1024
> > bytes and `kB` being the IEC standard for 1000 bytes.
> >   - Store available number of KiB in `available` so it can be more
> > easily queried and compared to ensure that there are enough KiB to
> > store the dump image on disk.
> >   - Print out the reserved space on disk, per `minfree`, so end-users
> > can troubleshoot why check_space(..) is reporting that there isn't
> > enough free space.
> >
> >   MFC after:7 weeks
> >   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
> >   Tested with:  positive/negative cases (see review); make tinderbox
> >   Sponsored by: Dell EMC Isilon
> >   Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

Is posible (in the future) to use multiple swaps (on multiple disks)
for save core dumps?
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316939 - head/lib/lib80211

[Adrian Chadd]

Author: adrian
Date: Fri Apr 14 20:10:18 2017
New Revision: 316939
URL: https://svnweb.freebsd.org/changeset/base/316939

Log:
  [lib80211] fix a missing cleanup path.
  
  PR:   lib/218655
  Reported by:  

Modified:
  head/lib/lib80211/lib80211_regdomain.c

Modified: head/lib/lib80211/lib80211_regdomain.c
==
--- head/lib/lib80211/lib80211_regdomain.c  Fri Apr 14 19:41:48 2017
(r316938)
+++ head/lib/lib80211/lib80211_regdomain.c  Fri Apr 14 20:10:18 2017
(r316939)
@@ -569,6 +569,7 @@ cleanup_bands(netband_head *head)
nb = LIST_FIRST(head);
if (nb == NULL)
break;
+   LIST_REMOVE(nb, next);
free(nb);
}
 }
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:04, Ngie Cooper (yaneurabeya)  
> wrote:
> 
> 
>> On Apr 14, 2017, at 13:02, Alan Somers  wrote:
> 
> ...
> 
>> Right: check_size() needs to know the on-disk size of the core dump.
>> But it doesn't have any way to know the on-disk size until after it's
>> written everything to disk.  That's why I simply disabled the check at
>> $WORK.  Another possibility would be for savecore to ignore the
>> up-front check, but handle ENOSPC by deleting the fragmentary vmcore
>> file.  I never tried to implement that.
> 
> Let me dig through the gzip format a bit. There should be a reasonably good 
> upper bound approximation that we can use for compressed mini dumps.

Also: deleting truncated dumps sounds great to me: their value when truncated 
is pretty low from what I remember.
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 13:02, Alan Somers  wrote:

...

> Right: check_size() needs to know the on-disk size of the core dump.
> But it doesn't have any way to know the on-disk size until after it's
> written everything to disk.  That's why I simply disabled the check at
> $WORK.  Another possibility would be for savecore to ignore the
> up-front check, but handle ENOSPC by deleting the fragmentary vmcore
> file.  I never tried to implement that.

Let me dig through the gzip format a bit. There should be a reasonably good 
upper bound approximation that we can use for compressed minidumps.
Thanks!
-Ngie



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 1:58 PM, Ngie Cooper (yaneurabeya)
 wrote:
>
>> On Apr 14, 2017, at 12:49, Alan Somers  wrote:
>>
>> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>>> Author: ngie
>>> Date: Fri Apr 14 19:41:48 2017
>>> New Revision: 316938
>>> URL: https://svnweb.freebsd.org/changeset/base/316938
>>>
>>> Log:
>>>  savecore: fix space calculation with respect to `minfree` in 
>>> check_space(..)
>>>
>>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>>representable data to INT_MAX. Check the values received from
>>>strtoll(3), trimming trailing whitespace off the end to maintain
>>>POLA.
>>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>>bytes and `kB` being the IEC standard for 1000 bytes.
>>>  - Store available number of KiB in `available` so it can be more
>>>easily queried and compared to ensure that there are enough KiB to
>>>store the dump image on disk.
>>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>>can troubleshoot why check_space(..) is reporting that there isn't
>>>enough free space.
>>>
>>>  MFC after:7 weeks
>>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>>  Tested with:  positive/negative cases (see review); make tinderbox
>>>  Sponsored by: Dell EMC Isilon
>>>  Differential Revision:D10379
>>
>> The free space calculation is still uselessly conservative, because it
>> doesn't account for the fact that core dumps will always be either
>> spare or compressed.  The result is that savecore will frequently
>> refuse to save corefiles even when there's plenty of space.  I
>> proposed removing the space check altogether in
>> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
>> after the compressed core dump feature was merged, because then mostly
>> accurate space checks will be possible.  AFAIK the compressed core
>> dump feature still hasn't been finished.
>
> That CR concerns me.
>
> The whole being able to check how much space we have on disk before dumping 
> is a wonderful tool because it prevents users from having /var/crash be full 
> of truncated full dumps, text dumps, and/or mini dumps.
>
> The real problem that you’re bringing up is that the size being passed in to 
> check_size(..) is wrong. It needs to be the on-disk size (which can be a 
> pessimization from gzip/zlib), not the in-memory size of all of the dumped 
> pages.
>
> Thanks for the note — this helps me with another related issue at $work that 
> will likely prevent me from being able to write proper tests to test out 
> panic dumps on OneFS :).
>
> Cheers!
> -Ngie

Right: check_size() needs to know the on-disk size of the core dump.
But it doesn't have any way to know the on-disk size until after it's
written everything to disk.  That's why I simply disabled the check at
$WORK.  Another possibility would be for savecore to ignore the
up-front check, but handle ENOSPC by deleting the fragmentary vmcore
file.  I never tried to implement that.
-Alan
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316938 - head/sbin/savecore

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 12:49, Alan Somers  wrote:
> 
> On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
>> Author: ngie
>> Date: Fri Apr 14 19:41:48 2017
>> New Revision: 316938
>> URL: https://svnweb.freebsd.org/changeset/base/316938
>> 
>> Log:
>>  savecore: fix space calculation with respect to `minfree` in check_space(..)
>> 
>>  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
>>representable data to INT_MAX. Check the values received from
>>strtoll(3), trimming trailing whitespace off the end to maintain
>>POLA.
>>  - Use `KiB` instead of `kB` when describing free space, total space,
>>etc. I am now fully aware of `KiB` being the IEC standard for 1024
>>bytes and `kB` being the IEC standard for 1000 bytes.
>>  - Store available number of KiB in `available` so it can be more
>>easily queried and compared to ensure that there are enough KiB to
>>store the dump image on disk.
>>  - Print out the reserved space on disk, per `minfree`, so end-users
>>can troubleshoot why check_space(..) is reporting that there isn't
>>enough free space.
>> 
>>  MFC after:7 weeks
>>  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>>  Tested with:  positive/negative cases (see review); make tinderbox
>>  Sponsored by: Dell EMC Isilon
>>  Differential Revision:D10379
> 
> The free space calculation is still uselessly conservative, because it
> doesn't account for the fact that core dumps will always be either
> spare or compressed.  The result is that savecore will frequently
> refuse to save corefiles even when there's plenty of space.  I
> proposed removing the space check altogether in
> https://reviews.freebsd.org/D2587.  However, I agreed to wait until
> after the compressed core dump feature was merged, because then mostly
> accurate space checks will be possible.  AFAIK the compressed core
> dump feature still hasn't been finished.

That CR concerns me.

The whole being able to check how much space we have on disk before dumping is 
a wonderful tool because it prevents users from having /var/crash be full of 
truncated full dumps, text dumps, and/or mini dumps.

The real problem that you’re bringing up is that the size being passed in to 
check_size(..) is wrong. It needs to be the on-disk size (which can be a 
pessimization from gzip/zlib), not the in-memory size of all of the dumped 
pages.

Thanks for the note — this helps me with another related issue at $work that 
will likely prevent me from being able to write proper tests to test out panic 
dumps on OneFS :).

Cheers!
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316938 - head/sbin/savecore

[Alan Somers]

On Fri, Apr 14, 2017 at 1:41 PM, Ngie Cooper  wrote:
> Author: ngie
> Date: Fri Apr 14 19:41:48 2017
> New Revision: 316938
> URL: https://svnweb.freebsd.org/changeset/base/316938
>
> Log:
>   savecore: fix space calculation with respect to `minfree` in check_space(..)
>
>   - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
> representable data to INT_MAX. Check the values received from
> strtoll(3), trimming trailing whitespace off the end to maintain
> POLA.
>   - Use `KiB` instead of `kB` when describing free space, total space,
> etc. I am now fully aware of `KiB` being the IEC standard for 1024
> bytes and `kB` being the IEC standard for 1000 bytes.
>   - Store available number of KiB in `available` so it can be more
> easily queried and compared to ensure that there are enough KiB to
> store the dump image on disk.
>   - Print out the reserved space on disk, per `minfree`, so end-users
> can troubleshoot why check_space(..) is reporting that there isn't
> enough free space.
>
>   MFC after:7 weeks
>   Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
>   Tested with:  positive/negative cases (see review); make tinderbox
>   Sponsored by: Dell EMC Isilon
>   Differential Revision:D10379

The free space calculation is still uselessly conservative, because it
doesn't account for the fact that core dumps will always be either
spare or compressed.  The result is that savecore will frequently
refuse to save corefiles even when there's plenty of space.  I
proposed removing the space check altogether in
https://reviews.freebsd.org/D2587.  However, I agreed to wait until
after the compressed core dump feature was merged, because then mostly
accurate space checks will be possible.  AFAIK the compressed core
dump feature still hasn't been finished.

-Alan
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Ngie Cooper (yaneurabeya)]

> On Apr 14, 2017, at 12:46, Peter Wemm  wrote:
> 
> On Friday, April 14, 2017 12:41:52 PM Maxim Sobolev wrote:
>> Thanks, Peter. I will try to look into this asap.
> 
> I don't understand what is going on yet. Presumably there must be other
> changes in play that affect udp/select sometime between the original 2015
> change and this. The syslogd -s code is Old(TM).  I'm also wondering whether
> the -s code even works at all since the 2015 / r285910 change...

syslogd has been refactored a lot on ^/head. I don’t think it’s safe to say 
that the ^/head and ^/stable/11 and earlier copies will function the same.
Thanks,
-Ngie


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Thanks, Peter. I will try to look into this asap.

-Max

On Apr 14, 2017 12:32 PM, "Peter Wemm"  wrote:

> On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> > On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > > Author: sobomax
> > > Date: Fri Apr 14 17:23:28 2017
> > > New Revision: 316874
> > > URL: https://svnweb.freebsd.org/changeset/base/316874
> > >
> > > Log:
> > >   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to
> be
> > >
> > > returned by the shutdown(2) system call. This ability has been lost as
> > > part
> > > of the svn revision 285910.
> > >
> > >   Reviewed by:  ed, rwatson, glebius, hiren
> > >   MFC after:2 weeks
> > >   Differential Revision:https://reviews.freebsd.org/D10351
> >
> > This appears to have broken syslogd and had a major change in behavior
> with
> > regards to select(2).
> >
> > If you run syslogd with the -s flag (which is default), it now spins at
> 100%
> > cpu as all the shutdown sockets now return readable from select.
> >
> > Old releases / jails also manifest this behavor.  If it wasn't for losing
> > the ability to run old branch binaries I'd suggest changing syslogd
> > instead, but we depend on this in the cluster and I expect others do too.
> >
> > I'm not 100% certain that this change is the culprit but a heads-up can't
> > hurt. I'll try reverting it on the freebsd cluster next, after fixing the
> > broken auditing changes.
> >
> > -Peter
>
> I can confirm that reverting r316874 fixes syslogd and backwards
> compatability
> with old branches.
>
> --
> Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com;
> KI6FJV
> UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316938 - head/sbin/savecore

[Ngie Cooper]

Author: ngie
Date: Fri Apr 14 19:41:48 2017
New Revision: 316938
URL: https://svnweb.freebsd.org/changeset/base/316938

Log:
  savecore: fix space calculation with respect to `minfree` in check_space(..)
  
  - Use strtoll(3) instead of atoi(3), because atoi(3) limits the
representable data to INT_MAX. Check the values received from
strtoll(3), trimming trailing whitespace off the end to maintain
POLA.
  - Use `KiB` instead of `kB` when describing free space, total space,
etc. I am now fully aware of `KiB` being the IEC standard for 1024
bytes and `kB` being the IEC standard for 1000 bytes.
  - Store available number of KiB in `available` so it can be more
easily queried and compared to ensure that there are enough KiB to
store the dump image on disk.
  - Print out the reserved space on disk, per `minfree`, so end-users
can troubleshoot why check_space(..) is reporting that there isn't
enough free space.
  
  MFC after:7 weeks
  Reviewed by:  Anton Rang  (earlier diff), cem (earlier diff)
  Tested with:  positive/negative cases (see review); make tinderbox
  Sponsored by: Dell EMC Isilon
  Differential Revision:D10379

Modified:
  head/sbin/savecore/savecore.c

Modified: head/sbin/savecore/savecore.c
==
--- head/sbin/savecore/savecore.c   Fri Apr 14 19:18:50 2017
(r316937)
+++ head/sbin/savecore/savecore.c   Fri Apr 14 19:41:48 2017
(r316938)
@@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -278,7 +279,7 @@ static int
 check_space(const char *savedir, off_t dumpsize, int bounds)
 {
FILE *fp;
-   off_t minfree, spacefree, totfree, needed;
+   off_t available, minfree, spacefree, totfree, needed;
struct statfs fsbuf;
char buf[100];
 
@@ -294,19 +295,37 @@ check_space(const char *savedir, off_t d
else {
if (fgets(buf, sizeof(buf), fp) == NULL)
minfree = 0;
-   else
-   minfree = atoi(buf);
+   else {
+   char *endp;
+
+   errno = 0;
+   minfree = strtoll(buf, , 10);
+   if (minfree == 0 && errno != 0)
+   minfree = -1;
+   else {
+   while (*endp != '\0' && isspace(*endp))
+   endp++;
+   if (*endp != '\0' || minfree < 0)
+   minfree = -1;
+   }
+   if (minfree < 0)
+   syslog(LOG_WARNING,
+   "`minfree` didn't contain a valid size "
+   "(`%s`). Defaulting to 0", buf);
+   }
(void)fclose(fp);
}
 
+   available = minfree > 0 ? spacefree - minfree : totfree;
needed = dumpsize / 1024 + 2;   /* 2 for info file */
needed -= saved_dump_size(bounds);
-   if ((minfree > 0 ? spacefree : totfree) - needed < minfree) {
+   if (available < needed) {
syslog(LOG_WARNING,
-   "no dump: not enough free space on device (%lldkB "
-   "available; need at least %lldkB)",
-   (long long)(minfree > 0 ? spacefree : totfree),
-   (long long)needed);
+   "no dump: not enough free space on device (need at least "
+   "%jdKiB for dump; %jdKiB available; %jdKiB reserved)",
+   (intmax_t)needed,
+   (intmax_t)available + minfree,
+   (intmax_t)minfree);
return (0);
}
if (spacefree - needed < 0)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 11:49:26 AM Peter Wemm wrote:
> On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> > Author: sobomax
> > Date: Fri Apr 14 17:23:28 2017
> > New Revision: 316874
> > URL: https://svnweb.freebsd.org/changeset/base/316874
> > 
> > Log:
> >   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to be
> > 
> > returned by the shutdown(2) system call. This ability has been lost as
> > part
> > of the svn revision 285910.
> > 
> >   Reviewed by:  ed, rwatson, glebius, hiren
> >   MFC after:2 weeks
> >   Differential Revision:https://reviews.freebsd.org/D10351
> 
> This appears to have broken syslogd and had a major change in behavior with
> regards to select(2).
> 
> If you run syslogd with the -s flag (which is default), it now spins at 100%
> cpu as all the shutdown sockets now return readable from select.
> 
> Old releases / jails also manifest this behavor.  If it wasn't for losing
> the ability to run old branch binaries I'd suggest changing syslogd
> instead, but we depend on this in the cluster and I expect others do too.
> 
> I'm not 100% certain that this change is the culprit but a heads-up can't
> hurt. I'll try reverting it on the freebsd cluster next, after fixing the
> broken auditing changes.
> 
> -Peter

I can confirm that reverting r316874 fixes syslogd and backwards compatability 
with old branches.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

svn commit: r316937 - in head: contrib/ofed/libcxgb4/src sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 19:18:50 2017
New Revision: 316937
URL: https://svnweb.freebsd.org/changeset/base/316937

Log:
  cxgbe/iw_cxgbe: Report accurate page_size_cap in ib_query_device.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/contrib/ofed/libcxgb4/src/t4.h
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/contrib/ofed/libcxgb4/src/t4.h
==
--- head/contrib/ofed/libcxgb4/src/t4.h Fri Apr 14 19:15:31 2017
(r316936)
+++ head/contrib/ofed/libcxgb4/src/t4.h Fri Apr 14 19:18:50 2017
(r316937)
@@ -102,7 +102,7 @@
 #define T4_MAX_CQ_DEPTH (T4_MAX_IQ_SIZE - 1)
 #define T4_MAX_NUM_STAG (1<<15)
 #define T4_MAX_MR_SIZE (~0ULL - 1)
-#define T4_PAGESIZE_MASK 0x000  /* 4KB-128MB */
+#define T4_PAGESIZE_MASK 0x000  /* 4KB-8TB */
 #define T4_STAG_UNSET 0x
 #define T4_FW_MAJ 0
 

Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
==
--- head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:15:31 2017
(r316936)
+++ head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:18:50 2017
(r316937)
@@ -70,7 +70,7 @@
 #define T4_MAX_QP_DEPTH (T4_MAX_RQ_SIZE - 1)
 #define T4_MAX_CQ_DEPTH (T4_MAX_IQ_SIZE - 1)
 #define T4_MAX_MR_SIZE (~0ULL - 1)
-#define T4_PAGESIZE_MASK 0x000  /* 4KB-128MB */
+#define T4_PAGESIZE_MASK 0x000  /* 4KB-8TB */
 #define T4_STAG_UNSET 0x
 #define T4_FW_MAJ 0
 #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316936 - head/sys/dev/cxgbe/iw_cxgbe

[Navdeep Parhar]

Author: np
Date: Fri Apr 14 19:15:31 2017
New Revision: 316936
URL: https://svnweb.freebsd.org/changeset/base/316936

Log:
  cxgbe/iw_cxgbe: hw supports 64K (not 32K) Protection Domains.
  
  MFC after:3 days
  Sponsored by: Chelsio Communications

Modified:
  head/sys/dev/cxgbe/iw_cxgbe/t4.h

Modified: head/sys/dev/cxgbe/iw_cxgbe/t4.h
==
--- head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 18:56:00 2017
(r316935)
+++ head/sys/dev/cxgbe/iw_cxgbe/t4.hFri Apr 14 19:15:31 2017
(r316936)
@@ -61,7 +61,7 @@
 
 #define T4_MAX_NUM_QP (1<<16)
 #define T4_MAX_NUM_CQ (1<<15)
-#define T4_MAX_NUM_PD (1<<15)
+#define T4_MAX_NUM_PD 65536
 #define T4_EQ_STATUS_ENTRIES (L1_CACHE_BYTES > 64 ? 2 : 1)
 #define T4_MAX_EQ_SIZE (65520 - T4_EQ_STATUS_ENTRIES)
 #define T4_MAX_IQ_SIZE (65520 - 1)
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316874 - head/sys/kern

[Peter Wemm]

On Friday, April 14, 2017 05:23:28 PM Maxim Sobolev wrote:
> Author: sobomax
> Date: Fri Apr 14 17:23:28 2017
> New Revision: 316874
> URL: https://svnweb.freebsd.org/changeset/base/316874
> 
> Log:
>   Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to be
> returned by the shutdown(2) system call. This ability has been lost as part
> of the svn revision 285910.
> 
>   Reviewed by:ed, rwatson, glebius, hiren
>   MFC after:  2 weeks
>   Differential Revision:  https://reviews.freebsd.org/D10351

This appears to have broken syslogd and had a major change in behavior with 
regards to select(2).

If you run syslogd with the -s flag (which is default), it now spins at 100% 
cpu as all the shutdown sockets now return readable from select.

Old releases / jails also manifest this behavor.  If it wasn't for losing the 
ability to run old branch binaries I'd suggest changing syslogd instead, but 
we depend on this in the cluster and I expect others do too.

I'm not 100% certain that this change is the culprit but a heads-up can't 
hurt. I'll try reverting it on the freebsd cluster next, after fixing the 
broken auditing changes.

-Peter
-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.

svn commit: r316878 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Apr 14 17:38:43 2017
New Revision: 316878
URL: https://svnweb.freebsd.org/changeset/base/316878

Log:
  Oops, the previous revision was missing the update of the shift variable.

Modified:
  head/sys/dev/syscons/scvgarndr.c

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cFri Apr 14 17:27:09 2017
(r316877)
+++ head/sys/dev/syscons/scvgarndr.cFri Apr 14 17:38:43 2017
(r316878)
@@ -1030,7 +1030,7 @@ draw_pxlmouse_planar(scr_stat *scp, int 
int line_width;
int xoff, yoff;
int ymax;
-   u_short m;
+   uint32_t m;
int i, j, k;
uint8_t m1;
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316874 - head/sys/kern

[Maxim Sobolev]

Author: sobomax
Date: Fri Apr 14 17:23:28 2017
New Revision: 316874
URL: https://svnweb.freebsd.org/changeset/base/316874

Log:
  Restore ability to shutdown DGRAM sockets, still forcing ENOTCONN to be 
returned
  by the shutdown(2) system call. This ability has been lost as part of the svn
  revision 285910.
  
  Reviewed by:  ed, rwatson, glebius, hiren
  MFC after:2 weeks
  Differential Revision:https://reviews.freebsd.org/D10351

Modified:
  head/sys/kern/uipc_socket.c

Modified: head/sys/kern/uipc_socket.c
==
--- head/sys/kern/uipc_socket.c Fri Apr 14 17:22:54 2017(r316873)
+++ head/sys/kern/uipc_socket.c Fri Apr 14 17:23:28 2017(r316874)
@@ -2343,13 +2343,27 @@ int
 soshutdown(struct socket *so, int how)
 {
struct protosw *pr = so->so_proto;
-   int error;
+   int error, soerror_enotconn;
 
if (!(how == SHUT_RD || how == SHUT_WR || how == SHUT_RDWR))
return (EINVAL);
+
+   soerror_enotconn = 0;
if ((so->so_state &
-   (SS_ISCONNECTED | SS_ISCONNECTING | SS_ISDISCONNECTING)) == 0)
-   return (ENOTCONN);
+   (SS_ISCONNECTED | SS_ISCONNECTING | SS_ISDISCONNECTING)) == 0) {
+   /*
+* POSIX mandates us to return ENOTCONN when shutdown(2) is
+* invoked on a datagram sockets, however historically we would
+* actually tear socket down. This is known to be leveraged by
+* some applications to unblock process waiting in recvXXX(2)
+* by other process that it shares that socket with. Try to meet
+* both backward-compatibility and POSIX requirements by forcing
+* ENOTCONN but still asking protocol to perform pru_shutdown().
+*/
+   if (so->so_type != SOCK_DGRAM)
+   return (ENOTCONN);
+   soerror_enotconn = 1;
+   }
 
CURVNET_SET(so->so_vnet);
if (pr->pr_usrreqs->pru_flush != NULL)
@@ -2360,11 +2374,12 @@ soshutdown(struct socket *so, int how)
error = (*pr->pr_usrreqs->pru_shutdown)(so);
wakeup(>so_timeo);
CURVNET_RESTORE();
-   return (error);
+   return ((error == 0 && soerror_enotconn) ? ENOTCONN : error);
}
wakeup(>so_timeo);
CURVNET_RESTORE();
-   return (0);
+
+   return (soerror_enotconn ? ENOTCONN : 0);
 }
 
 void
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316869 - head/sys/geom/mirror

[Mark Johnston]

Author: markj
Date: Fri Apr 14 17:13:57 2017
New Revision: 316869
URL: https://svnweb.freebsd.org/changeset/base/316869

Log:
  Rename two gmirror state flags to make their meanings slightly clearer.
  
  No functional change.
  
  MFC after:2 weeks
  Sponsored by: Dell EMC Isilon

Modified:
  head/sys/geom/mirror/g_mirror.c
  head/sys/geom/mirror/g_mirror.h

Modified: head/sys/geom/mirror/g_mirror.c
==
--- head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:13:49 2017
(r316868)
+++ head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:13:57 2017
(r316869)
@@ -1825,7 +1825,7 @@ g_mirror_try_destroy(struct g_mirror_sof
}
sc->sc_geom->softc = NULL;
sc->sc_sync.ds_geom->softc = NULL;
-   if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_WAIT) != 0) {
+   if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DRAIN) != 0) {
g_topology_unlock();
G_MIRROR_DEBUG(4, "%s: Waking up %p.", __func__,
>sc_worker);
@@ -2911,8 +2911,8 @@ g_mirror_destroy_delayed(void *arg, int 
sx_xlock(>sc_lock);
KASSERT((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROY) == 0,
("DESTROY flag set on %s.", sc->sc_name));
-   KASSERT((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROYING) != 0,
-   ("DESTROYING flag not set on %s.", sc->sc_name));
+   KASSERT((sc->sc_flags & G_MIRROR_DEVICE_FLAG_CLOSEWAIT) != 0,
+   ("CLOSEWAIT flag not set on %s.", sc->sc_name));
G_MIRROR_DEBUG(1, "Destroying %s (delayed).", sc->sc_name);
error = g_mirror_destroy(sc, G_MIRROR_DESTROY_SOFT);
if (error != 0) {
@@ -2939,7 +2939,7 @@ g_mirror_access(struct g_provider *pp, i
g_topology_unlock();
sx_xlock(>sc_lock);
if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROY) != 0 ||
-   (sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROYING) != 0 ||
+   (sc->sc_flags & G_MIRROR_DEVICE_FLAG_CLOSEWAIT) != 0 ||
LIST_EMPTY(>sc_disks)) {
if (acr > 0 || acw > 0 || ace > 0)
error = ENXIO;
@@ -2948,7 +2948,7 @@ g_mirror_access(struct g_provider *pp, i
sc->sc_provider_open += acr + acw + ace;
if (pp->acw + acw == 0)
g_mirror_idle(sc, 0);
-   if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROYING) != 0 &&
+   if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_CLOSEWAIT) != 0 &&
sc->sc_provider_open == 0)
g_post_event(g_mirror_destroy_delayed, sc, M_WAITOK, sc, NULL);
 end:
@@ -3068,7 +3068,7 @@ g_mirror_destroy(struct g_mirror_softc *
g_mirror_sync_stop(disk, 1);
}
}
-   sc->sc_flags |= G_MIRROR_DEVICE_FLAG_DESTROYING;
+   sc->sc_flags |= G_MIRROR_DEVICE_FLAG_CLOSEWAIT;
return (EBUSY);
case G_MIRROR_DESTROY_HARD:
G_MIRROR_DEBUG(1, "Device %s is still open, so it "
@@ -3079,7 +3079,7 @@ g_mirror_destroy(struct g_mirror_softc *
if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROY) != 0)
return (0);
sc->sc_flags |= G_MIRROR_DEVICE_FLAG_DESTROY;
-   sc->sc_flags |= G_MIRROR_DEVICE_FLAG_WAIT;
+   sc->sc_flags |= G_MIRROR_DEVICE_FLAG_DRAIN;
G_MIRROR_DEBUG(4, "%s: Waking up %p.", __func__, sc);
sx_xunlock(>sc_lock);
mtx_lock(>sc_queue_mtx);

Modified: head/sys/geom/mirror/g_mirror.h
==
--- head/sys/geom/mirror/g_mirror.h Fri Apr 14 17:13:49 2017
(r316868)
+++ head/sys/geom/mirror/g_mirror.h Fri Apr 14 17:13:57 2017
(r316869)
@@ -157,8 +157,8 @@ struct g_mirror_event {
 };
 
 #defineG_MIRROR_DEVICE_FLAG_DESTROY0x0100ULL
-#defineG_MIRROR_DEVICE_FLAG_WAIT   0x0200ULL
-#defineG_MIRROR_DEVICE_FLAG_DESTROYING 0x0400ULL
+#defineG_MIRROR_DEVICE_FLAG_DRAIN  0x0200ULL
+#defineG_MIRROR_DEVICE_FLAG_CLOSEWAIT  0x0400ULL
 #defineG_MIRROR_DEVICE_FLAG_TASTING0x0800ULL
 #defineG_MIRROR_DEVICE_FLAG_WIPE   0x1000ULL
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316867 - head/sys/geom/mirror

[Mark Johnston]

Author: markj
Date: Fri Apr 14 17:08:37 2017
New Revision: 316867
URL: https://svnweb.freebsd.org/changeset/base/316867

Log:
  Don't set the mirror GEOM softc to NULL in g_mirror_destroy().
  
  At this point we have not rendezvous'ed with the mirror worker thread, and
  I/O may still be in flight. Various I/O completion paths expect to be able
  to obtain a reference to the mirror softc from the GEOM, so setting it to
  NULL may result in various NULL pointer dereferences if the mirror is
  stopped with -f or the kernel is shut down while a mirror is
  synchronizing. The worker thread will clear the softc pointer before
  exiting.
  
  Tested by:pho
  MFC after:2 weeks
  Sponsored by: Dell EMC Isilon

Modified:
  head/sys/geom/mirror/g_mirror.c

Modified: head/sys/geom/mirror/g_mirror.c
==
--- head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:03:32 2017
(r316866)
+++ head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:08:37 2017
(r316867)
@@ -3076,15 +3076,8 @@ g_mirror_destroy(struct g_mirror_softc *
}
}
 
-   g_topology_lock();
-   if (sc->sc_geom->softc == NULL) {
-   g_topology_unlock();
+   if ((sc->sc_flags & G_MIRROR_DEVICE_FLAG_DESTROY) != 0)
return (0);
-   }
-   sc->sc_geom->softc = NULL;
-   sc->sc_sync.ds_geom->softc = NULL;
-   g_topology_unlock();
-
sc->sc_flags |= G_MIRROR_DEVICE_FLAG_DESTROY;
sc->sc_flags |= G_MIRROR_DEVICE_FLAG_WAIT;
G_MIRROR_DEBUG(4, "%s: Waking up %p.", __func__, sc);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316866 - head/sys/geom/mirror

[Mark Johnston]

Author: markj
Date: Fri Apr 14 17:03:32 2017
New Revision: 316866
URL: https://svnweb.freebsd.org/changeset/base/316866

Log:
  Check for a provider error before enqueuing mirror I/O.
  
  We are otherwise susceptible to a race with a concurrent teardown of the
  mirror provider, causing the I/O to be left uncompleted after the mirror
  started withering.
  
  Tested by:pho
  MFC after:2 weeks
  Sponsored by: Dell EMC Isilon

Modified:
  head/sys/geom/mirror/g_mirror.c

Modified: head/sys/geom/mirror/g_mirror.c
==
--- head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:02:24 2017
(r316865)
+++ head/sys/geom/mirror/g_mirror.c Fri Apr 14 17:03:32 2017
(r316866)
@@ -1176,6 +1176,11 @@ g_mirror_start(struct bio *bp)
return;
}
mtx_lock(>sc_queue_mtx);
+   if (bp->bio_to->error != 0) {
+   mtx_unlock(>sc_queue_mtx);
+   g_io_deliver(bp, bp->bio_to->error);
+   return;
+   }
bioq_insert_tail(>sc_queue, bp);
mtx_unlock(>sc_queue_mtx);
G_MIRROR_DEBUG(4, "%s: Waking up %p.", __func__, sc);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316865 - head/sys/dev/syscons

[Bruce Evans]

Author: bde
Date: Fri Apr 14 17:02:24 2017
New Revision: 316865
URL: https://svnweb.freebsd.org/changeset/base/316865

Log:
  Adjust shifting so that cursor widths up to 17 (was 9) work in vga planar
  mode.
  
  Direct mode always supported widths up to 32, except for its hard-coded
  16s matching the pixmap size.  Text mode is still limited to 9 its 2x2
  character cell method and missing adjustments for the gap between
  characters, if any.
  
  Cursor heights can be almost anything in graphics modes.

Modified:
  head/sys/dev/syscons/scvgarndr.c

Modified: head/sys/dev/syscons/scvgarndr.c
==
--- head/sys/dev/syscons/scvgarndr.cFri Apr 14 17:01:00 2017
(r316864)
+++ head/sys/dev/syscons/scvgarndr.cFri Apr 14 17:02:24 2017
(r316865)
@@ -1045,9 +1045,9 @@ draw_pxlmouse_planar(scr_stat *scp, int 
outw(GDCIDX, 0x0803);   /* data rotate/function select (and) */
p = scp->sc->adp->va_window + line_width*y + x/8;
for (i = y, j = 0; i < ymax; ++i, ++j) {
-   m = ~((mouse_and_mask[j] & ~mouse_or_mask[j]) >> xoff);
-   for (k = 0; k < 2; ++k) {
-   m1 = m >> (8 * (1 - k));
+   m = ~((mouse_and_mask[j] & ~mouse_or_mask[j]) << 8 >> xoff);
+   for (k = 0; k < 3; ++k) {
+   m1 = m >> (8 * (2 - k));
if (m1 != 0xff && x + 8 * k < scp->xpixel) {
readb(p + k);
writeb(p + k, m1);
@@ -1058,9 +1058,9 @@ draw_pxlmouse_planar(scr_stat *scp, int 
outw(GDCIDX, 0x1003);   /* data rotate/function select (or) */
p = scp->sc->adp->va_window + line_width*y + x/8;
for (i = y, j = 0; i < ymax; ++i, ++j) {
-   m = mouse_or_mask[j] >> xoff;
-   for (k = 0; k < 2; ++k) {
-   m1 = m >> (8 * (1 - k));
+   m = mouse_or_mask[j] << 8 >> xoff;
+   for (k = 0; k < 3; ++k) {
+   m1 = m >> (8 * (2 - k));
if (m1 != 0 && x + 8 * k < scp->xpixel) {
readb(p + k);
writeb(p + k, m1);
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316859 - head/sys/geom/mirror

[Mark Johnston]

Author: markj
Date: Fri Apr 14 16:54:50 2017
New Revision: 316859
URL: https://svnweb.freebsd.org/changeset/base/316859

Log:
  Stop mirror synchronization before draining the I/O queue.
  
  Regular I/O requests may be blocked by concurrent synchronization requests
  targeted to the same LBAs, in which case they are moved to a holding queue
  until the conflicting I/O completes. We therefore want to stop
  synchronization before completing pending I/O in g_mirror_destroy_provider()
  since this ensures that blocked I/O requests are completed as well.
  
  Tested by:pho
  MFC after:2 weeks
  Sponsored by: Dell EMC Isilon

Modified:
  head/sys/geom/mirror/g_mirror.c

Modified: head/sys/geom/mirror/g_mirror.c
==
--- head/sys/geom/mirror/g_mirror.c Fri Apr 14 16:53:34 2017
(r316858)
+++ head/sys/geom/mirror/g_mirror.c Fri Apr 14 16:54:50 2017
(r316859)
@@ -2170,6 +2170,11 @@ g_mirror_destroy_provider(struct g_mirro
KASSERT(sc->sc_provider != NULL, ("NULL provider (device=%s).",
sc->sc_name));
 
+   LIST_FOREACH(disk, >sc_disks, d_next) {
+   if (disk->d_state == G_MIRROR_DISK_STATE_SYNCHRONIZING)
+   g_mirror_sync_stop(disk, 1);
+   }
+
g_topology_lock();
g_error_provider(sc->sc_provider, ENXIO);
mtx_lock(>sc_queue_mtx);
@@ -2193,10 +2198,6 @@ g_mirror_destroy_provider(struct g_mirro
sc->sc_provider = NULL;
G_MIRROR_DEBUG(0, "Device %s: provider destroyed.", sc->sc_name);
g_topology_unlock();
-   LIST_FOREACH(disk, >sc_disks, d_next) {
-   if (disk->d_state == G_MIRROR_DISK_STATE_SYNCHRONIZING)
-   g_mirror_sync_stop(disk, 1);
-   }
 }
 
 static void
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316856 - head/cddl/contrib/opensolaris/cmd/zdb

[Alan Somers]

Author: asomers
Date: Fri Apr 14 16:30:37 2017
New Revision: 316856
URL: https://svnweb.freebsd.org/changeset/base/316856

Log:
  MFV 316855
  
  7900 zdb shouldn't print the path of a znode at verbosity < 5
  
  Reviewed by: Paul Dagnelie 
  Reviewed by: Matt Ahrens 
  Approved by: Dan McDonald 
  Author: Alan Somers 
  
  illumos/illumos-gate@e548d2fa41d1baa06662ed9abbb8bcec86e27dd9
  https://www.illumos.org/issues/7900
  
  MFC after:3 weeks
  Sponsored by: Spectra Logic Corp

Modified:
  head/cddl/contrib/opensolaris/cmd/zdb/zdb.c

Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c
==
--- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Fri Apr 14 16:18:53 2017
(r316855)
+++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Fri Apr 14 16:30:37 2017
(r316856)
@@ -1718,23 +1718,19 @@ dump_znode(objset_t *os, uint64_t object
return;
}
 
-   error = zfs_obj_to_path(os, object, path, sizeof (path));
-   if (error != 0) {
-   (void) snprintf(path, sizeof (path), "\?\?\?",
-   (u_longlong_t)object);
-   }
-   if (dump_opt['d'] < 3) {
-   (void) printf("\t%s\n", path);
-   (void) sa_handle_destroy(hdl);
-   return;
-   }
-
z_crtime = (time_t)crtm[0];
z_atime = (time_t)acctm[0];
z_mtime = (time_t)modtm[0];
z_ctime = (time_t)chgtm[0];
 
-   (void) printf("\tpath   %s\n", path);
+   if (dump_opt['d'] > 4) {
+   error = zfs_obj_to_path(os, object, path, sizeof (path));
+   if (error != 0) {
+   (void) snprintf(path, sizeof (path),
+   "\?\?\?", (u_longlong_t)object);
+   }
+   (void) printf("\tpath   %s\n", path);
+   }
dump_uidgid(os, uid, gid);
(void) printf("\tatime  %s", ctime(_atime));
(void) printf("\tmtime  %s", ctime(_mtime));
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316826 - head/sys/netpfil/ipfw/nat64

[Conrad Meyer]

Ian,

I don't know about "guaranteed," but it certainly looks like the
htonl/bswap macros intend to avoid double-evaluating macro parameters.

On Fri, Apr 14, 2017 at 8:53 AM, Ian Lepore  wrote:
> On Fri, 2017-04-14 at 08:32 -0700, Conrad Meyer wrote:
>> On Fri, Apr 14, 2017 at 4:58 AM, Andrey V. Elsukov 
>> wrote:
>> >
>> > Author: ae
>> > Date: Fri Apr 14 11:58:41 2017
>> > New Revision: 316826
>> > URL: https://svnweb.freebsd.org/changeset/base/316826
>> >
>> > Log:
>> >   Avoid undefined behavior.
>> >
>> >   The 'pktid' variable is modified while being used twice between
>> >   sequence points, probably due to htonl() is macro.
>> FYI — there are a ton of similar reports in sys/rpc due to the XDR
>> macros (which read a network value off a pointer and increment it).
>> See e.g., IXDR_GET_UINT32() macro.
>>
>> Best,
>> Conrad
>
> Aren't they all false positives, since the macros involved are g'teed
> not to evaluate their arguments more than once as written (because
> __builtin_constant_p always evaluates at compile time)?  Do we really
> want to churn our source code to eliminate false positives from some
> tool that appears to still be in its alpha-testing state?
>
> -- Ian
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316854 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Andriy Gapon]

On 14/04/2017 18:55, Alan Somers wrote:
> Are you sure it's a good idea to MFC this change?  Doing so will upset
> anybody currently setting that sysctl in /etc/sysctl.conf.

Well, this is a debug knob, so I expected that it is tweaked by hand on as
needed basis.  I don't expect that it is used as a permanent setting.
But to err on the side of caution I can add a compatibility sysctl name.

> On Fri, Apr 14, 2017 at 9:35 AM, Andriy Gapon  wrote:
>> Author: avg
>> Date: Fri Apr 14 15:35:07 2017
>> New Revision: 316854
>> URL: https://svnweb.freebsd.org/changeset/base/316854
>>
>> Log:
>>   rename vfs.zfs.debug_flags to vfs.zfs.debugflags
>>
>>   While the former name is easier to read, the "_flags" suffix has a special
>>   meaning for loader(8) and, thus, it was impossible to set the knob via
>>   loader.conf(5).  The loader interpreted the setting as flags that should
>>   be passed to a kernel module named "vfs.zfs.debug".
>>
>>   Discussed with:   smh
>>   MFC after:2 weeks
>>
>> Modified:
>>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
>>
>> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
>> ==
>> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri 
>> Apr 14 15:31:04 2017(r316853)
>> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri 
>> Apr 14 15:35:07 2017(r316854)
>> @@ -353,7 +353,7 @@ sysctl_vfs_zfs_debug_flags(SYSCTL_HANDLE
>> return (0);
>>  }
>>
>> -SYSCTL_PROC(_vfs_zfs, OID_AUTO, debug_flags,
>> +SYSCTL_PROC(_vfs_zfs, OID_AUTO, debugflags,
>>  CTLTYPE_UINT | CTLFLAG_MPSAFE | CTLFLAG_RWTUN, 0, sizeof(int),
>>  sysctl_vfs_zfs_debug_flags, "IU", "Debug flags for ZFS testing.");
>>
>>


-- 
Andriy Gapon
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316854 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Alan Somers]

Are you sure it's a good idea to MFC this change?  Doing so will upset
anybody currently setting that sysctl in /etc/sysctl.conf.

On Fri, Apr 14, 2017 at 9:35 AM, Andriy Gapon  wrote:
> Author: avg
> Date: Fri Apr 14 15:35:07 2017
> New Revision: 316854
> URL: https://svnweb.freebsd.org/changeset/base/316854
>
> Log:
>   rename vfs.zfs.debug_flags to vfs.zfs.debugflags
>
>   While the former name is easier to read, the "_flags" suffix has a special
>   meaning for loader(8) and, thus, it was impossible to set the knob via
>   loader.conf(5).  The loader interpreted the setting as flags that should
>   be passed to a kernel module named "vfs.zfs.debug".
>
>   Discussed with:   smh
>   MFC after:2 weeks
>
> Modified:
>   head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
>
> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
> ==
> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri 
> Apr 14 15:31:04 2017(r316853)
> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri 
> Apr 14 15:35:07 2017(r316854)
> @@ -353,7 +353,7 @@ sysctl_vfs_zfs_debug_flags(SYSCTL_HANDLE
> return (0);
>  }
>
> -SYSCTL_PROC(_vfs_zfs, OID_AUTO, debug_flags,
> +SYSCTL_PROC(_vfs_zfs, OID_AUTO, debugflags,
>  CTLTYPE_UINT | CTLFLAG_MPSAFE | CTLFLAG_RWTUN, 0, sizeof(int),
>  sysctl_vfs_zfs_debug_flags, "IU", "Debug flags for ZFS testing.");
>
>
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316826 - head/sys/netpfil/ipfw/nat64

[Ian Lepore]

On Fri, 2017-04-14 at 08:32 -0700, Conrad Meyer wrote:
> On Fri, Apr 14, 2017 at 4:58 AM, Andrey V. Elsukov 
> wrote:
> > 
> > Author: ae
> > Date: Fri Apr 14 11:58:41 2017
> > New Revision: 316826
> > URL: https://svnweb.freebsd.org/changeset/base/316826
> > 
> > Log:
> >   Avoid undefined behavior.
> > 
> >   The 'pktid' variable is modified while being used twice between
> >   sequence points, probably due to htonl() is macro.
> FYI — there are a ton of similar reports in sys/rpc due to the XDR
> macros (which read a network value off a pointer and increment it).
> See e.g., IXDR_GET_UINT32() macro.
> 
> Best,
> Conrad

Aren't they all false positives, since the macros involved are g'teed
not to evaluate their arguments more than once as written (because
__builtin_constant_p always evaluates at compile time)?  Do we really
want to churn our source code to eliminate false positives from some
tool that appears to still be in its alpha-testing state?

-- Ian
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316851 - head/sys/x86/iommu

[Conrad Meyer]

On Fri, Apr 14, 2017 at 8:16 AM, Konstantin Belousov  wrote:
> Author: kib
> Date: Fri Apr 14 15:16:41 2017
> New Revision: 316851
> URL: https://svnweb.freebsd.org/changeset/base/316851
>
> Log:
>   Correct calculation of the entry->free_down in the invariants-checking
>   code.
>
>   Reported by:  maxim
>   Found by: PVS studio scan


This was also reported by Coverity in CID 1304880 as an unused value.

Coverity also reports a NULL check (l. 547) after all paths have
dereferenced the pointer ("next") already in dmar_gas_alloc_region()
of the same file (CID 1199386).

Best,
Conrad
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316854 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 15:35:07 2017
New Revision: 316854
URL: https://svnweb.freebsd.org/changeset/base/316854

Log:
  rename vfs.zfs.debug_flags to vfs.zfs.debugflags
  
  While the former name is easier to read, the "_flags" suffix has a special
  meaning for loader(8) and, thus, it was impossible to set the knob via
  loader.conf(5).  The loader interpreted the setting as flags that should
  be passed to a kernel module named "vfs.zfs.debug".
  
  Discussed with:   smh
  MFC after:2 weeks

Modified:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c

Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c
==
--- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri Apr 
14 15:31:04 2017(r316853)
+++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c  Fri Apr 
14 15:35:07 2017(r316854)
@@ -353,7 +353,7 @@ sysctl_vfs_zfs_debug_flags(SYSCTL_HANDLE
return (0);
 }
 
-SYSCTL_PROC(_vfs_zfs, OID_AUTO, debug_flags,
+SYSCTL_PROC(_vfs_zfs, OID_AUTO, debugflags,
 CTLTYPE_UINT | CTLFLAG_MPSAFE | CTLFLAG_RWTUN, 0, sizeof(int),
 sysctl_vfs_zfs_debug_flags, "IU", "Debug flags for ZFS testing.");
 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r316826 - head/sys/netpfil/ipfw/nat64

[Conrad Meyer]

On Fri, Apr 14, 2017 at 4:58 AM, Andrey V. Elsukov  wrote:
> Author: ae
> Date: Fri Apr 14 11:58:41 2017
> New Revision: 316826
> URL: https://svnweb.freebsd.org/changeset/base/316826
>
> Log:
>   Avoid undefined behavior.
>
>   The 'pktid' variable is modified while being used twice between
>   sequence points, probably due to htonl() is macro.

FYI — there are a ton of similar reports in sys/rpc due to the XDR
macros (which read a network value off a pointer and increment it).
See e.g., IXDR_GET_UINT32() macro.

Best,
Conrad
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316853 - head/cddl/contrib/opensolaris/lib/libdtrace/common

[Andriy Gapon]

Author: avg
Date: Fri Apr 14 15:31:04 2017
New Revision: 316853
URL: https://svnweb.freebsd.org/changeset/base/316853

Log:
  dtrace: fix normalization of stddev aggregation
  
  To be upstreamed.
  
  Discussed with:   Bryan Cantrill 
  MFC after:2 weeks
  Sponsored by: Panzura

Modified:
  head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c

Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c
==
--- head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c Fri Apr 
14 15:22:00 2017(r316852)
+++ head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c Fri Apr 
14 15:31:04 2017(r316853)
@@ -389,8 +389,10 @@ dt_stddev(uint64_t *data, uint64_t norma
 * The standard approximation for standard deviation is
 * sqrt(average(x**2) - average(x)**2), i.e. the square root
 * of the average of the squares minus the square of the average.
+* When normalizing, we should divide the sum of x**2 by normal**2.
 */
dt_divide_128(data + 2, normal, avg_of_squares);
+   dt_divide_128(avg_of_squares, normal, avg_of_squares);
dt_divide_128(avg_of_squares, data[0], avg_of_squares);
 
norm_avg = (int64_t)data[1] / (int64_t)normal / (int64_t)data[0];
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316852 - head/sbin/fsck_ffs

[Konstantin Belousov]

Author: kib
Date: Fri Apr 14 15:22:00 2017
New Revision: 316852
URL: https://svnweb.freebsd.org/changeset/base/316852

Log:
  In fsck_ffs pass1, prevent the inosused variable from wrapping.
  
  The loop that scans the used inode map when soft updates is in use
  assumes that the inosused variable is signed.  However, ino_t is
  unsigned, so the loop invariant is incorrect and the check for
  inosused wrapping to < 0 can never be true.
  
  Instead of checking for wrap after the fact just prevent it from
  happening in the first place.
  
  PR:   218592
  Submitted by: Todd Miller 
  Reviewed by:  mckusick
  MFC after:1 week

Modified:
  head/sbin/fsck_ffs/pass1.c

Modified: head/sbin/fsck_ffs/pass1.c
==
--- head/sbin/fsck_ffs/pass1.c  Fri Apr 14 15:16:41 2017(r316851)
+++ head/sbin/fsck_ffs/pass1.c  Fri Apr 14 15:22:00 2017(r316852)
@@ -133,9 +133,14 @@ pass1(void)
 */
if ((preen || inoopt) && usedsoftdep && !rebuildcg) {
cp = _inosused(cgp)[(inosused - 1) / CHAR_BIT];
-   for ( ; inosused > 0; inosused -= CHAR_BIT, cp--) {
-   if (*cp == 0)
+   for ( ; inosused != 0; cp--) {
+   if (*cp == 0) {
+   if (inosused > CHAR_BIT)
+   inosused -= CHAR_BIT;
+   else
+   inosused = 0;
continue;
+   }
for (i = 1 << (CHAR_BIT - 1); i > 0; i >>= 1) {
if (*cp & i)
break;
@@ -143,8 +148,6 @@ pass1(void)
}
break;
}
-   if (inosused < 0)
-   inosused = 0;
}
/*
 * Allocate inoinfo structures for the allocated inodes.
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316851 - head/sys/x86/iommu

[Konstantin Belousov]

Author: kib
Date: Fri Apr 14 15:16:41 2017
New Revision: 316851
URL: https://svnweb.freebsd.org/changeset/base/316851

Log:
  Correct calculation of the entry->free_down in the invariants-checking
  code.
  
  Reported by:  maxim
  Found by: PVS studio scan
  Sponsored by: The FreeBSD Foundation
  MFC after:1 week

Modified:
  head/sys/x86/iommu/intel_gas.c

Modified: head/sys/x86/iommu/intel_gas.c
==
--- head/sys/x86/iommu/intel_gas.c  Fri Apr 14 15:09:30 2017
(r316850)
+++ head/sys/x86/iommu/intel_gas.c  Fri Apr 14 15:16:41 2017
(r316851)
@@ -198,7 +198,7 @@ dmar_gas_check_free(struct dmar_domain *
l->free_down));
} else {
v = MAX(entry->free_after, l->free_down);
-   v = MAX(entry->free_down, r->free_down);
+   v = MAX(v, r->free_down);
MPASS(entry->free_down == v);
}
}
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r316837 - head/sys/dev/usb/quirk

[Kevin Lo]

Author: kevlo
Date: Fri Apr 14 14:38:27 2017
New Revision: 316837
URL: https://svnweb.freebsd.org/changeset/base/316837

Log:
  Remove RTL8153 quirk since ure(4) supports this chip.

Modified:
  head/sys/dev/usb/quirk/usb_quirk.c

Modified: head/sys/dev/usb/quirk/usb_quirk.c
==
--- head/sys/dev/usb/quirk/usb_quirk.c  Fri Apr 14 14:34:48 2017
(r316836)
+++ head/sys/dev/usb/quirk/usb_quirk.c  Fri Apr 14 14:38:27 2017
(r316837)
@@ -95,7 +95,6 @@ static struct usb_quirk_entry usb_quirks
USB_QUIRK(SILICONPORTALS, YAPPHONE, 0x100, 0x100, UQ_AU_INP_ASYNC),
USB_QUIRK(LOGITECH, UN53B, 0x, 0x, UQ_NO_STRINGS),
USB_QUIRK(REALTEK, RTL8196EU, 0x, 0x, UQ_CFG_INDEX_1),
-   USB_QUIRK(REALTEK, RTL8153, 0x, 0x, UQ_CFG_INDEX_1),
USB_QUIRK(ELSA, MODEM1, 0x, 0x, UQ_CFG_INDEX_1),
USB_QUIRK(PLANEX2, MZKUE150N, 0x, 0x, UQ_CFG_INDEX_1),
USB_QUIRK(CISCOLINKSYS, USB3GIGV1, 0x, 0x, UQ_CFG_INDEX_1),
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"