Re: kern.maxswzone causing serious problems

[Curtis Villamizar]

Replying to myself.

Oops - error in the diff:

-+ (maxpages / SWAP_META_PAGES) * 2
++ (npages / SWAP_META_PAGES) * 2

The prior code worked but only because it doubled the allowable swap.  
This provides the right value for kern.maxswzone for the amount of swap 
you have for the complaint to go away.  I confirmed that by reducing the 
kern.maxswzone value by a small amount (under 1K) and the complaint 
returned.  I am currently running a regression test over the host and 
the entire set of VMs running on it to makes sure there are no more 
surprises. Since I generate configs it is easy to change the way 
kern.maxswzone is computed when generating boot/loader.conf for a given 
host or VM.


Will check back later after regression testing.  Apparently the best way 
to get this to get some attention is to file a bug so once the changes 
are fully verified in the regression testing I'll do that.  Just in case 
there is some further interaction with using more swap that recommended 
by the current code.


Curtis


On 03/28/18 23:36, Curtis Villamizar wrote:

I'm starting to upgrade a set of servers from FreeBSD 11.0-STABLE #0
r308356 (OK, rather old) to FreeBSD 11.1-STABLE #0 r331152 without
much success.  I'm getting the occasionally discussed kern.maxswzone
problems but I really do need to configure that swap space.

On an upgraded server I'm getting (line continuation added for
readability):

warning: total configured swap (5242880 pages) \
   exceeds maximum recommended amount (112288 pages).
warning: increase kern.maxswzone or reduce amount of swap.
warning: total configured swap (10485760 pages) \
   exceeds maximum recommended amount (112288 pages).
warning: increase kern.maxswzone or reduce amount of swap.

The value previously used was not working.  I ended up temporarily
cutting swap in half as well to get rid of the error.  This is only a
symptom of a greater problem.

This machine has for a long time run multiple VM which total 17GB
memory on a server with 8 GB of physical memory.  Normally things are
fine becuase most of these servers are idle most of the time and all
but a small memory footprint can be paged out.  I have for years had
40 GB swap, with 20GB each on two spindles (at least going back to
FreeBSD 9) on drives identical partitioning with mirrored partitions.

\begin{talesofwoe}  % ignore if busy

When just trying to install base system software using md devices on
at a time (no VM running yet) I had been getting "killed: out of swap
space" messages.  I got rid of this by reducing the size of the VM
root partitions (most VM run zfs on another partition with not much on
root so this was OK).  Next problem was running the VM.  I worked
around this by reducing some VM memory (-m in bhyve) by half.  I can
get the whole set of VM to boot but now installing additional software
(which involved just moving files with scp and running tar) doesn't
work.  At this point I have 9 GB of VM running on 8 GB physical memory
and still couldn't even install software.  Once the maxswzone message
went away, so did all of these problems except my VMs now have half as
much RAM each.  Now each of the VM are reporting swap problems.  And
this is just upgrading one server to 11.1.

\end{talesofwoe}

This is a major regression for FreeBSD 11.1.  The same value used in
FreeBSD 11.0 should just work or there should be some documentation on
how to set this (preferably the error message).  If nothing else, some
advice on converting a kern.maxswzone value from 11.0 to a working
value for 11.1 would be nice.  The entry in the loader(8) man page is
not very helpful.

btw- Reporting swap size in MB or KB in the error message would be
helpful.  In addition to pages would be fine.  Mentioning what the
highest value kern.maxswzone could be set to would also be helpful.
Changing "warning: increase kern.maxswzone" to "warning: increase
kern.maxswzone to %d" would be very helpful.

\begin{naiveananlysis}

The magic (or mess, depending on perspective) is mostly in "void
swap_pager_swap_init(void)" in the file vm/swap_pager.c between lines
484 and 563 (in current which is same as stable/11 in this function).
The diffs from known working r308356 to current show a diff at "@@
-538,21 +518,25 @@" which has swpctrie_zone and swblk_zone computed
based on maxswzone, then runs uma_zone_reserve_kva based on maxswzone
and potentially reduces it.

In the older code a "Swap zone entries reduced" message would be
produced if uma_zone_reserve_kva cut back (which is moved and the
message changed a bit).  But I didn't see this message so
uma_zone_reserve_kva ran fine the first time without reducing "n".  In
the new code swap_maxpages and swzone are then set.
swapon_check_swzone gives the warning but does nothing as far as I can
tell other than two printfs.  It doesn't appear to do any harm to have
too much swap and ignore this warning (you just can't use it).

There is a 

Re: kern.maxswzone causing serious problems

[Jeremy Chadwick]

I am not subscribed to -stable, so please keep me CC'd.

I mailed -stable about this problem, or a variation of it, earlier this
month:

https://lists.freebsd.org/pipermail/freebsd-stable/2018-March/088467.html

What isn't publicly visible is the list of individuals I CC'd on that
mail who had touched this code in recent days: k...@freebsd.org,
d...@freebsd.org, pluk...@freebsd.org, ead...@freebsd.org

I received no response from them on this matter.  At least two, however,
have been extremely busy commit-wise, so I imagine folks are just
swamped right now + have higher priorities.

I did not read or review your {naiveanalysis} section or your patch, as
tinkering with VM design/internals is *way* outside my comfort zone.

I will say that printing the sizes in a unit other than pages would be
generally helpful; I did try to figure out what value to use for
kern.maxswzone as a workaround by digging through kernel code but gave
up, as I wasn't able to truly determine what "pages" actually
represented (size-wise) in this specific context.

I hope someone with src commit bit will comment, as code slush for
11.2-RELEASE begins on April 20th:

https://www.freebsd.org/releases/11.2R/schedule.html

Else a separate PR can be opened if requested.

-- 
| Jeremy Chadwick   j...@koitsu.org |
| UNIX Systems Administratorhttp://jdc.koitsu.org/ |
| Making life hard for others since 1977. PGP 4BD6C0CB |

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: KBI unexpexted change in stable/11 ?

[Konstantin Belousov]

On Thu, Mar 29, 2018 at 09:21:43PM +0300, Slawa Olhovchenkov wrote:
> On Wed, Mar 28, 2018 at 11:25:08PM -0700, Kevin Oberman wrote:
> 
> > > > > r325665 is previos point and is good.
> > > > > r331615 crashed.
> > > > > Can I use some script for bisect?
> > > >
> > > > I'm not aware of a script for this.  The only tool I've used is "git
> > > > bisect", which is very handy if you're already familiar with git.
> > >
> > > You may want to try devel/p5-App-SVN-Bisect.  Never used it, so
> > > no idea if it's functional or helpful, just found it doing a quick
> > > search
> > 
> > It would be nice if this could be fixed, but it is the case.
> 
> r328475 bad (tzdata)
> r328469 in progress (kib, sys/vm)
> r328463 in progress (don't touch kernel)
> r328462 good
> 
> I mean r328469 break KBI.
This commit is basically required to avoid significant confusion between
HEAD and stable code.  Its absense might make other merges impossible or
quite hard.

In fact, my opinion is that the real bug is elsewere.  We do provide
vm_map_min/max KPI, but the KPI is not KBI-stable because it encodes
struct vm_map layout into binaries which (correctly) use KPI instead
of directly accessing struct vm_map.  This should be fixed, I put the
review https://reviews.freebsd.org/D14902 for it.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: KBI unexpexted change in stable/11 ?

[Slawa Olhovchenkov]

On Wed, Mar 28, 2018 at 11:25:08PM -0700, Kevin Oberman wrote:

> > > > r325665 is previos point and is good.
> > > > r331615 crashed.
> > > > Can I use some script for bisect?
> > >
> > > I'm not aware of a script for this.  The only tool I've used is "git
> > > bisect", which is very handy if you're already familiar with git.
> >
> > You may want to try devel/p5-App-SVN-Bisect.  Never used it, so
> > no idea if it's functional or helpful, just found it doing a quick
> > search
> 
> It would be nice if this could be fixed, but it is the case.

r328475 bad (tzdata)
r328469 in progress (kib, sys/vm)
r328463 in progress (don't touch kernel)
r328462 good

I mean r328469 break KBI.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Delegates Email list of Atlanta Apartment Association 2018

[Veronica Baker]

Hi,

 

Would you be interested in Attendees Email List Of Atlanta Apartment
Association 2018.? 

 

Attendees:

 

. Decision Makers

. Apartment owners

. Managers

. Suppliers, and Professionals

 

List includes: E-mails, contact number and other fields on an excel sheet.

 

Would you like to see counts and pricing details available.?

 

Looking forward for your email.

 

 

Regards,

Veronica Baker| B.D.Exe

 

If you do not wish to receive future emails from us, please reply as
"Exclude Me"

 

 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: KBI unexpexted change in stable/11 ?

[Kyle Evans]

On Thu, Mar 29, 2018 at 8:38 AM,   wrote:
>> From: Kevin Oberman
>
>> Regardless of intent, updates to port based modules can and and will break
>> when the kernel is updated.
>
> Are packages for pkg.freebsd.org built on -STABLE or latest -RELEASE?
>
> After 11.2 is released, if package for nvidia-driver
> is installed or upgraded on a machine with 11.1-RELEASE,
> will it crash the kernel?
>
> If the port nvidia-driver-304 is updated before 11.2 is released,
> will `pkg upgrade` crash the kernel?

Packages are built on the earliest supported release of a branch;
never from the stable branch. These packages will be built from 11.1
until it goes EOL a couple months after 11.2 release.

Breakage of packages from an earlier release of a branch is not ideal,
so it would still be most helpful if someone could bissect this so
that we can assess the breakage and figure out next steps before it
becomes a bigger problem with the impending release.

I've heard good things about the aforementioned
devel/p5-App-SVN-Bisect for these purposes.

Thanks,

Kyle Evans
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: KBI unexpexted change in stable/11 ?

[Lena]

> From: Kevin Oberman

> Regardless of intent, updates to port based modules can and and will break
> when the kernel is updated.

Are packages for pkg.freebsd.org built on -STABLE or latest -RELEASE?

After 11.2 is released, if package for nvidia-driver
is installed or upgraded on a machine with 11.1-RELEASE,
will it crash the kernel?

If the port nvidia-driver-304 is updated before 11.2 is released,
will `pkg upgrade` crash the kernel?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: KBI unexpexted change in stable/11 ?

[Kevin Oberman]

On Wed, Mar 28, 2018 at 9:46 AM, Gary Palmer  wrote:

> On Wed, Mar 28, 2018 at 11:16:49AM -0500, Eric van Gyzen wrote:
> > On 03/28/2018 10:35, Slawa Olhovchenkov wrote:
> > > On Wed, Mar 28, 2018 at 10:29:10AM -0500, Eric van Gyzen wrote:
> > >
> > >> On 03/28/2018 08:09, Slawa Olhovchenkov wrote:
> > >>> I am upgrade system to latest -STABLE and now see kernel crash:
> > >>>
> > >>> - loading virtualbox modules build on 11.1-RELEASE-p6
> > >>> - loading nvidia module build on 11.1-RELEASE-p6 and start xdm
> > >>>
> > >>> Is this expected? I am mean about loading modules builded on
> > >>> 11.1-RELEASE on any 11.1-STABLE.
> > >>
> > >> This is not expected.  Can you bisect to find the stable/11 commit
> that
> > >> broke this?
> > >>
> > >> If you can roll back to 11.1-RELEASE, you could probably just
> > >> buildkernel and installkernel from various points along stable/11.
> That
> > >> would save a lot of time by avoiding buildworld.
> > >
> > > r325665 is previos point and is good.
> > > r331615 crashed.
> > > Can I use some script for bisect?
> >
> > I'm not aware of a script for this.  The only tool I've used is "git
> > bisect", which is very handy if you're already familiar with git.
>
> You may want to try devel/p5-App-SVN-Bisect.  Never used it, so
> no idea if it's functional or helpful, just found it doing a quick
> search
>
> Regards,
>
> Gary
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
>

Regardless of intent, updates to port based modules can and and will break
when the kernel is updated. lsof breaks quite often, as well. It touches
things in the kernel that are intended for internal use only, so that is to
be expected. I can't say that virtualbox-ose-kmod has had to be rebuilt
because I put it into PORTS_MODULES a couple of years ago after having it
fail on several occasions. Every time it gets broken again thee are lots of
threads in STABLE like this one.

It would be nice if this could be fixed, but it is the case.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"