Re: Crashes in libthr?

[Larry Rosenman]

On 2016-03-15 12:21, Larry Rosenman wrote:

This may have been my screw up


On March 15, 2016 12:05:03 PM Bryan Drewery  
wrote:



On 3/14/16 8:03 PM, Larry Rosenman wrote:

On 2016-03-14 21:49, Larry Rosenman wrote:

On 2016-03-14 18:49, Larry Rosenman wrote:

On 2016-03-14 18:47, Steven Hartland wrote:

On 14/03/2016 22:28, Larry Rosenman wrote:

On 2016-03-14 17:25, Poul-Henning Kamp wrote:


In message <2016031428.ga1...@borg.lerctr.org>, Larry 
Rosenman

writes:


And sshd is busted.


FYI: I seeing no such issues on two systems running:

11.0-CURRENT #4 r296808: Sun Mar 13 22:39:59 UTC 2016
and
11.0-CURRENT #32 r296137: Sat Feb 27 11:34:01 UTC 2016

As I said it's this ONE box, even doing an install from the other
(RUNNING) boxes
/usr/src,/usr/obj).

This build was at:
borg.lerctr.org /usr/src $ svn info
Path: .
Working Copy Root Path: /usr/src
URL: https://svn.freebsd.org/base/head
Relative URL: ^/head
Repository Root: https://svn.freebsd.org/base
Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Revision: 296823
Node Kind: directory
Schedule: normal
Last Changed Author: adrian
Last Changed Rev: 296823
Last Changed Date: 2016-03-13 23:39:35 -0500 (Sun, 13 Mar 2016)

borg.lerctr.org /usr/src $


I can post the make.conf.

It's really weird.

Silly question your not building on an NFS FS are you?




No, this is local disk.  The "install from other machine" was via
NFS..



I found it.  A bad version (from march 8th or so) of
/lib/libprivatessh.so.5 that did NOT export the symbol, but the
version in /usr/lib/libprivatessh.so.5 DID export the symbol.

I wiped out the /lib/libprivate* and re-did installworld.

and all seems fine now.

I suspect I hit a time when the tree had bad stuff installing into
/lib/libprivate*



BTW, there were LOTS of OTHER things in /lib with the same bad date,
which I've now cleaned up.

make delete-old{-libs} did *NOT* clean this up.




These were dated March 8 2016?  I don't recall any recent changes
causing libraries to be installed to the wrong place.

--
Regards,
Bryan Drewery

I think I screwed up badly..

So, thank G-D for ZFS snapshots and Boot Environments.  I've reverted to 
an earler snap/root via

clone/promote.


--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@lerctr.org
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

[Bug 194744] [PATCH] allow to specify custom keymap when kbdmux used

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194744

o...@hardenedbsd.org changed:

   What|Removed |Added

 CC||o...@hardenedbsd.org

--- Comment #3 from o...@hardenedbsd.org ---
Not et all, without this patch you can't specify custom keyboard layout at
compile time to kbdmux, because the keyboard layout does not inherited to
kbdmux.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Adrian Chadd]

[snip]

It's not rsync itself. It's just triggering some odd behaviour.

I've poked alc; I'll work with him to see if this can be figured out.

Thanks! I'm glad I'm not the only person who has seen this behaviour!


-adrian
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

[Bug 194744] [PATCH] allow to specify custom keymap when kbdmux used

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194744

--- Comment #4 from o...@hardenedbsd.org ---
Yes, sorry, I looked the wrong PR. This  PR is a dup of 153459.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

[Bug 194744] [PATCH] allow to specify custom keymap when kbdmux used

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194744

--- Comment #2 from Ed Maste  ---
(In reply to Oliver Pinter from comment #0)
Is this patch functionally equivalent to the one in PR 153459?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Jeffrey Bouquet]

On Tue, 15 Mar 2016 09:30:11 -0600, Ian Lepore  wrote:

> On Tue, 2016-03-15 at 07:20 -0700, Jeffrey Bouquet wrote:
> > rsync... see bottom posting
> > 
> > On Tue, 15 Mar 2016 07:43:46 +0100, olli hauer  wrote:
> > 
> > > On 2016-03-14 15:19, Ian Lepore wrote:
> > > > On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote:
> > > > > On 13 March 2016 at 18:51, Mark Johnston 
> > > > > wrote:
> > > > > > On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I can reproduce this by doing a mkimage on a large
> > > > > > > destination
> > > > > > > file
> > > > > > > image. it looks like it causes all the desktop processes to
> > > > > > > get
> > > > > > > paged
> > > > > > > out whilst it's doing so, and then the whole UI freezes
> > > > > > > until it
> > > > > > > catches up.
> > > > > > 
> > > > > > mkimg(1) maps the destination file with MAP_NOSYNC, so if
> > > > > > it's
> > > > > > larger
> > > > > > than RAM, I think it'll basically force the pagedaemon to
> > > > > > write out
> > > > > > the
> > > > > > image as it tries to reclaim pages from the inactive queue.
> > > > > > This
> > > > > > can
> > > > > > cause stalls if the pagedaemon blocks waiting for some I/O to
> > > > > > complete.
> > > > > > The user/alc/PQ_LAUNDRY branch helps alleviate this problem
> > > > > > by
> > > > > > using a
> > > > > > different thread to launder dirty pages. I use mkimg on
> > > > > > various
> > > > > > desktop
> > > > > > machines to build bhyve images and have noticed the problem
> > > > > > you're
> > > > > > describing; PQ_LAUNDRY helps quite a bit in that case. But I
> > > > > > don't
> > > > > > know
> > > > > > why this would be a new problem.
> > > > > > 
> > > > > 
> > > > > That's why I'm confused. I just know that it didn't used to
> > > > > cause the
> > > > > whole UI to hang due to paging.
> > > > > 
> > > > 
> > > > I've been noticing this too.  This machine runs 10-stable and
> > > > this use
> > > > of the swap began happening recently when I updated from 10
> > > > -stable
> > > > around the 10.2 release time to 10-stable right about when the
> > > > 10.3
> > > > code freeze began.
> > > > 
> > > > In my case I have no zfs anything here.  I noticed the problem
> > > > bigtime
> > > > yesterday when rsync was syncing a ufs filesystem of about 500GB
> > > > from
> > > > one disk to another (probably 70-80 GB actually needed copying). 
> > > >  My
> > > > desktop apps were noticibly unresponsive when I activated a
> > > > window that
> > > > had been idle for a while (like it would take a couple seconds
> > > > for the
> > > > app to begin responding).  I could see lots of swap In happening
> > > > in top
> > > > during this unresponsiveness, and noticible amounts of Out
> > > > activity
> > > > when nothing was happening except the rsync.
> > > > 
> > > > This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it
> > > > at
> > > > the time.  Prior to the update around the 10.3 freeze, this
> > > > machine
> > > > would never touch the swap no matter what workload I threw at it
> > > > (this
> > > > rsync stuff happens every day, it's the usual workload).
> > > > 
> > > 
> > > I'm not sure if it is the same problem, or port related.
> > > 
> > > On two systems without zfs but with many files e.g. svn servers I
> > > see now
> > > from time to time they are running out of swap.
> > > 
> > >  kernel: swap_pager_getswapspace(9): failed
> > >  kernel: swap_pager_getswapspace(16): failed
> > >  ...
> > > 
> > > It also happened on one system during the nightly periodic tasks
> > > holding
> > > only millions of backup files.
> > > 
> > > $ freebsd-version -ku
> > >   10.2-RELEASE-p9
> > >   10.2-RELEASE-p13
> > > 
> > > 
> > > ___
> > > freebsd-current@freebsd.org mailing list
> > > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > > To unsubscribe, send any mail to "
> > > freebsd-current-unsubscr...@freebsd.org"
> > 
> > 
> > Just a point I've bought up elsewhere...
> > I've, if I recall, wrecked several filesystems (although EIDE) using
> > rsync at the normal bus rate, and sometimes
> > thumbdrives with whatever filesystem type on them.
> > 
> > I settled on --bwlimit=1500,  max for unattended  rsync usage and
> > almost every day
> > use --bwlimit=700.
> > 
> > The latter enables several resource-intensive processes ( music,
> > classical music videos, svn, pkg, browsing, etc) to
> > proceed apace concurrently on the desktop (SATA not EIDE) with nary a
> > hang nor slowdown.
> > 
> > If I recall, the usual speed is 1 so that is less than ten
> > percent, if I recall, of the usual speed.
> > 
> > YMMV.
> > 
> > J.
> > 
> > PS as an afterthough, it would be useful if that were more prominent
> > on the man page somewhere or even
> > in the port's pkg-message or pkg-description.  
> > The SATA more robust than EIDE on 

Re: how to recycle Inact memory more aggressively?

[olli hauer]

...

>> Just a point I've bought up elsewhere...
>> I've, if I recall, wrecked several filesystems (although EIDE) using
>> rsync at the normal bus rate, and sometimes
>> thumbdrives with whatever filesystem type on them.
>>
>> I settled on --bwlimit=1500,  max for unattended  rsync usage and
>> almost every day
>> use --bwlimit=700.

It happened also on VM's where the host is connected via FC to the storage
but only on the FreeBSD VM's.


>> The latter enables several resource-intensive processes ( music,
>> classical music videos, svn, pkg, browsing, etc) to
>> proceed apace concurrently on the desktop (SATA not EIDE) with nary a
>> hang nor slowdown.

I don't have any *NIX system with a gui, only around 110+ headless systems
and halve of them are running FreeBSD.


> I have no real idea what any of that is about, but before it turns into
> some kind of "rsync is bad" mythology, let me just say that I've been
> using rsync to copy gigabytes of backup data every day for years now. 
>  I've never had any kind of problem, especially system responsiveness
> problems, until this increased swapfile activity thing started
> happening on 10-stable in the past few months.
> 
> To reiterate: rsync is not in any way at fault here, and any suggestion
> that the unresponsiveness should be "fixed" by screwing around with
> rsync parms that have worked fine for a decade is just something I
> completely reject.
> 
> I'm sure I'd see the same kind of increased swapping with ANY process
> that read and wrote gigabytes of data in a short period of time.  And
> that's what this thread is about:  What has changed to cause this
> regression that multiple people are seeing where lots of IO now drives
> an unusual amount of swapfile activity on systems that used to NEVER
> write anything to swap?

All those systems are running already for years, for me it looks more
like a missing *free* (memory leak).

Looking at the net/rsync history the last update was in Dec. 2015. Perhaps
it is worth to test net/rsync r359474 for a while to get a comparsion, but
Gary reported the issue by using `cp' and not rsync.

-- 
olli
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Crashes in libthr?

[Larry Rosenman]

This may have been my screw up


On March 15, 2016 12:05:03 PM Bryan Drewery  wrote:


On 3/14/16 8:03 PM, Larry Rosenman wrote:

On 2016-03-14 21:49, Larry Rosenman wrote:

On 2016-03-14 18:49, Larry Rosenman wrote:

On 2016-03-14 18:47, Steven Hartland wrote:

On 14/03/2016 22:28, Larry Rosenman wrote:

On 2016-03-14 17:25, Poul-Henning Kamp wrote:


In message <2016031428.ga1...@borg.lerctr.org>, Larry Rosenman
writes:


And sshd is busted.


FYI: I seeing no such issues on two systems running:

11.0-CURRENT #4 r296808: Sun Mar 13 22:39:59 UTC 2016
and
11.0-CURRENT #32 r296137: Sat Feb 27 11:34:01 UTC 2016

As I said it's this ONE box, even doing an install from the other
(RUNNING) boxes
/usr/src,/usr/obj).

This build was at:
borg.lerctr.org /usr/src $ svn info
Path: .
Working Copy Root Path: /usr/src
URL: https://svn.freebsd.org/base/head
Relative URL: ^/head
Repository Root: https://svn.freebsd.org/base
Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Revision: 296823
Node Kind: directory
Schedule: normal
Last Changed Author: adrian
Last Changed Rev: 296823
Last Changed Date: 2016-03-13 23:39:35 -0500 (Sun, 13 Mar 2016)

borg.lerctr.org /usr/src $


I can post the make.conf.

It's really weird.

Silly question your not building on an NFS FS are you?




No, this is local disk.  The "install from other machine" was via
NFS..



I found it.  A bad version (from march 8th or so) of
/lib/libprivatessh.so.5 that did NOT export the symbol, but the
version in /usr/lib/libprivatessh.so.5 DID export the symbol.

I wiped out the /lib/libprivate* and re-did installworld.

and all seems fine now.

I suspect I hit a time when the tree had bad stuff installing into
/lib/libprivate*



BTW, there were LOTS of OTHER things in /lib with the same bad date,
which I've now cleaned up.

make delete-old{-libs} did *NOT* clean this up.




These were dated March 8 2016?  I don't recall any recent changes
causing libraries to be installed to the wrong place.

--
Regards,
Bryan Drewery



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Crashes in libthr?

[Bryan Drewery]

On 3/14/16 8:03 PM, Larry Rosenman wrote:
> On 2016-03-14 21:49, Larry Rosenman wrote:
>> On 2016-03-14 18:49, Larry Rosenman wrote:
>>> On 2016-03-14 18:47, Steven Hartland wrote:
 On 14/03/2016 22:28, Larry Rosenman wrote:
> On 2016-03-14 17:25, Poul-Henning Kamp wrote:
>> 
>> In message <2016031428.ga1...@borg.lerctr.org>, Larry Rosenman
>> writes:
>>
>>> And sshd is busted.
>>
>> FYI: I seeing no such issues on two systems running:
>>
>> 11.0-CURRENT #4 r296808: Sun Mar 13 22:39:59 UTC 2016
>> and
>> 11.0-CURRENT #32 r296137: Sat Feb 27 11:34:01 UTC 2016
> As I said it's this ONE box, even doing an install from the other
> (RUNNING) boxes
> /usr/src,/usr/obj).
>
> This build was at:
> borg.lerctr.org /usr/src $ svn info
> Path: .
> Working Copy Root Path: /usr/src
> URL: https://svn.freebsd.org/base/head
> Relative URL: ^/head
> Repository Root: https://svn.freebsd.org/base
> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> Revision: 296823
> Node Kind: directory
> Schedule: normal
> Last Changed Author: adrian
> Last Changed Rev: 296823
> Last Changed Date: 2016-03-13 23:39:35 -0500 (Sun, 13 Mar 2016)
>
> borg.lerctr.org /usr/src $
>
>
> I can post the make.conf.
>
> It's really weird.
 Silly question your not building on an NFS FS are you?


>>> No, this is local disk.  The "install from other machine" was via
>>> NFS..
>>
>>
>> I found it.  A bad version (from march 8th or so) of
>> /lib/libprivatessh.so.5 that did NOT export the symbol, but the
>> version in /usr/lib/libprivatessh.so.5 DID export the symbol.
>>
>> I wiped out the /lib/libprivate* and re-did installworld.
>>
>> and all seems fine now.
>>
>> I suspect I hit a time when the tree had bad stuff installing into
>> /lib/libprivate*
> 
> 
> BTW, there were LOTS of OTHER things in /lib with the same bad date,
> which I've now cleaned up.
> 
> make delete-old{-libs} did *NOT* clean this up.
> 
> 

These were dated March 8 2016?  I don't recall any recent changes
causing libraries to be installed to the wrong place.

-- 
Regards,
Bryan Drewery
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Gary Jennejohn]

On Sun, 13 Mar 2016 18:33:20 -0700
Mark Johnston  wrote:

> On Sat, Mar 12, 2016 at 09:38:35AM +0100, Gary Jennejohn wrote:
> > In the course of the last year or so the behavior of the vm system
> > has changed in regard to how aggressively Inact memory is recycled.
> > 
> > My box has 8GB of memory.  At the moment I'm copying 100s of gigabytes
> > from one file system to another one.  
> 
> How exactly are you copying them? How large are the files you're
> copying? Which filesystems are in use?
> 

cp(1) from one UFS filesystem to another for backup.

The files I copied were all movies on the order of 2GB to 4GB.  So
it seems unlikely that cp(1) would try to mmap them.

The aggregate total was on the order of 300GB.

To add further detail - I tend to keep an eye on top while doing
copies like this.  Previously, I would observe Inact getting up to
about 6GB, but it would quickly drop on the order of 3GB and Free
would increase correspondingly.

Now I see that Inact is pretty much stuck at 6GB and Free only
grows by a few 100MB at best, which are quickly used up.

In the good old days large file copies would only cause a few
MB to be swapped out, but now its on the order of 100MB.


-- 
Gary Jennejohn
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Ian Lepore]

On Tue, 2016-03-15 at 07:20 -0700, Jeffrey Bouquet wrote:
> rsync... see bottom posting
> 
> On Tue, 15 Mar 2016 07:43:46 +0100, olli hauer  wrote:
> 
> > On 2016-03-14 15:19, Ian Lepore wrote:
> > > On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote:
> > > > On 13 March 2016 at 18:51, Mark Johnston 
> > > > wrote:
> > > > > On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > I can reproduce this by doing a mkimage on a large
> > > > > > destination
> > > > > > file
> > > > > > image. it looks like it causes all the desktop processes to
> > > > > > get
> > > > > > paged
> > > > > > out whilst it's doing so, and then the whole UI freezes
> > > > > > until it
> > > > > > catches up.
> > > > > 
> > > > > mkimg(1) maps the destination file with MAP_NOSYNC, so if
> > > > > it's
> > > > > larger
> > > > > than RAM, I think it'll basically force the pagedaemon to
> > > > > write out
> > > > > the
> > > > > image as it tries to reclaim pages from the inactive queue.
> > > > > This
> > > > > can
> > > > > cause stalls if the pagedaemon blocks waiting for some I/O to
> > > > > complete.
> > > > > The user/alc/PQ_LAUNDRY branch helps alleviate this problem
> > > > > by
> > > > > using a
> > > > > different thread to launder dirty pages. I use mkimg on
> > > > > various
> > > > > desktop
> > > > > machines to build bhyve images and have noticed the problem
> > > > > you're
> > > > > describing; PQ_LAUNDRY helps quite a bit in that case. But I
> > > > > don't
> > > > > know
> > > > > why this would be a new problem.
> > > > > 
> > > > 
> > > > That's why I'm confused. I just know that it didn't used to
> > > > cause the
> > > > whole UI to hang due to paging.
> > > > 
> > > 
> > > I've been noticing this too.  This machine runs 10-stable and
> > > this use
> > > of the swap began happening recently when I updated from 10
> > > -stable
> > > around the 10.2 release time to 10-stable right about when the
> > > 10.3
> > > code freeze began.
> > > 
> > > In my case I have no zfs anything here.  I noticed the problem
> > > bigtime
> > > yesterday when rsync was syncing a ufs filesystem of about 500GB
> > > from
> > > one disk to another (probably 70-80 GB actually needed copying). 
> > >  My
> > > desktop apps were noticibly unresponsive when I activated a
> > > window that
> > > had been idle for a while (like it would take a couple seconds
> > > for the
> > > app to begin responding).  I could see lots of swap In happening
> > > in top
> > > during this unresponsiveness, and noticible amounts of Out
> > > activity
> > > when nothing was happening except the rsync.
> > > 
> > > This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it
> > > at
> > > the time.  Prior to the update around the 10.3 freeze, this
> > > machine
> > > would never touch the swap no matter what workload I threw at it
> > > (this
> > > rsync stuff happens every day, it's the usual workload).
> > > 
> > 
> > I'm not sure if it is the same problem, or port related.
> > 
> > On two systems without zfs but with many files e.g. svn servers I
> > see now
> > from time to time they are running out of swap.
> > 
> >  kernel: swap_pager_getswapspace(9): failed
> >  kernel: swap_pager_getswapspace(16): failed
> >  ...
> > 
> > It also happened on one system during the nightly periodic tasks
> > holding
> > only millions of backup files.
> > 
> > $ freebsd-version -ku
> >   10.2-RELEASE-p9
> >   10.2-RELEASE-p13
> > 
> > 
> > ___
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> > freebsd-current-unsubscr...@freebsd.org"
> 
> 
> Just a point I've bought up elsewhere...
> I've, if I recall, wrecked several filesystems (although EIDE) using
> rsync at the normal bus rate, and sometimes
> thumbdrives with whatever filesystem type on them.
> 
> I settled on --bwlimit=1500,  max for unattended  rsync usage and
> almost every day
> use --bwlimit=700.
> 
> The latter enables several resource-intensive processes ( music,
> classical music videos, svn, pkg, browsing, etc) to
> proceed apace concurrently on the desktop (SATA not EIDE) with nary a
> hang nor slowdown.
> 
> If I recall, the usual speed is 1 so that is less than ten
> percent, if I recall, of the usual speed.
> 
> YMMV.
> 
> J.
> 
> PS as an afterthough, it would be useful if that were more prominent
> on the man page somewhere or even
> in the port's pkg-message or pkg-description.  
> The SATA more robust than EIDE on FreeBSD that I've come across,
> though I prefer not to hint at because I
> believe it to be the fault of EIDE firmware rather than FreeBSD code.
> FWIW.

I have no real idea what any of that is about, but before it turns into
some kind of "rsync is bad" mythology, let me just say that I've been
using rsync to copy gigabytes of backup data every day 

Re: [CFT] packaging the base system with pkg(8)

[Jeffrey Bouquet]

On Tue, 15 Mar 2016 08:53:10 +0100, José Pérez  wrote:

> El 2016-03-03 11:27, Matthew Seaman escribió:
> > On 03/02/16 23:54, Glen Barber wrote:
> >> Also note (as repeated below), running 'pkg delete -a' will implicitly
> >> remove base system packages after they are installed.
> > 
> > This has the potential for many feet to be shot, given that up to now,
> > 'pkg delete -a' would always leave you with a viable system.
> 
> Agreed.
> 
> Suggested workaround (a las *grep): create two pkg binaries with 
> different names:
> - "pkg" does what it does now and works on non-base packages by default. 
> Need an extra
>arg to work on base system
> - "syspkg" (or something) works by default on base system
> 
> We'd need way less crutches.
> 
> Regards,
> 
> ---
> José Pérez
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Hmm... 
To reiterate this point..  (1)
As a wish here is for more code within pkg-install so that I do not encounter a 
situation such
as late last night whereupon I had to spend an extra half hour or so figuring 
out that hplip installed
a large number of unwanted additional qt4 ports alongside the cups upgrade with 
pkg, ... so
that a parameter or usual output would show NEW PORTS TO BE INSTALLED alongside 
each
one from WHICH port is the request to install the new dependency...
as a backdrop for this that I just thought of (2)...
  what if pkg on the base system deletes SOONER THAN THE USUAL 
make-delete-old
that PREVENTS/HALTS the successful completion of the pkg updating base?  
Something
critical to pkg itself proceeding?  As a typo or bug?..Maybe another 
cluster of testing 
machines and weeks of testing before each pkg-release-avail or pkg-stable-avail 
became 
known to FreeBSD users in emails... and that would maybe preclude pkg OF BASE 
from
being useful for CURRENT installs due to a lack of testing, and/or make current 
upgrades more
risky.  Unless of course pkg of base is NOT relevant to CURRENT builds.  In 
which case
please pardon the additional text slipping into this two-part food for 
thought... little time to
keep current on FreeBSD details vs FreeBSD ordinary usage cases.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Jeffrey Bouquet]

rsync... see bottom posting

On Tue, 15 Mar 2016 07:43:46 +0100, olli hauer  wrote:

> On 2016-03-14 15:19, Ian Lepore wrote:
> > On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote:
> >> On 13 March 2016 at 18:51, Mark Johnston  wrote:
> >>> On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote:
>  Hi,
> 
>  I can reproduce this by doing a mkimage on a large destination
>  file
>  image. it looks like it causes all the desktop processes to get
>  paged
>  out whilst it's doing so, and then the whole UI freezes until it
>  catches up.
> >>>
> >>> mkimg(1) maps the destination file with MAP_NOSYNC, so if it's
> >>> larger
> >>> than RAM, I think it'll basically force the pagedaemon to write out
> >>> the
> >>> image as it tries to reclaim pages from the inactive queue. This
> >>> can
> >>> cause stalls if the pagedaemon blocks waiting for some I/O to
> >>> complete.
> >>> The user/alc/PQ_LAUNDRY branch helps alleviate this problem by
> >>> using a
> >>> different thread to launder dirty pages. I use mkimg on various
> >>> desktop
> >>> machines to build bhyve images and have noticed the problem you're
> >>> describing; PQ_LAUNDRY helps quite a bit in that case. But I don't
> >>> know
> >>> why this would be a new problem.
> >>>
> >>
> >> That's why I'm confused. I just know that it didn't used to cause the
> >> whole UI to hang due to paging.
> >>
> > 
> > I've been noticing this too.  This machine runs 10-stable and this use
> > of the swap began happening recently when I updated from 10-stable
> > around the 10.2 release time to 10-stable right about when the 10.3
> > code freeze began.
> > 
> > In my case I have no zfs anything here.  I noticed the problem bigtime
> > yesterday when rsync was syncing a ufs filesystem of about 500GB from
> > one disk to another (probably 70-80 GB actually needed copying).  My
> > desktop apps were noticibly unresponsive when I activated a window that
> > had been idle for a while (like it would take a couple seconds for the
> > app to begin responding).  I could see lots of swap In happening in top
> > during this unresponsiveness, and noticible amounts of Out activity
> > when nothing was happening except the rsync.
> > 
> > This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it at
> > the time.  Prior to the update around the 10.3 freeze, this machine
> > would never touch the swap no matter what workload I threw at it (this
> > rsync stuff happens every day, it's the usual workload).
> > 
> 
> I'm not sure if it is the same problem, or port related.
> 
> On two systems without zfs but with many files e.g. svn servers I see now
> from time to time they are running out of swap.
> 
>  kernel: swap_pager_getswapspace(9): failed
>  kernel: swap_pager_getswapspace(16): failed
>  ...
> 
> It also happened on one system during the nightly periodic tasks holding
> only millions of backup files.
> 
> $ freebsd-version -ku
>   10.2-RELEASE-p9
>   10.2-RELEASE-p13
> 
> 
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Just a point I've bought up elsewhere...
I've, if I recall, wrecked several filesystems (although EIDE) using rsync at 
the normal bus rate, and sometimes
thumbdrives with whatever filesystem type on them.

I settled on --bwlimit=1500,  max for unattended  rsync usage and almost every 
day
use --bwlimit=700.

The latter enables several resource-intensive processes ( music, classical 
music videos, svn, pkg, browsing, etc) to
proceed apace concurrently on the desktop (SATA not EIDE) with nary a hang nor 
slowdown.

If I recall, the usual speed is 1 so that is less than ten percent, if I 
recall, of the usual speed.

YMMV.

J.

PS as an afterthough, it would be useful if that were more prominent on the man 
page somewhere or even
in the port's pkg-message or pkg-description.  
The SATA more robust than EIDE on FreeBSD that I've come across, though I 
prefer not to hint at because I
believe it to be the fault of EIDE firmware rather than FreeBSD code. FWIW.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: WITHOUT_CDDL prevents install of ctfconvert which breaks kernel build

[Julian H. Stacey]

I wrote:
> Hi current@ people
> src.conf WITHOUT_CDDL
> prevents installation of ctfconvert 
> lack of ctfconvert broke my GENERIC kernel build

Not just  /usr/src/cddl/usr.bin/ctfconvert
also need /usr/src/cddl/usr.bin/ctfmerge

> I could submit a send-pr (or whatever we call them on web now)
> so man src.conf warns of this, or do you have a better fix ?

Cheers,
Julian
--
Julian Stacey, BSD Linux Unix Sys Eng Consultant Munich http://berklix.eu/jhs/
 Mail plain text,  No quoted-printable, HTML, base64, MS.doc.
 Prefix old lines '> '  Reply below old, like play script.  Break lines by 80.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[Gary Jennejohn]

On Sun, 13 Mar 2016 16:41:17 +0100
Fabian Keil  wrote:

> Gary Jennejohn  wrote:
> 
> > In the course of the last year or so the behavior of the vm system
> > has changed in regard to how aggressively Inact memory is recycled.
> > 
> > My box has 8GB of memory.  At the moment I'm copying 100s of gigabytes
> > from one file system to another one.
> > 
> > Looking at top I observe that there are about 6GB of Inact memory.
> > This value hardly changes.  Instead of aggressively recycling the
> > Inact memory the vm now seems to prefer to swap.  
> 
> Are you using ZFS?
> 

No, only UFS, so it's not due to pressure caused by ZFS.

> > Last year, can't rmember excatly when, the behavior was totally
> > different.  The vm very aggessively recycled Inact memory and,
> > even when copying 100s of GB of files, the system hardly swapped.
> > 
> > It seems rather strange to me that the vm happily allows gigbytes
> > of Inact memory to be present and prefers swapping to recyclincg.
> >
> > Are there any sysctl's I can set to get the old behavior back?  
> 
> I don't think so.
> 
> I'm currently using this patch set to work around the issue:
> https://www.fabiankeil.de/sourcecode/electrobsd/vm-limit-inactive-memory-more-aggressively.diff
> 
> Patch 4 adds a couple of sysctls that can be used to let the ZFS
> ARC indirectly put pressure on the inactive memory until a given
> target is reached.
> 

Thanks, I'll take a closer look at it.

-- 
Gary Jennejohn
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: WITHOUT_CDDL prevents install of ctfconvert which breaks kernel build

[Howard Su]

​You need comment out the following line in GENRIC kernel conf file:
makeoptions WITH_CTF=1  # Run ctfconvert(1) for DTrace
support​


On Tue, Mar 15, 2016 at 6:49 PM, Julian H. Stacey  wrote:

> Hi current@ people
> src.conf WITHOUT_CDDL
> prevents installation of ctfconvert
> lack of ctfconvert broke my GENERIC kernel build
>
> I could submit a send-pr (or whatever we call them on web now)
> so man src.conf warns of this, or do you have a better fix ?
>
> Cheers,
> Julian
> --
> Julian Stacey, BSD Linux Unix Sys Eng Consultant Munich
> http://berklix.eu/jhs/
>  Mail plain text,  No quoted-printable, HTML, base64, MS.doc.
>  Prefix old lines '> '  Reply below old, like play script.  Break lines by
> 80.
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>



-- 
-Howard
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

WITHOUT_CDDL prevents install of ctfconvert which breaks kernel build

[Julian H. Stacey]

Hi current@ people
src.conf WITHOUT_CDDL
prevents installation of ctfconvert 
lack of ctfconvert broke my GENERIC kernel build

I could submit a send-pr (or whatever we call them on web now)
so man src.conf warns of this, or do you have a better fix ?

Cheers,
Julian
-- 
Julian Stacey, BSD Linux Unix Sys Eng Consultant Munich http://berklix.eu/jhs/
 Mail plain text,  No quoted-printable, HTML, base64, MS.doc.
 Prefix old lines '> '  Reply below old, like play script.  Break lines by 80.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[Miroslav Lachman]

Dag-Erling Smørgrav wrote on 03/14/2016 20:29:

Miroslav Lachman <000.f...@quip.cz> writes:

Bryan Drewery  writes:

https://github.com/freebsd/pkg/blob/master/scripts/pkg_tree.sh

Can you publish it as a port? I know there is one written in Perl but
I like your sh without dependencies.


It's not very useful, in my opinion.  The relationships between packages
form a directed acyclic graph, not a tree, so pkg_tree.sh will either
show too little (without -r) or far, far too much (with -r).  If you
want to visualize the package graph, you can feed the output of 'pkg
query "%n %dn"' into something like graphviz.  For other tasks, you are
better off learning how to use 'pkg query' and possibly creating your
own aliases or scripts.  It's not that difficult; feel free to ask for
help.

(Just for kicks, I tried running 'pkg_tree.sh -rn' on my desktop, which
has 934 packages installed.  It's been running for ten minutes and has
printed over 90,000 lines, with no end in sight.)


I know it. :) I had my own similar script "ports_tree.sh" to show me 
dependencies according to choosen options. I know it is too verbose and 
I use grep -v to exclude known packages from the output. The same will 
apply to pkg_tree.sh as well. I use pkg info -r, pkg info -d or pkg 
query often. The request for port of pkg_tree.sh has not high priority 
for me, it is just that this shell version is better than already 
existing pkg_tree in Perl. (which I don't use)


Miroslav Lachman

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

[José Pérez]

El 2016-03-03 11:27, Matthew Seaman escribió:

On 03/02/16 23:54, Glen Barber wrote:

Also note (as repeated below), running 'pkg delete -a' will implicitly
remove base system packages after they are installed.


This has the potential for many feet to be shot, given that up to now,
'pkg delete -a' would always leave you with a viable system.


Agreed.

Suggested workaround (a las *grep): create two pkg binaries with 
different names:
- "pkg" does what it does now and works on non-base packages by default. 
Need an extra

  arg to work on base system
- "syspkg" (or something) works by default on base system

We'd need way less crutches.

Regards,

---
José Pérez
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: how to recycle Inact memory more aggressively?

[olli hauer]

On 2016-03-14 15:19, Ian Lepore wrote:
> On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote:
>> On 13 March 2016 at 18:51, Mark Johnston  wrote:
>>> On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote:
 Hi,

 I can reproduce this by doing a mkimage on a large destination
 file
 image. it looks like it causes all the desktop processes to get
 paged
 out whilst it's doing so, and then the whole UI freezes until it
 catches up.
>>>
>>> mkimg(1) maps the destination file with MAP_NOSYNC, so if it's
>>> larger
>>> than RAM, I think it'll basically force the pagedaemon to write out
>>> the
>>> image as it tries to reclaim pages from the inactive queue. This
>>> can
>>> cause stalls if the pagedaemon blocks waiting for some I/O to
>>> complete.
>>> The user/alc/PQ_LAUNDRY branch helps alleviate this problem by
>>> using a
>>> different thread to launder dirty pages. I use mkimg on various
>>> desktop
>>> machines to build bhyve images and have noticed the problem you're
>>> describing; PQ_LAUNDRY helps quite a bit in that case. But I don't
>>> know
>>> why this would be a new problem.
>>>
>>
>> That's why I'm confused. I just know that it didn't used to cause the
>> whole UI to hang due to paging.
>>
> 
> I've been noticing this too.  This machine runs 10-stable and this use
> of the swap began happening recently when I updated from 10-stable
> around the 10.2 release time to 10-stable right about when the 10.3
> code freeze began.
> 
> In my case I have no zfs anything here.  I noticed the problem bigtime
> yesterday when rsync was syncing a ufs filesystem of about 500GB from
> one disk to another (probably 70-80 GB actually needed copying).  My
> desktop apps were noticibly unresponsive when I activated a window that
> had been idle for a while (like it would take a couple seconds for the
> app to begin responding).  I could see lots of swap In happening in top
> during this unresponsiveness, and noticible amounts of Out activity
> when nothing was happening except the rsync.
> 
> This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it at
> the time.  Prior to the update around the 10.3 freeze, this machine
> would never touch the swap no matter what workload I threw at it (this
> rsync stuff happens every day, it's the usual workload).
> 

I'm not sure if it is the same problem, or port related.

On two systems without zfs but with many files e.g. svn servers I see now
from time to time they are running out of swap.

 kernel: swap_pager_getswapspace(9): failed
 kernel: swap_pager_getswapspace(16): failed
 ...

It also happened on one system during the nightly periodic tasks holding
only millions of backup files.

$ freebsd-version -ku
  10.2-RELEASE-p9
  10.2-RELEASE-p13


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"