Re: (once again) freebsd-update cron and man whatis
On 08/11/15 08:21, Marko Cupać wrote:
- Next morning I get notification from freebsd-cron for the same update
I installed yesterday, but only for /usr/share/man/whatis. (not ok)
Did you by any chance happend to run the weekly/320.makewhatis periodic
script that night? It is enabled by default. However, you shouldn't
need it if you maintain your system entirely through freebsd-update.
Are there any differences between freebsd-update cron and
freebsd-update fetch, apart from random time offset?
freebsd-update is a shell script, so it's fairly easy to see the
difference between 'fetch' and 'cron' -- look for the functions
'cmd_fetch()' and 'cmd_cron()' in the code (they're right next to each
other.)
The differences are:
'fetch' tests that is is connected to a tty and complains about not
being run interactively if not
'cron' sleeps for a random number of seconds up to 1 hour, and it
logs everything it does to a temporary file in order to mail
back a report to the user if needed.
Otherwise, they both call exactly the same 'fetch_check_params()' and
'fetch_run()' functions.
Cheers,
Matthew
# Fetch command. Make sure that we're being called
# interactively, then run fetch_check_params and fetch_run
cmd_fetch () {
if [ ! -t 0 -a $NOTTYOK -eq 0 ]; then
echo -n `basename $0` fetch should not
echo be run non-interactively.
echo Run `basename $0` cron instead.
exit 1
fi
fetch_check_params
fetch_run || exit 1
}
# Cron command. Make sure the parameters are sensible; wait
# rand(3600) seconds; then fetch updates. While fetching updates,
# send output to a temporary file; only print that file if the
# fetching failed.
cmd_cron () {
fetch_check_params
sleep `jot -r 1 0 3600`
TMPFILE=`mktemp /tmp/freebsd-update.XX` || exit 1
if ! fetch_run ${TMPFILE} ||
! grep -q No updates needed ${TMPFILE} ||
[ ${VERBOSELEVEL} = debug ]; then
mail -s `hostname` security updates ${MAILTO} ${TMPFILE}
fi
rm ${TMPFILE}
}
signature.asc
Description: OpenPGP digital signature
(once again) freebsd-update cron and man whatis
Hi, I wrote about this some time ago here on the list: http://marc.info/?l=freebsd-stablem=143556175104543w=3 I am on 10.1-RELEASE-p17 amd64 now, and the problem still remains: - freebsd-update cron fetches new updates and notifies me by email. (ok) - I run freebsd-update install and reboot. freebsd-version shows the correct version. (ok) - Next morning I get notification from freebsd-cron for the same update I installed yesterday, but only for /usr/share/man/whatis. (not ok) Not 100% sure, but I think that running both freebsd-fetch and freebsd-install from an interactive ssh session corrects this until the next update. Are there any differences between freebsd-update cron and freebsd-update fetch, apart from random time offset? Regards, -- Marko Cupać https://www.mimar.rs/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: (once again) freebsd-update cron and man whatis
On Tue, 11 Aug 2015 09:31:04 +0100 Matthew Seaman matt...@freebsd.org wrote: On 08/11/15 08:21, Marko Cupać wrote: - Next morning I get notification from freebsd-cron for the same update I installed yesterday, but only for /usr/share/man/whatis. (not ok) Did you by any chance happend to run the weekly/320.makewhatis periodic script that night? It is enabled by default. However, you shouldn't need it if you maintain your system entirely through freebsd-update. Hi Matthew, thank you for looking into it. I do maintain most of my systems entirely through freebsd-update, but with modified freebsd-update.conf so that it does not fetch sources. I have two systems which are redownloading /usr/share/man/whatis since Sunday update to -p17. Would things be clearer if I ran weekly/320.makewhatis and observed if that prevented re-downloads? What would be long-term solution for this? I guess /usr/share/man/whatis is compiled from src. As I have removed 'src' from freebsd-update.conf, and I have nothing under /usr/src, should I add /usr/share/man/whatis to IgnorePaths? I see it is already there for IDSIgnorePaths. Regards, -- Marko Cupać https://www.mimar.rs/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: (once again) freebsd-update cron and man whatis
On Tue, 2015-08-11 at 11:48 +0200, Marko Cupać wrote: On Tue, 11 Aug 2015 09:31:04 +0100 Matthew Seaman matt...@freebsd.org wrote: On 08/11/15 08:21, Marko Cupać wrote: - Next morning I get notification from freebsd-cron for the same update I installed yesterday, but only for /usr/share/man/whatis. (not ok) Did you by any chance happend to run the weekly/320.makewhatis periodic script that night? It is enabled by default. However, you shouldn't need it if you maintain your system entirely through freebsd-update. Hi Matthew, thank you for looking into it. I do maintain most of my systems entirely through freebsd-update, but with modified freebsd-update.conf so that it does not fetch sources. I have two systems which are redownloading /usr/share/man/whatis since Sunday update to -p17. Would things be clearer if I ran weekly/320.makewhatis and observed if that prevented re-downloads? What would be long-term solution for this? I guess /usr/share/man/whatis is compiled from src. As I have removed 'src' from freebsd-update.conf, and I have nothing under /usr/src, should I add /usr/share/man/whatis to IgnorePaths? I see it is already there for IDSIgnorePaths. Regards, I too have freebsd-update cron trying to replace /usr/share/man/whatis on a subset of my installs. I presume that the 320.makewhatis is generating the new whatis, but the 320.makewhatis is more complete. # diff whatis.freebsdupdate whatis.makewhatis 365a366 bcd(6), ppt(6) - reformat input as punch cards or paper tape 471a473 caesar(6), rot13(6) - decrypt caesar ciphers 848a851 factor(6), primes(6) - factor a number, generate primes 942a946 fortune(6) - print a random, hopefully interesting, adage 1143a1148 grdc(6) - grand digital clock (curses) 1886a1892 morse(6) - reformat input as morse code 2093a2100 number(6)- convert Arabic numerals to English 2379a2387 pom(6) - display the phase of the moon 2536a2545 random(6)- random lines from a file or random numbers 2956a2966 strfile(8), unstr(8) - create a random access file for storing strings This is what I get on a DigitalOcean FreeBSD 10.1-RELEASE which I have only ever administered with freebsd-update and pkgng. I also note all of these including strfile(8) and unstr(8) reside in /usr/games. I would like to echo Marko's question what is the 'correct' way to handle this situation? Regards, Michael B. Eichorn smime.p7s Description: S/MIME cryptographic signature
Continuously increasing L2ARC header size
Dear FreeBSD community, We're facing a somewhat odd issue, perhaps similar to what is discussed here: https://forums.freebsd.org/threads/l2arc-degraded.47540/ The issue is that the L2ARC header seems to grow without limit, similar to a memory leak, pressuring more and more memory over time out of the ARC. For example, the output of zpool iostat -v 1 capacity operationsbandwidth pool alloc free read write read write - - - - - - syspool 1.15G 275G 0 0 0 0 mirror 1.15G 275G 0 0 0 0 gpt/zfs0 - - 0 0 0 0 gpt/zfs1 - - 0 0 0 0 - - - - - - tank 1.21T 1.51T229 1.99K 3.67M 9.48M mirror 124G 154G 67125 787K 503K da0 - - 20 27 440K 503K da1 - - 45 28 379K 503K [...] mirror 124G 154G 34164 454K 612K da18 - - 26 12 417K 612K da19 - - 6 13 58.8K 612K logs - - - - - - mirror 117M 74.4G 0109 0 1.75M da21 - - 0109 0 1.75M da22 - - 0109 0 1.75M cache - - - - - - da231.67T 16.0E302 7 2.85M 223K - - - - - - Here the cache shows 1.67T, in use and 16.0E free. The cache is a 373GB Intel SSD. # diskinfo -v da23 da23 512 # sectorsize 400088457216# mediasize in bytes (373G) 781422768 # mediasize in sectors 4096# stripesize 0 # stripeoffset 48641 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. BTTV4234089C400HGN # Disk ident. id1,enc@n500e004aaa3e/type@0/slot@18# Physical path The L2ARC stats section from zfs-stats -a: L2 ARC Summary: (DEGRADED) Passed Headroom:133.33m Tried Lock Failures:4.90b IO In Progress: 313.63k Low Memory Aborts: 1.52k Free on Write: 589.79k Writes While Full: 34.57k R/W Clashes:46.95k Bad Checksums: 408.40m IO Errors: 151.99m SPA Mismatch: 632.00m L2 ARC Size: (Adaptive) 1.89TiB Header Size:0.88% 16.98 GiB L2 ARC Evicts: Lock Retries: 1.27k Upon Reading: 2 L2 ARC Breakdown: 2.10b Hit Ratio: 32.89% 691.15m Miss Ratio: 67.11% 1.41b Feeds: 3.70m L2 ARC Buffer: Bytes Scanned: 10.70 PiB Buffer Iterations: 3.70m List Iterations:236.30m NULL List Iterations: 24.86m L2 ARC Writes: Writes Sent:100.00% 3.38m Here we can see that currently the Header Size is almost 17gb. This header size grows continuously without (apparent) limit. Also zfs appears to think it's holding 1.89 TiB inside the L2ARC, which seems very very unlikely. # freebsd-version 10.1-RELEASE-p13 # uname -a FreeBSD servername 10.1-RELEASE-p10 FreeBSD 10.1-RELEASE-p10 #0: Wed May 13 06:54:13 UTC 2015 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 # uptime 4:35PM up 42 days, 15:24, 1 user, load averages: 1.35, 0.96, 0.84 Does anyone know how we can alleviate the issue? We originally thought the issue was caused by https://www.freebsd.org/security/advisories/FreeBSD-EN-15:07.zfs.asc We have updated our Servers since but the header size seems to keep growing still. For reference, we have multiple bsd fileservers which are used mostly over NFS, all with identical configuration (but varying workload). They all still show these symptoms. Any tips/hints/pointers are appreciated! With kind regards, Daniel ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Continuously increasing L2ARC header size
Hello everyone, looking closer and comparing our servers, we infact have 2 servers that behave differently, but they also have a different workload (mail instead of file storage). They have identical hardware, but were installed at a later time, compared to the servers which do have the issue. They have a stable l2arc cache size, not like the one i described previously, where the l2arc size is bigger than the dataset in the pool. Nonleaking server: L2 ARC Summary: (HEALTHY) Passed Headroom:63.05m Tried Lock Failures:198.57m IO In Progress: 53.57k Low Memory Aborts: 32 Free on Write: 21.40k Writes While Full: 16.50k R/W Clashes:3.50k Bad Checksums: 0 IO Errors: 0 SPA Mismatch: 613.80m L2 ARC Size: (Adaptive) 443.42 GiB Header Size:0.27% 1.22GiB L2 ARC Evicts: Lock Retries: 1.33k Upon Reading: 0 L2 ARC Breakdown: 191.36m Hit Ratio: 28.27% 54.09m Miss Ratio: 71.73% 137.27m Feeds: 1.68m L2 ARC Buffer: Bytes Scanned: 4.28PiB Buffer Iterations: 1.68m List Iterations:107.55m NULL List Iterations: 1.16m L2 ARC Writes: Writes Sent:100.00% 915.04k No bad checksums or IO Errors. The l2arc size of 443gb is sensible compared to it's actual size (373gb). Yet aside of the workload I cannot find any difference between the 2 mail fileservers and the other 6+ web fileservers doing storage for websites. They're identical in every regard, aside of the workload and the time of installation. The mail fileservers were installed much later in comparison. I just wanted to add this, as it maybe very relevant. With kind regards, Daniel On 08/11/2015 04:42 PM, Daniel Genis wrote: Dear FreeBSD community, We're facing a somewhat odd issue, perhaps similar to what is discussed here: https://forums.freebsd.org/threads/l2arc-degraded.47540/ The issue is that the L2ARC header seems to grow without limit, similar to a memory leak, pressuring more and more memory over time out of the ARC. For example, the output of zpool iostat -v 1 capacity operationsbandwidth pool alloc free read write read write - - - - - - syspool 1.15G 275G 0 0 0 0 mirror 1.15G 275G 0 0 0 0 gpt/zfs0 - - 0 0 0 0 gpt/zfs1 - - 0 0 0 0 - - - - - - tank 1.21T 1.51T229 1.99K 3.67M 9.48M mirror 124G 154G 67125 787K 503K da0 - - 20 27 440K 503K da1 - - 45 28 379K 503K [...] mirror 124G 154G 34164 454K 612K da18 - - 26 12 417K 612K da19 - - 6 13 58.8K 612K logs - - - - - - mirror 117M 74.4G 0109 0 1.75M da21 - - 0109 0 1.75M da22 - - 0109 0 1.75M cache - - - - - - da231.67T 16.0E302 7 2.85M 223K - - - - - - Here the cache shows 1.67T, in use and 16.0E free. The cache is a 373GB Intel SSD. # diskinfo -v da23 da23 512 # sectorsize 400088457216# mediasize in bytes (373G) 781422768 # mediasize in sectors 4096# stripesize 0 # stripeoffset 48641 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. BTTV4234089C400HGN # Disk ident. id1,enc@n500e004aaa3e/type@0/slot@18# Physical path The L2ARC stats section from zfs-stats -a: L2 ARC Summary: (DEGRADED) Passed Headroom:133.33m Tried Lock Failures:4.90b IO In Progress: 313.63k Low Memory Aborts: 1.52k Free on Write: 589.79k Writes While Full: 34.57k R/W Clashes:
