Re: my build time impact of clang 5.0
Andy Farkas writes: > Perhaps you could hack src/tools/tools/whereintheworld/whereintheworld.pl > > -andyf So, I creatd a slightly different build script in perl; kinda works but needs to be optimized: https://github.com/danmack/freebsd-buildtools Basic output looks like with timing on each "section": Building FreeBSD (svn: 324242) Build Time : 20171003.174621 SRC URL: https://svn.freebsd.org/base/stable/11 KERNEL CONF: GENERIC BUILD UUID : 20171003.174621|https://svn.freebsd.org/base/stable/11|GENERIC|/usr/src SRC DIR: /usr/src LOG DIR: /var/log/bsdbuild/324242 ... building phase buildworld ... >>> World build started ... 00:00:02 : 00:00:04 >>> Rebuilding the temporary build tree ... 00:00:00 : 00:00:06 >>> stage 1.1: legacy release compatibility s ... 00:00:00 : 00:00:08 >>> stage 1.2: bootstrap tools... 00:00:00 : 00:00:10 >>> stage 2.1: cleaning up the object tree... 00:03:43 : 00:03:55 >>> stage 2.2: rebuilding the object tree ... 00:01:01 : 00:04:58 >>> stage 2.3: build tools... 00:00:21 : 00:05:21 >>> stage 3: cross tools ... 00:00:04 : 00:05:28 >>> stage 3.1: recording compiler metadata... 00:00:44 : 00:06:14 >>> stage 4.1: building includes ... 00:00:00 : 00:06:16 >>> stage 4.2: building libraries ... 00:00:19 : 00:06:37 Dan ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: firefox compilation problem
03.10.2017, 22:13, "Kurt Jaeger" : > Hi! > >> > I've got the following problem: the firefox building from ports ends with >> errors: > > [...] > > There's a problem report for this: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222641 > Hi Kurt, thank you for your response. I can confirm the reference above describes a possible solution of my problem. -- Regards, S.Grigoriev. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: firefox compilation problem
Hi, * Kurt Jaeger [171003 21:03]: >> I've got the following problem: the firefox building from ports ends with >> errors: >> -- >> x86_64-unknown-freebsd/release/libgkrust.a: could not read symbols: File >> format not recognized > Same here, with a build in poudriere. > http://people.freebsd.org/~pi/logs/firefox-56.0_1,1.log maybe this is related to the recent switch to LLVM 5 and associated libraries? I rebuilt all ports on my system after updating stable past the llvm switch and all ports including firefox built without problem. Wolfgang ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: firefox compilation problem
Hi! > > I've got the following problem: the firefox building from ports ends with > > errors: [...] There's a problem report for this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222641 -- p...@opsec.eu+49 171 3101372 3 years to go ! ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: firefox compilation problem
Hi! > I've got the following problem: the firefox building from ports ends with > errors: > > -- > x86_64-unknown-freebsd/release/libgkrust.a: could not read symbols: File > format not recognized Same here, with a build in poudriere. http://people.freebsd.org/~pi/logs/firefox-56.0_1,1.log -- p...@opsec.eu+49 171 3101372 3 years to go ! ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
firefox compilation problem
Hi list, I've got the following problem: the firefox building from ports ends with errors: -- x86_64-unknown-freebsd/release/libgkrust.a: could not read symbols: File format not recognized c++: error: linker command failed with exit code 1 (use -v to see invocation) gmake[7]: *** [/usr/ports/www/firefox/work/firefox-56.0/config/rules.mk:719: libxul.so] Error 1 gmake[7]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0/obj-x86_64-unknown-freebsd11.1/toolkit/library' gmake[6]: *** [/usr/ports/www/firefox/work/firefox-56.0/config/recurse.mk:73: toolkit/library/target] Error 2 gmake[6]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0/obj-x86_64-unknown-freebsd11.1' gmake[5]: *** [/usr/ports/www/firefox/work/firefox-56.0/config/recurse.mk:33: compile] Error 2 gmake[5]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0/obj-x86_64-unknown-freebsd11.1' gmake[4]: *** [/usr/ports/www/firefox/work/firefox-56.0/config/rules.mk:453: default] Error 2 gmake[4]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0/obj-x86_64-unknown-freebsd11.1' gmake[3]: *** [/usr/ports/www/firefox/work/firefox-56.0/client.mk:419: realbuild] Error 2 gmake[3]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0' gmake[2]: *** [/usr/ports/www/firefox/work/firefox-56.0/client.mk:170: build] Error 2 gmake[2]: Leaving directory '/usr/ports/www/firefox/work/firefox-56.0' *** Error code 1 -- My system is: FreeBSD 11.1-STABLE #0 r324239: Tue Oct 3 18:10:29 MSK 2017 root@am:/usr/obj/usr/src/sys/VNET11 amd64 Any tips are appreciated. -- Regards, S.Grigoriev. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: my build time impact of clang 5.0
Jakub Lach writes: > On the other hand, I'm having tremendous increases in Unixbench scores > comparing to > 11-STABLE in the April (same machine, clang 4 then, clang 5 now) (about > 40%). > > I have never seen something like that, and I'm running Unixbench on -STABLE > since > 2008. Agree; clang/llvm and friends have added a lot of value. It's worth it I think. It is however getting harder to continue with a source based update model, which I prefer even though most people just use package managers today. I still like to read the commits and understand what's changing, why, and select the version I am comfortable with given the nuances of my configuration(s). I think that's why 'knock-on-wood' I've been able to track mostly CURRENT and/or STABLE without any outages since about 1998 on production systems :-) ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
Bezüglich Harry Schmalzbauer's Nachricht vom 03.10.2017 16:39 (localtime): > Bezüglich Andriy Gapon's Nachricht vom 03.10.2017 16:28 (localtime): >> On 03/10/2017 17:19, Harry Schmalzbauer wrote: >>> Have tried several different txg IDs, but the latest 5 or so lead to the >>> panic and some other random picked all claim missing devices... >>> Doh, if I only knew about -T some days ago, when I had all 4 devices >>> available. >> I don't think that the error is really about the missing devices. >> Most likely the real problem is that you are going too far back in history >> where >> the data required to import the pool is not present. It's just that there >> is no >> special error code to report that condition distinctly, so it gets >> interpreted >> as a missing device condition. > Sounds reasonable. > When the RAM-corruption happened, a live update was started, where > several pool availability checks were done. No data write. > Last data write were view KBytes some minutes before the corruption, and > the last significant ammount written to that pool was long time before that. > So I still have hope to find an importable txg ID. > > Are they strictly serialized? Seems so. Just for the records, I couldn't recover any data yet, but in general, if a pool isn't damaged that much, the following promising steps were the ones I got closest: I have attached dumps of the physical disks as md2 and md3. 'zpool import' offers cetusPsysDEGRADED mirror-0 DEGRADED 8178308212021996317 UNAVAIL cannot open md3 ONLINE mirror-1 DEGRADED md2p5ONLINE 4036286347185017167 UNAVAIL cannot open Which is ḱnown to be corrupt. This time I also attached zdb(8) dumps (sparse files) of the remaining two disks, resp. partition. Now import offers this: pool: cetusPsys id: 13207378952432032998 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: cetusPsys ONLINE mirror-0 ONLINE md5 ONLINE md3 ONLINE mirror-1 ONLINE md2p5 ONLINE md4 ONLINE 'zdb -ue cetusPsys' showed me the latest txg ID (3757573 in my case). So I decremented the txg ID by one and repeated until the following fatal panicing indicator vanished: loading space map for vdev 1 of 2, metaslab 108 of 109 ... WARNING: blkptr at 0x80e0ead00 has invalid CHECKSUM 1 WARNING: blkptr at 0x80e0ead00 has invalid COMPRESS 0 WARNING: blkptr at 0x80e0ead00 DVA 0 has invalid VDEV 2337865727 WARNING: blkptr at 0x80e0ead00 DVA 1 has invalid VDEV 289407040 WARNING: blkptr at 0x80e0ead00 DVA 2 has invalid VDEV 3959586324 Which was 'zdb -c -t 3757569 -AAA -e cetusPsys': Traversing all blocks to verify metadata checksums and verify nothing leaked ... loading space map for vdev 1 of 2, metaslab 108 of 109 ... 89.0M completed ( 6MB/s) estimated time remaining: 3hr 34min 47sec zdb_blkptr_cb: Got error 122 reading <69, 0, 0, c> -- skipping 86.8G completed ( 588MB/s) estimated time remaining: 0hr 00min 00sec Error counts: errno count 122 1 leaked space: vdev 0, offset 0xa01084200, size 512 leaked space: vdev 0, offset 0xd0dc23c00, size 512 leaked space: vdev 0, offset 0x2380182200, size 3072 leaked space: vdev 0, offset 0x2380189a00, size 1536 leaked space: vdev 0, offset 0x2380183000, size 1536 leaked space: vdev 0, offset 0x238039a200, size 2560 leaked space: vdev 0, offset 0x238039be00, size 18944 leaked space: vdev 0, offset 0x23801b3200, size 9216 leaked space: vdev 0, offset 0x33122a8800, size 512 leaked space: vdev 1, offset 0x2808f1600, size 512 leaked space: vdev 1, offset 0x2808f1e00, size 512 leaked space: vdev 1, offset 0x2808f2e00, size 4096 leaked space: vdev 1, offset 0x2808f1a00, size 512 leaked space: vdev 1, offset 0x9010e6c00, size 512 leaked space: vdev 1, offset 0x23c5ad9c00, size 512 leaked space: vdev 1, offset 0x2e00ad4800, size 512 leaked space: vdev 1, offset 0x2f0030b200, size 50176 leaked space: vdev 1, offset 0x2f000ca800, size 512 leaked space: vdev 1, offset 0x2f003a9800, size 15360 leaked space: vdev 1, offset 0x2f003af600, size 13312 leaked space: vdev 1, offset 0x2f00715c00, size 1024 leaked space: vdev 1, offset 0x2f003adc00, size 6144 leaked space: vdev 1, offset 0x2f00363600, size 38912 block traversal size 93540302336 != alloc 93540473344 (leaked 171008) bp count: 3670624 ganged count: 0 bp logical:96083156992 avg: 26176 bp physical: 93308853248 avg: 25420 compression: 1.03 bp allocated: 93540302336 avg: 25483 compression: 1.03 bp deduped: 0ref>1: 0 deduplication: 1.00 SPA allocated: 93540473344 used: 19.98% additional, non-pointer bps of type 0: 48879 Dittoed blocks on same vdev: 23422 In my case, import didn't work with the highest non-panicing txg ID: zpool imp
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
Bezüglich Andriy Gapon's Nachricht vom 03.10.2017 16:28 (localtime): > On 03/10/2017 17:19, Harry Schmalzbauer wrote: >> Have tried several different txg IDs, but the latest 5 or so lead to the >> panic and some other random picked all claim missing devices... >> Doh, if I only knew about -T some days ago, when I had all 4 devices >> available. > > I don't think that the error is really about the missing devices. > Most likely the real problem is that you are going too far back in history > where > the data required to import the pool is not present. It's just that there is > no > special error code to report that condition distinctly, so it gets interpreted > as a missing device condition. Sounds reasonable. When the RAM-corruption happened, a live update was started, where several pool availability checks were done. No data write. Last data write were view KBytes some minutes before the corruption, and the last significant ammount written to that pool was long time before that. So I still have hope to find an importable txg ID. Are they strictly serialized? Dou you know any other possibility to get data from a dataset (by objetct id) without importing the whole pool? zdz successfully checks the one datset (object ID) I'm interested in; the rest of the pool isnt much an issue... Thank you very much for your help! -harry ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
On 03/10/2017 17:19, Harry Schmalzbauer wrote: > Have tried several different txg IDs, but the latest 5 or so lead to the > panic and some other random picked all claim missing devices... > Doh, if I only knew about -T some days ago, when I had all 4 devices > available. I don't think that the error is really about the missing devices. Most likely the real problem is that you are going too far back in history where the data required to import the pool is not present. It's just that there is no special error code to report that condition distinctly, so it gets interpreted as a missing device condition. -F/-X/-T is not guaranteed to work as the old (freed, overwritten) data is not kept indefinitely. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
Bezüglich Andriy Gapon's Nachricht vom 03.10.2017 11:20 (localtime): > On 03/10/2017 11:43, Harald Schmalzbauer wrote: > ... >> action: The pool can be imported despite missing or damaged devices. The >> fault tolerance of the pool may be compromised if imported. > ... >> Is it impossible to import degraded pools in general, or only together> with >> "-X -T"? > It should be possible to import degraded pools... > Perhaps the pool originally had more devices? Like log devices. > Or maybe there is some issue with the txg you picked. > > By the way, I think that you didn't have to provide -T option for -F or -X. > It's either -F or -X or -T , the first two try to figure out txg > automatically. But I could be wrong. You're right that -T works without F[X] flag, but -X needs -F. Unfortunately not specifying a distinct txg leads to panic. Specifying one leads to "device is missing". Which is true, but only redundant data... It's abosultely sure that this pool never had any log or cache or other device than the two mirror vdevs. zdb -l confirms that. Have tried several different txg IDs, but the latest 5 or so lead to the panic and some other random picked all claim missing devices... Doh, if I only knew about -T some days ago, when I had all 4 devices available. I haven't expected problems due to missing redundant mirrors. Can anybody imagine why degraded import doesn't work in my case and how to work arround? Will try to provide the sparse zdb dumps in addition, maybe that changes anything. But I'm sure these don't have much data., dump time was within 3 seconds at most. Thanks, -harry ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: my build time impact of clang 5.0
On the other hand, I'm having tremendous increases in Unixbench scores comparing to 11-STABLE in the April (same machine, clang 4 then, clang 5 now) (about 40%). I have never seen something like that, and I'm running Unixbench on -STABLE since 2008. -- Sent from: http://freebsd.1045724.x6.nabble.com/freebsd-stable-f3932046.html ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: my build time impact of clang 5.0
On Oct 03 09:46, Andriy Gapon wrote: On 02/10/2017 21:34, Dan Mack wrote: Another significant change in build times this week - not complaining, just my observations on build times; same server doing buildworld during the various phases of compiler changes over the last year or so FWIW: |--+--+---+--+---| | Ver (svn-id) | World (mins) | Kernel (mins) | Relative | Comment | |--+--+---+--+---| | 292733 | 90 |16 | 0.5 | | | 299948 | 89 |16 | 0.5 | | | 322724 | 174 |21 | 1.0 | clang 4.x | | 323310 | 175 |21 | 1.0 | clang 4.x | | 323984 | 175 |21 | 1.0 | clang 4.x | | 324130 | 285 |21 | 1.6 | clang 5.x | | 324204 | 280 |21 | 1.6 | clang 5.x | |--+--+---+--+---| It shocked me to a realize that I can build several platforms that do not have clang support yet (like powerpc) one after another in a fraction of time required to build just amd64. It is insane now how long it takes on lesser powered hardware. When we had gcc as the system compiler I could do a full buildworld/kernel in around 3 hours. With the first version of clang that we had imported I think it increased to around 6 hours. Clang4 made it around 8 hours. And now clang5 it took 12 hours! This is on an Intel Atom D525 with amd64. And I have a few things disabled too like lib32 and profiled libraries. -- Matt ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
On 03/10/2017 11:43, Harald Schmalzbauer wrote: ... > action: The pool can be imported despite missing or damaged devices. The > fault tolerance of the pool may be compromised if imported. ... > Is it impossible to import degraded pools in general, or only together> with > "-X -T"? It should be possible to import degraded pools... Perhaps the pool originally had more devices? Like log devices. Or maybe there is some issue with the txg you picked. By the way, I think that you didn't have to provide -T option for -F or -X. It's either -F or -X or -T , the first two try to figure out txg automatically. But I could be wrong. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: panic: Solaris(panic): blkptr invalid CHECKSUM1
Bezüglich Harry Schmalzbauer's Nachricht vom 02.10.2017 20:28 (localtime): > Bezüglich Andriy Gapon's Nachricht vom 02.10.2017 13:49 (localtime): >> On 01/10/2017 00:38, Harry Schmalzbauer wrote: >>> Now my striped mirror has all 4 devices healthy available, but all >>> datasets seem to be lost. >>> No problem for 450G (99,9_%), but there's a 80M dataset which I'm really >>> missing :-( >> If it's not too late now, you may try to experiment with an "unwind" / >> "extreme >> unwind" import using -F -n / -X -n. Or manually specifying a txg number for >> import (in read-only mode). > Thanks for your reply! > > I had dumped one of each mirror's drive and attaching it as memory disk > works as intended. > So "zfs import" offers me the corrupt backup (on the host with a already > recreated pool). > > Unfortunately my knowledge about ZFS internals (transaction group number > relations to (ü)uberblocks) doesn't allow me to follow your hint. > > How can I determine the last txg#, resp. the ones before the last? > I guess 'zpool import -t' is the tool/parameter to use. > ZFS has wonderful documentation, but although this was a perfect reason > to start learning the details about my beloved ZFS, I don't have the > time to. > > Is there a zdb(8) aequivalent of 'zpool import -t', so I can issue the > zdb check, wich doesn't crash the kernel but only zdb(8)? > > For regular 'zpool import', 'zdb -ce' seems to be such a synonym. At > least the crash report is identical, see my reply to Scott Bennett's post.. Made some progress. Unfortunately the zpool(8) man page doesn't mention some available options for the 'import' command. -T was the important one for me some days ago... Utilizing internet search engines would have discovered -T, but I haven't tried... This post describes the -T (-t for zdb) option very well: https://lists.freebsd.org/pipermail/freebsd-hackers/2013-July/043131.html After attaching the dumps from two of the 4 drives as memory disk, 'zpool import' offers me: pool: cetusPsys id: 13207378952432032998 state: DEGRADED status: The pool was last accessed by another system. action: The pool can be imported despite missing or damaged devices. The fault tolerance of the pool may be compromised if imported. see: http://illumos.org/msg/ZFS-8000-EY config: cetusPsysDEGRADED mirror-0 DEGRADED 8178308212021996317 UNAVAIL cannot open md3 ONLINE mirror-1 DEGRADED md2p5ONLINE 4036286347185017167 UNAVAIL cannot open With 'zdb -lu /dev/md3' I picked a TXGid. The following doesn't panic the kernel: zpool import -o readonly=on -f -R /net -F -X -n -T 3757541 13207378952432032998 cetusPalt But: As soon as I don't tell test-run (not specifying -n), it fails with: cannot import 'cetusPsys': one or more devices is currently unavailable I'm clueless again :-( Is it impossible to import degraded pools in general, or only together with "-X -T"? Any tricks to convince zpool import to ignore the missing devices? I dumped only 2 of 4 disks, the physical disks are in usage. But people made me hope I have chances to recover data :-) Seems -T had really offered a way to do that, but I wasn't aware of it some days ago, so I only have these two tumps, but all data needed should be available – I thought... Thanks, -harry signature.asc Description: OpenPGP digital signature