Re: [gentoo-dev] New project: LLVM
On Sat, Aug 20, 2016 at 5:41 AM, james wrote: > On 08/19/2016 05:05 PM, C Bergström wrote: >> >> On Sat, Aug 20, 2016 at 4:52 AM, james wrote: >> >> >> > > You removed your rude remark::: > " Sorry to be the party crasher, but..." > > So let's put it back, just for clarity. I'll forgive you because it seems English may not be your native language, but this expression is rarely to never seen as offensive. Further, you should be able to infer I was talking about *my* own party. I was clarifying about the limitations of using "my" compiler in a system context. This had nothing to do you. So again please keep your tone civil, adult and if possible on-topic.
Re: [gentoo-dev] New project: LLVM
On 08/19/2016 05:05 PM, C Bergström wrote: On Sat, Aug 20, 2016 at 4:52 AM, james wrote: You removed your rude remark::: " Sorry to be the party crasher, but..." So let's put it back, just for clarity. Back to my own glass house.. It will take a few years, but I am trying to make it easier (internally) to expose in some clear way all the pieces which compose a fine tuning per-processor. If this was "just" scheduling models it would be really easy, but it's not.. Those latencies and other magic bits decide things like.. "should I unroll this loop or do something else" and then you venture into the land of accelerators where a custom regalloc may be what you really need and *nothing* off the shelf fits to meet your goals.. (projects like that can take 9 months and in the end only give a general 1-5% median performance gain..) If this is your mantra, I resend the generous comments. Cray use to work that way, milking the Petroleum Industry for tons of money, but, things have changed and the change is accelerating, rapidly. Perhaps too much off those Cray patents that your company owns are leaking toxins into the brain-trust where you park? Vendor walk-back is sad, imho. ymmv. Best of luck to your company's 5-year plan I have no idea what on earth you were trying to say in most of your reply. I am speaking only from a compiler perspective. Nothing more and nothing less.. it may be my difficultly to describe the target level and processor specific optimization choices a compiler *must* make. Nobody disagreed with the principals you espoused. If compiler development and optimizations all occurred glacially as you suggest for all things related, we'd be nowhere near where the industry currently is. In fact this sort of work is greatly accelerating. ymmv. So, I included some prose to 'encourage' folks that this work is open to all and a very prosperous area. Just because I dipped a bit into the financial status of folks involved, is no reason for you to be insulted. Facts are facts and the assimilation of diverse data usually enhances one's personal evaluation of where they should spend their technical attention. Beyond not understanding your email, I found it insulting. If you did not understand what I wrote, how could you be insulted? > So please keep rude comments to yourself. Right back at you. Your opening remark "Sorry to be the party crasher, but..." Was found by me to be highly insulting, inaccurate and discouraging to folks to learn about some of the inner goals of HPC. It was totally rude and discouraging. Keep that sort of attitude to yourself, thank you. It's simple to ignore what you do not like or disagree with, most of us do that all the time. YOU direct an insult directly at me, high or low brow, your gonna get 'domain data', that sheds light, right back at you. So again to try to explain the technical side of this - We can't and have no desire to optimize for every class of processor on the planet. We have a narrow band of focus on mostly HPC centric code patterns and processors which are are typically used in HPC workloads. I'd love to expand past this, but we're a small company and that's our niche. There's no walking back or trying to claim to be something we're not.. this is pure honest transparency. (imagine it like - do one thing and do it well) The only special note I'd add on to this - the CPU isn't where we spend most of our time tuning, it's by far more on the accelerator support. hth, James
Re: [gentoo-dev] New project: LLVM
On Sat, Aug 20, 2016 at 4:52 AM, james wrote: >> Back to my own glass house.. It will take a few years, but I am trying >> to make it easier (internally) to expose in some clear way all the >> pieces which compose a fine tuning per-processor. If this was "just" >> scheduling models it would be really easy, but it's not.. Those >> latencies and other magic bits decide things like.. "should I unroll >> this loop or do something else" and then you venture into the land of >> accelerators where a custom regalloc may be what you really need and >> *nothing* off the shelf fits to meet your goals.. (projects like that >> can take 9 months and in the end only give a general 1-5% median >> performance gain..) > > > If this is your mantra, I resend the generous comments. Cray use to work > that way, milking the Petroleum Industry for tons of money, but, things have > changed and the change is accelerating, rapidly. Perhaps too much off those > Cray patents that your company owns are leaking toxins into the brain-trust > where you park? > > Vendor walk-back is sad, imho. ymmv. > > Best of luck to your company's 5-year plan I have no idea what on earth you were trying to say in most of your reply. I am speaking only from a compiler perspective. Nothing more and nothing less.. it may be my difficultly to describe the target level and processor specific optimization choices a compiler *must* make. Beyond not understanding your email, I found it insulting. So please keep rude comments to yourself. So again to try to explain the technical side of this - We can't and have no desire to optimize for every class of processor on the planet. We have a narrow band of focus on mostly HPC centric code patterns and processors which are are typically used in HPC workloads. I'd love to expand past this, but we're a small company and that's our niche. There's no walking back or trying to claim to be something we're not.. this is pure honest transparency. (imagine it like - do one thing and do it well) The only special note I'd add on to this - the CPU isn't where we spend most of our time tuning, it's by far more on the accelerator support.
Re: [gentoo-dev] New project: LLVM
On 08/19/2016 02:20 PM, C Bergström wrote: Sorry to be the party crasher, but... I'd love to have optimizations for everything out there, but it takes a lot of work to fine tune for something specific. Agreed. Right now on Armv8 alone, there are dozens of teams working on the identical concepts presented in this thread. Most are also targeting specific domains. At some point there with pathways, just like in Computational Chemistry, where the optimization pathway for new silicon is fast and previous work helps tremendously. That is, you are not alone in your quests, far, far from it. Right now I see a few variants of ARMv8 ARM reference stuff - A57 cores and the newer bits.. The scheduling and stuff seems more-or-less similar enough that one tuning could probably work for the vast majority of these parts. Cavium ThunderX - It's ground up and quite different from the ARM reference stuff under the hood APM - Mustang, again ground up and different. I don't have enough hands on to know how different from reference. Broadcom - Coming Soon(tm) - Again no hands on or any data, but certainly very interesting.. ... now add in every variant of ground up implementation and you have 50 shades of gray.. And billions of dollars financing those efforts in parallel. It's an arms race, (like the pun?). Wonder why a Japanese conglomerate offered to purchase ARM ltd. for such a large figure? Wonder why intel has arm licenses now? Your group might only be able to focus on a few ARM offerings, but there are dozens and dozens of ARM teams alone that would dispute your arithmetic above. - Soo.. depending on your target hardware, you may be better off with gcc if the end goal is general all-around performance. (It does a quite respectable job of being generic) I realize a lot of people have strong feelings for or against it. I leave that to the reader to decide.. You misconstrue concepts. Nobody, especially me, implies that one pathway (to a Unikernel [1] if you like) suites all near-optimized solutions. That would be pointless. What you allude to, already exists in some of the more progressive data/cloud vendor clouds. We are talking about a unikernel for different classes of problems, across arm8 and x86-64 and GPU architectures, not thousands of (arch) processor variants. However, those other processor (arch) variants and the folks that earn a living off of those variants, are not sitting back idle, either. Back to my own glass house.. It will take a few years, but I am trying to make it easier (internally) to expose in some clear way all the pieces which compose a fine tuning per-processor. If this was "just" scheduling models it would be really easy, but it's not.. Those latencies and other magic bits decide things like.. "should I unroll this loop or do something else" and then you venture into the land of accelerators where a custom regalloc may be what you really need and *nothing* off the shelf fits to meet your goals.. (projects like that can take 9 months and in the end only give a general 1-5% median performance gain..) If this is your mantra, I resend the generous comments. Cray use to work that way, milking the Petroleum Industry for tons of money, but, things have changed and the change is accelerating, rapidly. Perhaps too much off those Cray patents that your company owns are leaking toxins into the brain-trust where you park? Vendor walk-back is sad, imho. ymmv. Best of luck to your company's 5-year plan [2] http://unikernel.org/ hth, James -- On Sat, Aug 20, 2016 at 2:02 AM, james wrote: On 08/19/2016 11:15 AM, C Bergström wrote: On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato wrote: BTW is pathscale ready to be used as system compiler as well? I wish, but no. We have known issues when building grub2, glibc and the Linux kernel at the very least. Someone* did report a long time ago that with their unofficial port, were able to build/boot the NetBSD kernel. (*A community dev we trusted with our sources and was helping us with portability across platforms) The stuff with grub2 may potentially be fixed in the "near" future... the others are more tricky. In general if clang can do it, we have a strong chance as well. As a philosophy - "we" aren't really trying to be the best generic compiler in the world. We aim more on optimizing as much for known targets. So if by system you mean, a compiler that would produce an "OS" which only runs on a single class of hardware, then yeah it could work at some point in the future. Specifically, on x86 we default on host CPU optimizations. So on newer Intel hardware it's easy to get a binary that won't run on AMD or older 64bit Intel. More recently on ARMv8 - we turn on processor specific tuning. So while it may "run", the difference between APM's mustang and Cavium ThunderX is pretty big and running binaries intended for A and ran on B would certainly take a hit.. (t
Re: [gentoo-dev] New project: LLVM
Sorry to be the party crasher, but... I'd love to have optimizations for everything out there, but it takes a lot of work to fine tune for something specific. Right now I see a few variants of ARMv8 ARM reference stuff - A57 cores and the newer bits.. The scheduling and stuff seems more-or-less similar enough that one tuning could probably work for the vast majority of these parts. Cavium ThunderX - It's ground up and quite different from the ARM reference stuff under the hood APM - Mustang, again ground up and different. I don't have enough hands on to know how different from reference. Broadcom - Coming Soon(tm) - Again no hands on or any data, but certainly very interesting.. ... now add in every variant of ground up implementation and you have 50 shades of gray.. - Soo.. depending on your target hardware, you may be better off with gcc if the end goal is general all-around performance. (It does a quite respectable job of being generic) I realize a lot of people have strong feelings for or against it. I leave that to the reader to decide.. Back to my own glass house.. It will take a few years, but I am trying to make it easier (internally) to expose in some clear way all the pieces which compose a fine tuning per-processor. If this was "just" scheduling models it would be really easy, but it's not.. Those latencies and other magic bits decide things like.. "should I unroll this loop or do something else" and then you venture into the land of accelerators where a custom regalloc may be what you really need and *nothing* off the shelf fits to meet your goals.. (projects like that can take 9 months and in the end only give a general 1-5% median performance gain..) -- On Sat, Aug 20, 2016 at 2:02 AM, james wrote: > On 08/19/2016 11:15 AM, C Bergström wrote: >> >> On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato wrote: >>> >>> BTW is pathscale ready to be used as system compiler as well? >> >> >> I wish, but no. We have known issues when building grub2, glibc and >> the Linux kernel at the very least. Someone* did report a long time >> ago that with their unofficial port, were able to build/boot the >> NetBSD kernel. >> (*A community dev we trusted with our sources and was helping us with >> portability across platforms) >> >> The stuff with grub2 may potentially be fixed in the "near" future... >> the others are more tricky. In general if clang can do it, we have a >> strong chance as well. >> >> As a philosophy - "we" aren't really trying to be the best generic >> compiler in the world. We aim more on optimizing as much for known >> targets. So if by system you mean, a compiler that would produce an >> "OS" which only runs on a single class of hardware, then yeah it could >> work at some point in the future. Specifically, on x86 we default on >> host CPU optimizations. So on newer Intel hardware it's easy to get a >> binary that won't run on AMD or older 64bit Intel. >> >> More recently on ARMv8 - we turn on processor specific tuning. So >> while it may "run", the difference between APM's mustang and Cavium >> ThunderX is pretty big and running binaries intended for A and ran on >> B would certainly take a hit.. (this is just the tip of the iceberg) >> >> For general scalar OS code it isn't likely to matter... the real >> impact being like 1-10% difference (being very general.. it could be >> less or more in the real world..) >> >> For HPC codes or anything where you get loops or computationally >> complex - the gloves are off and I could see big differences... (again >> being general and maybe a bit dramatic for fun) > > > > OK (actually fantastic!). Looking at the pathscale site pages and github, > perhaps a cheap arm embedded board where llvm is the centerpiece of > compiling a minimal system to entice gentoo-llvm testers, would be possible > in the near future?. I have a 96boards, HiKey arm64v8 that I could dedicate > to gentoo+armv8-llvm testing, if that'd help. [1] > > Perhaps a baseline bootstrap iso (or such) version targeted at > llvm-centric testers on x86-64 or armv8 ? Skip grub2 and use grub-legacy or > lilo or (?), since there seems to be issues with llvm-grub2. > > > [1] http://dev.gentoo.org/~tgall/ > > > No matter how you slice it, from someone who is focused on building > minimized and embedded (bare metal) systems that are customized and > coalesced into a heterogeneous gentoo cluster for HPC, this is wonderful > news. Finally a vendor in the cluster space, with some vision and > common-sense, imho. Heterogeneous and open HPC is where is at, imho. If > there is a forum where the community and pathscale folks discuss issues, > point that out as I could not find one for deeper reading > > > hth, > James >
Re: [gentoo-dev] New project: LLVM
On 08/19/2016 11:15 AM, C Bergström wrote: On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato wrote: BTW is pathscale ready to be used as system compiler as well? I wish, but no. We have known issues when building grub2, glibc and the Linux kernel at the very least. Someone* did report a long time ago that with their unofficial port, were able to build/boot the NetBSD kernel. (*A community dev we trusted with our sources and was helping us with portability across platforms) The stuff with grub2 may potentially be fixed in the "near" future... the others are more tricky. In general if clang can do it, we have a strong chance as well. As a philosophy - "we" aren't really trying to be the best generic compiler in the world. We aim more on optimizing as much for known targets. So if by system you mean, a compiler that would produce an "OS" which only runs on a single class of hardware, then yeah it could work at some point in the future. Specifically, on x86 we default on host CPU optimizations. So on newer Intel hardware it's easy to get a binary that won't run on AMD or older 64bit Intel. More recently on ARMv8 - we turn on processor specific tuning. So while it may "run", the difference between APM's mustang and Cavium ThunderX is pretty big and running binaries intended for A and ran on B would certainly take a hit.. (this is just the tip of the iceberg) For general scalar OS code it isn't likely to matter... the real impact being like 1-10% difference (being very general.. it could be less or more in the real world..) For HPC codes or anything where you get loops or computationally complex - the gloves are off and I could see big differences... (again being general and maybe a bit dramatic for fun) OK (actually fantastic!). Looking at the pathscale site pages and github, perhaps a cheap arm embedded board where llvm is the centerpiece of compiling a minimal system to entice gentoo-llvm testers, would be possible in the near future?. I have a 96boards, HiKey arm64v8 that I could dedicate to gentoo+armv8-llvm testing, if that'd help. [1] Perhaps a baseline bootstrap iso (or such) version targeted at llvm-centric testers on x86-64 or armv8 ? Skip grub2 and use grub-legacy or lilo or (?), since there seems to be issues with llvm-grub2. [1] http://dev.gentoo.org/~tgall/ No matter how you slice it, from someone who is focused on building minimized and embedded (bare metal) systems that are customized and coalesced into a heterogeneous gentoo cluster for HPC, this is wonderful news. Finally a vendor in the cluster space, with some vision and common-sense, imho. Heterogeneous and open HPC is where is at, imho. If there is a forum where the community and pathscale folks discuss issues, point that out as I could not find one for deeper reading hth, James
Re: [gentoo-dev] New project: LLVM
On 19/08/16 17:15, C Bergström wrote: > On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato wrote: >> BTW is pathscale ready to be used as system compiler as well? > > I wish, but no. We have known issues when building grub2, glibc and > the Linux kernel at the very least. Someone* did report a long time > ago that with their unofficial port, were able to build/boot the > NetBSD kernel. > (*A community dev we trusted with our sources and was helping us with > portability across platforms) > > The stuff with grub2 may potentially be fixed in the "near" future... > the others are more tricky. In general if clang can do it, we have a > strong chance as well. I see, it is getting quite close =) > As a philosophy - "we" aren't really trying to be the best generic > compiler in the world. We aim more on optimizing as much for known > targets. So if by system you mean, a compiler that would produce an > "OS" which only runs on a single class of hardware, then yeah it could > work at some point in the future. Specifically, on x86 we default on > host CPU optimizations. So on newer Intel hardware it's easy to get a > binary that won't run on AMD or older 64bit Intel. > > More recently on ARMv8 - we turn on processor specific tuning. So > while it may "run", the difference between APM's mustang and Cavium > ThunderX is pretty big and running binaries intended for A and ran on > B would certainly take a hit.. (this is just the tip of the iceberg) This is not a problem for Gentoo, actually sounds a good match for one of our many use-cases =) > For HPC codes or anything where you get loops or computationally > complex - the gloves are off and I could see big differences... (again > being general and maybe a bit dramatic for fun) I started to do some decoding benchmark across compiler version some time ago, I should try to put in the mix your compiler as well and eventually blog about it =) lu
Re: [gentoo-dev] New project: LLVM
On 19/08/16 19:13, C Bergström wrote: > I finally got it to build and here's the size numbers > 952K./lib/libc++abi.a > 616K./lib/libc++abi.so.1.0 > > If the above isn't enough motivation and you really want benchmarks > which prove it's a pig... I'll try to figure something else > > Not exactly a 1:1 comparison because I think other things are mixed in, but... > 352K/usr/lib/gcc/x86_64-linux-gnu/4.9/libsupc++.a > 356K/usr/lib/gcc/x86_64-linux-gnu/5/libsupc++.a > > In the land of HPC we frequently statically link stuff... not that > 864KB is big by any sort of modern definition, but it does raise > questions.. > We aren't in love with any specific implementation of it so it is nice to have some comparison. We could probably start a page in the wiki about it. As said, the only part that makes uncomfortable about libcxxrt seems the lack of versions and releases. Surely we can cut another snapshot out of it and be happy about it. lu
Re: [gentoo-dev] New project: LLVM
I finally got it to build and here's the size numbers 952K./lib/libc++abi.a 616K./lib/libc++abi.so.1.0 If the above isn't enough motivation and you really want benchmarks which prove it's a pig... I'll try to figure something else Not exactly a 1:1 comparison because I think other things are mixed in, but... 352K/usr/lib/gcc/x86_64-linux-gnu/4.9/libsupc++.a 356K/usr/lib/gcc/x86_64-linux-gnu/5/libsupc++.a In the land of HPC we frequently statically link stuff... not that 864KB is big by any sort of modern definition, but it does raise questions.. On Sat, Aug 20, 2016 at 12:54 AM, Lei Zhang wrote: > 2016-08-19 11:11 GMT+08:00 C Bergström : >> I think you're getting a bit confused >> >> libsupc++ is the default now, from GNU >> >> libcxxabi is the bloated runtime from Apple >> >> libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which >> PathScale and FreeBSD use by default. We don't need a version number >> because it's pretty much rock solid stable for a while. >> I'd encourage you to consider libcxxrt for at least the code size and >> performance reasons. Build it and you'll see. Locally my unoptimized >> libcxxrt.so is like 88K. How much is your libcxxabi (static and >> shared) >> >> 88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so >> 140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a >> // This seems larger than I remember and I need to check why. >> >> https://github.com/pathscale/libcxxrt > > Currently libcxxrt is the default ABI lib for libc++ in Gentoo. I mean > to replace it with libc++abi in that context. > > I'm interested in benchmarking to reveal the claimed difference in > performance. Perhaps I can build the same program against libcxxrt and > libc++abi respectively and see how it behaves. Do you have some hints > on what kind of programs I should test? > > > Thanks, > Lei >
Re: [gentoo-dev] New project: LLVM
2016-08-19 11:11 GMT+08:00 C Bergström : > I think you're getting a bit confused > > libsupc++ is the default now, from GNU > > libcxxabi is the bloated runtime from Apple > > libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which > PathScale and FreeBSD use by default. We don't need a version number > because it's pretty much rock solid stable for a while. > I'd encourage you to consider libcxxrt for at least the code size and > performance reasons. Build it and you'll see. Locally my unoptimized > libcxxrt.so is like 88K. How much is your libcxxabi (static and > shared) > > 88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so > 140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a > // This seems larger than I remember and I need to check why. > > https://github.com/pathscale/libcxxrt Currently libcxxrt is the default ABI lib for libc++ in Gentoo. I mean to replace it with libc++abi in that context. I'm interested in benchmarking to reveal the claimed difference in performance. Perhaps I can build the same program against libcxxrt and libc++abi respectively and see how it behaves. Do you have some hints on what kind of programs I should test? Thanks, Lei
Re: [gentoo-dev] Packages up for grabs
On 08/19/2016 06:07 AM, Pacho Ramos wrote: This packages are now up for grabs: app-crypt/efitools app-crypt/pesign app-doc/linux-kernel-in-a-nutshell dev-util/cgvg dev-util/kup dev-util/xxdi net-misc/bti app-doc/linux-kernel-in-a-nutshell is quite interesting. I went to greg's site, looked around and built this package. It shows up, in chapter form under /usr/share. I used 'elogv' to look at the directional output. Scant. So I'm not sure how to discover the gui/tools one would used, other and manually looking at the chapters one at a time. I quick look at the gentoo wiki revealed little how to access this or other /usr/share/ documents; perhaps I have missed the obvious reference pages on those documentation systems? Sure, it's an old doc, but it could be useful? I bring this up as maybe a tie-in doc to the GSoC kernel build work that is currently being performed. [1] Personally, with gentoo being a strong embedded distro, and the IoT movement bringing in lots of new hardware to the linux world, I think these old documents are still valuable for the learning the basics process, particularly related to small and minimized gentoo systems. As such, I am willing to help, but, first I think a comprehensive discussion on documentation on gentoo to tie in all the new, wonderful additions in the gentoo wiki, to some/most of the traditionally available documentation sources, is warranted. Decide what to keep and integrate those resoureces so that they 'flow' one to another for the gentoo community. There are also, many new documentation resources on github and similar sites, so those need to be 'integrated' too. A pattern or template for devs to follow possible? [1] https://wiki.gentoo.org/wiki/Google_Summer_of_Code/2016/Ideas/kernelconfig hth, James
Re: [gentoo-dev] Developers, please work on underlinking issues!
On 08/19/2016 03:49 AM, Rich Freeman wrote: On Fri, Aug 19, 2016 at 12:58 AM, Michał Górny wrote: On Thu, 18 Aug 2016 15:21:16 +0200 Alexis Ballier wrote: On Thu, 18 Aug 2016 08:13:14 -0400 Rich Freeman wrote: If you just check your packages occassionally to make sure they build with gold it completely achieves the goal, and it will actually result in fewer bugs using the non-gold linker as well. That's what a tinderbox is for. The only QA problem I see here is that QA doesn't automate that kind of checks anymore since Diego left. Maybe QA should ask Toralf to run a ld.gold tinderbox and avoid asking people to randomly test random packages ? Yes, tinderboxing makes a lot of sense if the bugs are afterwards ignored by package maintainers. Or in the best case, the maintainer tells reporter (Toralf) to file the bug upstream. TBH, these are really two different problems. 1. I think raising awareness of underlinking is good. 2. I think encouraging developers to test their own packages with the gold linker is good, because it helps accomplish #1, and increases their awareness in general. 3. I think that having a tinderbox systematically testing using the gold linker is also good. 4. I think that hitting devs with a cluebat when they ignore valid bugs is good. The flip side of this is that we're not necessarily better off if maintainers just abandon packages because they have terrible build systems. At some point you need to work with them. However, if they're not willing to at least stick in a slot operator dependency when asked to, then sure we should have a talk with them. (A slot op dep will of course help by triggering rebuilds, but it doesn't actually directly fix the underlinking issue, which would require fixing the build system.) I think the big thing is acknowledging that packages that are missing dependencies or which are underlinked are defective. Sure, it would be nice if somebody else came along and helped find our mistakes. However, that in itself doesn't excuse us from having made them in the first place. And it certainly doesn't excuse giving people a hard time when they politely point them out. +1 Perhaps much of the mechanics and ordinary part of these aforementioned task, would make for fertile ground of skills diversification for the 'proxy-maintainers' ? Understanding and routine usage of a full suite of tools available under gentoo, could easily be missed during the proxy period. In fact, putting the tinderbox out there as part of the proxy training and perhaps available to a portion of the wider gentoo-user base, might also be a fertile area for technical growth or the gentoo community? Access to there and other dev tools might be a powerful incentive, if packaged up attactively, for the gentoo user community to participate more in the less risky parts of gentoo development workflows? hth, James
Re: [gentoo-dev] New project: LLVM
On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato wrote: > BTW is pathscale ready to be used as system compiler as well? I wish, but no. We have known issues when building grub2, glibc and the Linux kernel at the very least. Someone* did report a long time ago that with their unofficial port, were able to build/boot the NetBSD kernel. (*A community dev we trusted with our sources and was helping us with portability across platforms) The stuff with grub2 may potentially be fixed in the "near" future... the others are more tricky. In general if clang can do it, we have a strong chance as well. As a philosophy - "we" aren't really trying to be the best generic compiler in the world. We aim more on optimizing as much for known targets. So if by system you mean, a compiler that would produce an "OS" which only runs on a single class of hardware, then yeah it could work at some point in the future. Specifically, on x86 we default on host CPU optimizations. So on newer Intel hardware it's easy to get a binary that won't run on AMD or older 64bit Intel. More recently on ARMv8 - we turn on processor specific tuning. So while it may "run", the difference between APM's mustang and Cavium ThunderX is pretty big and running binaries intended for A and ran on B would certainly take a hit.. (this is just the tip of the iceberg) For general scalar OS code it isn't likely to matter... the real impact being like 1-10% difference (being very general.. it could be less or more in the real world..) For HPC codes or anything where you get loops or computationally complex - the gloves are off and I could see big differences... (again being general and maybe a bit dramatic for fun)
Re: [gentoo-dev] New project: LLVM
On 19/08/16 05:11, C Bergström wrote: > I think you're getting a bit confused > > libsupc++ is the default now, from GNU > > libcxxabi is the bloated runtime from Apple > > libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which > PathScale and FreeBSD use by default. We don't need a version number > because it's pretty much rock solid stable for a while. C++ is evolving so it will be needed in the future =) Please consider adding some versions even if it is a bourden. > I'd encourage you to consider libcxxrt for at least the code size and > performance reasons. Build it and you'll see. Locally my unoptimized > libcxxrt.so is like 88K. How much is your libcxxabi (static and > shared) > > 88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so > 140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a > // This seems larger than I remember and I need to check why. > > https://github.com/pathscale/libcxxrt BTW is pathscale ready to be used as system compiler as well? lu
[gentoo-dev] Packages up for grabs
This packages are now up for grabs: app-arch/unmakeself app-crypt/md5deep app-editors/leafpad x11-misc/revelation x11-themes/vanilla-dmz-aa-xcursors x11-themes/vanilla-dmz-xcursors
Re: [gentoo-dev] Developers, please work on underlinking issues!
On Fri, 19 Aug 2016 06:58:44 +0200 Michał Górny wrote: > On Thu, 18 Aug 2016 15:21:16 +0200 > Alexis Ballier wrote: > > > On Thu, 18 Aug 2016 08:13:14 -0400 > > Rich Freeman wrote: > > > > > If you just check your packages occassionally to make sure they > > > build with gold it completely achieves the goal, and it will > > > actually result in fewer bugs using the non-gold linker as > > > well. > > > > That's what a tinderbox is for. The only QA problem I see here is > > that QA doesn't automate that kind of checks anymore since Diego > > left. Maybe QA should ask Toralf to run a ld.gold tinderbox and > > avoid asking people to randomly test random packages ? > > Yes, tinderboxing makes a lot of sense if the bugs are afterwards > ignored by package maintainers. "ignored" is rather strong here; considering people join gentoo and basically work for free, I'd rather say "not considered important enough" which yields basically the same result but without accusing people of refusing to fix a bug. With that in mind, everybody is free to submit patches to bugzie, and if it is a QA goal, QA is free to apply it after some time. Nothing is blocked here, expect when people prefer to hit fellow developers "with a cluebat" rather than getting things done. > Or in the best case, the maintainer > tells reporter (Toralf) to file the bug upstream. It's not because Toralf reported it that he is the only one that has the right to report it upstream. Obtaining fixes from those that know the package best is good, isn't it ? I thought we learned from the past and tried to avoid, e.g., fixing valgrind warnings in libssl by our own :) Alexis.
Re: [gentoo-dev] rfc: /etc/init.d/modules loading modules defined in files
On 2016-08-17 01:49, Matthias Maier wrote: >> I have received a request to implement a feature in OpenRC to allow >> multiple software packages to drop files in a directory, /etc/modules.d >> for example, which would define modules the /etc/init.d/modules script >> would load. > > What about /etc/modules-load.d containing various files that list one > module to load per line? With this, OpenRC's behavior would be > compatible with systemd-modules-load [1]. +1 I wanted to ask the same after working on bug 480018 [1]. [1] https://bugs.gentoo.org/480018 -- Regards, Thomas aka Whissi signature.asc Description: OpenPGP digital signature
[gentoo-dev] Packages up for grabs
This packages are now up for grabs: app-crypt/efitools app-crypt/pesign app-doc/linux-kernel-in-a-nutshell dev-util/cgvg dev-util/kup dev-util/xxdi net-misc/bti
Re: [gentoo-dev] Developers, please work on underlinking issues!
On Fri, Aug 19, 2016 at 12:58 AM, Michał Górny wrote: > On Thu, 18 Aug 2016 15:21:16 +0200 > Alexis Ballier wrote: > >> On Thu, 18 Aug 2016 08:13:14 -0400 >> Rich Freeman wrote: >> >> > If you just check your packages occassionally to make sure they build >> > with gold it completely achieves the goal, and it will actually result >> > in fewer bugs using the non-gold linker as well. >> >> That's what a tinderbox is for. The only QA problem I see here is that >> QA doesn't automate that kind of checks anymore since Diego left. Maybe >> QA should ask Toralf to run a ld.gold tinderbox and avoid asking people >> to randomly test random packages ? > > Yes, tinderboxing makes a lot of sense if the bugs are afterwards > ignored by package maintainers. Or in the best case, the maintainer > tells reporter (Toralf) to file the bug upstream. > TBH, these are really two different problems. 1. I think raising awareness of underlinking is good. 2. I think encouraging developers to test their own packages with the gold linker is good, because it helps accomplish #1, and increases their awareness in general. 3. I think that having a tinderbox systematically testing using the gold linker is also good. 4. I think that hitting devs with a cluebat when they ignore valid bugs is good. The flip side of this is that we're not necessarily better off if maintainers just abandon packages because they have terrible build systems. At some point you need to work with them. However, if they're not willing to at least stick in a slot operator dependency when asked to, then sure we should have a talk with them. (A slot op dep will of course help by triggering rebuilds, but it doesn't actually directly fix the underlinking issue, which would require fixing the build system.) I think the big thing is acknowledging that packages that are missing dependencies or which are underlinked are defective. Sure, it would be nice if somebody else came along and helped find our mistakes. However, that in itself doesn't excuse us from having made them in the first place. And it certainly doesn't excuse giving people a hard time when they politely point them out. -- Rich
