Re: [gentoo-dev] New project: LLVM

[C Bergström]

On Sat, Aug 20, 2016 at 5:41 AM, james  wrote:
> On 08/19/2016 05:05 PM, C Bergström wrote:
>>
>> On Sat, Aug 20, 2016 at 4:52 AM, james  wrote:
>>
>> 
>>
>
> You removed your rude remark:::
> " Sorry to be the party crasher, but..."
>
> So let's put it back, just for clarity.

I'll forgive you because it seems English may not be your native
language, but this expression is rarely to never seen as offensive.
Further, you should be able to infer I was talking about *my* own
party. I was clarifying about the limitations of using "my" compiler
in a system context. This had nothing to do you. So again please keep
your tone civil, adult and if possible on-topic.

Re: [gentoo-dev] New project: LLVM

[C Bergström]

On Sat, Aug 20, 2016 at 4:52 AM, james  wrote:



>> Back to my own glass house.. It will take a few years, but I am trying
>> to make it easier (internally) to expose in some clear way all the
>> pieces which compose a fine tuning per-processor. If this was "just"
>> scheduling models it would be really easy, but it's not.. Those
>> latencies and other magic bits decide things like.. "should I unroll
>> this loop or do something else" and then you venture into the land of
>> accelerators where a custom regalloc may be what you really need and
>> *nothing* off the shelf fits to meet your goals.. (projects like that
>> can take 9 months and in the end only give a general 1-5% median
>> performance gain..)
>
>
> If this is your mantra, I resend the generous comments. Cray use to work
> that way, milking the Petroleum Industry for tons of money, but, things have
> changed and the change is accelerating, rapidly. Perhaps too much off those
> Cray patents that your company owns are leaking toxins into the brain-trust
> where you park?
>
> Vendor walk-back is sad, imho. ymmv.
>
> Best of luck to your company's  5-year plan

I have no idea what on earth you were trying to say in most of your
reply. I am speaking only from a compiler perspective. Nothing more
and nothing less.. it may be my difficultly to describe the target
level and processor specific optimization choices a compiler *must*
make.

Beyond not understanding your email, I found it insulting. So please
keep rude comments to yourself.

So again to try to explain the technical side of this - We can't and
have no desire to optimize for every class of processor on the planet.
We have a narrow band of focus on mostly HPC centric code patterns and
processors which are are typically used in HPC workloads. I'd love to
expand past this, but we're a small company and that's our niche.
There's no walking back or trying to claim to be something we're not..
this is pure honest transparency. (imagine it like - do one thing and
do it well)

The only special note I'd add on to this - the CPU isn't where we
spend most of our time tuning, it's by far more on the accelerator
support.

Re: [gentoo-dev] New project: LLVM

[C Bergström]

Sorry to be the party crasher, but...

I'd love to have optimizations for everything out there, but it takes
a lot of work to fine tune for something specific.

Right now I see a few variants of ARMv8

ARM reference stuff - A57 cores and the newer bits.. The scheduling
and stuff seems more-or-less similar enough that one tuning could
probably work for the vast majority of these parts.

Cavium ThunderX - It's ground up and quite different from the ARM
reference stuff under the hood

APM - Mustang, again ground up and different. I don't have enough
hands on to know how different from reference.

Broadcom - Coming Soon(tm) - Again no hands on or any data, but
certainly very interesting..

... now add in every variant of ground up implementation and you have
50 shades of gray..
-
Soo.. depending on your target hardware, you may be better off with
gcc if the end goal is general all-around performance. (It does a
quite respectable job of being generic) I realize a lot of people have
strong feelings for or against it. I leave that to the reader to
decide..

Back to my own glass house.. It will take a few years, but I am trying
to make it easier (internally) to expose in some clear way all the
pieces which compose a fine tuning per-processor. If this was "just"
scheduling models it would be really easy, but it's not.. Those
latencies and other magic bits decide things like.. "should I unroll
this loop or do something else" and then you venture into the land of
accelerators where a custom regalloc may be what you really need and
*nothing* off the shelf fits to meet your goals.. (projects like that
can take 9 months and in the end only give a general 1-5% median
performance gain..)
--


On Sat, Aug 20, 2016 at 2:02 AM, james  wrote:
> On 08/19/2016 11:15 AM, C Bergström wrote:
>>
>> On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato  wrote:
>>>
>>> BTW is pathscale ready to be used as system compiler as well?
>>
>>
>> I wish, but no. We have known issues when building grub2, glibc and
>> the Linux kernel at the very least. Someone* did report a long time
>> ago that with their unofficial port, were able to build/boot the
>> NetBSD kernel.
>> (*A community dev we trusted with our sources and was helping us with
>> portability across platforms)
>>
>> The stuff with grub2 may potentially be fixed in the "near" future...
>> the others are more tricky. In general if clang can do it, we have a
>> strong chance as well.
>>
>> As a philosophy - "we" aren't really trying to be the best generic
>> compiler in the world. We aim more on optimizing as much for known
>> targets. So if by system you mean, a compiler that would produce an
>> "OS" which only runs on a single class of hardware, then yeah it could
>> work at some point in the future. Specifically, on x86 we default on
>> host CPU optimizations. So on newer Intel hardware it's easy to get a
>> binary that won't run on AMD or older 64bit Intel.
>>
>> More recently on ARMv8 - we turn on processor specific tuning. So
>> while it may "run", the difference between APM's mustang and Cavium
>> ThunderX is pretty big and running binaries intended for A and ran on
>> B would certainly take a hit.. (this is just the tip of the iceberg)
>>
>> For general scalar OS code it isn't likely to matter... the real
>> impact being like 1-10% difference (being very general.. it could be
>> less or more in the real world..)
>>
>> For HPC codes or anything where you get loops or computationally
>> complex - the gloves are off and I could see big differences... (again
>> being general and maybe a bit dramatic for fun)
>
>
>
> OK (actually fantastic!). Looking at the pathscale site pages and github,
> perhaps a cheap arm embedded board where llvm is the centerpiece of
> compiling a minimal system to entice gentoo-llvm testers, would be possible
> in the near future?. I have a 96boards, HiKey arm64v8  that I could dedicate
> to gentoo+armv8-llvm testing, if that'd help. [1]
>
> Perhaps a  baseline bootstrap iso (or such) version  targeted at
> llvm-centric testers on x86-64 or armv8 ? Skip grub2 and use grub-legacy or
> lilo or (?), since there seems to be issues with llvm-grub2.
>
>
> [1] http://dev.gentoo.org/~tgall/
>
>
> No matter how you slice it, from someone who is focused on building
> minimized and embedded (bare metal) systems that are customized and
> coalesced into a heterogeneous gentoo cluster for HPC, this is wonderful
> news. Finally a vendor in the cluster space, with some vision and
> common-sense, imho. Heterogeneous and open  HPC is where is at, imho. If
> there is a forum where the community and pathscale folks discuss issues,
> point that out as I could not find one for deeper reading
>
>
> hth,
> James
>

Re: [gentoo-dev] New project: LLVM

[C Bergström]

I finally got it to build and here's the size numbers
952K./lib/libc++abi.a
616K./lib/libc++abi.so.1.0

If the above isn't enough motivation and you really want benchmarks
which prove it's a pig... I'll try to figure something else

Not exactly a 1:1 comparison because I think other things are mixed in, but...
352K/usr/lib/gcc/x86_64-linux-gnu/4.9/libsupc++.a
356K/usr/lib/gcc/x86_64-linux-gnu/5/libsupc++.a

In the land of HPC we frequently statically link stuff... not that
864KB is big by any sort of modern definition, but it does raise
questions..


On Sat, Aug 20, 2016 at 12:54 AM, Lei Zhang  wrote:
> 2016-08-19 11:11 GMT+08:00 C Bergström :
>> I think you're getting a bit confused
>>
>> libsupc++ is the default now, from GNU
>>
>> libcxxabi is the bloated runtime from Apple
>>
>> libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which
>> PathScale and FreeBSD use by default. We don't need a version number
>> because it's pretty much rock solid stable for a while.
>> I'd encourage you to consider libcxxrt for at least the code size and
>> performance reasons. Build it and you'll see. Locally my unoptimized
>> libcxxrt.so is like 88K. How much is your libcxxabi (static and
>> shared)
>>
>> 88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so
>> 140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a
>> // This seems larger than I remember and I need to check why.
>>
>> https://github.com/pathscale/libcxxrt
>
> Currently libcxxrt is the default ABI lib for libc++ in Gentoo. I mean
> to replace it with libc++abi in that context.
>
> I'm interested in benchmarking to reveal the claimed difference in
> performance. Perhaps I can build the same program against libcxxrt and
> libc++abi respectively and see how it behaves. Do you have some hints
> on what kind of programs I should test?
>
>
> Thanks,
> Lei
>

Re: [gentoo-dev] New project: LLVM

[C Bergström]

On Fri, Aug 19, 2016 at 11:01 PM, Luca Barbato  wrote:
> BTW is pathscale ready to be used as system compiler as well?

I wish, but no. We have known issues when building grub2, glibc and
the Linux kernel at the very least. Someone* did report a long time
ago that with their unofficial port, were able to build/boot the
NetBSD kernel.
(*A community dev we trusted with our sources and was helping us with
portability across platforms)

The stuff with grub2 may potentially be fixed in the "near" future...
the others are more tricky. In general if clang can do it, we have a
strong chance as well.

As a philosophy - "we" aren't really trying to be the best generic
compiler in the world. We aim more on optimizing as much for known
targets. So if by system you mean, a compiler that would produce an
"OS" which only runs on a single class of hardware, then yeah it could
work at some point in the future. Specifically, on x86 we default on
host CPU optimizations. So on newer Intel hardware it's easy to get a
binary that won't run on AMD or older 64bit Intel.

More recently on ARMv8 - we turn on processor specific tuning. So
while it may "run", the difference between APM's mustang and Cavium
ThunderX is pretty big and running binaries intended for A and ran on
B would certainly take a hit.. (this is just the tip of the iceberg)

For general scalar OS code it isn't likely to matter... the real
impact being like 1-10% difference (being very general.. it could be
less or more in the real world..)

For HPC codes or anything where you get loops or computationally
complex - the gloves are off and I could see big differences... (again
being general and maybe a bit dramatic for fun)

Re: [gentoo-dev] New project: LLVM

[C Bergström]

I think you're getting a bit confused

libsupc++ is the default now, from GNU

libcxxabi is the bloated runtime from Apple

libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which
PathScale and FreeBSD use by default. We don't need a version number
because it's pretty much rock solid stable for a while.
I'd encourage you to consider libcxxrt for at least the code size and
performance reasons. Build it and you'll see. Locally my unoptimized
libcxxrt.so is like 88K. How much is your libcxxabi (static and
shared)

88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so
140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a
// This seems larger than I remember and I need to check why.

https://github.com/pathscale/libcxxrt

On Fri, Aug 19, 2016 at 10:40 AM, Lei Zhang  wrote:
> 2016-08-19 10:07 GMT+08:00  :
>> That seems a lot like what we've already done. I guess a GSOC student is 
>> working on the libcxxabi piece.
>
> I am that GSoC student :)
>
> I'm currently trying to push libc++abi to replace libcxxrt as the
> default runtime: https://github.com/gentoo/gentoo/pull/2048
>
> The reason is I think libc++abi blends in more naturally with other
> LLVM components, and it has a clear version number as opposed to
> libcxxrt.
>
>> The only advantage to using our runtime, libcxxrt, is performance and code 
>> size.‎
>
> Honestly I don't know what essential difference these two libs have; I
> can't find any decent comparison of them on the internet. Do you have
> some real numbers to show the difference in performance and code size?
>
>
> Lei
>

Re: [gentoo-dev] New project: LLVM

[C Bergström]

@mgorny may be able to help with some of this and has quite a bit of
experience building clang/llvm.  Where I work we use a "wrapper" that
helps coordinate a lot of the moving pieces.

https://github.com/pathscale/llvm-suite/

This may not be the perfect "gentoo" way to handle it, but the
approach would produce a clean and correct compiler. With llvm
dependencies getting more and more complicated, I'm not sure if it
would be possible to have both a gnu-free and also perfect
1-project-source-repo:1-ebuild ratio.

I'm sure there's llvm/clang ebuilds already and curious what others think..



On Thu, Aug 18, 2016 at 9:48 PM, Ian Bloss  wrote:
> Woot! Don't tell Stallman lol.
>
>
> On Tue, Aug 16, 2016, 09:22 Michał Górny  wrote:
>>
>> Hello, everyone.
>>
>> I have the pleasure of announcing that after the long period of split
>> maintenance, we are forming an united LLVM project [1] to maintain all
>> of LLVM packages in Gentoo and work on establishing improved support for
>> a healthy, gcc-free ecosystem.
>>
>> [1]:https://wiki.gentoo.org/wiki/Project:LLVM
>>
>> --
>> Best regards,
>> Michał Górny
>> 

Re: [gentoo-dev] impending c++11 clusterfuck?

[C Bergström]

One thing to point out.. trying to detect and using vX are just hacks
for what this really is - Adding abi (C++STL/ABI) information to the
ebuilds/packages.

To extend this - what happens when you have a compiler that isn't
compatible with the system default? When the package is merged should
some information from the toolchain like g++-48-cpp03 be inserted for
the package? The only downside I could think about is a crazy world
where libs end up being installed in.

/usr/lib64/{CPP-ABI}
/usr/lib64/g++-48-cpp03/

In this case the default system compiler could be gcc5 and then an
"alt" compiler which provides a different ABI could be used based on
some "restrict" or explicit information in the ebuild. That alt
compiler would have to build all the dependencies, but as long as it
knew about /usr/lib64/g++-48-cpp03/ and not /usr/lib64/g++-52-cpp11/
things would be ok...

If you want c++11 and 03 to exist, link together and be perfect at the
same time.. I think you're asking for trouble or the impossible.

While we're solving this problem - the same sort of "ABI" may also
help other languages suffering from similar problems like python
(Which now "fixed" their problem, but it's still not a perfect world)

Nothing is to say it won't pop up again in the future (Java?)...

Re: [gentoo-dev] impending c++11 clusterfuck?

[C Bergström]

On Mon, Nov 30, 2015 at 10:56 AM, Gregory M. Turner  wrote:
> I'm quoting myself from bug #566328 here.  These were off-the-cuff
> remarks that got away from me and became a call-to-arms...
>
> (In reply to Michał Górny from comment #7)
>> This is never this simple. C++11 can change the ABI. So the point kinda is,
>> we need to ensure that all C++ libraries in a depgraph use the same C++
>> version.
>
> This is pretty awful when you really think about it.  I feel like I'm
> watching a train-wreck in super slow motion.
>
> I'm not sure we're taking this seriously enough -- sooner or later it
> seems destined to become a major clusterfuck if we don't do something
> proactive about it now while the drawing-board is relatively
> uncluttered.
>
> The only thing I can think of that has this kind of two-way depgraph
> magic property are the major "abi" USE_EXPAND values (multilib-build
> and python-r1, in other words).
>
> But those rely on fancy framework-generated USE-flag deps, which seem
> like overkill and likely to incur unjustifiable user-experience-costs.
>
> Perhaps a solution to this cxx11 clusterfuck can be found that works
> more like perl?  By that I mean, pick your poison (respectively, your
> cxx11 ABI of preference or your major perl version of choice), rely on
> inbuilt portage features do the trick most of the time, and, when it
> breaks, run "magically-fix-everything.sh," grab a caffeinated beverage
> or three and fire up your favorite VOD client while the mess gets
> magically cleaned up by robots somehow.

I'm not sure I fully understand the problem, but what about
a. Always define the c++11 flag
b. Not support a system with mixed (and incompatible) c++ standards

I personally don't think it seems reasonable to try to carry both
c++03 and c++11 at the same time. This especially seems like nonsense
to me in the gentoo world. gcc has or will soon switch to c++11 by
default. Packages which can't be built will start having issues
anyway.

Re: [gentoo-dev] Re: Git workflow

[C Bergström]

On Mon, Jul 6, 2015 at 2:15 AM, William Hubbs  wrote:
> On Sun, Jul 05, 2015 at 07:17:26PM +0400, Jason Zaman wrote:
>> On Sun, Jul 05, 2015 at 12:03:29PM +0700, C Bergström wrote:
>> > On Sun, Jul 5, 2015 at 11:31 AM, Duncan <1i5t5.dun...@cox.net> wrote:
>> > > C Bergström posted on Sun, 05 Jul 2015 01:17:41 +0700 as excerpted:
>> > >
>> > >> I super don't like "merge" workflows.
>> > >> 1) "merge commits" are confusing at best and normal tools don't display
>> > >> and work with them as you'd always expect
>> > >
>> > > git log --graph, as others have mentioned.
>> >
>> > we are not talking about the same thing.
>> >
>> > I want to see the "diff" - not the graph.
>> >
>> > svn diff -r 1234
>> > git show 
>> >
>> > show me the "merge" commit in diff format
>>
>> So this isn't a good comparison. You are asking for a merge commit in
>> git and a normal commit in svn. Svn can branch but it is so complicated
>> that no one ever does it. If you were similarly to never ever make
>> branches in git its not a huge deal. (There are not *that* many pushes
>> to the tree, if you look at #gentoo-commits there is plenty of time
>> between commits.)
>>
>> While I personally rebase almost all of my stuff, merges are important
>> when taking contributions. A good example would be the main linux kernel
>> tree, if Linus were to merge everything it would be incredibly difficult
>> to figure anything out.
>
> I'm with Duncan on this. I think I understand what he's asking for...
>
> I think he is asking the question, "What changed in commit ".
>
> If you use the hash of a merge commit with "git show", you get nothing, so
> the merge commit is useless in terms of following changes.

That comment was from me (not Duncan)

I have responded to every point - 1 by 1, but the "passionate people"
(most polite term I can find) haven't addressed most of the "problems"
or why commit reordering is a particular problem in gentoo's typical
case.

To avoid the ire of the bystanders - I'm out of this thread and maybe
those with more tact and tolerance carry this forward.

Re: [gentoo-dev] Re: Git workflow

[C Bergström]

On Sun, Jul 5, 2015 at 11:31 AM, Duncan <1i5t5.dun...@cox.net> wrote:
> C Bergström posted on Sun, 05 Jul 2015 01:17:41 +0700 as excerpted:
>
>> I super don't like "merge" workflows.
>> 1) "merge commits" are confusing at best and normal tools don't display
>> and work with them as you'd always expect
>
> git log --graph, as others have mentioned.

we are not talking about the same thing.

I want to see the "diff" - not the graph.

svn diff -r 1234
git show 

show me the "merge" commit in diff format

>
> Works fine, at the console, is clear enough to follow, and is part of git
> itself and commonly used, so is definitely a "normal" tool for those
> using git.
>
> FWIW, to follow kernel developments during the merge window, I'll
> actually run git log showing merges-only (generally confined to author
> linus, as well, which pretty effectively limits me to only the master
> merges).  The merge comments give me a summary of what was committed in
> that merge, and I can both skip entire merge categories after reading
> only the title or merged tree name, where they're not of interest, and
> drill down into merges if I find something covered there of particular
> interest.
>
>> 2) merge commits lead to multiple parents, which breaks a clean and
>> simple to follow linear history -
>
> Ugh!  Development isn't linear if there's more than one person working on
> a shared tree and making commits to their respective local trees to be
> pushed later, at once, and making it appear linear is, plainly stated, a
> lie.  It's not clean and simple.  It's a lie, and maintaining a lie
> arguably always ends up more complex than simply stating the truth.

Your truth and my truth are are clearly not the same thing.

You want to preserve (for some reason) the original
commit+date+original parent. To me, development is not "done" at that
point. It's just getting started. It's not done in my mind until it's
ready to be pushed. For something simple it doesn't really matter does
it?

I do see your point - I just think in the gentoo model it's unlikely
to be relevant

WHY is the original parent so important? If someone is working in foo/
category of ebuilds and you're working in bar/ - the parent commit is
in all probability in a different category. The original parent commit
probably has no value.. (who cares!! honest question)


>
> If you're not using one of the biggest and best features of git, its
> ability to accurately and clearly portray parallel development and where
> branches actually branched and merged, why are you using git in the first
> place?

Huh? Git is "nice" because it's distributed. Using rebase doesn't take
away any of the benefits to that.

  Simply because it's used 'most everywhere else and thus people
> tend to have some familiarity with it?  That's not an invalid reason, of
> course, but if you have ten thousand nails to nail, and a nail gun
> plugged in and ready, why are you insisting on using a screwdriver to
> pound them in, instead?  If you're going to use a screwdriver, why not
> use screws and a screw gun (svn, I guess, its commits are even
> sequentially numbered, which ought to be heaven, by your definition!)?

This is a passionate reply, but it's not very accurate. nail gun vs
screw driver? (really?)


>
> To be clear, since the policy states rebase if possible, I'd do it, but
> I'd be internally grousing about it every single time, and that's not
> very conducive to remaining happy at one's job over a period of any
> significant length, absolutely critical if that job's a volunteer job.

Happiness is a state of mind - if this conflicts so strongly with a
relgious view - sure I could see it being such a burden. Is doing git
pull --rebase before you push really so bad?

>
> (I'd probably deal with it by scripting it to the extent possible and do
> my best to forget about the horrible misuse of perfectly good tool
> resources, tho I'd be cursing every time I went to use git log and
> couldn't use it properly due to all the lying going on.)

Again I don't see it as "lying" - (you're still working on stuff until
you push.. development isn't done) The ability to do micro or
incremental commits instead of the svn's forced wait approach is the
benefit here.

Hopefully you could share those scripts with others. Making this
workflow easier on everyone is a great thing.
I've seen people alias this and I think there's ways in git config to
do this as well.

Re: [gentoo-dev] Git workflow

[C Bergström]

On Sun, Jul 5, 2015 at 6:54 AM, Peter Stuge  wrote:
> C Bergström wrote:

>> 3) Ever tried to make a patch of the *actual* merge commit? Can one of
>> the advocates of merge show me the git command to do that? (Sure you
>> can diff between 2 commits, but the "merge" commit likes to avoid
>> being seen)
>
> If there are no conflicts when merging then the merge commit does not
> contain any changes, so how could there be a patch? Do you actually
> know the Git data model? I wrote it down the other day.
>
> http://peter.stuge.se/git-data-model

You deflect the question so nicely. If there is no merge conflict then
a rebase would be just fine?

You clearly know git very well - much better than I, why not just
answer the question?

>
>
>> 4) I disagree that even a very active repo will have a problem with
>> rebase - Why?
>> a. If devs aren't working in the same area there will be no conflicts
>> b. If
>
> It's not about conflicts.
>
>
>> and a "git pull --rebase" before push will be clean and fast (no
>> problems).
>
> Q: How do you deal with the two commits from other developers that
> were pushed while you were rebasing?
>
> A: You end up having to spin on the remote repo. That's really clumsy.

Pragmatically - the answer to you question is the same as it exists
today. You're scared of something which gentoo devs have been dealing
with and resolving for years. It's not like CVS/SVN handles this
magically better.

>
>
>> 5) More about linear commits and "history" - I need to double check,
>> but I don't think rebase changes the actual commit date (I could be
>> mistaken).
>
> You are mistaken, and should have double checked before you argued.
>
> Arguing without checking makes you look bad.

How? I didn't claim to know and clearly not knowing didn't seem
important (to me). I'm not trying to overstate anything. I'm just
trying to passionately bring this up. I ***wish*** someone with some
guts would actually take charge of this on the gentoo side, have a
vote or make some executive decision which is stronger than this wimpy
policy we have now.

>
>
>> The only advantage to merge is you could see that the commit
>> happened on the "1st" ${DATE-TIME} of the month
>
> That's not correct.
>
>> I can't say I've ever cared to know the date of a commit, but I can
>> say I have cared a lot about knowing which commit came 1st. Rebase,
>> for better or worse, forces something to be 1st and it's clear and
>> easy to see.
>
> A rebase, like a cherry-pick, loses the very information you claim to
> care about; the original parent of the commit.

To clarify - I don't (clearly) give a pooh what the original parent
was. I care much more about a linear history. I rarely can see any
value of the at-to-time of original commit compared to the final
at-push-time commit (what's rebased). The value of linear history
outweighs the arbitrary point you started from. imho a commit and
development is not done until it's ready to be pushed.

>
>
>> If I controlled the gentoo git server, I'd
>
> I think it's a good thing that you don't, since you seem to still
> need to study Git in more detail.

Peter - I'm ashamed of you - don't be a dick. We can keep this civil
without subtle insults. I understand you are one of the people who
feels strongly about this in the opposing way. I wish I could flip
your way of thinking, but git brings out a lot of strong
religious-like views. (immutable)

Re: [gentoo-dev] Git workflow

[C Bergström]

On Sun, Jul 5, 2015 at 3:33 AM, Alon Bar-Lev  wrote:
> On 4 July 2015 at 23:28, Alexandre Rostovtsev  wrote:
>>
>> On Sun, 2015-07-05 at 02:16 +0700, C Bergström wrote:
>> > 2) I don't understand your comment about signatures.
>>
>> Gpg commit signatures [1] which are a requirement for any gentoo git
>> workflow. Rebasing breaks the author's signature afaict, so the user
>> who is doing rebasing needs to re-sign the commit using his own key.
>>
>> [1] 
>> https://git-scm.com/book/tr/v2/Git-Tools-Signing-Your-Work#Signing-Commits
>>
>
> Maybe this is the root cause of all issues, and simpler was to remain
> with signed manifests.
> Just a thought... Not every git feature out there should be actually
> be leveraged.
> Doing so would enable rebase without loosing data, more secure (than
> SHA-1) signatures, using code review tools such as gerrit without an
> issue, migration out of git in future and probably more.
>

Gpg commit signatures - lol... really? (sorry I realize this is a
serious comment)
--
I'd agree that the point of security failure would probably be better
at actually ensuring the content to the users is correct and valid.

+1 for gerrit, but I realize that may be overkill

Re: [gentoo-dev] Git workflow

[C Bergström]

On Sun, Jul 5, 2015 at 1:56 AM, hasufell  wrote:
> On 07/04/2015 08:17 PM, C Bergström wrote:
>> I realize that this is subject to lots of different opinions and that
>> my input doesn't carry much weight - At least I thought it's a topic
>> that should be brought up (again?)
>> -
>> To start I hate git.. I have used it for years now and the
>> multitude of ways that are possible to accomplish nearly the same
>> thing are *annoying* at best.. With that rant out of the way on to the
>> point..
>> -
>> I super don't like "merge" workflows.
>> 1) "merge commits" are confusing at best and normal tools don't
>> display and work with them as you'd always expect
>>
>> 2) merge commits lead to multiple parents, which breaks a clean and
>> simple to follow linear history
>> -
>>
>> What I personally prefer is a rebase workflow.
>> The downsides
>> 1) Requires to you rebase before pushing. Which can be slightly more
>> work than just a lazy resolution of the conflicting "merge commit"
>> (depending on if you flatten your commits)
>>
>> 2) May not be the most popular git work-flow.
>>
>> 3) Due to #2 from above - may have detractors and or less people who
>> are familiar with it.
>> --
>> I'm of the mindset that if you're going to keep something under
>> revision - the history should be clear and clean. Linear is the only
>> way to fly for that. For those using cvs or svn - that's what they'll
>> be familiar with and probably expect to find in a git log. You can
>> start with forcing rebase and keep a clean history - if one day it
>> proves too problematic  you can switch over to "merging" - the other
>> way isn't really possible to do cleanly, depending on your tools..
>>
>> Thanks for the minute
>>
>
> Forcing a rebase-only workflow on developers will increase the amount of
> bad commits. Because non-trivial conflicts in rebases are difficult to
> resolve, since you fix conflicts for _every_ commit separately.

Not true - you have the choice to flatten the commit. This may not be
ideal, but I consider that way more optimal than some
hack-to-make-it-work "merge" commit.

To be honest and pragmatic - I don't really see tons of conflicts in
the typical gentoo dev workday.

The people who own ebuilds and eclasses won't be clobbering each
other. That doesn't happen today and why would switching to git
magically make it start?

With a rebase workflow - you
a. Rebase frequently for long running branches
b. Branch, rebase and push to master
--
Lastly - IF for whatever reason gentoo wants to switch to a different
VCS for whatever reason - a linear history would absolutely make that
easier (possible). Lets think 10+ years from now..

I'm not a git fanboy - I hope one day it's replaced by something
superior (the same thing could be said about almost any piece of
software and given enough time - it's probably true)

Re: [gentoo-dev] Git workflow

[C Bergström]

On Sun, Jul 5, 2015 at 1:42 AM, Rich Freeman  wrote:
> On Sat, Jul 4, 2015 at 2:17 PM, C Bergström  wrote:
>>
>> What I personally prefer is a rebase workflow.
>
> The recommendation is to rebase when practical.
>
> Rebasing makes the history look clean, but it sometimes does this by
> obscuring the real history.  It also discards original author commits
> with their signatures and rewrites them without signatures (in theory
> it could add new signatures if the person doing the rebase is the
> author).
>

Without having this conversation - then no logical decision or points
will be brought up. Using git isn't magic. There are downsides to
different work-flows and not having some decision or plan on this will
lead to shit and confusion. Leaders should establish rules, guidelines
or some policy before the migration (!!!)

1) Rebase doesn't obscure history, but it may reorder things out of
chronological order. (Unless you flatten the commit - which makes
things easier and then yes does "lose" history, but that's a developer
choice

2) I don't understand your comment about signatures. I have never seen
author tags lost in a rebase. merge will skrew up git blame more than
rebase any day.

3) Ever tried to make a patch of the *actual* merge commit? Can one of
the advocates of merge show me the git command to do that? (Sure you
can diff between 2 commits, but the "merge" commit likes to avoid
being seen)

4) I disagree that even a very active repo will have a problem with
rebase - Why?
a. If devs aren't working in the same area there will be no conflicts
and a "git pull --rebase" before push will be clean and fast (no
problems).
b. If multiple devs have a conflict - then it would be only the 1st
dev to push to have the easy job. The 2nd dev - regardless of merge or
rebase would have to resolve the conflicts. IF the conflicts can be
auto resolved - that would happen with a merge commit or rebase
anyway. Same amount of work - someone would still have to resolve
problems

5) More about linear commits and "history" - I need to double check,
but I don't think rebase changes the actual commit date (I could be
mistaken). The only advantage to merge is you could see that the
commit happened on the "1st" ${DATE-TIME} of the month - wouldn't have
to manually figure out if that commit was before or after another
commit in the graph. I can't say I've ever cared to know the date of a
commit, but I can say I have cared a lot about knowing which commit
came 1st. Rebase, for better or worse, forces something to be 1st and
it's clear and easy to see.

If I controlled the gentoo git server, I'd set it to forbid merge
commits - at least to start. If that poses *real* problems (not just
random crying) then it could always be relaxed.

[gentoo-dev] Git workflow

[C Bergström]

I realize that this is subject to lots of different opinions and that
my input doesn't carry much weight - At least I thought it's a topic
that should be brought up (again?)
-
To start I hate git.. I have used it for years now and the
multitude of ways that are possible to accomplish nearly the same
thing are *annoying* at best.. With that rant out of the way on to the
point..
-
I super don't like "merge" workflows.
1) "merge commits" are confusing at best and normal tools don't
display and work with them as you'd always expect

2) merge commits lead to multiple parents, which breaks a clean and
simple to follow linear history
-

What I personally prefer is a rebase workflow.
The downsides
1) Requires to you rebase before pushing. Which can be slightly more
work than just a lazy resolution of the conflicting "merge commit"
(depending on if you flatten your commits)

2) May not be the most popular git work-flow.

3) Due to #2 from above - may have detractors and or less people who
are familiar with it.
--
I'm of the mindset that if you're going to keep something under
revision - the history should be clear and clean. Linear is the only
way to fly for that. For those using cvs or svn - that's what they'll
be familiar with and probably expect to find in a git log. You can
start with forcing rebase and keep a clean history - if one day it
proves too problematic  you can switch over to "merging" - the other
way isn't really possible to do cleanly, depending on your tools..

Thanks for the minute

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

On Sat, May 23, 2015 at 1:18 PM, J. Roeleveld  wrote:
> On 11 May 2015 15:59:40 CEST, Rich Freeman  wrote:
>>On Mon, May 11, 2015 at 9:37 AM, C Bergström 
>>wrote:
>>> Sorry to shoot and run, but I think you're trying to tackle this
>>> problem in the wrong way. The problem isn't to drop the mail. The
>>> solution is to change email hosting providers. As a non-profit I
>>> believe Google hosted apps would be an option (free).
>>
>>In general we try to stick to our social contract, and that means
>>trying to avoid depending on proprietary technologies such as gmail.
>>
>>Now, I could see just using a FOSS-based IMAP/SMTP/POP provider,
>>perhaps which allows things like forwarding and such, which allows us
>>to have a copy of all our configuration and such in case we want to
>>migrate.  I'm not super-familiar with the wordpress.com model but
>>something like that also seems reasonable - we leverage donations of
>>hosting services but we aren't bound to anything proprietary and have
>>the ability to migrate off.
>>
>>I'd REALLY like to see a FOSS alternative to Gmail (a good one, that
>>is), and ditto for Google docs (or whatever the latest branding for
>>that is). There is nothing magical about cloud-based services any more
>>than there is anything magical about letting somebody else host your
>>website.  The key is to ensure that the technologies are open so that
>>you aren't bound to a single provider.
>
> Rich,
>
> If you are thinking of a FOSS email provider. Maybe investigate Fastmail?
>
> They use postfix and cyrus. And they also handle a lot of the development of 
> the latter.
>
> Not sure if they would fit in with the rest, but I would trust them sooner 
> then Google.

Trust? LOL - If by trust you mean the government can man-in-the-middle
attack them easily - then sure.. gmail always uses encryption.. does
fastmail force that as well? Google has a much stronger means to push
back short term and long term against government spying (snowden).
fastmail would have to comply just the same and if you go back in
history - you'll see other providers who didn' t comply and the only
outcome was for them to go out of business... (happened once?)

social contract shouldn't be a religious contract - It's not like I
ever suggested we use something which isn't blessed by the pope. I
guess I should just be quiet since everyone has their own religion...

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

On Mon, May 11, 2015 at 11:55 PM, Rich Freeman  wrote:
> On Mon, May 11, 2015 at 11:21 AM, C Bergström  
> wrote:
>> On Mon, May 11, 2015 at 9:59 PM, Rich Freeman  wrote:
>>> On Mon, May 11, 2015 at 10:44 AM, C Bergström  
>>> wrote:
>>>> What I'm describing is not "gmail" - it's everything that gmail has
>>>> and offers, but @gentoo.org domain. I'm using it right now in fact.
>>>>
>>>> You get the web interface, IMAP, POP, 2 token authentication (if you
>>>> want to enabled it) and lots of other things. etc etc
>>>
>>> How about the source code?
>>
>> Do you have the source for github?
>>
>
> That would be why I point out that we have debates over its use.
> Right now we're accepting it mainly because it is an alternative.  I
> suspect there would be more concern if it were proposed that we move
> to it exclusively.
>
>>
>> There are "free" alternatives and this is the exact same thing as
>> github. IMAP and POP are comparable to git as google hosted apps is
>> comparable to github.
>
> I think that is actually your best line of argument.  They're just a
> black box that accepts SMTP and makes available POP, and that is all
> Gentoo uses.  They just happen to offer an MUA on the side, which g.o
> devs/staff can use at their discretion.
>
> I'm not convinced it is the way to go, all the same.
>
>> There's a line between being passionate and
>> ignoring a sensible good alternative. I can't say where to draw that
>> line, but imho I hope pragmatic people will take a look instead of
>> just dismissing it.
>>
>
> I have nothing against Gmail and I'm composing this email using it.
> Even so, unless we were simply unable to host our own mail, I'm not
> sure I'd advocate moving g.o over to Google apps.
>
> That doesn't mean I'm not willing to be pragmatic.  I just think that
> making a change like this really should be done only out of necessity.
> There are other Gentoo services that I'd sooner see go than email.
>
>> Oh and btw - the whole problem comes because people are forwarding to
>> gmail. Is that open source? It's clear a large number of people
>> already use and depend on the exact same service I'm suggesting. How
>> on earth could those same people object... (I don't see the open
>> source communit up in arms over yahoo mail and gmail..)
>
> I can use Gmail personally and still object to forcing everybody else
> to use it.  I don't see that as hypocrisy at all.
>
>> Do you own a phone that connects to this email? Android, iOS.. etc
>> aren't "open source", but somehow we survive..
>
> Android is more or less open-source.  There are some blobs in it, but
> not all that much more than you'd find on a typical PC running Gentoo.
> But again, Gentoo doesn't provide cell phones or require that anybody
> own one.  If somebody wants to read dev.g.o on their iPhone, I don't
> have a problem with that.  If we were to force anybody who wanted to
> subscribe to dev.g.o to own an iPhone I would have a problem with
> that.

Android is much less open than people think or want to believe.
Entirely besides that point - I hope this could be proposed and at
least looked at as an option. I have ran a postfix setup with
graylisting and spam filtering. It's quite a bit of maintenance to
ensure it's done in a high quality manner. The time/cost/energy in
that vs something which "just works" is hard to pass up. I realize
philosophies and religions differ - I respect your view and hope a
good solution is found in the end. If not now - I suspect at some
point gentoo will relinquish their email to a service provider..

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

Look at the forwarding which is already happening. They are already
giving that big company the emails. That big company gets a copy of
every email which is posted publicly already.

Are you concerned about their privacy policy? Are you concerned about
them complying to a government demand or ads.. What's your exact
concern here..

Keep in mind that github has access to your data and their EULA
probably allows them to run whatever analysis they want. A that the
high level it's essentially the same thing. The difference is that
github isn't smart enough to tie revenue into this. (Such as
displaying ads to C/C++ devs)

"we" (most technically savvy people who own a smart phone) can't avoid
these big companies. It's just a fact of life. For a small
organization like gentoo - it should be respected when time saving
choices are made that allow everyone to just get things done.

I hope "social contract" doesn't mean zealot and instead means someone
who is pragmatic... What's the real concern here.. pragmatically.
(honest question)

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

On Mon, May 11, 2015 at 9:59 PM, Rich Freeman  wrote:
> On Mon, May 11, 2015 at 10:44 AM, C Bergström  
> wrote:
>> What I'm describing is not "gmail" - it's everything that gmail has
>> and offers, but @gentoo.org domain. I'm using it right now in fact.
>>
>> You get the web interface, IMAP, POP, 2 token authentication (if you
>> want to enabled it) and lots of other things. etc etc
>
> How about the source code?

Do you have the source for github?

>
>>
>> It used to be free, but now google charges for it with an exception
>> for non-profits.
>
> The social contract isn't about free-of-cost.  In fact, Gentoo pays
> for a number of services (often below commercial rates, but not
> everybody can afford to donate 100% of what we need).  We've even paid
> for a bug bounty on one occasion.  The social contract is about
> free-as-in-freedom.  We don't depend on proprietary services as much
> as possible.
>
> We even have debates over the use of github, since the pull request
> side isn't really FOSS.  It is tolerated mainly because we have FOSS
> alternatives as well, and bugzilla is still the primary bug
> tracker/etc.  To the extent that github is just used as a hosting
> provider for git it is completely compatible with the social contract,
> and would be so even if we were paying for it.

There are "free" alternatives and this is the exact same thing as
github. IMAP and POP are comparable to git as google hosted apps is
comparable to github. There's a line between being passionate and
ignoring a sensible good alternative. I can't say where to draw that
line, but imho I hope pragmatic people will take a look instead of
just dismissing it.

Oh and btw - the whole problem comes because people are forwarding to
gmail. Is that open source? It's clear a large number of people
already use and depend on the exact same service I'm suggesting. How
on earth could those same people object... (I don't see the open
source communit up in arms over yahoo mail and gmail..)

/* I'm just trying to level the conversation in terms of "social
contract" and what people generally find acceptable */
Do you own a phone that connects to this email? Android, iOS.. etc
aren't "open source", but somehow we survive..

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

What I'm describing is not "gmail" - it's everything that gmail has
and offers, but @gentoo.org domain. I'm using it right now in fact.

You get the web interface, IMAP, POP, 2 token authentication (if you
want to enabled it) and lots of other things. etc etc

It used to be free, but now google charges for it with an exception
for non-profits. The admin could test it and see if it provides all
the features you need. If you do SSO it may even be possible to
integrate with existing infrastructure.. blah blah.. I'd highly
recommend someone who actually has to manage this take a look.

Re: [gentoo-dev] Anti-spam changes: proposal to drop spammy mail

[C Bergström]

Sorry to shoot and run, but I think you're trying to tackle this
problem in the wrong way. The problem isn't to drop the mail. The
solution is to change email hosting providers. As a non-profit I
believe Google hosted apps would be an option (free). Then it would be
possible to simply leverage that service and not have to worry about
the forwarding. (You'd maybe save time and get (great?) spam filtering
in the process)

Re: [gentoo-dev] [OT] pkgcore bountry or contract work

[C Bergström]

On Sun, Feb 22, 2015 at 4:36 AM, Brian Dolbec  wrote:
> On Sun, 22 Feb 2015 03:32:58 +0700
> C Bergström  wrote:
>
>> PathScale is interested to hire a full time dev (for at least a few
>> months) in order to bring pkgcore back to life.
>>
>> General goals
>> 1) Make it capable of parsing/handling the current portage tree (We'll
>> contribute all this work upstream/open source)
>>
>> 2) Improve the web based front-end
>> https://github.com/dol-sen/CoreBuilder/tree/master/corebuilder
>> (some, but not all of this work may be open sourced)
>>
>
>
> I will give commit capability to whomever does do the work on
> CoreBuilder, I will also create the porthole/backends portion of
> porthole's SF repo there for as much of the backend changes that are
> not Pathscale specific.  I did have intention of splitting the backends
> portion from porthole's main pkg anyway.

Thanks I really appreciate this. Would you ever consider moving this
away from SF and to github instead?

[gentoo-dev] [OT] pkgcore bountry or contract work

[C Bergström]

PathScale is interested to hire a full time dev (for at least a few
months) in order to bring pkgcore back to life.

General goals
1) Make it capable of parsing/handling the current portage tree (We'll
contribute all this work upstream/open source)

2) Improve the web based front-end
https://github.com/dol-sen/CoreBuilder/tree/master/corebuilder
(some, but not all of this work may be open sourced)

misc other python coding.. etc
--
If we fail to find an interested and qualified candidate I'll probably
post bounties

Please reply offlist

Christopher

codestr0m on irc freenode

Re: [gentoo-dev] arm64 update

[C Bergström]

On Thu, Feb 19, 2015 at 3:27 AM, Tom Gall  wrote:

> So first, for those interested in cheap arm64 hardware, the first 96 board
> is going to start shipping in March for ~$129. The HiKey board is an 8 way
> 64 bit ARM board with 8 A53 cores. (No A57s bummer!)  Only had a gig of
> memory on the board but it’s not a bad device. I’ve had one for about 2-3
> weeks now. I’m basically running off a USB hard disk for the moment and
> thankfully the kernel continues to improve.
>
> The one downside is you really really need a 1.8v USB -> UART cable. You
> can get them from digikey and connect it on the board. If working with raw
> cables makes you squeamish take a breath, you’ll be fine.
>
> Info and links at:
>
> https://www.96boards.org
>
>
> Ok so hardware aside here’s the arm64 gentoo plan.
>
> 1) new stage3 put together and get that onto the mirrors for consumption.
> 2) Get the handbook extended to include arm64
> 3) continued package stabilization
>
>
If devs don't mind working out of a chroot - I can potentially get access
to some hardware or possibly help test after things are a bit more "stable".

If possible I do really recommend getting something with an A57 core (vs
a53) - there is quite a bit of difference from the compiler side of how
they compare.

How much are the APM xgene1 systems selling for? Has anyone tried to
request hw from: APM, ARM, Cavium or AMD?

Re: [gentoo-dev] gentoo git workflow

[C. Bergström]

On 09/15/14 02:34 AM, hasufell wrote:

William Hubbs:

On Sun, Sep 14, 2014 at 08:04:12PM +0200, Andreas K. Huettel wrote:

Deciding on a _commit policy_ should be fairly straightforward and we
already have one point
* gpg sign every commit (unless it's a merged branch, then we only care
about the merge commit)

+1

Merge commits only happen if we allow non-fast-forward merges. I would
personally be against allowing merge commits on the master branch.


Allowing fast-forward merges will break signature verification if you
fetched from a user repo.
If we don't allow merge commits, then _every_ commit hast to be signed
by a gentoo dev (e.g. by using git-am). I don't see much sense in this.
It will rather complicate workflow.

The currently proposed verification script skips branch 'B', so what
matters is the signature of the merge commit which say "yes, I have
reviewed the users branch(es) and it's fine".

Merging from branches holds useful information. A linear history isn't
necessarily easier to understand, so from me linear history gets a

-1

It just isn't really "git" to me. But it also requires people to know
when to avoid merge commits.


Rebases involving commits that are already pushed to master probably
shouldn't be allowed.


Of course, yes. That has to be documented in a gentoo developer git guide.
Pretty please do NOT allow "merge" commits.. they are the bane of evil 
for the long term ability to have any sane work-flow. Trying browsing a 
commit history after a big merge commit.. or following the parent..


lastly - the "merge" commit itself could be very confusing to some 
people when viewed in github. (At least personally I find them 
frequently unreadable)


After 5 years of git where I work - they are now banned (policy) and I 
wish github would allow them to be banned (non-fast forward) to avoid 
mistakes


There's a big debate between merge vs rebase.. I'm not trying to go down 
the benefits of one workflow vs the other.. However, if rebase fails.. 
you can allow merge commits in the future.. The opposite isn't easily 
accomplished without squashing history and losing stuff..

Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)

[C. Bergström]

On 09/14/14 08:24 PM, Jauhien Piatlicki wrote:

14.09.14 15:23, Jauhien Piatlicki написав(ла):

Another question: will it be possible to maintain a copy of tree on github to 
make contributions for users simpler (similarly to e.g. science overlay)? (Can 
it somehow be combined with proposed signing mechanism?)



Or well, have our own pull requests review tool.
NIH? What would be the benefit of that.. before going down this path.. I 
think there's some good tools around which may at least serve as a base 
to (fork) from before starting a ground up project.


Sorry to jump in the middle of the conversation, but I know 1st hand how 
much is involved here.

Re: [gentoo-dev] Packages up for grabs

[C. Bergström]

On 06/ 3/14 02:50 AM, Parker Schmitt wrote:
I think we need to keep the opencl stuff. In a few weeks I'll have 
time to help.

I work for PathScale and can probably take on

dev-lang/ekopath

path64 - while I'd like it to continue - it could(should?) be retired
-
I'd need someone to help proxy the version bumps on ekopath though. Join 
#pathscale on freenode or email me offlist to coordinate please.


Thanks

Re: [gentoo-dev] Re: LTO use in the tree

[C. Bergström]

On 04/28/14 06:14 AM, Joshua Kinard wrote:

On 04/27/2014 19:08, Rich Freeman wrote:

On Sun, Apr 27, 2014 at 6:56 PM, Joshua Kinard  wrote:

My curiosity, as I have not attempted LTO yet on any machine, is what are
the RAM requirements?  Is it a hard limit, wherein the compiler simply fails
if there isn't enough RAM, or does it just start hitting swap real hard?

It just allocates RAM, and the OS does the rest.  I've seen it invoke
the OOM killer.  That was back when I only had 8GB of RAM.  Now I have
16GB and I only need to disable LTO on the really big packages.

Of course, if you set an appropriate ulimit then the process will just
terminate more gracefully.  I'd highly recommend doing just that if
you have a lot of swap available.

My favourite, starting long compiles on slow boxen, only to wake up to
discover they failed in the final five minutes of the build over something
as trite as low memory :)



Those of us using older archs where the RAM is limited might have to be more
cautious w/ LTO.  I.e., my SGI O2 maxes right now at 512MB.  It can go to
1GB if the odd memory/PROM issue is ever worked out.  But 512MB is it for
now, so what are my odds of successfully using LTO on that?

About zero.  Well, I'm sure it will work fine for hello.c, especially
if you eliminate any function calls inside of it.

About zero?  So, some floating point value infinitely between 0 and 1?  Hmm,
maybe I'll try it once I get my SGI Octane to boot Linux again.



Especially if LTO helps to reduce the final binary size, that's less data
being shuffled around main memory and the CPU caches, which, although means
slower compile times, might hake such a machine a bit snippier.  Though, I
dread how long GCC will take to build itself w/ LTO.  The O2 already needs
~18hrs for 4.8.  I haven't tried 4.9 on it yet.

Yeah, good luck with that...  :)

I'd be curious as to what you find.  You can always try it out by
picking a small package and doing a CFLAGS=foo emerge bar.  Be sure to
only use -j1 -flto=1 as well.

O2 only has one CPU, so it's always -j1.  SMP on my other MIPS machines
doesn't work yet (either Linux isn't supported, or I haven't debugged SMP
code yet).
On those old SGI MIPS machines use MIPSPro. It had better (LTO/whole 
program) optimizations than GCC more than 10 years ago (imho and gcc may 
have caught up now in 4.9). Just add the -ipa flag and test. In fairness 
there is primarily 3 limitations with MIPSPro IPA


1) It set a rather low (by modern standards) cutoff for when IPA 
wouldn't be really turned on (like 1MLOC or something)
2) GCC and others will do IPA on a single file (Module) - whereas 
MIPSPro required whole program to do some similar optimizations
3) MIPSPro is/was also a very very slow compiler (I do not recommend 
this for large c++ codes)


Most "sane" compilers who are dealing with IPA will have some cutoff in 
order to avoids insane levels of memory usage. The compiler will bail 
internally on the problem, but the compilation won't fail. It's also in 
theory possible to segment the problem and work on chunks of it. This 
would also in theory move to longer compile times, but lower memory 
constraints. "We" aren't doing this yet and I don't know anyone who is, 
but I'm possibly just uninformed.


In terms of general performance gains using LTO - The #1 candidate would 
be the linux kernel actually. See if anyone can get that to work ;)


While this thread is fun - I should exit() here since it doesn't seem 
productive to discuss further..


Thanks

Re: [gentoo-dev] Re: LTO use in the tree

[C. Bergström]

On 04/27/14 06:23 PM, Rich Freeman wrote:

On Sat, Apr 26, 2014 at 10:37 PM, "C. Bergström"
 wrote:

#2 The only reference to anything which the compiler could impact is
"Use Boyer-Moore (and unroll its inner loop a few times)." Finding out which
flag controls that for ${CC} would have some importance. It's almost
certainly combined with -O3 and or some standalone loop related
optimization. (Nothing depending on LTO). If they were really clever or
determined  - there's probably a few GCC or other pragma which could give a
hint about unrolling.

So, I'll certainly agree that package-specific CFLAG tuning will
always be superior to just setting some flag at the system level and
walking away.

And yet, in the same paragraph you mention -O3, which is tantamount to
just setting a flag and walking away.  That turns on 14 things you
probably don't really need.
I was trying to give a simplified example... no need to nitpick my reply 
(Every compiler defines -O3 differently and even the flag to unroll 
loops and that threshold may be different.. ...)


I run -flto at the system level since in my experience it only causes
problems with a handful of packages, and when it does provide a
benefit I get it.
Can you name a single package that you use which receives a measurable 
benefit from LTO? (Just asking)


I don't disagree about enabling it, filing bug reports or many other 
things. I'm just curious if you have any hard numbers... (You seem 
passionate and sorry if this seems like I'm putting you on the spot)


/*
Side note
IPA (aka whole program and LTO) is by far the hardest optimizations I've 
ever personally had to debug/engineer/tune in a compiler. Making it 
robust needs passionate users who file good reduced test cases. While 
for a single source you have creduce or delta - what options are there 
for automated reduction of whole program problems..

*/

Re: [gentoo-dev] Re: LTO use in the tree

[C. Bergström]

On 04/27/14 09:14 AM, Alex Xu wrote:

On 26/04/14 08:34 PM, "C. Bergström" wrote:

Pragmatically nobody gives a f* if grep has been optimized to the max
since it's usually not the bottleneck.

http://lists.freebsd.org/pipermail/freebsd-current/2010-August/019310.html

My point about grep + LTO still stands

#1 This isn't gentoo FreeBSD so it's probably irrelevant from the start 
- (the comparison is gnu vs bsd grep. Further - LTO won't save your butt 
from poor programming practices or magically turn things into efficient 
syscalls)


#2 The only reference to anything which the compiler could impact is
"Use Boyer-Moore (and unroll its inner loop a few times)." Finding out 
which flag controls that for ${CC} would have some importance. It's 
almost certainly combined with -O3 and or some standalone loop related 
optimization. (Nothing depending on LTO). If they were really clever or 
determined  - there's probably a few GCC or other pragma which could 
give a hint about unrolling.

---
The color of my bikeshed is __

Re: [gentoo-dev] Re: LTO use in the tree

[C. Bergström]

On 04/27/14 02:58 AM, Martin Vaeth wrote:

Rich Freeman  wrote:

FWIW the list of packages I have issues with include:

Not sure whether this is the right place to post it.
It's interesting to see that rather lengthy list. From a compiler 
engineer perspective I'd like to toss in my opinion

-
Compiler flags are typically meant to do one or two things. Improve 
performance or reduce binary (code size). Pragmatically nobody gives a 
f* if grep has been optimized to the max since it's usually not the 
bottleneck. Having LTO and whole program optimizations turned on for 
every package will probably not give you a noticeably faster system, but 
will certainly slow your build down. (Due to rather large link times)


The packages which it really *should* be turned on for - anything which 
is computationally complex, Fortran and stuff where performance matters. 
I don't know gcc's LTO or what it's capable of, but in our compiler it 
would also potentially improve large c++ applications a lot. (It should 
help inline more aggressively and remove c++ layer overhead). In 
practice though - the most important c++ applications tend to be too 
huge and end up hitting bugs. They will also typically be very very long 
link times. (I've seen 30+ minutes - system specs and all things being 
relative of course..)


Go ping the gentoo-science guys and get their feedback - they may have 
the most experience with this...

--

Not to be a smart-ass, but will someone start a thread on global PGO 
(profile guided optimizations) next? imho it would be interesting and 
great to have some general training data already contributed next to the 
ebuilds. For the science stuff I wouldn't recommend it, but who knows..

Re: [gentoo-dev] [OT] pkgcore bikeshed (was Portage team)

[C. Bergström]

On 01/14/14 12:37 AM, Greg KH wrote:

On Mon, Jan 13, 2014 at 04:15:37PM +0700, "C. Bergström" wrote:

At the end of the day we have one codebase which is "engineered" and
another which has "evolved".

I'll take an "evolved" codebase over "engineered" anyday.

You do realize that is exactly why Linux has succeeded, right?  The
kernel has evolved, and was never "engineered".  There's lots people
should be learning from biology...

So you are using the benifits of evolution right now on your system,
don't knock it, it's proven to work.
I'll bite - While I don't think nature stopped to properly design 
interfaces along the way. I bet you Linus wouldn't agree with your 
comment very much.


1) I expect quite a bit of time has gone into (Solaris and Linux) kernel 
interfaces
2) Any larger or invasive changes require quite a bit of planning, 
review and testing. (Possibly with tests (public/private) to cover a 
large amount of the new/existing feature


To clarify - by engineering I mean significant forethought into the 
design and implementation. I'm not trying to take away the achievements 
of any particular piece of software. In this case it just one of those 
"you know it when you see it". If I wasn't clear or you don't understand 
what I meant - that's ok because I meant no offense to anyone.

Re: [gentoo-dev] [OT] pkgcore bikeshed (was Portage team)

[C. Bergström]

On 01/13/14 04:31 PM, Fabio Erculiani wrote:

On Mon, Jan 13, 2014 at 10:15 AM, "C. Bergström"
 wrote:

On 01/13/14 03:43 PM, Alexander Berntsen wrote:
Where I work uses pkgcore[1], but not the areas which are generally
beneficial to the whole community. (We use it as part of a web application
to handle testsuites which have build dependencies.) We can blah blah about
performance of resolving package dependencies all day long,
[...]

Not sure about what you mean with "blah blah". But given the amount of
both disk caches (metadata, vdb cache) and memory caches (the
in-memory aux_db cache that portage loads using pickle (it's a dict)
takes like 70-100Mb of RAM on an average desktop system), Portage can
still take *minutes* to calculate the merge queue of a pkg with all
its deps satisfied. Ironically, launching the same emerge command
twice, will take more or less the same time.
Yeah, this is probably bad design...
ack - I know the benefits (and downsides) of pkgcore compared to 
portage, but I leave that up to others who would like to voice their 
opinion. It would be great to get pkgcore up to feature parity with 
portage, but I don't have the resources to help with that. (In the 
future, possibly next month, I will try to put some bounties)

Re: [gentoo-dev] [OT] pkgcore bikeshed (was Portage team)

[C. Bergström]

On 01/13/14 03:43 PM, Alexander Berntsen wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 13/01/14 09:39, C. Bergström wrote:

Drive-by trolling comment but I wish the effort to keep porkage
alive would have instead been directed towards pkgcore.

Realistically, we have to keep updating them both in parallel. pkgcore
needs to be brought up to portage-level functionality, and we need to
keep fixing portage bugs for its many users. We could have a technical
discussion on the merits of pkgcore vs. portage, but in the end, it's
about the users.

For the record, I would be very happy if you hacked on pkgcore. Just
as happy as if you hacked on portage, even. So please do. :-)
Where I work uses pkgcore[1], but not the areas which are generally 
beneficial to the whole community. (We use it as part of a web 
application to handle testsuites which have build dependencies.) We can 
blah blah about performance of resolving package dependencies all day 
long, but it's clearly not a game changer feature for users. (or they 
just don't know)  Long term the API to pkgcore could be beneficial, but 
again I'm not sure it's a game changer for users. Dare I turn this from 
a trolling comment into a laundry list of pros/cons of pkgcore and why 
now may be a good time to invest in the future. At the end of the day we 
have one codebase which is "engineered" and another which has "evolved". 
If gentoo only ever wants to fetch tarballs and build source.. who 
cares.. If you ever want to build something on top of that - you will 
painfully realize that there's only 1 choice.


On 01/13/14 03:59 PM, Dirkjan Ochtman wrote:
If you know your email is going to be drive-by trolling, maybe just 
hold it in next time? 

Yeah I know better, but this time was like a fart - (half attempt joking)



[1] /* Side details */
In a nutshell we make it possible to track the results of "make check" 
or equivalent test coverage for every source package. There's a little 
work involved for setting up each package, but it gives the easy ability 
to *know* and prove that between xyz ${FLAGS} there's no regressions. 
For example: How do you *know* that fftw or ssl is regression free when 
you enable avx, fma or some other higher level of optimization (-O3). By 
knowing I don't mean just result correctness, but also potentially 
runtime performance, code size and potentially compile times. (If those 
are things you care about)

Re: [gentoo-dev] Portage team, Zac's development break and stepping down as lead

[C . Bergström]

Drive-by trolling comment but I wish the effort to keep porkage alive would have instead been directed towards pkgcore.

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/19/13 03:35 PM, Michał Górny wrote:

Dnia 2013-12-19, o godz. 15:28:46
"C. Bergström"  napisał(a):


On 12/19/13 03:20 PM, Michał Górny wrote:

Dnia 2013-12-19, o godz. 00:56:31
"C. Bergström"  napisał(a):


On 12/19/13 12:47 AM, Kent Fredric wrote:

On 19 December 2013 06:33, Jan Kundrát  wrote:

I'm worried by the cost of such a policy, though, because we would suddenly
have to patch some unknown amount of software

Given the nature that changing that CXX Flag globally for all users
could cause many packages to spontaneously fail to build, wouldn't
that imply that changing that flag would essentially be de-stabilizing
the whole tree, and a package being (arch) would no longer be an
indication of sane, tested behaviour?

This is really the perk of the USE driven process, the granular
piecemeal approach that does only as much as necessary, without
changing things that are already stable.

In practice wouldn't that mean you'd have to add c++11 USE flag to every
C++11 application and lib?

No. Only the libs that change their ABI in C++11.


"Best case" both build and you end up with a linker problem (can be
worked around with compiler patches)
/usr/lib64/libboost.so
/usr/lib64-c++11/libboost.so

What's wrong with this solution:

1. distro-specific compiler patching is wrong,

Pragmatically, this needs to be upstream and should have been there
already. Get some feedback to see if gcc people are receptive to the
idea before testing a gentoo-only patch. If they accept it upstream -
backport it. If they tell you f* off - get their feedback on how to deal
with it - more belo

(this is not a gentoo only problem - this discussion should happen on a
more global level...)

And how is this an issue to the major distributions? Binary distros can
do a simple switch with standard all-package upgrade and forget about
it. Like they usually do. Only people who built from sources have to
think about it.

Umm..  no? Lets use a hypothetical example...

libboost.so (or any really popular lib.. Qt..) built with -std=c++11 
breaks abi


If they don't do some sort of multilib approach - they are only going to 
build it once and then any consumer of that outside the distro is stuck 
with their decision. That's probably fine in the predominately C++03 
world we have today, but for how long? I expect users on the binary 
distro just do what they have to work around the problem (go build their 
whole dependency chain from source). It didn't solve the problem - just 
made it work for distro packages and pushed it off to the user.


My -L rant would depend on the above being used - that's all

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/19/13 03:20 PM, Michał Górny wrote:

Dnia 2013-12-19, o godz. 00:56:31
"C. Bergström"  napisał(a):


On 12/19/13 12:47 AM, Kent Fredric wrote:

On 19 December 2013 06:33, Jan Kundrát  wrote:

I'm worried by the cost of such a policy, though, because we would suddenly
have to patch some unknown amount of software

Given the nature that changing that CXX Flag globally for all users
could cause many packages to spontaneously fail to build, wouldn't
that imply that changing that flag would essentially be de-stabilizing
the whole tree, and a package being (arch) would no longer be an
indication of sane, tested behaviour?

This is really the perk of the USE driven process, the granular
piecemeal approach that does only as much as necessary, without
changing things that are already stable.

In practice wouldn't that mean you'd have to add c++11 USE flag to every
C++11 application and lib?

No. Only the libs that change their ABI in C++11.


"Best case" both build and you end up with a linker problem (can be
worked around with compiler patches)
/usr/lib64/libboost.so
/usr/lib64-c++11/libboost.so

What's wrong with this solution:

1. distro-specific compiler patching is wrong,
Pragmatically, this needs to be upstream and should have been there 
already. Get some feedback to see if gcc people are receptive to the 
idea before testing a gentoo-only patch. If they accept it upstream - 
backport it. If they tell you f* off - get their feedback on how to deal 
with it - more below.


(this is not a gentoo only problem - this discussion should happen on a 
more global level...)


Unfortunately, it's going to be really hard to tell what will break ABI 
and what won't. I guess for ABI compatible packages 
/usr/lib64-c++11/libfoobar.so would be a symlink back to 
/usr/lib64/libfoobar.so


2. kinda FHS deviation, at least in spirit of lib directory.

We could go with '-L' but this is very fragile anyway. It's *very easy*
for the compiler to link the 'wrong' library due to -L/usr/lib64 being
added by some kind of foo-config.
-L would likely mean you also need -nostdlib to make it work - which is 
more hacky than the above. pretty please don't do this.. plaassse

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/19/13 12:47 AM, Kent Fredric wrote:

On 19 December 2013 06:33, Jan Kundrát  wrote:

I'm worried by the cost of such a policy, though, because we would suddenly
have to patch some unknown amount of software


Given the nature that changing that CXX Flag globally for all users
could cause many packages to spontaneously fail to build, wouldn't
that imply that changing that flag would essentially be de-stabilizing
the whole tree, and a package being (arch) would no longer be an
indication of sane, tested behaviour?

This is really the perk of the USE driven process, the granular
piecemeal approach that does only as much as necessary, without
changing things that are already stable.
In practice wouldn't that mean you'd have to add c++11 USE flag to every 
C++11 application and lib?


I just sent an email with some crazy thoughts - Your point is totally 
correct - this "migration" needs to happen while not breaking the whole 
tree. Logistically - what's the best way to maintain both those "ABI" at 
the same time?


"Best case" both build and you end up with a linker problem (can be 
worked around with compiler patches)

/usr/lib64/libboost.so
/usr/lib64-c++11/libboost.so

Worst case only 1 builds
this breaks down into generally 2 cases
1. Programs/libs which have intentionally adopted c++11 and don't 
care about C++03 (clang/llvm/lldb)

2. Programs/libs which can't be compiles with c++11 mode


/usr/lib64/libfoo.so

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/19/13 12:33 AM, Jan Kundrát wrote:

On Wednesday, 18 December 2013 18:05:46 CEST, "C. Bergström" wrote:
If moving to C++11 - Isn't that considered just part of the work 
along the path? There's some clang tools to help with the migration, 
but I don't think anyone expects it to be zero work. The flag is just 
a way to a) enable building programs that can be built with c++11 b) 
flush out the culprits in the cases it can't be. If (b) is a bug - 
how else to find it easily?


This perspective is interesting (and I admit that I tend to like it) 
-- considering packages which won't build with C++11 to be buggy.


I'm worried by the cost of such a policy, though, because we would 
suddenly have to patch some unknown amount of software (and I'm pretty 
sure some upstreams would reject these patches anyway). If we were an 
enterprise distro with binary compatibility requirements, we would 
also have to worry about that and either assume that the ABI changes 
are non-issue in real world, or provide two versions of all C++ 
libraries. I tried to check how RHEL7 will deal with it, but I wasn't 
able to find any information about that. It also seems that Fedora 
hasn't addressed this yet, either.


Either way, it is reasonable to assume that some users would like to 
build their own software and link it with system libraries. It is not 
reasonable to force these users to build in the C++11 mode, IMHO.

I want my cake and eat it too!

/* Off the cuff crazy ideas and not meant to be taken too seriously */

In my mind this is almost similar to ABI (incompatibilities) between 32 
and 64bit. Why not just (ab)use the multilib approach for c++11?


/usr/lib64-c11/

It would possibly (likely) require some patching to clang/g++ when the 
--std=c++11 flag is used, but might allow the extension of a c++11 
library universe while leaving the things which are working today 
unbroken. The main problem I see with this is proliferation of more crap 
in /usr/lib* (4 variations instead of just the 2 (32vs64) we have now). 
(Personally, I'd typically build *only* 64bit versions and 32bit x86 can 
go to hell...)


To get support from the "enterprise" distros - I'd try to move the 
discussion of this problem upstream to the LSB level. They will at some 
point need to solve this same problem as well. It's not like it'll be 
gentoo specific forever. (You guys are just pioneering and ahead of the 
curve.. this isn't the same thing as -Omgfast)


With this approach - would it make sense to create a new profile?

I don't know if a USE flag would in general get pushback, but if the 
c++11 use flag was used - it would put those libs in /usr/lib64-c11/

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/18/13 11:50 PM, Jan Kundrát wrote:

On Wednesday, 18 December 2013 17:37:56 CEST, "C. Bergström" wrote:

 From the perspective of a compiler vendor - I must ask why not?


There is code out there which builds fine under C++98, but fails to 
build when C++11 is enabled (as but one exmaple, have a look at [1]).
If moving to C++11 - Isn't that considered just part of the work along 
the path? There's some clang tools to help with the migration, but I 
don't think anyone expects it to be zero work. The flag is just a way to 
a) enable building programs that can be built with c++11 b) flush out 
the culprits in the cases it can't be. If (b) is a bug - how else to 
find it easily?

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/18/13 11:29 PM, Jan Kundrát wrote:

On Wednesday, 18 December 2013 14:58:07 CEST, hero...@gentoo.org wrote:

I think it is better achieved by a (simple and stupid) global
CXXFLAGS. Adding an extra USE flag feels a little over-engineering.


What compiler flag do you propose to use? Note that --std=c++11 will 
not work.

From the perspective of a compiler vendor - I must ask why not?

Re: [gentoo-dev] How to support C++11 in libraries?

[C. Bergström]

On 12/18/13 02:54 PM, Michał Górny wrote:

Hello, folks.


Hi



Basically, I've hit this with sys-devel/llvm. A user has requested lldb
support to be enabled in the ebuild [2]. Sadly, lldb requires C++11 to
be used, and this means that whole LLVM needs to become C++11 enabled.
And then, it would be at least recommended that all reverse deps become
C++11 enabled as well.

/*
Personally, I think lldb is pooh (bloated mess that has poor internal 
design, offers little or no logical features that can't be accomplished 
with source access+printf and takes a needlessly long time to compile as 
a result)

*/

If the only driving motivation is lldb then I think this isn't worth the 
effort and I wonder what may be incompatible as a result. Long term it 
certainly should happen - I can't/won't argue or disagree with the long 
term merits, but when.. and who will do all that work..


Just a heads up that clang/llvm will (have in svn trunk) force building 
with c++11 for the next major release (6 months from now). So unless 
some 3rd party goes and backports or removes the c++11 pieces - this 
will add to the list of c++11 only software in the near future.

Re: [gentoo-dev] git-r3: initial draft for review

[C. Bergström]

Do you have any plans to add support for sparse checkout?

Something like this

|cd 
git clone -n 
cd 
git remote add –f  
git config core.sparsecheckout true
echo // >> .git/info/sparse-checkout
git checkout 

(Credit goes to : 
http://stackoverflow.com/questions/15827117/git-sparse-checkout-for-simple-web-deployment
 )
|


What about subtrees?

Nice work!

Re: [gentoo-dev] New eclass cuda.eclass

[C. Bergström]

On 11/26/12 12:59 AM, Rick "Zero_Chaos" Farina wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/25/2012 11:47 AM, Justin wrote:

Hi,

I would like to introduce a new eclass for packages using the nvidia
cuda compiler suite. Currently the eclass simply sanitize the NVCCFLAGS.
May be extended in the future.

Two problems come up with using nvcc:

* Each version only supports a limited number of gcc versions. Therefore
we need to pass the path to a supported gcc bindir

* nvcc calls CXX but doesn't pass CXXFLAGS on.
I don't know if this is helpful feedback, but it would be great if our 
"CUDA" compiler was also detected in your eclass.


Notable differences : Probably a lot faster compilation times (needs 
benchmarking), no external userland dependencies (gcc or nvidia), we 
don't support nvcc flags and we have a driver just for the CUDA language


Example:
pathcu deadbeef.cu
./a.out

(Unfortunately our recent nightly builds started to do a license check, 
but any interested open source dev just ping me to get setup)


To validate simple things you could use an older nightly
http://c591116.r16.cf2.rackcdn.com/enzo/nightly/Linux/enzo-2012-11-18-installer.run

chmod +x enzo-2012-11-18-installer.run ; ./enzo-2012-11-18-installer.run 
--mode unattended # --prefix /opt/foobar


./C

Re: [gentoo-dev] Six month major project on Gentoo

[C. Bergström]

On 12/15/11 01:05 AM, Christian Ruppert wrote:

On Wednesday 14 December 2011 16:36:42 Gaurav Saxena wrote:

Hello all,
I am interested in doing my final year computer scence project on gentoo. I
would be having a duration of six months to work on the project. Could you
please suggest me some good project ideas that would be helpful to me as
well as gentoo. I am interested in parallel computing, data structures ,
operating system. I am well versed in C/C++. I think  there might be
projects which need to be done, I would like to work on them.
Not directly gentoo, but certainly would impact all gentoo users - 
toolchain/compilers/path64
There's a number of ways the compiler could be improved for better SIMD 
vectorization and parallel computing
/* I'm biased and work on this project - ping me on irc if you're 
interested */


./C

#pathscale

[gentoo-dev] EKOPath/Path64 compiler ebuilds + support

[C. Bergström]

Hi

Recently a couple new ebuilds were added to the portage tree and I felt 
it's worthwhile to give a friendly heads up.


So without further ado let me introduce EKOPath and Path64.

EKOPath - This is a binary installer that comes from one of the nightly 
PathScale builds.  The source to the compiler/debuger/libraries.. etc 
are available on github.  It's meant as a convenience to new users and 
should be portable across many distributions.


Path64 - This is the source build of the compiler that will warm your 
house in the winter.  While not required I would generally recommend 
that you do a Release build.  If you don't the compiler will be *slower* 
than mud.  It may not be so bad if you're not building any C++ code, but 
you've been warned.

-
Documentation - We're working on it, but help is appreciated.  
Converting a 300 page user guide from $proprietary discontinued format 
to latex isn't fun.  (Send me a private email if you're interested to help)


Support - Our mailing lists are quiet, but most questions will get a 
response

http://lists.pathscale.com/mailman/listinfo

If you're on irc feel free to say hi or ask questions on freenode - 
#pathscale
# Warning: The channel is generally quite friendly unless you say 
something stupid or piss off an OP

--

Bug reports
For now we're using github issues.  # Yes it sucks and we'll get a jira 
instance setup some time in the future

https://github.com/path64/compiler/issues/

We welcome any and all bug reports - performance regressions, ICE, 
incorrect results. etc.  The best way to get your bug fixed is by having 
the smallest possible test case which reproduces the problem.


We generally recommend the tool Delta for getting a good automatic start 
before hand reducing

http://delta.tigris.org/

Why care?
PathScale traditionally cared a lot about the performance for HPC and 
scientific codes.  If you have some Fortran code I'd highly recommend 
you give it a try.  C++ is getting a lot of improvements and we're 
aiming to take the lead before the end of the year.  That leaves C and 
anything else which people really care about
(OpenSSL, web servers, database.. etc)  Try to build it, file good bug 
reports, tweak the flags for best performance and yell at us if we're 
not the fastest.


I'm around if you have any questions or just generally need someone to 
yell at :)



./C

@CTOPathScale

Re: [gentoo-dev] sandbox-1.3 testing request

[C. Bergström]

Mike Frysinger wrote:
can people who feel adventurous unmask sandbox-1.3.2 and give it a spin on 
their systems before i unmask it for everyone ...


if you hit a bug, use bugzilla mmkay
-mike
  
I pulled the latest git sources about a week ago and seemed like I hit a 
regression in the autoconf.. (I'm sure this isn't the case and probably 
a regression in my perl)  Sorry I only have fuzzy details right now, but 
tested it about a week ago.  If anyone is interested to take a look at 
it for Solaris I'm more than happy to try to provide ssh access to a dev 
machine with everything in place and ready to go.  My results prior to 
the last ones were quite optimistic.. (feel free to pm me off list)


btw.. Thanks Mike for the great work on this..


./C

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Fabian Groffen wrote:

On 17-09-2008 10:41:07 +0200, "C. Bergström" wrote:
  

By the way, I'm against this stuff.  I rather see a PATH solution
involved.  Portage already has a DEFAULT_PATH, and if someone refuses to
install patch, one could always use a special directory with symlinks to
the g-versions, e.g. patch -> /usr/sfw/bin/gpatch such that
Portage/eclass/ebuilds don't have to bother about this at all.
  
  
patch is installed and I would agree with you, but in certain  
circumstances using the GNU tools are broken.



Then if that is the case, Portage/eclass/ebuild relies on that
brokenness.  I'm not saying you should have the same PATH as Portage.
  
GNU tools always behaved as expected on Linux.  The brokeness is 
platform specific in my case.  Anyway, I'm pretty sure you use Solaris 
in production and I'm just trying to make the situation easier.  If 
someone will review the patch I'll make it up, but don't want to waste 
more of anyone's time on this.


Thanks

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Fabian Groffen wrote:

On 17-09-2008 10:21:17 +0200, Santiago M. Mola wrote:
  

Why not simply alias patch=gpatch in profile.bashrc?
See the FreeBSD profile for an example.



I'd like to package portage for OpenSolaris and have it just drop-in work so
modifications like what you suggest wouldn't be required.
  

You'd still need to create an OpenSolaris profile. While you're at it,
you can create a profile.bashrc with the required modifications.

I don't see any reason to not do the gpatch change, but it looks like
unecessary to me because you already have simpler ways to solve the
problem. So, requiring others to do a significant useless amount of
work when you can solve it with just a line is not fair.



>From some experience, I can tell that an alias is not sufficient to
cover all cases, and will result in random failures because you only
notice too late patch is used and not gpatch.

By the way, I'm against this stuff.  I rather see a PATH solution
involved.  Portage already has a DEFAULT_PATH, and if someone refuses to
install patch, one could always use a special directory with symlinks to
the g-versions, e.g. patch -> /usr/sfw/bin/gpatch such that
Portage/eclass/ebuilds don't have to bother about this at all.
  
patch is installed and I would agree with you, but in certain 
circumstances using the GNU tools are broken.

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Santiago M. Mola wrote:

On Wed, Sep 17, 2008 at 9:59 AM, "C. Bergström"
<[EMAIL PROTECTED]> wrote:
  

Ulrich Mueller wrote:


On Wed, 17 Sep 2008, C. Bergström wrote:


  

Here's another idea and I don't know why I didn't think of it
sooner.. Instead of any system change to the patch ebuild.. Inside
the eutils.eclass do a quick check for gpatch and if it exists use
that vs patch.



Why not simply alias patch=gpatch in profile.bashrc?
See the FreeBSD profile for an example.

  

I'd like to package portage for OpenSolaris and have it just drop-in work so
modifications like what you suggest wouldn't be required.



You'd still need to create an OpenSolaris profile. While you're at it,
you can create a profile.bashrc with the required modifications.

I don't see any reason to not do the gpatch change, but it looks like
unecessary to me because you already have simpler ways to solve the
problem. So, requiring others to do a significant useless amount of
work when you can solve it with just a line is not fair.
  

If I don't have to introduce a new bash dependency I'd like to avoid it.

I'll do the work. The patch is a couple lines and I don't think it would 
be significant or impact maintenance of the eclass.

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Fabian Groffen wrote:

On 17-09-2008 09:59:42 +0200, "C. Bergström" wrote:
  

Why not simply alias patch=gpatch in profile.bashrc?
See the FreeBSD profile for an example.
  
  
I'd like to package portage for OpenSolaris and have it just drop-in  
work so modifications like what you suggest wouldn't be required.   
Hopefully, base-system maintainers don't mind my request, but I can  
understand it's beyond the Linux use case.



As I understood it, you want to package (Prefix) Portage for
(Open)Solaris, but without USERLAND=GNU like Gentoo Prefix does.  You
want all eclasses, ebuilds and portage itself to work with the Solaris
userland tools.

  

Not exactly..

1) yes I'm patching (prefix) portage for OpenSolaris.. I say OpenSolaris 
because while I'm doing best effort to make it as portable as possible, 
but can't test and don't much care about prior versions w/o source.


2) OpenSolaris comes with most GNU tools by default (except sed/find) , 
but appends a g to the front

File: `/usr/gnu/bin/sed' -> `../../bin/gsed'

I'd happily not bring this issue and bother anyone, but on every sync 
that eclass is going to get overwritten and the change is pretty minimal.


In regards to the ebuilds I will manage an overlay containing mostly 
repackaged sources from upstream.  The end goal being able to repackage 
the entire OpenSolaris distribution built with portage from source.


Thanks for the minute

./Chrisotpher

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Ulrich Mueller wrote:

On Wed, 17 Sep 2008, C. Bergström wrote:



  

Here's another idea and I don't know why I didn't think of it
sooner.. Instead of any system change to the patch ebuild.. Inside
the eutils.eclass do a quick check for gpatch and if it exists use
that vs patch.



Why not simply alias patch=gpatch in profile.bashrc?
See the FreeBSD profile for an example.
  
I'd like to package portage for OpenSolaris and have it just drop-in 
work so modifications like what you suggest wouldn't be required.  
Hopefully, base-system maintainers don't mind my request, but I can 
understand it's beyond the Linux use case.

Re: [gentoo-dev] Re: Request for feedback on GNU Patch change

[C. Bergström]

Duncan wrote:

"C. Bergström" <[EMAIL PROTECTED]> posted
[EMAIL PROTECTED], excerpted below, on  Tue, 16 Sep 2008
22:30:51 +0200:

  

1) Add a symlink in GNU patch ebuild to symlink patch to gpatch



You mean the other way, right?  gpatch -> patch , since we already have 
patch, and need gpatch if the below is to work.


Or did you mean change the sys-devel/patch ebuild so the executable is 
named gpatch instead of patch, then create a symlink patch -> gpatch as 
suggested above.


  

2) Change references to patch in eclass/eutils.eclass to gpatch



This makes sense to me.

  
Here's another idea and I don't know why I didn't think of it sooner..  
Instead of any system change to the patch ebuild.. Inside the 
eutils.eclass do a quick check for gpatch and if it exists use that vs 
patch.  I'm trying to think of any circumstances where this would fail..

[gentoo-dev] Request for feedback on GNU Patch change

[C. Bergström]

To start.. I humbly ask for no response vs starting a flamewar..

Short version:
The proposed change will require changes in two places, but may cause 
other breakage and or simply not settle well with general consensus.


1) Add a symlink in GNU patch ebuild to symlink patch to gpatch

2) Change references to patch in eclass/eutils.eclass to gpatch
-

Benefit is that on systems != Linux where GNU patch exists in the 
subpath to some crappy legacy thing which probably shouldn't even be 
called patch.. On these systems calling gpatch explicitly means GNU 
patch and avoids the whole PATH and incompatibility issue.


Would this benefit those working on the FBSD?
Anyone interested in OpenSolaris?

I have other changes I'm making to remove GNUisms, but this one I'm not 
seeing much hope for and will pop up on every emerge --sync..


patch -p0 -g0 -E --no-backup-if-mismatch < /opt/foo/bar.patch

Suggestions?

Thanks

./Christopher

[gentoo-dev] OT: Dickheads, but good devs Was: Re: Re: gentoo-dev vs lkml

[C. Bergström]

Since there's a select few people here who feel it's their duty to keep 
posting non-technical discussion to this list.


1) Someone much more senior than me please step in and take a leader role.
2) Everyone wrapped up in please take a step back and see what's 
actually trying to be accomplished.


The facts are great.. I appreciate reading about some of the recent 
history, technical hurdles/shortcomings/foo and the personalities 
involved.  On the other hand there needs to be a point and ranting has 
continued for hours now.


Please bring this back to a technical relevant discussion or move it 
offlist.


Thanks :)

C.
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: gentoo-dev vs lkml?

[C. Bergström]

This is a great link for the leaders, developers and just about anyone 
else involved in our community.  While this is solely my opinion I do 
humbly ask anyone with a spare few minutes to step back and take a look.


http://video.google.com/videoplay?docid=-4216011961522818645&q=poisonous+people


C.
--
gentoo-dev@gentoo.org mailing list