date:20140109

Hello Alec,

Thursday, January 9, 2014, 12:12:18 PM, you wrote:

On Wed, Jan 8, 2014 at 11:24 PM, LTHR lanthrus...@gmail.com wrote:
Hi All,

I want to start off by discussing your premise, before embarking on the overall
goals.

You wrote:
I'm with Gentoo for many years. For various reasons many techs were not
implemented and now Gentoo is in a kind of stagnation. But we can give Gentoo a
new birth with relatively little efforts and bring the distro to the whole new
level.

I don't actually believe your premise of stagnation. But I can put aside my
disagreement for now.

True. There is no data to tell what happens with Gentoo (to give that data is
one of the goals of the
project). We only have some formal esteems from unreliable sources.

According to distro watch:

http://web.archive.org/web/2013070100*/http://distrowatch.com/dwres.php?resource=popularity

In February 2012, Gentoo distro was in 19th place.
In December 2012, Gentoo went to 22nd place.
In December 2013, Gentoo is down to 32nd place

According to Linux Counter
http://web.archive.org/web/2012010100*/http://linuxcounter.net/distributions/stats.html

In January 2012, Gentoo distro had 5.32%
In January 2012, Gentoo had 4.04%
In November 2013, Gentoo had 4,21%

And from my experience of Gentoo forums, gentoo.wiki - I vote for Gentoo at
least not gaining new users.
If in several years the number of users is not increased - we can tell about
stagnation.

Why new users are not with Gentoo and how to get them - good question, let's
find out!

Lets talk about how the overall goal of 'bringing the distro to a whole new
level' and how 'Portage QoS' will help us get there. I don't think you covered
these points well in your post (I will talk about the goals more below...)

I'll re-phrase the goals.

It's all not very well thought after at this stage but immediate goals are like
this:

* Knowledge of the number of Gentoo distros installed world wide - knowing the
trend how many users choose
Gentoo and where Gentoo is really going down|up|stands still.

You can then try different features and see how a feature is met - if the
number of systems increase
then this feature is probably useful. It's a strategical job, somebody at the
very top of the project should
analyze databases and make conclusions.

* Knowledge of the ebuild popularity - what ebuilds are popular and what are
not - what ebuild to give an extra focus
and what ebuild could wait

* Knowledge of ebuild quality. If some ebuilds fail on many systems - something
is wrong and ebuild and may be portage
need fixing. It's especially useful to make sure that all ebuilds have
correct dependencies, missing dependencies, etc.

* A formal esteem of portage quality
PortageQ = (the number of successful ebuilds/the number of all ebuild
attempts)

Portage speed efficiency:
Average time before build starts (or download starts)

How many times portage fails itself.

* Immediate problem detection. If the number of PortageQ went down last day -
there is some problem.
(then you go to ebuild stats and see what is failing)

* Reducing load on bugtracker folks - the build problems will be detected
automatically and solved according
to their importance. There will be no need to supply bug tracker with ebuild
logs and emerge --info if
somebody wants to report a problem.

* Team efficiency esteem. The stats will tell what ebuilds are failing most
often.

* Team automated info. When failure rate of a certain ebuild increase the
maintainer is automatically
informed and he can login in web-interface and see details how exactly ebuild
failed.
The same for the portage itself. Next day a maintainer could push a new
ebuild in the portage and the
problem might be solved.

It's not possible not to make mistakes. But it's possible to react on their
consequences fast.

* Knowledge what kernels are used by Gentoo users, how often they update their
systems, what flags
are used

2nd turn goals:

* to integrate forums.gentoo.org and bug tracker. People are offering great
workarounds and solutions. But
they're not known to the majority of Gentoo users.

If a e-build fails - may be there is already a solution - and we can offer
the solutions automatically from
portage. Like:

There might be some work-arounds on this problem:
[Gentoo Forum - qt-core ebuild fails - SOLVED]
htpp://forums.gentoo.org/.

There is a known bug on this ebuild:
[Gentoo Bug - qt-core ebuild fails]
htpp://forums.gentoo.org/.

* to make Bug Tracker almost unmanned. We can use gathered infromation on
failed e-builds to
create bugs in Bug Tracker automatically and automatically set priorities
according to the
severity.

The severity could be assigned automatically from package popularity and
failure rate stats.

The users with the problems could receive e-mail automatically to follow up
the quick

[gentoo-dev] Last rites: net-libs/libguac net-libs/libguac-client-rdp net-libs/libguac-client-vnc net-misc/guacd www-apps/guacamole-0.8.3

2014-01-09 Thread Andreas Schuerch

Due to bug 497262, I will mask the following packages for removal in 30 days.

net-libs/libguac-0.6.3
net-libs/libguac-0.7.0
net-libs/libguac-client-rdp-0.6.2
net-libs/libguac-client-rdp-0.7.0
net-libs/libguac-client-rdp-0.7.1
net-libs/libguac-client-vnc-0.6.1
net-libs/libguac-client-vnc-0.7.0
net-misc/guacd-0.6.2
net-misc/guacd-0.7.0
www-apps/guacamole-0.6.2
www-apps/guacamole-0.7.0
www-apps/guacamole-0.8.0

Since guacamole-0.8.3, only net-misc/guacamole-server  www-apps/guacamole are 
needed.

Br 
Andreas

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Christopher Schwan

Hi,

you motivate your proposal by claiming the Gentoo Project stagnates which you 
relate with its decline in popularity:

 According to Linux Counter
 http://web.archive.org/web/2012010100*/http://linuxcounter.net/distribut
 ions/stats.html
 
 In January 2012, Gentoo distro had 5.32%
 In January 2012, Gentoo had 4.04%
 In November 2013, Gentoo had 4,21%
 
 And from my experience of Gentoo forums, gentoo.wiki - I vote for Gentoo at
 least not gaining new users. If in several years the number of users is not
 increased - we can tell about stagnation.

But let me ask this question: Is the number of users really that important to 
Gentoo? Since it does not strive for world domination I think all that matters 
is to keep the current userbase happy. From your thread I do not understand 
whats wrong on that side:

 For various reasons many techs were not implemented and now Gentoo is in a 
kind of stagnation.

What do you mean by that in particular? And what is wrong with 
bugs.gentoo.org? Wouldn't it be better to talk about how attract more 
developers? I guess a lot unsolved bugs stem from the fact that there are too 
few that can take care of them.

Cheers,
Christopher


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] Portage Feature Request: making thirdpartymirrors easier to manage

2014-01-09 Thread Ben Kohler

On Wed, Jan 8, 2014 at 12:37 PM, Alex Xu alex_y...@yahoo.ca wrote:


 Eww. Geographically-close files should be made available through
 GENTOO_MIRRORS and the regular distfiles system.


I think you may be missing the point of this proposal, or are unaware of
how profiles/thirdpartymirrors and SRC_URI=mirror://.. work.

Readability and maintanence issues aside (which themselves are huge), the
current setup with a list of literally hundreds of geographically-random
mirrors for one source, is quite often doing a disservice to our users.
 Most of the very large upstream mirror list sources are already sorted
geographically, it would be great to take advantage of this.

And to me, it seems like the proposed thirdpartymirrors/mirrorname/Region
setup seems quite elegant.  I'm sure it would be optional and mirror lists
that can't take advantage of this would just use a flat file at
thirdpartymirrors/mirrorname.

-Ben

Re: [gentoo-dev] Portage Feature Request: making thirdpartymirrors easier to manage

2014-01-09 Thread Ben Kohler

On Mon, Jan 6, 2014 at 2:20 PM, Robin H. Johnson robb...@gentoo.org wrote:

 This is a small feature request, but it will require a modification to
 PMS, so I describe it here.

 The present thirdpartymirrors file is unwieldy, and difficult to manage
 due to it's format with very long lines. It also doesn't permit easy
 comments. Presently commits to it look very ugly, because diffs are
 line-based, and we pack a lot into each line.

 I would like to make it a directory instead of a single file, and extend
 the internal syntax.

 I am very excited about this whole idea, this thirdpartymirrors setup
badly needs some reworking.   To me it makes the most sense to turn
thirdpartymirrors into a dir, with a file structure like:

thirdpartymirrors/mirror1
thirdpartymirrors/mirror2
thirdpartymirrors/mirror3/
thirdpartymirrors/mirror3/Asia
thirdpartymirrors/mirror3/Europe
thirdpartymirrors/mirror3/Etc
thirdpartymirrors/mirror4

I'm not sure I see much real value in allowing individual profiles to
add/remove mirrors from each group, to be honest.  Maybe I'm just not
thinking of the right scenarios.

But I really do believe just the basic changes like splitting the file so
each group gets its own file, one server per line, with comments, etc...
will be a huge help in using and maintaining these groups.

-Ben

Re: [gentoo-dev] Portage QOS

Hello Christopher,

Thursday, January 9, 2014, 6:12:37 PM, you wrote:


 you motivate your proposal by claiming the Gentoo Project stagnates which you
 relate with its decline in popularity:

 According to Linux Counter
 http://web.archive.org/web/2012010100*/http://linuxcounter.net/distribut
 ions/stats.html

 In January 2012, Gentoo distro had 5.32%
 In January 2012, Gentoo had 4.04%
 In November 2013, Gentoo had 4,21%

 And from my experience of Gentoo forums, gentoo.wiki - I vote for Gentoo at
 least not gaining new users. If in several years the number of users is not
 increased - we can tell about stagnation.

 But let me ask this question: Is the number of users really that important to
 Gentoo? 

Should be. It looks important for Gentoo and the Gentoo developers.

For Gentoo - the more users Gentoo has the more resources it can deploy 
in development and support. May be the world domination isn't the correct 
word but a steady growth of Gentoo systems globally is a good goal. If Gentoo 
looses 1% annually in 5 years there would be no new developers motivated enough 
to push the project ahead and the old ones would think bad about investing 
their 
life in what is not gaining popularity and they will not be that enthusiastic, 
getting demoralized. Without fresh blood the crucial people will get old/tired 
and alas. 
I'm sure you don't need Gentoo only for yourself as nobody else here does. It's 
a project 
for people, the people get it running and it's the treasure the Gentoo has so 
why not 
to turn towards users? 

Try build a house, tell friends about the house, unite them in one goal and 
then build it for free and for everybody, then if the house is not working well 
- you have friends no more, next time you're alone in one huge house, 
nobody needs, nobody will help and you can't just support it on your own. 

 Since it does not strive for world domination I think all that matters
 is to keep the current userbase happy.

True. That is the goal.
 
 From your thread I do not understand
 whats wrong on that side:

 For various reasons many techs were not implemented and now Gentoo is in a 
 kind of stagnation.

 What do you mean by that in particular? 

Gentoo stopped. The work is done but it's not a game changer. The 
ebuilds have approximately the same time to install, the failure rate is about 
the same, 
emerge is getting slower. The number of users decrease. 
It looks like this is because it's not clear where to go and what to change. 
What to change 
exactly to bring more users? Not clear. No information.

Apparently the criteria is timing. When you develop a human access interface 
the most 
reliable thing to check your work against is the time required for an average 
user to achieve the goal. 

Time is the most important in our lifes and this is the criteria that always 
works. There 
are a variety of users, with different genetics, different views, education and 
skills but you 
can find an interface that unites them all and everyone is feeling like it's 
easy. It's not an
easy task because what looks easy for a developer might look alien for his 
neighbour. 

If we decrease time required for the users to maintain Gentoo and decrease time 
for developers 
to push the project - then Gentoo will grow.

 And what is wrong with 
 bugs.gentoo.org? Wouldn't it be better to talk about how attract more 
 developers?  

Good question. 
You can't attract enough supporters not being successful or
without paying them. Supporters are the same users if you're loosing
users the number of supporters are decreasing.

The times are changed. The projects are so complicated nowadays that 
keeping them manually is practically impossible. Why drudgery? There 
are numerous jobs with which robots can do better. Human should focus 
on what machines can't do better. 

 I guess a lot unsolved bugs stem from the fact that there are too
 few that can take care of them.

And from all these bugs only 10% are critical 20% are affecting like 
1% of all users and it's not clear what to fix first. 

It's a self balancing system. How do you plan to attract more developers? 
What to offer them?

-- 
Best regards,
 Igormailto:lanthrus...@gmail.com

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Ben Kohler

On Thu, Jan 9, 2014 at 6:44 AM, Igor lanthrus...@gmail.com wrote:



 According to distro watch:

 ...
 According to Linux Counter

 ...


What are distro watch and linux counter and who cares what their opt-in
stats gathering says?
-most Gentoo users I've ever talked to

I think if you drop the premise Gentoo is dying, how do we fix it? and
just focus on Here are some ideas on how we can improve Gentoo, you'll
get a better response here.  From what I see (IRC activity, new ebuild
activity, bugzilla activity, etc), our community seems stronger than ever.
 I think a lot of us are puzzled that you think Gentoo has stopped.

You have some great ideas but this is not a sinking ship scenario.

-Ben

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Jeroen Roovers

On Thu, 9 Jan 2014 19:26:24 +0400
Igor lanthrus...@gmail.com wrote:

  For various reasons many techs were not implemented and now Gentoo
  is in a 
  kind of stagnation.
 
  What do you mean by that in particular? 
 
 Gentoo stopped.

https://bugs.gentoo.org/show_bug.cgi?id=298754
https://bugs.gentoo.org/show_bug.cgi?id=439378
https://bugs.gentoo.org/show_bug.cgi?id=455908
https://bugs.gentoo.org/show_bug.cgi?id=495002

It looks like the only thing that is stagnating is your Gentoo Linux
install.


 jer

Re: [gentoo-dev] Portage QOS

Hello Ben,

Thursday, January 9, 2014, 7:49:28 PM, you wrote:

True, thanks for noting that.

 What are distro watch and linux counter and who cares what their opt-in 
 stats gathering says?

 -most Gentoo users I've ever talked to

 I think if you drop the premise Gentoo is dying, how do we fix
 it? and just focus on Here are some ideas on how we can improve
 Gentoo, you'll get a better response here.  From what I see (IRC
 activity, new ebuild activity, bugzilla activity, etc), our
 community seems stronger than ever.  I think a lot of us are puzzled
 that you think Gentoo has stopped.  
  


 You have some great ideas but this is not a sinking ship scenario.



-- 
Best regards,
 Igormailto:lanthrus...@gmail.com

Re: [gentoo-dev] Portage QOS

Hello Jeroen,

Thursday, January 9, 2014, 7:55:42 PM, you wrote:

I was expecting you a few hours earlier, Jeroen. I knew you
wouldn't resist a terrible temptation remembering the Python Bug
that I filed from the old kernel gentoo.

For your information this is a confirmed bug in Python right now and
it is going to be fixed in 3.5

http://bugs.python.org/issue20065

Jeroen, tell me how many users world wide do not prefer to upgrade Gentoo
on automated basis? There are important servers, and there are many
cases when after upgrade server stops. Do you remember that recent udev
change? And there are many similar cases. Imagine that your server
is running a reactor. So what would you prefer to keep it running the
reactor as it did flawlessly for 8 years or launch an upgrade taking
the risks to blast yourself?

Many be it's not only me, but somebody else who is thinking the same?
Are you sure that the majority of Gentoo users are indulged in
paranoid automated upgrade and then spending time fixing damage
that upgrade did?

Do you have a car? Why you don't change EVERY detail in your car on a new
version on daily basis automatically?

Why don't you change car as soon as a new version is released? Why not
changing the new mouse, new keyboard, new monitor, new supply daily as
soon as there is a new version?

Not to mention that you can change daily appearances.

I belive that each users has the right to decide how to use Gentoo.
I have my reasons not to upgrade some servers.

And you, being a support team has no right to mock on your users
weather they're right or not. You don't make users feel bad, you
help them. One mocked user, another and there is no users to mock
about.

And you need users to mock more than they need you.

  For various reasons many techs were not implemented and now Gentoo
  is in a 
  kind of stagnation.
 
  What do you mean by that in particular? 
 
 Gentoo stopped.

 https://bugs.gentoo.org/show_bug.cgi?id=298754
 https://bugs.gentoo.org/show_bug.cgi?id=439378
 https://bugs.gentoo.org/show_bug.cgi?id=455908
 https://bugs.gentoo.org/show_bug.cgi?id=495002

 It looks like the only thing that is stagnating is your Gentoo Linux
 install.





-- 
Best regards,
 Igormailto:lanthrus...@gmail.com

[gentoo-dev] Re: Portage QOS

2014-01-09 Thread Duncan

Igor posted on Thu, 09 Jan 2014 16:44:02 +0400 as excerpted:

 There is no data to tell what happens with Gentoo (to give that data is
 one of the goals of the project). We only have some formal esteems from
 unreliable sources.
 
 According to distro watch:
 
 In February 2012, Gentoo distro was in 19th place.
 In December 2012, Gentoo went to 22nd place.
 In December 2013, Gentoo is down to 32nd place

There was some discussion of this previously.  The conclusion was 
basically that gentooers don't tend to be the trend-watching type, nor, 
really, are we really interested in the trend-watching type, as that's 
not gentoo's base or target user. Instead, our users tend to be 
independent customizers that aren't so interested in what the majority 
thinks, but, rather find gentoo's general support for letting them make 
of their gentoo installation what they will a very good match for their 
needs.  If that's not the best match or if their needs change, the fact 
that gentoo takes more work than many distros because you have to 
actually configure and build it, tends to have them quickly off to some 
other distro that's a better fit for their less time/interest, more 
cookie-cutter needs.

In a way, then, gentoo in the Linux ecosystem is what Linux itself is in 
the more general OS ecosystem, and gentoo tends to get only the self-
selecting independent thinkers who value the ability to make their OS 
what they want while never-the-less automating much of the process (thus 
we aren't Linux from Scratch), in the same way that the same group, but 
to a somewhat lessor extent, tend to be Linux users.

And just as a significant subset of those Linux users and devs value 
their (software) freedom and independence enough to not be willing to 
sacrifice it just to have Linux more popular and maybe exceed MS, so a 
lot of Gentoo users and devs aren't willing to compromise on gentoo's 
ideals of highly customizable individuality just to see us rise in 
rankings such as distro-watch.  If it happens, great, but it won't 
greatly affect the way gentoo is developed, and if it doesn't happen, no 
big deal anyway, since that's not something we consider significant or 
important, particularly /because/ we recognize that sort of user isn't 
what gentoo's targeting in the first place.

 According to Linux Counter
 
 In January 2012, Gentoo distro had 5.32%
 In January 2012, Gentoo had 4.04%
 In November 2013, Gentoo had 4,21%

I guess one of those January 2012s is supposed to be something 
different...

Same thing here, really, tho the reason is a bit different.

I know *I* certainly haven't registered with linuxcounter, and don't 
expect I ever will, either.  I see it as useless at best, and yet another 
way to be tracked at worst.  (I /do/ deliberately keep my browser's user-
agent string set to Linux instead of setting it to say the latest MS 
Windows version, and deliberately kept 64-bit back when 32-bit was the 
norm for similar reasons, so sites that I visit and thus care about can 
count that, but I most certainly do NOT let the various third-party 
tracing sites do their thing, using tools such as firefox plugins 
noscript, request-policy and cookie permissions, as well as privoxy, to 
help me keep that information out of third-party-tracker's hands.)

Tho interestingly, that does show percentage stabilizing or even 
increasing a bit between the second and third samples.  What it means, 
however, I'm not going to attempt to guess.  For all I know it simply 
means a few gentooers don't object to being tracked as strongly as they 
once did, which is actually slightly disturbing to me, tho it's their 
life so they get to decide, not me.

 And from my experience of Gentoo forums, gentoo.wiki - I vote for Gentoo
 at least not gaining new users.

That would be a more interesting number, there.  But you don't provide 
stats for that one, and personal perception such as yours above for those 
constantly involved is notoriously inaccurate.  Someone who left for a 
couple years and came back tends to see changes much better, for the same 
reason you don't tend to notice changes in a friend as you grow old 
together, unless you're separated for a few years and then meet again.

I wonder what the forums stats counts are.  I know there's mailing list 
activity stats as I've seen them posted occasionally, but I'm not sure if 
there's anything like that for the forums...  That would give us some 
concrete numbers to work with.

 If in several years the number of users is not increased - we can tell
 about stagnation.

As I've personally argued about Linux, if popularity comes at the cost of 
loss of freedom, etc, it's not worth it.  There's worse things than 
seeing numbers stagnate, and losing our ideals in a likely futile pursuit 
of popularity (what's the chances of gentoo topping Red Hat even if we 
forsook all that makes gentoo gentoo and tried? that's not what we're 
good at or care about) is one of them.

 It's all not very

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread yac

On Thu, 9 Jan 2014 11:24:25 +0400
LTHR lanthrus...@gmail.com wrote:

 Hi All,
 
 What do you think about implementing this:
 
 http://forums.gentoo.org/viewtopic.php?p=7477494
 
 I've system design in my head and could write it down with the
 implementation details. Then may be we could all review it and get to
 something we all agree upon then I could try getting a team and
 implement it.
 
 Just a brief question - does anyone know how many ebuilds are
 assembled world wide each second?
 

Hi,

There are some ideas that may be worth pursuing and by bottom up
approach we (me and mainly naota) started turning gentwoo [1]_ [2]_ into
a package usage stats [3]_

.. [1] http://gentwoo.elisp.net/
.. [2] https://github.com/naota/gentwoo
.. [3] https://github.com/gentoo/GenTwoo-backend

---
Jan Matějka| Gentoo Developer
https://gentoo.org | Gentoo Linux
GPG: A33E F5BC A9F6 DAFD 2021  6FB6 3EBF D45B EEB6 CA8B


signature.asc
Description: PGP signature

Re: [gentoo-dev] Re: About pam herd status

2014-01-09 Thread Mike Frysinger

On Monday 09 December 2013 16:32:09 Markos Chandras wrote:
 On 12/09/2013 02:21 PM, Diego Elio Pettenò wrote:
  On Mon, Dec 9, 2013 at 2:20 PM, Pacho Ramos pa...@gentoo.org wrote:
  Hello
  
  Is pam team still active? I wonder about this as, recently, we have
  needed to go ahead and fix some bugs related, for example, with pambase
  and pam_ssh
  
  Thanks for the info :)
 
 So I guess it's time to call for maintainers or we should consider
 merging this herd with another one (base?) to avoid unattended bugs for
 a long time.

well, the sep herd was kind of by design ... i didn't want it cluttering up 
base-system@ and it is super convenient to abdicate all PAM decisions to a 
single herd.
-mike


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] Re: About pam herd status

2014-01-09 Thread Diego Elio Pettenò

On 9 January 2014 20:20, Mike Frysinger vap...@gentoo.org wrote:


 well, the sep herd was kind of by design ... i didn't want it cluttering up
 base-system@ and it is super convenient to abdicate all PAM decisions to a
 single herd.


Yeah the problem has been that the herd has been fundamentally a single
person, and when said single person asked for the rest of the users
of/providers for PAM to accentrate on a single package, they ignored/worked
around problems instead of reporting/discussing them.

Diego Elio Pettenò — Flameeyes
flamee...@flameeyes.eu — http://blog.flameeyes.eu/

Re: [gentoo-dev] Re: About pam herd status

2014-01-09 Thread Markos Chandras

On 01/09/2014 08:20 PM, Mike Frysinger wrote:
 On Monday 09 December 2013 16:32:09 Markos Chandras wrote:
 On 12/09/2013 02:21 PM, Diego Elio Pettenò wrote:
 On Mon, Dec 9, 2013 at 2:20 PM, Pacho Ramos pa...@gentoo.org wrote:
 Hello

 Is pam team still active? I wonder about this as, recently, we have
 needed to go ahead and fix some bugs related, for example, with pambase
 and pam_ssh

 Thanks for the info :)

 So I guess it's time to call for maintainers or we should consider
 merging this herd with another one (base?) to avoid unattended bugs for
 a long time.
 
 well, the sep herd was kind of by design ... i didn't want it cluttering up 
 base-system@ and it is super convenient to abdicate all PAM decisions to a 
 single herd.
 -mike
 
I understand that but if nobody is tracking pam@ then what's the point?

-- 
Regards,
Markos Chandras

Re: [gentoo-dev] Re: Portage QOS

Hello Duncan,

Thursday, January 9, 2014, 9:59:50 PM, you wrote:

Thank you for the reply. I started to comment first... but it was more
philosophy a mature and grown up, experienced man and I don't think
I have right to comment it.

Statistically if you have more users the probability of the system
survival of any architecture, philosophy or type is higher. People
learn, they're not fixed and if they at the beginning do not share
the philosophy of the system but they can use it - they may like it,
understand it and follow it and support later. Many people I asked
are not minding to help Gentoo getting better by turning on
feedback. If you remember - feedback worked well for Perl once and
many used it and Perl is very traditional.

It's like a chess game. You have the system in it's prime. There is
already one fork from Gentoo. There will be more. It's inevitable. You
have to understand that not all the developers share the same
philosophy - and it OK.
And they may fork Gentoo with time and pull half of the team to their
side.

When there is a competition between systems with equal philosophy the
only thing that stands between who is going to live and who is going
to die is the number of users.
The fight will focus not around philosophy or system but around gaining
user support. The competitor can build a better, more friendly system
sharing basically the same design and he will win it over.

To keep in power it's in your deepest interest to close the open gates that
invite competition while the power is in your hands. This is a failure
many grown up companies made they belive they're forever and gods. I could
share with you privately with several examples that prove that concept
wrong.

Your competitors will build basically the same system targeting the
same philosophy but more user oriented, friendly. User oriented - means
each user opinion matters.

There might be millions of users but each is treated like he is the only one.


PortageQOS is small step, it's not everything or main part of the
system, it's a just small contribution. But it will close the door and
you'll have another peaceful 8 years to rule.

What we need is a vote YES or NO. If you against it - vote NO. It's
perfectly normal, if there would be no NO there would be no need voting.


 Actually, in that regard it's very possible that gentoo's long planned
 and worked toward cvs-to-git conversion will help finally bust that 
 barrier for gentoo as well.  Time will tell I guess, but that's one more
 reason to try to help make it happen.




-- 
Best regards,
 Igormailto:lanthrus...@gmail.com

[gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Magnus Granberg

Hi

Some time ago we discussed that we should enable stack smashing 
(-fstack-protector) by default.  So we opened a bug to track this [1].  
The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, ppc, 
ppc64 and arm will be affected by this change. 

You can turn off ssp by using the nossp USE flag or by adding 
-fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
patch as Debian/Ubuntu but with some Gentoo fixes.

The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and 
ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will 
make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn 
it on or off with hardened_gcc_works() that will make some sanity checks.

/Magnus
2013-12-31  Magnus Granberg  zo...@gentoo.org

	# 484714
	We Add -fstack-protector as default

--- a/eclass/toolchain.eclass	2013-12-30 21:21:05.431832881 +0100
+++ b/eclass/toolchain.eclass	2013-12-31 11:34:00.720993536 +0100
@@ -473,7 +473,9 @@ toolchain_src_prepare() {
 	do_gcc_PIE_patches
 	epatch_user
 
-	use hardened  make_gcc_hard
+	if ( tc_version_is_at_least 4.8 || use hardened )  ! use vanilla ; then
+		make_gcc_hard
+	fi
 
 	# install the libstdc++ python into the right location
 	# http://gcc.gnu.org/PR51368
@@ -606,6 +608,12 @@ do_gcc_PIE_patches() {
 		epatch ${WORKDIR}/piepatch/def
 	fi
 
+	BRANDING_GCC_PKGVERSION=${BRANDING_GCC_PKGVERSION}, pie-${PIE_VER}
+}
+
+# configure to build with the hardened GCC specs as the default
+make_gcc_hard() {
+	
 	# we want to be able to control the pie patch logic via something other
 	# than ALL_CFLAGS...
 	sed -e '/^ALL_CFLAGS/iHARD_CFLAGS = ' \
@@ -618,38 +626,38 @@ do_gcc_PIE_patches() {
 -i ${S}/gcc/Makefile.in
 	fi
 
-	BRANDING_GCC_PKGVERSION=${BRANDING_GCC_PKGVERSION}, pie-${PIE_VER}
-}
-
-# configure to build with the hardened GCC specs as the default
-make_gcc_hard() {
-	# defaults to enable for all hardened toolchains
-	local gcc_hard_flags=-DEFAULT_RELRO -DEFAULT_BIND_NOW
-
-	if hardened_gcc_works ; then
-		einfo Updating gcc to use automatic PIE + SSP building ...
-		gcc_hard_flags+= -DEFAULT_PIE_SSP
-	elif hardened_gcc_works pie ; then
-		einfo Updating gcc to use automatic PIE building ...
-		ewarn SSP has not been enabled by default
-		gcc_hard_flags+= -DEFAULT_PIE
-	elif hardened_gcc_works ssp ; then
-		einfo Updating gcc to use automatic SSP building ...
-		ewarn PIE has not been enabled by default
-		gcc_hard_flags+= -DEFAULT_SSP
+	# defaults to enable for all toolchains
+	local gcc_hard_flags=
+	if use hardened ; then
+		if hardened_gcc_works ; then
+			einfo Updating gcc to use automatic PIE + SSP building ...
+			gcc_hard_flags+= -DEFAULT_PIE_SSP
+		elif hardened_gcc_works pie ; then
+			einfo Updating gcc to use automatic PIE building ...
+			ewarn SSP has not been enabled by default
+			gcc_hard_flags+= -DEFAULT_PIE
+		elif hardened_gcc_works ssp ; then
+			einfo Updating gcc to use automatic SSP building ...
+			ewarn PIE has not been enabled by default
+			gcc_hard_flags+= -DEFAULT_SSP
+		else
+			# do nothing if hardened is't supported, but don't die either
+			ewarn hardened is not supported for this arch in this gcc version
+			return 0
+		fi
+		# rebrand to make bug reports easier
+		BRANDING_GCC_PKGVERSION=${BRANDING_GCC_PKGVERSION/Gentoo/Gentoo Hardened}
 	else
-		# do nothing if hardened isnt supported, but dont die either
-		ewarn hardened is not supported for this arch in this gcc version
-		ebeep
-		return 0
+		if hardened_gcc_works ssp ; then
+			einfo Updating gcc to use automatic SSP building ...
+			gcc_hard_flags+= -DEFAULT_SSP
+		fi
 	fi
 
 	sed -i \
 		-e /^HARD_CFLAGS = /s|=|= ${gcc_hard_flags} | \
 		${S}/gcc/Makefile.in || die
 
-	# rebrand to make bug reports easier
-	BRANDING_GCC_PKGVERSION=${BRANDING_GCC_PKGVERSION/Gentoo/Gentoo Hardened}
 }
 
 # This is a historical wart.  The original Gentoo/amd64 port used:

Re: [gentoo-dev] Re: Portage QOS

2014-01-09 Thread Chris Reffett

On 01/09/2014 03:42 PM, Igor wrote:
 Hello Duncan,
 
 Thursday, January 9, 2014, 9:59:50 PM, you wrote:
 
 Thank you for the reply. I started to comment first... but it was more
 philosophy a mature and grown up, experienced man and I don't think
 I have right to comment it.
 
 Statistically if you have more users the probability of the system
 survival of any architecture, philosophy or type is higher. People
 learn, they're not fixed and if they at the beginning do not share
 the philosophy of the system but they can use it - they may like it,
 understand it and follow it and support later. Many people I asked
 are not minding to help Gentoo getting better by turning on
 feedback. If you remember - feedback worked well for Perl once and
 many used it and Perl is very traditional.
 
 It's like a chess game. You have the system in it's prime. There is
 already one fork from Gentoo. There will be more. It's inevitable. You
 have to understand that not all the developers share the same
 philosophy - and it OK.
 And they may fork Gentoo with time and pull half of the team to their
 side.
 
 When there is a competition between systems with equal philosophy the
 only thing that stands between who is going to live and who is going
 to die is the number of users.
 The fight will focus not around philosophy or system but around gaining
 user support. The competitor can build a better, more friendly system
 sharing basically the same design and he will win it over.
 
 To keep in power it's in your deepest interest to close the open gates that
 invite competition while the power is in your hands. This is a failure
 many grown up companies made they belive they're forever and gods. I could
 share with you privately with several examples that prove that concept
 wrong.
 
 Your competitors will build basically the same system targeting the
 same philosophy but more user oriented, friendly. User oriented - means
 each user opinion matters.
 
 There might be millions of users but each is treated like he is the only one.
 
 
 PortageQOS is small step, it's not everything or main part of the
 system, it's a just small contribution. But it will close the door and
 you'll have another peaceful 8 years to rule.
 
Right here is the big problem: you're not looking at this from the
perspective of the average Gentoo developer. We don't care about market
share. We don't care whether we're on top for another few years. There
are several forks of Gentoo. I doubt most devs care about them. I
personally know that we're not going to compete with Debian, which has a
huge contributor, or Ubuntu or Red Hat, which have whole companies
behind them. You're selling this as if you're selling to a company which
wants to be on the top of the market and beating out competitors, and
that's not what we are. We are a source-based distro that requires some
effort from users, and people want that or they don't want it.
 What we need is a vote YES or NO. If you against it - vote NO. It's
 perfectly normal, if there would be no NO there would be no need voting.
 
 
 Actually, in that regard it's very possible that gentoo's long planned
 and worked toward cvs-to-git conversion will help finally bust that 
 barrier for gentoo as well.  Time will tell I guess, but that's one more
 reason to try to help make it happen.
 
 
 
 
Chris Reffett

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 03:58 PM, Magnus Granberg wrote:
 Hi
 
 Some time ago we discussed that we should enable stack smashing 
 (-fstack-protector) by default.  So we opened a bug to track this [1].  
 The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, ppc, 
 ppc64 and arm will be affected by this change. 
 
 You can turn off ssp by using the nossp USE flag or by adding 
 -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
 patch as Debian/Ubuntu but with some Gentoo fixes.

Please avoid noblah use flags.

http://devmanual.gentoo.org/general-concepts/use-flags/

ssp flag that defaults to on is fine.

Thanks,
Zero

 
 The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and 
 ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will 
 make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn 
 it on or off with hardened_gcc_works() that will make some sanity checks.
 
 /Magnus
 

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSzxCAAAoJEKXdFCfdEflKohoP/iDVryOAlunTvMtrhHYlZAXn
LTPbJRNMQ9bJB8bAd9ECRJ8Q92pAyQt+NyNgUFLtatI+UuiZM6t+dz4K0LcmMQka
n5i6ILdJ14V0BRLGhRz+Xa0ZjpnYRJjCAWrFENTDY6wHc5ni0hSb32xBg84L6j/3
HzNFIWnQOp9dw3hH5OPNQrPXndPVYZMjYTfIADSGx8/4dL9A0GWPY6AXOz08NwuC
oC+zkQi2xSXCb7eHTfkKcIW0TIOF7mFp8Z5LsdW5dm/8nCCLDVH7yxmxVyymyMpb
RnraqCghiv5WOJVsysgivtNPzBwR3ATrNk4dl2qigZSGpJcDEgF2AtSv+YAVb1Ux
wcGOJ5ZJizkMBdAo2pqUBeekXPT/LLWg1EtRvhFI3OLInhbwaHaF9kMWEmhwq2d9
sX6ZfoCmtvn4ZtG3fL++RqyK5QevKOXFCtN2pK9DVMjjgHgp6OtnmVXVCMuZTztI
uqI3XGVvDc0dNIwgxI6KIEfV4/S05Q599hLI49YJVniPknp+sUnsYIdQ+mKztwDf
XON5Fq/Yzt07G1FqZMutEpVMjeTjVckpcLgbFlWz+lO6/xIrqJUgnLUNTXTQf34r
n54Rw+WWIGUWAQqwK3bDh9N2etLzG8p8TqhzEXSCMKXP0sjbX1zzJiYl1roMMpu3
nTFjVJbwpgoaw8OR6FW4
=tSUd
-END PGP SIGNATURE-

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Pacho Ramos

El jue, 09-01-2014 a las 21:58 +0100, Magnus Granberg escribió:
 Hi
 
 Some time ago we discussed that we should enable stack smashing 
 (-fstack-protector) by default.  So we opened a bug to track this [1].  
 The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, ppc, 
 ppc64 and arm will be affected by this change. 
 
 You can turn off ssp by using the nossp USE flag or by adding 
 -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
 patch as Debian/Ubuntu but with some Gentoo fixes.
 
 The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and 
 ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will 
 make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn 
 it on or off with hardened_gcc_works() that will make some sanity checks.
 
 /Magnus

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag? 

Thanks a lot for the info :)

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.


On 01/09/2014 04:57 PM, Pacho Ramos wrote:

El jue, 09-01-2014 a las 21:58 +0100, Magnus Granberg escribió:

Hi

Some time ago we discussed that we should enable stack smashing
(-fstack-protector) by default.  So we opened a bug to track this [1].
The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, ppc,
ppc64 and arm will be affected by this change.

You can turn off ssp by using the nossp USE flag or by adding
-fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same
patch as Debian/Ubuntu but with some Gentoo fixes.

The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and
ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will
make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn
it on or off with hardened_gcc_works() that will make some sanity checks.

/Magnus

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag?

Thanks a lot for the info :)




There are some cases where ssp could break things.  I know of once case 
right now, but its somewhat exotic.  Also, sometimes we *want* to break 
things for testing.  I'm thinking here of instance where we want to test 
a pax hardened kernel to see if it catches abuses of memory which would 
otherwise be caught by executables emitted from a hardened toolchain.  
Take a look at the app-admin/paxtest suite.



--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Magnus Granberg

torsdag 09 januari 2014 22.57.09 skrev  Pacho Ramos:
 El jue, 09-01-2014 a las 21:58 +0100, Magnus Granberg escribió:
  Hi
  
  Some time ago we discussed that we should enable stack smashing
  (-fstack-protector) by default.  So we opened a bug to track this [1].
  The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips,
  ppc, ppc64 and arm will be affected by this change.
  
  You can turn off ssp by using the nossp USE flag or by adding
  -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same
  patch as Debian/Ubuntu but with some Gentoo fixes.
  
  The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and
  ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will
  make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn
  it on or off with hardened_gcc_works() that will make some sanity checks.
  
  /Magnus
 
 What are the advantages of disabling SSP to deserve that special
 handling via USE flag or easily disabling it appending the flag?
 
 Thanks a lot for the info :)

If you want Gcc not to build stuff with ssp as default you turn on the nossp 
flag and rebuild Gcc.

/Magnus

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Pacho Ramos

El jue, 09-01-2014 a las 17:06 -0500, Anthony G. Basile escribió:
 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
  El jue, 09-01-2014 a las 21:58 +0100, Magnus Granberg escribió:
  Hi
 
  Some time ago we discussed that we should enable stack smashing
  (-fstack-protector) by default.  So we opened a bug to track this [1].
  The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, 
  ppc,
  ppc64 and arm will be affected by this change.
 
  You can turn off ssp by using the nossp USE flag or by adding
  -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same
  patch as Debian/Ubuntu but with some Gentoo fixes.
 
  The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and
  ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will
  make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn
  it on or off with hardened_gcc_works() that will make some sanity checks.
 
  /Magnus
  What are the advantages of disabling SSP to deserve that special
  handling via USE flag or easily disabling it appending the flag?
 
  Thanks a lot for the info :)
 
 
 
 There are some cases where ssp could break things.  I know of once case 
 right now, but its somewhat exotic.  Also, sometimes we *want* to break 
 things for testing.  I'm thinking here of instance where we want to test 
 a pax hardened kernel to see if it catches abuses of memory which would 
 otherwise be caught by executables emitted from a hardened toolchain.  
 Take a look at the app-admin/paxtest suite.
 
 

OK, thanks a lot, I was wondering if I would need to disable SSP on some
of the machines I maintain for some reason. Looks like keeping it
enabled is preferred instead ;)

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread William Hubbs

On Thu, Jan 09, 2014 at 04:11:28PM -0500, Rick Zero_Chaos Farina wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 01/09/2014 03:58 PM, Magnus Granberg wrote:
  Hi
  
  Some time ago we discussed that we should enable stack smashing 
  (-fstack-protector) by default.  So we opened a bug to track this [1].  
  The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, 
  ppc, 
  ppc64 and arm will be affected by this change. 
  
  You can turn off ssp by using the nossp USE flag or by adding 
  -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
  patch as Debian/Ubuntu but with some Gentoo fixes.
 
 Please avoid noblah use flags.
 
 http://devmanual.gentoo.org/general-concepts/use-flags/
 
 ssp flag that defaults to on is fine.

Agreed; please do not introduce this with a nossp use flag.

Thanks,

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Michał Górny

Dnia 2014-01-09, o godz. 17:06:52
Anthony G. Basile bluen...@gentoo.org napisał(a):

 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
  What are the advantages of disabling SSP to deserve that special
  handling via USE flag or easily disabling it appending the flag?
 
 There are some cases where ssp could break things.  I know of once case 
 right now, but its somewhat exotic.  Also, sometimes we *want* to break 
 things for testing.  I'm thinking here of instance where we want to test 
 a pax hardened kernel to see if it catches abuses of memory which would 
 otherwise be caught by executables emitted from a hardened toolchain.  
 Take a look at the app-admin/paxtest suite.

Just to be clear, are we talking about potential system-wide breakage
or single, specific packages being broken by SSP? In other words, are
there cases when people will really want to disable SSP completely?

Unless I'm misunderstanding something, your examples sound like you
just want -fno-stack-protector per-package. I don't really think you
actually want to rebuild whole gcc just to do some testing on a single
package...

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 05:21 PM, Michał Górny wrote:
 Dnia 2014-01-09, o godz. 17:06:52
 Anthony G. Basile bluen...@gentoo.org napisał(a):
 
 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
 What are the advantages of disabling SSP to deserve that special
 handling via USE flag or easily disabling it appending the flag?

 There are some cases where ssp could break things.  I know of once case 
 right now, but its somewhat exotic.  Also, sometimes we *want* to break 
 things for testing.  I'm thinking here of instance where we want to test 
 a pax hardened kernel to see if it catches abuses of memory which would 
 otherwise be caught by executables emitted from a hardened toolchain.  
 Take a look at the app-admin/paxtest suite.
 
 Just to be clear, are we talking about potential system-wide breakage
 or single, specific packages being broken by SSP? In other words, are
 there cases when people will really want to disable SSP completely?
 
 Unless I'm misunderstanding something, your examples sound like you
 just want -fno-stack-protector per-package. I don't really think you
 actually want to rebuild whole gcc just to do some testing on a single
 package...
 
Or just as easily set -fno-stack-protector in CFLAGS in make.conf.

I never felt manipulating cflags with use flags was a great idea, but in
this case is does feel extra pointless.

Personally I don't feel this is needed, and the added benefit of
clearing up a bogus noblah use flag makes me smile.

Zorry, do we really need this flag?

- -Zero
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSzyLGAAoJEKXdFCfdEflKo44QAJ4wYfgHdLYffTPaiXZe2ZJn
3jPxaZX47m7BjgdtePZZncClby5h84cG+Jchb0pn/a6K6TknpiFXLQzArsJbMH0N
th7cuuuS4iFQMw2xq8hNAvGAdMF4R0+/OSpBkzlskakcCVRHgV+KCz9llimny4hB
RbTXK9Irva0bwYb5IkmTq+/JVqHjB5DyXUYMu32vdvgz8uxTPXXHHO5HtPlkLeiq
YsumFhnHFb5d+yPvPzZ3YSMJyuHHtBeZFCOJoirtxL08+yr5dZhgppEbqkMJcHIG
r1xKxPqFSmccHSJ8mCZ+l3mvrSL7Akd7D9c7Rk6hZ8RpMQnxCTNb3/Twq6oqAqKm
JfcU1B6rKIDz6eZOtMmVMyVcfnlo7MHO/resmFCS/BYN5AyqyfHgn+I4OU4IVCvQ
2jaZOwxeXGePgkwK37ebK/274N63lSkQAbaB0K43oqvsmtNuq+qZQQEm7jkY+0Vu
SYKc3y4hXeLvexxteiR559fB7zJ1zPIsvvOWrqVP7euYezPMI7cjamz+7VHJYyH4
3RdGpro4Qg7OOTr42naBsWBW20nRTecWpm6kg0jyJo9eSD5YPzLq5r9ITcv7c6mB
/6OxgqbVubzxpo9+kpY/11rEgQx5li4wKbLVsBY3n/f+tDCi1GNTu2k6ottdgrt2
XgsAKT4j/dUq8dhh80P7
=9pZW
-END PGP SIGNATURE-

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.


On 01/09/2014 05:21 PM, Michał Górny wrote:

Dnia 2014-01-09, o godz. 17:06:52
Anthony G. Basile bluen...@gentoo.org napisał(a):


On 01/09/2014 04:57 PM, Pacho Ramos wrote:

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag?

There are some cases where ssp could break things.  I know of once case
right now, but its somewhat exotic.  Also, sometimes we *want* to break
things for testing.  I'm thinking here of instance where we want to test
a pax hardened kernel to see if it catches abuses of memory which would
otherwise be caught by executables emitted from a hardened toolchain.
Take a look at the app-admin/paxtest suite.

Just to be clear, are we talking about potential system-wide breakage
or single, specific packages being broken by SSP? In other words, are
there cases when people will really want to disable SSP completely?

Unless I'm misunderstanding something, your examples sound like you
just want -fno-stack-protector per-package. I don't really think you
actually want to rebuild whole gcc just to do some testing on a single
package...

Correct, you'd only want to turn off ssp per package and then only in 
rare cases.  You should never have to rebuild gcc for this.  With ssp on 
by default, gcc specs would add -fstack-protector to all builds.  If you 
don't want a package build with ssp, then just do 
CFLAGS=-fno-stack-protector and you're building without ssp.


--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.


On 01/09/2014 05:29 PM, Rick Zero_Chaos Farina wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 05:21 PM, Michał Górny wrote:

Dnia 2014-01-09, o godz. 17:06:52
Anthony G. Basile bluen...@gentoo.org napisał(a):


On 01/09/2014 04:57 PM, Pacho Ramos wrote:

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag?

There are some cases where ssp could break things.  I know of once case
right now, but its somewhat exotic.  Also, sometimes we *want* to break
things for testing.  I'm thinking here of instance where we want to test
a pax hardened kernel to see if it catches abuses of memory which would
otherwise be caught by executables emitted from a hardened toolchain.
Take a look at the app-admin/paxtest suite.

Just to be clear, are we talking about potential system-wide breakage
or single, specific packages being broken by SSP? In other words, are
there cases when people will really want to disable SSP completely?

Unless I'm misunderstanding something, your examples sound like you
just want -fno-stack-protector per-package. I don't really think you
actually want to rebuild whole gcc just to do some testing on a single
package...


Or just as easily set -fno-stack-protector in CFLAGS in make.conf.

I never felt manipulating cflags with use flags was a great idea, but in
this case is does feel extra pointless.

Personally I don't feel this is needed, and the added benefit of
clearing up a bogus noblah use flag makes me smile.

Zorry, do we really need this flag?




toolchain.eclass currently uses nossp as well as nopie.  You'd have to 
rework that to get rid of the flag.


--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.


On 01/09/2014 05:29 PM, Rick Zero_Chaos Farina wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 05:21 PM, Michał Górny wrote:

Dnia 2014-01-09, o godz. 17:06:52
Anthony G. Basile bluen...@gentoo.org napisał(a):


On 01/09/2014 04:57 PM, Pacho Ramos wrote:

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag?

There are some cases where ssp could break things.  I know of once case
right now, but its somewhat exotic.  Also, sometimes we *want* to break
things for testing.  I'm thinking here of instance where we want to test
a pax hardened kernel to see if it catches abuses of memory which would
otherwise be caught by executables emitted from a hardened toolchain.
Take a look at the app-admin/paxtest suite.

Just to be clear, are we talking about potential system-wide breakage
or single, specific packages being broken by SSP? In other words, are
there cases when people will really want to disable SSP completely?

Unless I'm misunderstanding something, your examples sound like you
just want -fno-stack-protector per-package. I don't really think you
actually want to rebuild whole gcc just to do some testing on a single
package...


Or just as easily set -fno-stack-protector in CFLAGS in make.conf.



I just reread this and we'd better be clear here.  With ssp on by 
default in gcc, if you put CFLAGS=...  -fno-stack-protector in 
make.conf you will build your *entire* system with no ssp.  You probably 
don't want this.  You'll probably only want ssp off on a per package 
basis, in which case, add a line to package.env and set the CFLAGS for 
only that package.


--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 06:01 PM, Anthony G. Basile wrote:
 On 01/09/2014 05:21 PM, Michał Górny wrote:
 Dnia 2014-01-09, o godz. 17:06:52
 Anthony G. Basile bluen...@gentoo.org napisał(a):

 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
 What are the advantages of disabling SSP to deserve that special
 handling via USE flag or easily disabling it appending the flag?
 There are some cases where ssp could break things.  I know of once case
 right now, but its somewhat exotic.  Also, sometimes we *want* to break
 things for testing.  I'm thinking here of instance where we want to test
 a pax hardened kernel to see if it catches abuses of memory which would
 otherwise be caught by executables emitted from a hardened toolchain.
 Take a look at the app-admin/paxtest suite.
 Just to be clear, are we talking about potential system-wide breakage
 or single, specific packages being broken by SSP? In other words, are
 there cases when people will really want to disable SSP completely?

 Unless I'm misunderstanding something, your examples sound like you
 just want -fno-stack-protector per-package. I don't really think you
 actually want to rebuild whole gcc just to do some testing on a single
 package...

 Correct, you'd only want to turn off ssp per package and then only in
 rare cases.  You should never have to rebuild gcc for this.  With ssp on
 by default, gcc specs would add -fstack-protector to all builds.  If you
 don't want a package build with ssp, then just do
 CFLAGS=-fno-stack-protector and you're building without ssp.
 
This reads very much like the nossp use flag is useless.

Not that Zorry needs to fix that (preexisting and all that) but it
sounds to me like it's safe to remove these types of use flags from
toolchain.

I'm really interested in dirtyepic's opinion though... sir?

Thanks,
Zero
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSzy0dAAoJEKXdFCfdEflKZJEP/3P/Gq3sD6aB9XDcsLxUAVqC
vg10PuwhmNpJK6HiYO2F/C5TNv3J+hpkiYPDMgjChOTw+JvqGCeIYYKvKuumtIXV
fjnHDW9IRD8BGHlNFF9xx3sGV9VMPYDNICkK3oeNQJPlZOVSbnVEWsaTju/CEA7e
tMkeA93ULed9pSzSZ3OBAIwLH906Kh8hO+o/gcJDyBa9/tJrXKfS+jtd6zTMbVtO
8ruLjRUDTsYwt61uMFhV7R/eWlSagGIFDGbxop0JyhTZaB+zxvbm8wzmZck4Tc2J
HFO4A289zFBVZESaDA4SHAYJHQTSMND1fzAB8X4sPEwNebmLwOinneuA7XYVRsHW
svu/I3tUPjNTKimTSmjMySi7f+3QDYLIxQ8UY0PUCPKjdlNZMQruqCR52lTsjy8F
n0EpLMqodD61B+aCkkBpdrt1sx/BJ4AISq8R51yiJecujPoSk1oj5gG1aFOPK/mG
BIQqLL1c6TvbB4ECLVMh6YAfxRKcyCT8tlMZqu2rTRqtxQ+YlUnxwvIQV7ivQ5sL
M8eC/HjVjd0In9v5GVxePa3NFfwwuswnFipi2mivniajmZYi8M8avSVLpv54Kvi0
cAysdf/FP4WA+iVCd5J+MKGygKKSmbyYZ9IHyE4yCyCNK+0+ZZcFm9YNy9nx8rAJ
4ctTVxoCTtA+B9p3MBnL
=6a0w
-END PGP SIGNATURE-

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 09 Jan 2014 16:11:28 -0500
Rick \Zero_Chaos\ Farina zeroch...@gentoo.org wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 01/09/2014 03:58 PM, Magnus Granberg wrote:
  Hi
  
  Some time ago we discussed that we should enable stack smashing 
  (-fstack-protector) by default.  So we opened a bug to track this [1].  
  The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips,
  ppc, ppc64 and arm will be affected by this change. 
  
  You can turn off ssp by using the nossp USE flag or by adding 
  -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
  patch as Debian/Ubuntu but with some Gentoo fixes.
 
 Please avoid noblah use flags.
 
 http://devmanual.gentoo.org/general-concepts/use-flags/
 
 ssp flag that defaults to on is fine.

This flag already exists and has always worked this way.  We don't have USE
defaults yet.


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 06:09 PM, Anthony G. Basile wrote:
 On 01/09/2014 05:29 PM, Rick Zero_Chaos Farina wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 01/09/2014 05:21 PM, Michał Górny wrote:
 Dnia 2014-01-09, o godz. 17:06:52
 Anthony G. Basile bluen...@gentoo.org napisał(a):

 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
 What are the advantages of disabling SSP to deserve that special
 handling via USE flag or easily disabling it appending the flag?
 There are some cases where ssp could break things.  I know of once case
 right now, but its somewhat exotic.  Also, sometimes we *want* to break
 things for testing.  I'm thinking here of instance where we want to
 test
 a pax hardened kernel to see if it catches abuses of memory which would
 otherwise be caught by executables emitted from a hardened toolchain.
 Take a look at the app-admin/paxtest suite.
 Just to be clear, are we talking about potential system-wide breakage
 or single, specific packages being broken by SSP? In other words, are
 there cases when people will really want to disable SSP completely?

 Unless I'm misunderstanding something, your examples sound like you
 just want -fno-stack-protector per-package. I don't really think you
 actually want to rebuild whole gcc just to do some testing on a single
 package...

 Or just as easily set -fno-stack-protector in CFLAGS in make.conf.

 
 I just reread this and we'd better be clear here.  With ssp on by
 default in gcc, if you put CFLAGS=...  -fno-stack-protector in
 make.conf you will build your *entire* system with no ssp.  You probably
 don't want this.  You'll probably only want ssp off on a per package
 basis, in which case, add a line to package.env and set the CFLAGS for
 only that package.
 
Of course this is EXACTLY the same as building gcc[nossp] which is what
we are discussing. So afaict you and I are in total agreement on all fronts.

- -Zero
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSzy6AAAoJEKXdFCfdEflKOY0P/2dfvjVAFTq9NyZqMgJe0j1/
sENGtTCAAxKWh3eoqPywDJpEarPYoIsctMUGbuM2Dx6kC1zv20klXiT9Oec5j8aG
qnAogeCubAQD/AjDLI5VjDU5dAH7xUEEQKWPEEdjqfV1xWstW91f+tfPg2JkxpMS
zeQtSAIhJJMRdcFXmmWIvbh8zAUczdxsEcdGBHSt97utbMnbJMOE1eGEWGqAfzWm
vFYLnA8R/TZO//wkbkqNTAQjL3JV8DKScaqVyFxh5wQhTCLMN4QFVqnlSJGDiZPS
bddylShRtMXXsqPmFmLIsFf9tY7N03+2U8Ex3l1ToEpBATK6kkwBtuVCv0tOPvp8
EYOOXjmHZSmsG37SUFMgZpsAfNCf6H030G1i9NEC2zOnW5i9vHWmL1rAVpVYGdu2
N3rW2QYPEQzIBjNOojsXp515okIzPt8biXcWGT1R+te2BUoEeNwLNco9zCJecL1H
YZNSmmA0fwc/vgvKOh1kfV4VAFwmM/cHAlI7UPG9ypM6Fo/3dn7zZgUaXdQU2KeL
g+UNaFDj2p8ob+2vIc5N0lNwSNgY/vms2DehXRAV52vwogxNBgTftJZwwQv+j25u
g1JWGf/MOXbh7mfDDK5Xr10fHEui6hpeSofC3BZC8pQ6k1duB1rKituWhBzBJBPF
w8AeXL74ZvsUwwUxwi4A
=AtZz
-END PGP SIGNATURE-

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 09 Jan 2014 17:29:26 -0500
Rick \Zero_Chaos\ Farina zeroch...@gentoo.org wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 01/09/2014 05:21 PM, Michał Górny wrote:
  Dnia 2014-01-09, o godz. 17:06:52
  Anthony G. Basile bluen...@gentoo.org napisał(a):
  
  On 01/09/2014 04:57 PM, Pacho Ramos wrote:
  What are the advantages of disabling SSP to deserve that special
  handling via USE flag or easily disabling it appending the flag?
 
  There are some cases where ssp could break things.  I know of once case 
  right now, but its somewhat exotic.  Also, sometimes we *want* to break 
  things for testing.  I'm thinking here of instance where we want to test 
  a pax hardened kernel to see if it catches abuses of memory which would 
  otherwise be caught by executables emitted from a hardened toolchain.  
  Take a look at the app-admin/paxtest suite.
  
  Just to be clear, are we talking about potential system-wide breakage
  or single, specific packages being broken by SSP? In other words, are
  there cases when people will really want to disable SSP completely?
  
  Unless I'm misunderstanding something, your examples sound like you
  just want -fno-stack-protector per-package. I don't really think you
  actually want to rebuild whole gcc just to do some testing on a single
  package...
  
 Or just as easily set -fno-stack-protector in CFLAGS in make.conf.
 
 I never felt manipulating cflags with use flags was a great idea, but in
 this case is does feel extra pointless.
 
 Personally I don't feel this is needed, and the added benefit of
 clearing up a bogus noblah use flag makes me smile.
 
 Zorry, do we really need this flag?

Yes, we do.  I want a way to disable it at a toolchain level.


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.


On 01/09/2014 06:13 PM, Rick Zero_Chaos Farina wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 06:01 PM, Anthony G. Basile wrote:

On 01/09/2014 05:21 PM, Michał Górny wrote:

Dnia 2014-01-09, o godz. 17:06:52
Anthony G. Basile bluen...@gentoo.org napisał(a):


On 01/09/2014 04:57 PM, Pacho Ramos wrote:

What are the advantages of disabling SSP to deserve that special
handling via USE flag or easily disabling it appending the flag?

There are some cases where ssp could break things.  I know of once case
right now, but its somewhat exotic.  Also, sometimes we *want* to break
things for testing.  I'm thinking here of instance where we want to test
a pax hardened kernel to see if it catches abuses of memory which would
otherwise be caught by executables emitted from a hardened toolchain.
Take a look at the app-admin/paxtest suite.

Just to be clear, are we talking about potential system-wide breakage
or single, specific packages being broken by SSP? In other words, are
there cases when people will really want to disable SSP completely?

Unless I'm misunderstanding something, your examples sound like you
just want -fno-stack-protector per-package. I don't really think you
actually want to rebuild whole gcc just to do some testing on a single
package...


Correct, you'd only want to turn off ssp per package and then only in
rare cases.  You should never have to rebuild gcc for this.  With ssp on
by default, gcc specs would add -fstack-protector to all builds.  If you
don't want a package build with ssp, then just do
CFLAGS=-fno-stack-protector and you're building without ssp.


This reads very much like the nossp use flag is useless.

I was not referring to the nossp flag.  I was simply answering Michał's 
concern about ssp and system wide breakage.  My point is that ssp on by 
default will NOT lead to system wide breakage, only per package and then 
only very very rarely.


--
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Andreas K. Huettel

Am Freitag, 10. Januar 2014, 00:26:03 schrieb Ryan Hill:

  Please avoid noblah use flags.
  
  http://devmanual.gentoo.org/general-concepts/use-flags/
  
  ssp flag that defaults to on is fine.
 
 This flag already exists and has always worked this way. 

already exists and has always worked this way is not really a good argument. 
The rest of the tree sticks to guidelines, so why not here?

 We don't have USE defaults yet.

Now if you had said we can't use USE defaults yet since current ebuilds are 
still at EAPI=0... that would have been slightly more informative. :)

(Yes I've seen that there is work going on, and I think that's good. Being 
careful when modernizing makes sense here of course.)


-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfri...@gentoo.org
http://www.akhuettel.de/



signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread William Hubbs

On Fri, Jan 10, 2014 at 12:30:04AM +0100, Andreas K. Huettel wrote:
 Am Freitag, 10. Januar 2014, 00:26:03 schrieb Ryan Hill:
 
   Please avoid noblah use flags.
   
   http://devmanual.gentoo.org/general-concepts/use-flags/
   
   ssp flag that defaults to on is fine.
  
  This flag already exists and has always worked this way. 
 
 already exists and has always worked this way is not really a good 
 argument. 
 The rest of the tree sticks to guidelines, so why not here?

Agreed again. Saying that something has always worked a certain way in
the past is not justification for keeping it working that way,
especially when there are preferred ways of doing the same thing that
are different.

  We don't have USE defaults yet.
 
 Now if you had said we can't use USE defaults yet since current ebuilds are 
 still at EAPI=0... that would have been slightly more informative. :)
 
 (Yes I've seen that there is work going on, and I think that's good. Being 
 careful when modernizing makes sense here of course.)

Right, I thought someone had updated toolchain.eclass to use a modern
eapi, but I hadn't seen any more on that in some time. What's the
latest?

Thanks,

William



signature.asc
Description: Digital signature

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 09 Jan 2014 21:58:46 +0100
Magnus Granberg zo...@gentoo.org wrote:

 - use hardened  make_gcc_hard
 + if ( tc_version_is_at_least 4.8 || use hardened )  ! use vanilla ; 
 then

s/4.8/4.8.2

Or should we wait until the next release (4.8.3 or 4.9.0)?  I think I'd
prefer it but I don't have a good reason.

What gcc-config profiles get installed after this patch?


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Rich Freeman

On Thu, Jan 9, 2014 at 5:29 PM, Rick Zero_Chaos Farina
zeroch...@gentoo.org wrote:
 I never felt manipulating cflags with use flags was a great idea, but in
 this case is does feel extra pointless.


Tend to agree, though one place I could see it being hypothetically
useful is if we need to set a use-dep.  That is, package bar might
have a dep on dev-lib/libfoo[-ssp] (or nossp or whatever).

However, what I don't know is whether that will be helpful.  If it is,
then it might make sense to make an exception, otherwise I agree that
overloading CFLAGS in USE flags is bad.

Rich

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 9 Jan 2014 17:41:08 -0600
William Hubbs willi...@gentoo.org wrote:

 On Fri, Jan 10, 2014 at 12:30:04AM +0100, Andreas K. Huettel wrote:
  Am Freitag, 10. Januar 2014, 00:26:03 schrieb Ryan Hill:
  
Please avoid noblah use flags.

http://devmanual.gentoo.org/general-concepts/use-flags/

ssp flag that defaults to on is fine.
   
   This flag already exists and has always worked this way. 
  
  already exists and has always worked this way is not really a good
  argument. The rest of the tree sticks to guidelines, so why not here?
 
 Agreed again. Saying that something has always worked a certain way in
 the past is not justification for keeping it working that way,
 especially when there are preferred ways of doing the same thing that
 are different.

Sure, I'm just pointing out that nothing is changing here.  It sounded like
people were objecting to the addition of a new no* flag.  I agree we should
change them once we can but that shouldn't block this patch.


   We don't have USE defaults yet.
  
  Now if you had said we can't use USE defaults yet since current ebuilds
  are still at EAPI=0... that would have been slightly more informative. :)
  
  (Yes I've seen that there is work going on, and I think that's good. Being 
  careful when modernizing makes sense here of course.)
 
 Right, I thought someone had updated toolchain.eclass to use a modern
 eapi, but I hadn't seen any more on that in some time. What's the
 latest?

I did, but I can't start using new features until I bring all the ebuilds up to
a minimum EAPI.  I'm going to start that this weekend.



-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 9 Jan 2014 17:30:46 -0600
Ryan Hill dirtye...@gentoo.org wrote:

 On Thu, 09 Jan 2014 17:29:26 -0500
 Rick \Zero_Chaos\ Farina zeroch...@gentoo.org wrote:
 
  -BEGIN PGP SIGNED MESSAGE-
  Hash: SHA1
  
  On 01/09/2014 05:21 PM, Michał Górny wrote:
   Dnia 2014-01-09, o godz. 17:06:52
   Anthony G. Basile bluen...@gentoo.org napisał(a):
   
   On 01/09/2014 04:57 PM, Pacho Ramos wrote:
   What are the advantages of disabling SSP to deserve that special
   handling via USE flag or easily disabling it appending the flag?
  
   There are some cases where ssp could break things.  I know of once case 
   right now, but its somewhat exotic.  Also, sometimes we *want* to break 
   things for testing.  I'm thinking here of instance where we want to test 
   a pax hardened kernel to see if it catches abuses of memory which would 
   otherwise be caught by executables emitted from a hardened toolchain.  
   Take a look at the app-admin/paxtest suite.
   
   Just to be clear, are we talking about potential system-wide breakage
   or single, specific packages being broken by SSP? In other words, are
   there cases when people will really want to disable SSP completely?
   
   Unless I'm misunderstanding something, your examples sound like you
   just want -fno-stack-protector per-package. I don't really think you
   actually want to rebuild whole gcc just to do some testing on a single
   package...
   
  Or just as easily set -fno-stack-protector in CFLAGS in make.conf.
  
  I never felt manipulating cflags with use flags was a great idea, but in
  this case is does feel extra pointless.
  
  Personally I don't feel this is needed, and the added benefit of
  clearing up a bogus noblah use flag makes me smile.
  
  Zorry, do we really need this flag?
 
 Yes, we do.  I want a way to disable it at a toolchain level.

Let me clarify.  I would like to be able to disable it without relying on
CFLAGS or anything the user could fiddle with.  I need a big red off switch,
at least for now.


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread heroxbd

Igor lanthrus...@gmail.com writes:

 The ebuilds have approximately the same time to install, the failure
 rate is about the same, emerge is getting slower.

I am curious about the slowness of emerge.

How about profile the portage and rewrite the time-crucial part in
C/C++, or ideally, borrowing the counterpart from paludis? How feasible
is that?

I guess the dep-tree calculation is the slowest part.

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread heroxbd

Hey Igor,

Igor lanthrus...@gmail.com writes:

 Jeroen, tell me how many users world wide do not prefer to upgrade Gentoo
 on automated basis? There are important servers, and there are many
 cases when after upgrade server stops. Do you remember that recent udev
 change? And there are many similar cases. Imagine that your server
 is running a reactor. So what would you prefer to keep it running the
 reactor as it did flawlessly for 8 years or launch an upgrade taking
 the risks to blast yourself?

 Many be it's not only me, but somebody else who is thinking the same?
 Are you sure that the majority of Gentoo users are indulged in
 paranoid automated upgrade and then spending time fixing damage
 that upgrade did?

 Do you have a car? Why you don't change EVERY detail in your car on a new
 version on daily basis automatically?

 Why don't you change car as soon as a new version is released? Why not
 changing the new mouse, new keyboard, new monitor, new supply daily as
 soon as there is a new version?

 Not to mention that you can change daily appearances.

IMHO, the bleeding-edgeness and stability form a balance. We cannot
achieve both. Taking RHEL for example, it uses ancient software for the
sake of stability. Gentoo is way off the other extreme.

For the udev change, the upstream has been doing evil and eudev is not
introduced as the default for Gentoo (yet).

New software breaks things, and security-updated old software needs
extra care: That's the fundamental problem we couldn't circumvent.

Cheers,
Benda

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Patrick Lauer

On 01/10/2014 08:16 AM, hero...@gentoo.org wrote:
 Igor lanthrus...@gmail.com writes:
 
 The ebuilds have approximately the same time to install, the failure
 rate is about the same, emerge is getting slower.
 
 I am curious about the slowness of emerge.
 
 How about profile the portage and rewrite the time-crucial part in
 C/C++, or ideally, borrowing the counterpart from paludis? How feasible
 is that?

Last I checked paludis wasn't faster - on average portage was a few
percents faster.

For python things you really want  python or C instead of C++...

So, what you wanted to ask was:
Which parts of pkgcore can be migrated into portage?

 I guess the dep-tree calculation is the slowest part.
Yes, it's doing lots of silly dynamic things (backtracking), and portage
codebase
on average is not designed for speed.

As a first step I would recommend profiling it and removing unneeded
stuff (do less work!), rewriting parts in C is a lot of work and not
needed for the first round of speedups.

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Tom Wijsman

On Fri, 10 Jan 2014 09:16:47 +0900
hero...@gentoo.org wrote:

 Igor lanthrus...@gmail.com writes:
 
  The ebuilds have approximately the same time to install, the failure
  rate is about the same, emerge is getting slower.
 
 I am curious about the slowness of emerge.

Try a --backtrack=0 approach, I no longer need to increase it. :)

 How about profile the portage and rewrite the time-crucial part in
 C/C++, or ideally, borrowing the counterpart from paludis? How
 feasible is that?

http://dev.gentoo.org/~tomwij/files/portage-2.2.7-python-2.7-backtrack-0.png
http://dev.gentoo.org/~tomwij/files/portage-2.2.7-python-2.7-backtrack-0-hot.png
http://dev.gentoo.org/~tomwij/files/portage-2.2.7-python-3.3-backtrack-0.png

(hot is the hotshot profiler, it internally checks on the line level
instead; 3.3's profiler is obstructed by module loading, no idea why)

 I guess the dep-tree calculation is the slowest part.

Affirmative.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Tom Wijsman

On Fri, 10 Jan 2014 08:31:21 +0800
Patrick Lauer patr...@gentoo.org wrote:

 On 01/10/2014 08:16 AM, hero...@gentoo.org wrote:
  Igor lanthrus...@gmail.com writes:
  
  The ebuilds have approximately the same time to install, the
  failure rate is about the same, emerge is getting slower.
  
  I am curious about the slowness of emerge.
  
  How about profile the portage and rewrite the time-crucial part in
  C/C++, or ideally, borrowing the counterpart from paludis? How
  feasible is that?
 
 Last I checked paludis wasn't faster - on average portage was a few
 percents faster.

Besides that, a different Python version can also differ; I've found
Python 3.3 to be faster for both emerge and repoman. Maybe PyPy can end
up even being faster than that, but that's another subjective story;
though introducing multiple threads in Portage would be really nice,
but the GIL sits in the way:

https://wiki.python.org/moin/GlobalInterpreterLock
http://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil-why

 For python things you really want  python or C instead of C++...

Why is this a Python thing? What's the reason to exclude a language?

 So, what you wanted to ask was:
 Which parts of pkgcore can be migrated into portage?

Or rather: What does it take to migrate parts of pkgcore into portage?

  I guess the dep-tree calculation is the slowest part.
 Yes, it's doing lots of silly dynamic things (backtracking),

Exactly, without backtracking (which I can live without) it takes
around half a minute as compared to a lot of minutes; when it started
to take almost half an hour it was time to completely nuke backtracking.

Now I happily live without ever needing to enable backtracking again. :)

 and portage codebase on average is not designed for speed.

Yes, inspecting the profiler results has me worried; though I find half
a minute acceptable for the amount of packages that are involved on my
system, so, I'm less concerned but it is still interesting that there
is the possibility to do things faster. It's just some work to actually
get it to do that, which requires much more dedication, time and
knowledge than doing an occasional commit.

 As a first step I would recommend profiling it and removing unneeded
 stuff (do less work!), rewriting parts in C is a lot of work and not
 needed for the first round of speedups.

See my other mail 20140110020218.0f6244d5@TOMWIJ-GENTOO for recent
profiling images; we should indeed start with dealing with the low
hanging fruit (there are quite some 0-2% speed improvements possible),
as that can get us to speed it up faster than trying to deal with some
algorithmic complexity that needs far more insight to redesign, let
alone to properly re-implement it.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Patrick McLean

On Fri, 10 Jan 2014 02:19:03 +0100
Tom Wijsman tom...@gentoo.org wrote:

 On Fri, 10 Jan 2014 08:31:21 +0800
 Patrick Lauer patr...@gentoo.org wrote:
 
  On 01/10/2014 08:16 AM, hero...@gentoo.org wrote:
   Igor lanthrus...@gmail.com writes:
   
   The ebuilds have approximately the same time to install, the
   failure rate is about the same, emerge is getting slower.
   
   I am curious about the slowness of emerge.
   
   How about profile the portage and rewrite the time-crucial part in
   C/C++, or ideally, borrowing the counterpart from paludis? How
   feasible is that?
  
  Last I checked paludis wasn't faster - on average portage was a few
  percents faster.
 

  For python things you really want  python or C instead of C++...
 
 Why is this a Python thing? What's the reason to exclude a language?
 
  So, what you wanted to ask was:
  Which parts of pkgcore can be migrated into portage?
 
 Or rather: What does it take to migrate parts of pkgcore into
 portage?

Why not just switch to using pkgcore as the default package manager.
radhermit has been doing a lot of work lately getting EAPI 5 support
added, and generally fixing bugs etc.

Since we no longer have anyone intimately familiar with the
portage code, we should just switch to a more modern and readable
system. Rather than having random people trying to learn the convoluted
portage code, let's concentrate on getting pkgcore to a point where
we can replace portage with it.

   I guess the dep-tree calculation is the slowest part.
  Yes, it's doing lots of silly dynamic things (backtracking),
 
 Exactly, without backtracking (which I can live without) it takes
 around half a minute as compared to a lot of minutes; when it started
 to take almost half an hour it was time to completely nuke
 backtracking.
 
 Now I happily live without ever needing to enable backtracking
 again. :)
 
  and portage codebase on average is not designed for speed.
 
 Yes, inspecting the profiler results has me worried; though I find
 half a minute acceptable for the amount of packages that are involved
 on my system, so, I'm less concerned but it is still interesting that
 there is the possibility to do things faster. It's just some work to
 actually get it to do that, which requires much more dedication, time
 and knowledge than doing an occasional commit.
 
  As a first step I would recommend profiling it and removing unneeded
  stuff (do less work!), rewriting parts in C is a lot of work and not
  needed for the first round of speedups.
 
 See my other mail 20140110020218.0f6244d5@TOMWIJ-GENTOO for recent
 profiling images; we should indeed start with dealing with the low
 hanging fruit (there are quite some 0-2% speed improvements possible),
 as that can get us to speed it up faster than trying to deal with some
 algorithmic complexity that needs far more insight to redesign, let
 alone to properly re-implement it.

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread Tom Wijsman

On Thu, 9 Jan 2014 17:52:16 -0800
Patrick McLean chutz...@gentoo.org wrote:

 On Fri, 10 Jan 2014 02:19:03 +0100
 Tom Wijsman tom...@gentoo.org wrote:
 
  Or rather: What does it take to migrate parts of pkgcore into
  portage?
 
 Why not just switch to using pkgcore as the default package manager.

Has anyone switched to pkgcore already?

It needs to work well and be wide tested before it can become a default.

 radhermit has been doing a lot of work lately getting EAPI 5 support
 added, and generally fixing bugs etc.

Are we there yet?

The thing about pkgcore is that it is perceived as running behind; and
with the lack of interest in it due to that, it seems that having it
will take some time before it lifts off the ground. Don't get me wrong,
it eventually will if we pursue it; but, when? It can take time.

 Since we no longer have anyone intimately familiar with the
 portage code, we should just switch to a more modern and readable
 system.

Agreed, but we also should keep Portage alive and kicking until then.

 Rather than having random people trying to learn the
 convoluted portage code, let's concentrate on getting pkgcore to a
 point where we can replace portage with it.

Either way, people need to learn something; and if we can't guarantee
the short term future of Portage before pkgcore becomes usable, the
long term future could rather be out of reach before you know it.

In the short term we should focus on Portage, but in the long term we
should indeed look at one or another replacement; and indeed does
pkgcore soon appear to be the most interesting option here.

To satisfy QA and Portage teams at the same time; I'm going to start
digging into repoman soon as there are 80+ bugs open for it, so in the
short term (days / weeks), I have no plans for pkgcore myself.

From there on I plan to do a software re-engineering style inspection on
the Portage base to learn and understand its convoluted structure; at
that point we can make more educated choices as to which way to go
forward, but until then ... let's just fix as much bugs as time permits.

Don't get me wrong; pkgcore is great, but it takes time for attention
to shift to it as Portage's slowly runs into more legacy code problems.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : tom...@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D


signature.asc
Description: PGP signature

[gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

On Thu, 09 Jan 2014 21:58:46 +0100
Magnus Granberg zo...@gentoo.org wrote:

 Some time ago we discussed that we should enable stack smashing 
 (-fstack-protector) by default.  So we opened a bug to track this [1].  
 The affected Gcc version will be 4.8.2 and newer. Only amd64, x86, mips, ppc, 
 ppc64 and arm will be affected by this change. 
 
 You can turn off ssp by using the nossp USE flag or by adding 
 -fno-stack-protector to the CFLAGS and/or CXXFLAGS. We are using the same 
 patch as Debian/Ubuntu but with some Gentoo fixes.
 
 The patch will move the sed for the HARD_CFLAGS, ALLCFLAGS and 
 ALLCXXFLAGS from do_gcc_PIE_patches() to make_gcc_hard().  We will 
 make_gcc_hard() the default for all Gcc versions 4.8 and newer, and turn 
 it on or off with hardened_gcc_works() that will make some sanity checks.

I went ahead and spun a new patchset for the compiler-side stuff if anyone
wants to start playing around.

- apply the eclass patch from bug #484714 (the one attached to Magnus' email
  wouldn't apply for me but maybe my mailer mangled it)
- in gcc-4.8.2.ebuild do:

-PATCH_VER=1.3
+PATCH_VER=1.4-ssptest
 
-PIE_VER=0.5.8
+PIE_VER=0.5.9-ssptest

BTW Magnus, thanks for doing this.


-- 
Ryan Hillpsn: dirtyepic_sk
   gcc-porting/toolchain/wxwidgets @ gentoo.org

47C3 6D62 4864 0E49 8E9E  7F92 ED38 BD49 957A 8463


signature.asc
Description: PGP signature

Re: [gentoo-dev] Portage Feature Request: making thirdpartymirrors easier to manage

2014-01-09 Thread Michał Górny

Dnia 2014-01-06, o godz. 20:20:03
Robin H. Johnson robb...@gentoo.org napisał(a):

 2.4. For stack repos/overlays:
 2.4.1. No prefix: replace all prior mirrors from masters with new URLS in 
 this file.
 2.4.2. - prefix: remove this URL from the list from masters.
 2.4.2. + prefix: append this URL to the list from masters.

Just to be clear, what is the exact use case for this? I can't think
of a really good reason to manipulate mirror lists in subsequent repos.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

2014-01-09 Thread Michał Górny

Dnia 2014-01-09, o godz. 18:59:26
Rich Freeman ri...@gentoo.org napisał(a):

 On Thu, Jan 9, 2014 at 5:29 PM, Rick Zero_Chaos Farina
 zeroch...@gentoo.org wrote:
  I never felt manipulating cflags with use flags was a great idea, but in
  this case is does feel extra pointless.
 
 
 Tend to agree, though one place I could see it being hypothetically
 useful is if we need to set a use-dep.  That is, package bar might
 have a dep on dev-lib/libfoo[-ssp] (or nossp or whatever).
 
 However, what I don't know is whether that will be helpful.  If it is,
 then it might make sense to make an exception, otherwise I agree that
 overloading CFLAGS in USE flags is bad.

We're talking about the ssp (nossp) flag on gcc, not target ebuilds.
Ebuilds have to do stuff like '-fno-stack-protector'. The flag on gcc
rebuilds whole gcc just to change the global default.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature

Re: [gentoo-dev] Portage Feature Request: making thirdpartymirrors easier to manage

2014-01-09 Thread Alec Warner

On Thu, Jan 9, 2014 at 8:58 PM, Michał Górny mgo...@gentoo.org wrote:

 Dnia 2014-01-06, o godz. 20:20:03
 Robin H. Johnson robb...@gentoo.org napisał(a):

  2.4. For stack repos/overlays:
  2.4.1. No prefix: replace all prior mirrors from masters with new URLS
 in this file.
  2.4.2. - prefix: remove this URL from the list from masters.
  2.4.2. + prefix: append this URL to the list from masters.

 Just to be clear, what is the exact use case for this? I can't think
 of a really good reason to manipulate mirror lists in subsequent repos.


I can mostly think of bad reasons, where instead of manipulating the list,
the user should do something else (like use a local mirror.)

-A


 --
 Best regards,
 Michał Górny

Re: [gentoo-dev] Portage QOS

On Thu, 2014-01-09 at 17:52 -0800, Patrick McLean wrote:
 On Fri, 10 Jan 2014 02:19:03 +0100
 Tom Wijsman tom...@gentoo.org wrote:
 
  On Fri, 10 Jan 2014 08:31:21 +0800
  Patrick Lauer patr...@gentoo.org wrote:
  
   On 01/10/2014 08:16 AM, hero...@gentoo.org wrote:
   Last I checked paludis wasn't faster - on average portage was a few
   percents faster.
  
 
   For python things you really want  python or C instead of C++...
  
  Why is this a Python thing? What's the reason to exclude a language?
  
   So, what you wanted to ask was:
   Which parts of pkgcore can be migrated into portage?
  
  Or rather: What does it take to migrate parts of pkgcore into
  portage?
 
 Why not just switch to using pkgcore as the default package manager.
 radhermit has been doing a lot of work lately getting EAPI 5 support
 added, and generally fixing bugs etc.
 
 Since we no longer have anyone intimately familiar with the
 portage code, we should just switch to a more modern and readable
 system. Rather than having random people trying to learn the convoluted
 portage code, let's concentrate on getting pkgcore to a point where
 we can replace portage with it.
 

Patrick  If you wish to be a part of getting pkgcore's eapi 5 fully
working your welcome to start helping out.  I can set you up with access
in the main google code repo.  Radhermit has been working mostly out of
his github account, updating the main repo.

I would very much love to get pkgcore fully functional in eapi 5.

It's speed runs circles around portage and paludis.

I have not been able to help radhermit out as yet, I have been trying to
get some other project cleaned up before delving deeper into pkgcore
code.  I have just recently taken over lead of portage (an interim
position) due to Zac stepping down and away.  So I have that added to my
todo list at the moment too. 

-- 
Brian Dolbec dol...@gentoo.org


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 07:12 PM, Ryan Hill wrote:
 On Thu, 9 Jan 2014 17:41:08 -0600
 William Hubbs willi...@gentoo.org wrote:
 
 On Fri, Jan 10, 2014 at 12:30:04AM +0100, Andreas K. Huettel wrote:
 Am Freitag, 10. Januar 2014, 00:26:03 schrieb Ryan Hill:

 Please avoid noblah use flags.

 http://devmanual.gentoo.org/general-concepts/use-flags/

 ssp flag that defaults to on is fine.

 This flag already exists and has always worked this way. 

 already exists and has always worked this way is not really a good
 argument. The rest of the tree sticks to guidelines, so why not here?

 Agreed again. Saying that something has always worked a certain way in
 the past is not justification for keeping it working that way,
 especially when there are preferred ways of doing the same thing that
 are different.
 
 Sure, I'm just pointing out that nothing is changing here.  It sounded like
 people were objecting to the addition of a new no* flag.  I agree we should
 change them once we can but that shouldn't block this patch.

To be clear, I don't believe the QA team has any desire to block forward
progress including this patch.
I think everyone is chomping at the bit here to see some improvement on
toolchain getting more up to date, and more with the QA policies that
have been in place for years. I realize eapi 2 wasn't that long ago
for some of you, but seriously, it was that long ago.

More to the point, this specific use flag appears to have no purpose
what-so-ever.  If a user can do exactly the same with
CFLAGS=-fno-stack-protector in make.conf, and it would be INSANE for a
package to dep on gcc[nossp] then this is has got to be one of the most
useless use flags in gentoo.

Not saying I would block this patch, not saying it has to be this
second, but I see this use flag as a small example of things in
toolchain which could probably be cleaned up if fresh eyes were to see
things.

- -Zero
 
 
 We don't have USE defaults yet.

 Now if you had said we can't use USE defaults yet since current ebuilds
 are still at EAPI=0... that would have been slightly more informative. :)

 (Yes I've seen that there is work going on, and I think that's good. Being 
 careful when modernizing makes sense here of course.)

 Right, I thought someone had updated toolchain.eclass to use a modern
 eapi, but I hadn't seen any more on that in some time. What's the
 latest?
 
 I did, but I can't start using new features until I bring all the ebuilds up 
 to
 a minimum EAPI.  I'm going to start that this weekend.
 
 
 

-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSz5SdAAoJEKXdFCfdEflKq6wQAILiZN+BVZh+8XrEBsd4a0om
aOk6Inj4zWMK2y5LvI+T29u1xMvko18lu4Vny4dv13w6OMXsE+nip+1nOhxNyJJG
lCwiVWC603pzYw5am/q/XGg/GncjQFkx3FUSRlM8rJrRCQOyLinronojTtIG0GeV
e4k4eHih+wx73agAHXdvLrXP0Ps11yYxY5+U1Rkjf9p4LwMCtJTAwidm0458YZSp
7+ZYJHiBQvLOpG+evcTx8r+7WqfROjIPpFsCXfuPvZiTbGalXK0hEp1rWZ3aDSsw
wZyjo7cuucTGGDn58QRUIH5KLDZtPC0SVUZl9TVbT+rVbv/ugrboIH2rA33UxYr0
yUFj96gZCgVOHgmxsuOUhiR4R2yIDoFOFg7GEU1TL7ydnqPbxZ9FiYuwOTO5/6oN
hqofWQgC9DgjVDB8V/9m4SON7xZbCsmUhlXM1BCCaDx4HlfsgyHn2QQThRwYM4Oq
HHIA8dCBZytyhlZZ/E8qFlebkbBc7CmsU52htp/iI/eSVMBs7856ljzVbToyY95Q
ClGHIF7zRRWW2tGNo9EurKt+U+esuJS6h2buRwRzWVW8nJYy3z11BnkzGp9vwTAc
1GO3kfruHDTtuvB7esSJAfCdz5WDQ/i7rdj5DkaSISrRL0sIQsgeGPDP2Z7+V4cq
0ItbZIIb/50u8JHNiucS
=lrYq
-END PGP SIGNATURE-

Re: [gentoo-dev] Re: [PATCH] To enable ssp default in Gcc the toolchain.eclass need some changes.

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2014 07:17 PM, Ryan Hill wrote:
 On Thu, 9 Jan 2014 17:30:46 -0600 Ryan Hill dirtye...@gentoo.org
 wrote:
 
 On Thu, 09 Jan 2014 17:29:26 -0500 Rick \Zero_Chaos\ Farina
 zeroch...@gentoo.org wrote:
 
 -BEGIN PGP SIGNED MESSAGE- Hash: SHA1
 
 On 01/09/2014 05:21 PM, Michał Górny wrote:
 Dnia 2014-01-09, o godz. 17:06:52 Anthony G. Basile
 bluen...@gentoo.org napisał(a):
 
 On 01/09/2014 04:57 PM, Pacho Ramos wrote:
 What are the advantages of disabling SSP to deserve that
 special handling via USE flag or easily disabling it
 appending the flag?
 
 There are some cases where ssp could break things.  I know
 of once case right now, but its somewhat exotic.  Also,
 sometimes we *want* to break things for testing.  I'm
 thinking here of instance where we want to test a pax
 hardened kernel to see if it catches abuses of memory which
 would otherwise be caught by executables emitted from a
 hardened toolchain. Take a look at the app-admin/paxtest
 suite.
 
 Just to be clear, are we talking about potential system-wide
 breakage or single, specific packages being broken by SSP? In
 other words, are there cases when people will really want to
 disable SSP completely?
 
 Unless I'm misunderstanding something, your examples sound
 like you just want -fno-stack-protector per-package. I don't
 really think you actually want to rebuild whole gcc just to
 do some testing on a single package...
 
 Or just as easily set -fno-stack-protector in CFLAGS in
 make.conf.
 
 I never felt manipulating cflags with use flags was a great
 idea, but in this case is does feel extra pointless.
 
 Personally I don't feel this is needed, and the added benefit
 of clearing up a bogus noblah use flag makes me smile.
 
 Zorry, do we really need this flag?
 
 Yes, we do.  I want a way to disable it at a toolchain level.
 
 Let me clarify.  I would like to be able to disable it without
 relying on CFLAGS or anything the user could fiddle with.  I need a
 big red off switch, at least for now.
 
 
I think if you clarify this last statement a lot of the arguments will
vanish.  I believe most of us are happy to hear your thoughts in a
little more detail, and will be swayed by them.

- -Zero
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSz5W+AAoJEKXdFCfdEflKle4P/3pwjbSDuPy56TXvyH43/Ucg
LvTtyycvGF5pw4UzNBPoao+E7F4E84MFIrfLGE3XpkH1J4v4SLXwh+CuHJHaSxJg
Zx1C6zhk0HnAS2LZHDCuS14+zyOpF/VELOZAimr8V1UBTot3gyZUVvBIsXoJwKx6
3nb+abTHHIFDcSGJz6KM36dy67MMW2kpcKTx5Wu7W91232K8WY3v1KuNRtLBQkd/
/YNcpkW3fkA5Plk7sMTFb8iL6k2oajNw/3PKvX9L/MSBf+PmGKB7yA3Yt5Qu2Grw
gUdUel/fCtXyhv1Had3pjeqZCgK7mFUuw6pAieQen2cYmAypLTC8D7JpaIBkJ9G0
cAz4lBB4EXc+l5mh2SS9D4LqL/4bWu3f7xFdeSuGcoxU1aL8/pgAvz0LpP7/o8ib
BMdvwPcy5zn9W5+jkx5IZXidIpEBHzJKEnSR5+Mf39xn9z0nOrEvzGWxEHIerlWc
hlHNU8x5wsZdnfJsY9tOfb5/hCOZW6uTtdvSR7eDFC6+xx2kBuiS+lC9Dtg76t1q
VAiDOQIb1qgM9D6IUA+Q6WG2sWj3aTmZxfRYYQZ0DBGG266Rr2HaQaVDmWjU8W2u
etxretxm0emYr8vHBPJD1XKwOCs577u4W5sVrQbZl+VutEVH+pqJTg0NOYxTeSTW
CWAZGdV093+dQAa58/M4
=YP+O
-END PGP SIGNATURE-

Re: [gentoo-dev] Portage QOS

2014-01-09 Thread heroxbd

Patrick Lauer patr...@gentoo.org writes:

 For python things you really want  python or C instead of C++...

Well, we have boost-python to do python extensions in C++. And yes,
introducing boost as a dependency to portage is not cool.

 I guess the dep-tree calculation is the slowest part.
 Yes, it's doing lots of silly dynamic things (backtracking), and
 portage codebase on average is not designed for speed.

 As a first step I would recommend profiling it and removing unneeded
 stuff (do less work!), rewriting parts in C is a lot of work and not
 needed for the first round of speedups.

Cython[1] can be used to generate C code quickly to avoid spending to much
time coding in C.

1. http://www.cython.org

[gentoo-portage-dev] Announcing New wiki Project pages

I have mostly migrated the Portage project page to the wiki.

Please look it over, edit any errors you see (only gentoo devs have edit
capabilities).

I have started a couple sub-pages:  Ongoing-TODO and Proposals

links: 

https://wiki.gentoo.org/wiki/Project:Portage

https://wiki.gentoo.org/wiki/Project:Portage/Ongoing-TODO

https://wiki.gentoo.org/wiki/Project:Portage/Proposals

-- 
Brian Dolbec dol...@gentoo.org


signature.asc
Description: This is a digitally signed message part

[gentoo-portage-dev] New proposed modular sync system

First off, I know many of you think portage needs to do everything on
it's own.  NO outside dependencies.

BUT!  Please hear me out.

While working on many of gentoo's tools gentoolkit's equery, eclean,
enalyze, Layman, mirrorselect, etc..  I have found that there was a lot
of needless code duplication.  Both within the apps themselves and
copying/re-inventing the nearly identical code in other apps.

I want to move many of gentoo's core applications to using some more
shared libraries.  But that in itself is not a topic to discuss in this
thread.  Please, do not sidetrack this thread.  The above is only for
background info leading to this proposed change.  This proposal follows
along that thinking.

Many years ago after re-writing ecleans code I began re-writing
portage's emaint code so that it could be imported and used internally
in eclean to re-generate the Packages binpkg index file.  In that code I
found a todo which said make a plug-in system so we don't have to keep
modifying this code  or something along those lines.  So I did.  It
took 4 years+, rebasing many times and migrating vcs's, but Zac finally
merged it.  It has been used in portage for more than a year. That
plug-in system I designed to be very flexible and basic.  It can be used
for many areas of portage to simplify it's code and the maintenance
required to keep current with changes in repository types, and transfer
methods.

I have started a Proposals sub-page under the Portage project page in
the wiki.  It has a link to a diagram I made showing how the plug-in
system is laid out.  This thread will be used to discuss the proposal
and the details needed for the module_spec dictionary that is used to
provide the manager with the details of what a module provides, options,
etc..

For a working example of the system in action, just try out emaint -h,
and some other actions it provides.  Then move one of it's modules out
of the the emaint/modules/ directory and re-try emaint -h.  You will see
the options for that module and it's actions are no longer available.
No code editing was required, no errors occurred,...  Put it back,
retry...

The modular sync system I propose would re-use that plug-in manager to
interface the sync manager code with the modules that perform the sync
actions.  This sytem is easily extended with new modules for different
transfer methods or VCS specific modules.

This also makes it possible for other gentoo applications like layman to
install a layman specific module which syncs all layman overlays that it
installs.  All this done with out effort on the part of the portage team
either for maintenance or code changes to support it's addition.  The
same can be said for Funtoo, if our git module is not suiotable, they
just install their own custom sync module.  Done :)

These add-on sync modules can also be installed via their own ebuild.

For example:  Zac has for years had a custom script that syncs the
portage tree and makes a squashfs version of it which he uses for his
PORTDIR. it is faster than a normal one.  He, or someone else could make
a app-portage/squashfs-sync ebuild that installs a module for that
purpose.  The user then just needs to edit his repos.conf file and
change the sync-type to squashfs.  Then emerge --sync will do it
automatically without user intervention and auxiliary scripts run
manually or cron job.

This system would also make it possible to modify emerge-s cli to accept
target repos to sync. and not any others also installed.  Similarly to
layman -s some-overlay, emerge --sync some-overlay
 
P.S. sorry for such a long email 
-- 
Brian Dolbec dol...@gentoo.org


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-portage-dev] New proposed modular sync system

2014-01-09 Thread Sebastian Luther

Am 09.01.2014 18:33, schrieb Brian Dolbec:
 introduction
 
 history of the plugin system

I fully agree with idea behind the plugin system. That is, to keep
things that are separate, separate. It probably wouldn't have occurred
to me to implement it that way (i.e. with auto-detection) but that's
just fine.

I don't see a problem with reusing that system here.

 I have started a Proposals sub-page under the Portage project page
 in the wiki.  It has a link to a diagram I made showing how the
 plug-in system is laid out.  This thread will be used to discuss
 the proposal and the details needed for the module_spec dictionary
 that is used to provide the manager with the details of what a
 module provides, options, etc..
 

The layout makes sense. Except the problems I see with where the
modules are installed (see later).

Not sure about module_spec yet.

 [...]
 
 stuff about ebuilds installing plugins
 
While I see a valid use case here (especially with your squashfs
example), I wonder how that is supposed to work.

1) Where should the ebuild install the plugin?
2) How does portage find those plugins.
3) How does portage's behavior to run with the currently selected
python version, mix with the fact that ebuilds usually install only
for some set of python versions? (especially python 2 vs 3).

 This system would also make it possible to modify emerge-s cli to
 accept target repos to sync. and not any others also installed.
 Similarly to layman -s some-overlay, emerge --sync
 some-overlay

emerge --sync some-overlay already works.


 P.S. sorry for such a long email

Re: [gentoo-portage-dev] New proposed modular sync system

On Thu, 2014-01-09 at 09:33 -0800, Brian Dolbec wrote:

 I have started a Proposals sub-page under the Portage project page in
 the wiki.  It has a link to a diagram I made showing how the plug-in
 system is laid out.  This thread will be used to discuss the proposal
 and the details needed for the module_spec dictionary that is used to
 provide the manager with the details of what a module provides, options,
 etc..
 
 Forgot to add the link:

https://wiki.gentoo.org/wiki/Project:Portage/Proposals

It has the link to the diagram I created on that page


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-portage-dev] New proposed modular sync system

2014-01-09 Thread Alec Warner

On Thu, Jan 9, 2014 at 9:33 AM, Brian Dolbec dol...@gentoo.org wrote:

 First off, I know many of you think portage needs to do everything on
 it's own.  NO outside dependencies.

 BUT!  Please hear me out.

 While working on many of gentoo's tools gentoolkit's equery, eclean,
 enalyze, Layman, mirrorselect, etc..  I have found that there was a lot
 of needless code duplication.  Both within the apps themselves and
 copying/re-inventing the nearly identical code in other apps.

 I want to move many of gentoo's core applications to using some more
 shared libraries.  But that in itself is not a topic to discuss in this
 thread.  Please, do not sidetrack this thread.  The above is only for
 background info leading to this proposed change.  This proposal follows
 along that thinking.

 Many years ago after re-writing ecleans code I began re-writing
 portage's emaint code so that it could be imported and used internally
 in eclean to re-generate the Packages binpkg index file.  In that code I
 found a todo which said make a plug-in system so we don't have to keep
 modifying this code  or something along those lines.  So I did.  It
 took 4 years+, rebasing many times and migrating vcs's, but Zac finally
 merged it.  It has been used in portage for more than a year. That
 plug-in system I designed to be very flexible and basic.  It can be used
 for many areas of portage to simplify it's code and the maintenance
 required to keep current with changes in repository types, and transfer
 methods.

 I have started a Proposals sub-page under the Portage project page in
 the wiki.  It has a link to a diagram I made showing how the plug-in
 system is laid out.  This thread will be used to discuss the proposal
 and the details needed for the module_spec dictionary that is used to
 provide the manager with the details of what a module provides, options,
 etc..

 For a working example of the system in action, just try out emaint -h,
 and some other actions it provides.  Then move one of it's modules out
 of the the emaint/modules/ directory and re-try emaint -h.  You will see
 the options for that module and it's actions are no longer available.
 No code editing was required, no errors occurred,...  Put it back,
 retry...

 The modular sync system I propose would re-use that plug-in manager to
 interface the sync manager code with the modules that perform the sync
 actions.  This sytem is easily extended with new modules for different
 transfer methods or VCS specific modules.

 This also makes it possible for other gentoo applications like layman to
 install a layman specific module which syncs all layman overlays that it
 installs.  All this done with out effort on the part of the portage team
 either for maintenance or code changes to support it's addition.  The
 same can be said for Funtoo, if our git module is not suiotable, they
 just install their own custom sync module.  Done :)

 These add-on sync modules can also be installed via their own ebuild.

 For example:  Zac has for years had a custom script that syncs the
 portage tree and makes a squashfs version of it which he uses for his
 PORTDIR. it is faster than a normal one.  He, or someone else could make
 a app-portage/squashfs-sync ebuild that installs a module for that
 purpose.  The user then just needs to edit his repos.conf file and
 change the sync-type to squashfs.  Then emerge --sync will do it
 automatically without user intervention and auxiliary scripts run
 manually or cron job.

 This system would also make it possible to modify emerge-s cli to accept
 target repos to sync. and not any others also installed.  Similarly to
 layman -s some-overlay, emerge --sync some-overlay


I think the opposition to this idea primarily falls on reliability. There
have been a number of hacks to portage over the years to keep it
functioning during upgrades of itself, even down to the non-standard place
where its .py files are stored (/usr/lib/portage/pym?) So the real question
is when we move to a plug-in architecture, how do we add, remove, upgrade
the plugins without breaking the actual package manager?

-A




 P.S. sorry for such a long email
 --
 Brian Dolbec dol...@gentoo.org

Re: [gentoo-portage-dev] New proposed modular sync system

On Thu, 2014-01-09 at 19:46 +0100, Sebastian Luther wrote:

 The layout makes sense. Except the problems I see with where the
 modules are installed (see later).
 
 Not sure about module_spec yet.
 
  [...]
  

The module_spec is a means to make available to the operational manager
what the module supplies and any options it provides.

The second thing it does is not import the files and modules it
supplies.  This reduces the memory requirements by not loading modules,
files that are not needed.  It also speeds up run time as a result.

  stuff about ebuilds installing plugins
  
 While I see a valid use case here (especially with your squashfs
 example), I wonder how that is supposed to work.
 
 1) Where should the ebuild install the plugin?
 2) How does portage find those plugins.

That is to be determined.  It doesn't matter where to the plugin
manager.  It needs a path to the plug-in directory on initialization.
It does not need to be in a subdirectory where it is located.


 3) How does portage's behavior to run with the currently selected
 python version, mix with the fact that ebuilds usually install only
 for some set of python versions? (especially python 2 vs 3).
 

The plug-in must be able to work with all supported python versions in
one form or another.

In esearch, it supports python 3, but layman does not (yet), so when you
use the -l switch to sync layman overlays as well as the main tree.  It
first tries to import the layman modules, upon failure it falls back to
calling layman using a subprocess call.  Just like portage currently
calls rsync and git in a subprocess now.

see working code:
https://github.com/fuzzyray/esearch/blob/master/esearch/sync.py
layman_sync()  line # 149


If a proposed module is to be added to the tree, it should be capable of
working in all configurations a user may have withing portage's
capability.  If not, if it isn't shipped by and with portage, then it is
a user issue.  Not ours.  In such a case we (the portage-tools team)
would insist on ewarns in the ebuild stating such constraints, etc..

  This system would also make it possible to modify emerge-s cli to
  accept target repos to sync. and not any others also installed.
  Similarly to layman -s some-overlay, emerge --sync
  some-overlay
 
 emerge --sync some-overlay already works.
 
 
  P.S. sorry for such a long email
  
 
 



signature.asc
Description: This is a digitally signed message part

Re: [gentoo-portage-dev] New proposed modular sync system