Re: mach has landed

[Nicholas Nethercote]

On Tue, Oct 2, 2012 at 7:12 PM, Jeff Gilbert  wrote:
> As an additional data point, my experience is that the interactivity of my 
> machine is not noticeably impacted when I overcommit with -j12 on my 
> 4core/8thread i7 windows machine.

Right, I think we've discussed this issue enough.  Everyone with
strong opinions about this can simply change the default.  Good work,
Gregor.

Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: mach has landed

[Jeff Gilbert]

As an additional data point, my experience is that the interactivity of my 
machine is not noticeably impacted when I overcommit with -j12 on my 
4core/8thread i7 windows machine. Task manager shows the cores often pegged at 
100%, but the machine basically behaves normally. Neither is my Ubuntu 
8core/16thread/-j16 (sometimes -j24) linux box noticeably affected when it's 
fully pegged.

Clearly it depends what you're trying to do, but for my uses, I just spin up a 
build and do other things normally until it's done, regardless of how 
over-committed it is.

- Original Message -
From: "Karl Tomlinson" 
To: dev-platform@lists.mozilla.org
Sent: Tuesday, October 2, 2012 3:05:55 PM
Subject: Re: mach has landed

On Fri, 28 Sep 2012 11:44:56 -0700, Gary Kwong wrote:

>> http://blog.johnford.org/new-mac-builders-ssds-j-settings/
>
> Quoted from that blog:
>
> "I did find that it is better to set the -j setting too high than
> it is to set it too low."
>
> -Gary

Better in terms of build time, which is the right metric for a
dedicated build machine.

But bear in mind that a developer is usually trying to use the
machine at the same time as doing a build.  Adding more context
switching load to the system without significantly improving build
times is not necessarily a win for the developer.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Snappy Meeting, Thurs. Oct 4, 11am PT

[Lawrence Mandel]

Correction. PB&J is booked this week. Those in MV should find another room. 
We'll use the ProgramManagement vidyo room.

- Original Message -
> With people off helping B2G let's spend some time this week taking a
> good look at the open projects and the progress that we expect to
> make in Q4.
> 
> Please add your items and status to the agenda before the call.
> https://etherpad.mozilla.org/snappy
> 
> Dial-in: conference# 95346
> US/International: +1 650 903 0800 x92 Conf# 95346
> US toll free: +1 800 707 2533 (pin 369) Conf# 95346
> Canada: +1 416 848 3114 x92 Conf# 95346
> Vidyo: PB&J
> IRC: #perf
> 
> Lawrence
> ___
> dev-planning mailing list
> dev-plann...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
> 
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Snappy Meeting, Thurs. Oct 4, 11am PT

[Lawrence Mandel]

With people off helping B2G let's spend some time this week taking a good look 
at the open projects and the progress that we expect to make in Q4.

Please add your items and status to the agenda before the call.
https://etherpad.mozilla.org/snappy

Dial-in: conference# 95346
US/International: +1 650 903 0800 x92 Conf# 95346
US toll free: +1 800 707 2533 (pin 369) Conf# 95346
Canada: +1 416 848 3114 x92 Conf# 95346
Vidyo: PB&J
IRC: #perf

Lawrence
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: try: -p all considered harmful?

[Ehsan Akhgari]

On 2012-10-02 6:00 PM, Karl Tomlinson wrote:

Ben Hearsum writes:


On 09/29/12 12:58 AM, Kannan Vijayan wrote:

1. A patch that is expected to succeed, but you want to run it through
try to verify.



For "optimistic" pushes, we expect that the patch goes from green =>
green.  For "pessimistic" pushes we expect a listing of all oranges that
arise from that push.



I believe that these patches are exactly what inbound was designed for.


Bear in mind that as well as the temporary cost of backout in
inbound, there is also the close-to-eternal cost on archaeology
when trying to wade through annotations of several versions for
several similar landings.

inbound landings should be more certain than "optimistic".


In an ideal world, yes.  Note that we are currently under extremely high 
load, which makes us consider trade-offs.


Cheers,
Ehsan

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to remove support for the -t all trychooser syntax

[Ehsan Akhgari]

On 2012-10-02 6:17 PM, Bobby Holley wrote:

The general problem as I see it is that talos try doesn't go orange if
there's a regression (because we detect regressions over time), and
checking the results against a baseline revision is kind of a pain, even
with talos-compare. So I think most developers ignore it, and those who
have the gumption to compare all those numbers can probably be bothered to
explicitly write in the platforms they want.


Indeed.  This is in fact the point of this proposal.

Cheers,
Ehsan

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to remove support for the -t all trychooser syntax

[Bobby Holley]

The general problem as I see it is that talos try doesn't go orange if
there's a regression (because we detect regressions over time), and
checking the results against a baseline revision is kind of a pain, even
with talos-compare. So I think most developers ignore it, and those who
have the gumption to compare all those numbers can probably be bothered to
explicitly write in the platforms they want.

On Wed, Oct 3, 2012 at 12:13 AM, Karl Tomlinson  wrote:

> Ehsan Akhgari writes:
>
> > Over in bug 796087, I'm proposing for us to remove the -t all try server
> > flag.  The rationale is that the set of Talos tests vary greatly and most
> > of the people who test performance are only interested in a subset of
> Talos
> > tests.
>
> Frequently enough people change code that I would expect could
> change any talos results except perhaps dromaeojs.
>
> I'm wondering whether there is a smaller subset that should be
> considered.
>
> Are there other suites that should not be affected by graphics
> code changes, for example?
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to remove support for the -t all trychooser syntax

[Karl Tomlinson]

Ehsan Akhgari writes:

> Over in bug 796087, I'm proposing for us to remove the -t all try server
> flag.  The rationale is that the set of Talos tests vary greatly and most
> of the people who test performance are only interested in a subset of Talos
> tests.

Frequently enough people change code that I would expect could
change any talos results except perhaps dromaeojs.

I'm wondering whether there is a smaller subset that should be
considered.

Are there other suites that should not be affected by graphics
code changes, for example?
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: mach has landed

[Karl Tomlinson]

On Fri, 28 Sep 2012 11:44:56 -0700, Gary Kwong wrote:

>> http://blog.johnford.org/new-mac-builders-ssds-j-settings/
>
> Quoted from that blog:
>
> "I did find that it is better to set the -j setting too high than
> it is to set it too low."
>
> -Gary

Better in terms of build time, which is the right metric for a
dedicated build machine.

But bear in mind that a developer is usually trying to use the
machine at the same time as doing a build.  Adding more context
switching load to the system without significantly improving build
times is not necessarily a win for the developer.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: try: -p all considered harmful?

[Karl Tomlinson]

Ben Hearsum writes:

> On 09/29/12 12:58 AM, Kannan Vijayan wrote:
>> 1. A patch that is expected to succeed, but you want to run it through
>> try to verify.
>
>> For "optimistic" pushes, we expect that the patch goes from green =>
>> green.  For "pessimistic" pushes we expect a listing of all oranges that
>> arise from that push.
>>
>
> I believe that these patches are exactly what inbound was designed for.

Bear in mind that as well as the temporary cost of backout in
inbound, there is also the close-to-eternal cost on archaeology
when trying to wade through annotations of several versions for
several similar landings.

inbound landings should be more certain than "optimistic".
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to remove support for the -t all trychooser syntax

[Steve Fink]

On 10/01/2012 04:40 PM, Jeff Hammel wrote:

On 10/01/2012 04:32 PM, Ehsan Akhgari wrote:

Over in bug 796087, I'm proposing for us to remove the -t all try server
flag.  The rationale is that the set of Talos tests vary greatly and 
most
of the people who test performance are only interested in a subset of 
Talos

tests.  So I'm suggesting that we should disallow -t all as it is a very
easy way to consume a lot of infrastructure resources that one most 
likely
does not need.  As philor mentioned in the bug, there are platforms 
such as

mobile where Talos is more than merely a performance test, and in those
platforms developers can use -t foo,bar,baz to run multiple Talos suites
(or all of them for that matter).

This will make it a bit more verbose to run all of the Talos tests if
that's what you're looking for, but for the majority of pushes that 
should
not be a burden, and the ability of actually running all Talos suites 
will

remain intact with the more verbose syntax.

Does anybody have an objection about this?

Thanks!
--
Ehsan


This will make things much harder for me to test talos changes.  I 
realize, however, that I am the < 1% use-case here though :( If there 
is a programmatic way of getting all of '-t foo,bar,baz,etc' then I 
wouldn't care, but I doubt this is easily doable.


I see no problem in just renaming -t all to -t 
my-name-is-jhammel-hear-me-roar, or any other string making it clear 
that people should rarely if ever be using it.


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: try: -p all considered harmful?

[Justin Lebar]

> Boxes which run performance tests are separate from boxes which run
> unit tests, right?  If so, I'd be interested to know what the
> breakdown is ignoring Talos, since that's not usually on the critical
> path for m-i or try, and since that would presumably skew the load
> measured above towards m-i.

Actually, bhearsum tells me that talos and unit tests run on the same
pool of machines, so the 40% m-i to 31% try breakdown seems like a
reasonable measure.

On Tue, Oct 2, 2012 at 9:58 AM, Justin Lebar  wrote:
>> That's just the load in terms of # of pushes. Load in terms of % time spent
>> on the various branches is pretty different. For the past 30 days, here's
>> the breakdown of time of builds and tests spent on the top 4 branches:
>> 39.93% mozilla-inbound
>> 31.00% try
>> 6.91% mozilla-central
>> 6.22% mozilla-aurora
>>
>> Actual machine load from try is disproportionally lower than its share of
>> pushes would indicate. I would guess this is due to not running the full set
>> of builds/tests via try syntax, or from cancelling jobs early. It's also
>> interesting because jobs on try never coalesce, whereas this happens quite a
>> bit on mozilla-inbound.
>
> Thank you so much for data.
>
> Boxes which run performance tests are separate from boxes which run
> unit tests, right?  If so, I'd be interested to know what the
> breakdown is ignoring Talos, since that's not usually on the critical
> path for m-i or try, and since that would presumably skew the load
> measured above towards m-i.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: try: -p all considered harmful?

[Justin Lebar]

> That's just the load in terms of # of pushes. Load in terms of % time spent
> on the various branches is pretty different. For the past 30 days, here's
> the breakdown of time of builds and tests spent on the top 4 branches:
> 39.93% mozilla-inbound
> 31.00% try
> 6.91% mozilla-central
> 6.22% mozilla-aurora
>
> Actual machine load from try is disproportionally lower than its share of
> pushes would indicate. I would guess this is due to not running the full set
> of builds/tests via try syntax, or from cancelling jobs early. It's also
> interesting because jobs on try never coalesce, whereas this happens quite a
> bit on mozilla-inbound.

Thank you so much for data.

Boxes which run performance tests are separate from boxes which run
unit tests, right?  If so, I'd be interested to know what the
breakdown is ignoring Talos, since that's not usually on the critical
path for m-i or try, and since that would presumably skew the load
measured above towards m-i.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: try: -p all considered harmful?

[Chris AtLee]

On 01/10/12 10:51 PM, Justin Lebar wrote:

For case 1., an idea that has been floated here and again (in Automation and
Tools and Release Engineering, anyway) is landing directly from try ->
inbound (or central) for green try pushes. However, this isn't a small
endeavor, both for the reasons of building the infra + software to do this
as well as the fact that since the average random oranges/push is high
landing is such a manual process.  If this is something we want, though, we
should ticket it and estimate what this will really take.


A very simple way to approximate this would be the following policy:

   * If you have a recent green try build, you can land on inbound with
DONTBUILD.

I think this would likely result in even more headaches for sheriffs
and is optimizing for load on the wrong tree (m-i is only 25% of our
load, while try is 50%), but it would be easy to try out, so maybe we
should do it for a day or two and see if the result is chaos.


That's just the load in terms of # of pushes. Load in terms of % time 
spent on the various branches is pretty different. For the past 30 days, 
here's the breakdown of time of builds and tests spent on the top 4 
branches:

39.93% mozilla-inbound
31.00% try
6.91% mozilla-central
6.22% mozilla-aurora

Actual machine load from try is disproportionally lower than its share 
of pushes would indicate. I would guess this is due to not running the 
full set of builds/tests via try syntax, or from cancelling jobs early. 
It's also interesting because jobs on try never coalesce, whereas this 
happens quite a bit on mozilla-inbound.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Proposal to remove support for the -t all trychooser syntax

[Bobby Holley]

I think this is a great idea.

On Tue, Oct 2, 2012 at 1:32 AM, Ehsan Akhgari wrote:

> Over in bug 796087, I'm proposing for us to remove the -t all try server
> flag.  The rationale is that the set of Talos tests vary greatly and most
> of the people who test performance are only interested in a subset of Talos
> tests.  So I'm suggesting that we should disallow -t all as it is a very
> easy way to consume a lot of infrastructure resources that one most likely
> does not need.  As philor mentioned in the bug, there are platforms such as
> mobile where Talos is more than merely a performance test, and in those
> platforms developers can use -t foo,bar,baz to run multiple Talos suites
> (or all of them for that matter).
>
> This will make it a bit more verbose to run all of the Talos tests if
> that's what you're looking for, but for the majority of pushes that should
> not be a burden, and the ability of actually running all Talos suites will
> remain intact with the more verbose syntax.
>
> Does anybody have an objection about this?
>
> Thanks!
> --
> Ehsan
> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform