Re: Autoland to mozilla-central updates currently blocked

[Carsten Book]

Hi,

all resolved now and will update now autoland to m-c on a regular base!

Thanks to gps for resolving the problem!

Cheers,
- tomcat

On Mon, Jul 4, 2016 at 10:08 AM, Carsten Book  wrote:

> Hi,
>
> just in case someone wonders (and since i got pinged by a couple of
> people).
>
> Currently due to https://bugzilla.mozilla.org/show_bug.cgi?id=1283798#c5
> we cannot update/merge/rebase autoland to mozilla-central currently.
>
> We are working on this and gps is aware of this - but due to the US
> holiday today it might could take some time today to get that fixed.
>
> Thanks for you patience and sorry for the inconvenience.
>
> - Tomcat
>
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Xidorn Quan]

I hope it could support MSVC one day as well, and support distribute any
job to macOS machines as well.

In my case, I use Windows as my main development environment, and I have
a personally powerful enough MacBook Pro. (Actually I additionally have
a retired MBP which should still work.) And if it is possible to
distribute Windows builds to Linux machines, I would probably consider
purchasing another machine for Linux.

I would expect MSVC to be something not too hard to run with wine. When
I was in my university, I ran VC6 compiler on Linux to test my homework
without much effort. I guess the situation shouldn't be much worse with
VS2015. Creating the environment tarball may need some work, though.

- Xidorn

On Tue, Jul 5, 2016, at 07:36 AM, Benoit Girard wrote:
> In my case I'm noticing an improvement with my mac distributing jobs to a
> single Ubuntu machine but not compiling itself (Right now we don't
> support
> distributing mac jobs to other mac, primarily because we just want to
> maintain one homogeneous cluster).
> 
> On Mon, Jul 4, 2016 at 5:12 PM, Gijs Kruitbosch
> 
> wrote:
> 
> > On 04/07/2016 22:06, Benoit Girard wrote:
> >
> >> So to emphasize, if you compile a lot and only have one or two machines
> >> on your 100mps or 1gbps LAN you'll still see big benefits.
> >>
> >
> > I don't understand how this benefits anyone with just one machine (that's
> > compatible...) - there's no other machines to delegate compile tasks to (or
> > to fetch prebuilt blobs from). Can you clarify? Do you just mean "one extra
> > machine"? Am I misunderstanding how this works?
> >
> > ~ Gijs
> >
> >
> >
> >> On Mon, Jul 4, 2016 at 4:39 PM, Gijs Kruitbosch  >> >
> >> wrote:
> >>
> >> What about people not lucky enough to (regularly) work in an office,
> >>> including but not limited to our large number of volunteers? Do we intend
> >>> to set up something public for people to use?
> >>>
> >>> ~ Gijs
> >>>
> >>>
> >>> On 04/07/2016 20:09, Michael Layzell wrote:
> >>>
> >>> If you saw the platform lightning talk by Jeff and Ehsan in London, you
>  will know that in the Toronto office, we have set up a distributed
>  compiler
>  called `icecc`, which allows us to perform a clobber build of
>  mozilla-central in around 3:45. After some work, we have managed to get
>  it
>  so that macOS computers can also dispatch cross-compiled jobs to the
>  network, have streamlined the macOS install process, and have refined
>  the
>  documentation some more.
> 
>  If you are in the Toronto office, and running a macOS or Linux machine,
>  getting started using icecream is as easy as following the instructions
>  on
>  the wiki:
> 
> 
>  https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream
> 
>  If you are in another office, then I suggest that your office starts an
>  icecream cluster! Simply choose one linux desktop in the office, run the
>  scheduler on it, and put its IP in the Wiki, then everyone can connect
>  to
>  the network and get fast builds!
> 
>  If you have questions, myself, BenWa, and jeff are probably the ones to
>  talk to.
> 
> 
>  ___
> >>> dev-platform mailing list
> >>> dev-platform@lists.mozilla.org
> >>> https://lists.mozilla.org/listinfo/dev-platform
> >>>
> >>>
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Benoit Girard]

In my case I'm noticing an improvement with my mac distributing jobs to a
single Ubuntu machine but not compiling itself (Right now we don't support
distributing mac jobs to other mac, primarily because we just want to
maintain one homogeneous cluster).

On Mon, Jul 4, 2016 at 5:12 PM, Gijs Kruitbosch 
wrote:

> On 04/07/2016 22:06, Benoit Girard wrote:
>
>> So to emphasize, if you compile a lot and only have one or two machines
>> on your 100mps or 1gbps LAN you'll still see big benefits.
>>
>
> I don't understand how this benefits anyone with just one machine (that's
> compatible...) - there's no other machines to delegate compile tasks to (or
> to fetch prebuilt blobs from). Can you clarify? Do you just mean "one extra
> machine"? Am I misunderstanding how this works?
>
> ~ Gijs
>
>
>
>> On Mon, Jul 4, 2016 at 4:39 PM, Gijs Kruitbosch > >
>> wrote:
>>
>> What about people not lucky enough to (regularly) work in an office,
>>> including but not limited to our large number of volunteers? Do we intend
>>> to set up something public for people to use?
>>>
>>> ~ Gijs
>>>
>>>
>>> On 04/07/2016 20:09, Michael Layzell wrote:
>>>
>>> If you saw the platform lightning talk by Jeff and Ehsan in London, you
 will know that in the Toronto office, we have set up a distributed
 compiler
 called `icecc`, which allows us to perform a clobber build of
 mozilla-central in around 3:45. After some work, we have managed to get
 it
 so that macOS computers can also dispatch cross-compiled jobs to the
 network, have streamlined the macOS install process, and have refined
 the
 documentation some more.

 If you are in the Toronto office, and running a macOS or Linux machine,
 getting started using icecream is as easy as following the instructions
 on
 the wiki:


 https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

 If you are in another office, then I suggest that your office starts an
 icecream cluster! Simply choose one linux desktop in the office, run the
 scheduler on it, and put its IP in the Wiki, then everyone can connect
 to
 the network and get fast builds!

 If you have questions, myself, BenWa, and jeff are probably the ones to
 talk to.


 ___
>>> dev-platform mailing list
>>> dev-platform@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-platform
>>>
>>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Michael Layzell]

I'm pretty sure he means one extra machine. For example, if you have a
laptop and a desktop, just adding the desktop into the network at home will
still dramatically improve build times (I think).

On Mon, Jul 4, 2016 at 5:12 PM, Gijs Kruitbosch 
wrote:

> On 04/07/2016 22:06, Benoit Girard wrote:
>
>> So to emphasize, if you compile a lot and only have one or two machines
>> on your 100mps or 1gbps LAN you'll still see big benefits.
>>
>
> I don't understand how this benefits anyone with just one machine (that's
> compatible...) - there's no other machines to delegate compile tasks to (or
> to fetch prebuilt blobs from). Can you clarify? Do you just mean "one extra
> machine"? Am I misunderstanding how this works?
>
> ~ Gijs
>
>
>
>> On Mon, Jul 4, 2016 at 4:39 PM, Gijs Kruitbosch > >
>> wrote:
>>
>> What about people not lucky enough to (regularly) work in an office,
>>> including but not limited to our large number of volunteers? Do we intend
>>> to set up something public for people to use?
>>>
>>> ~ Gijs
>>>
>>>
>>> On 04/07/2016 20:09, Michael Layzell wrote:
>>>
>>> If you saw the platform lightning talk by Jeff and Ehsan in London, you
 will know that in the Toronto office, we have set up a distributed
 compiler
 called `icecc`, which allows us to perform a clobber build of
 mozilla-central in around 3:45. After some work, we have managed to get
 it
 so that macOS computers can also dispatch cross-compiled jobs to the
 network, have streamlined the macOS install process, and have refined
 the
 documentation some more.

 If you are in the Toronto office, and running a macOS or Linux machine,
 getting started using icecream is as easy as following the instructions
 on
 the wiki:


 https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

 If you are in another office, then I suggest that your office starts an
 icecream cluster! Simply choose one linux desktop in the office, run the
 scheduler on it, and put its IP in the Wiki, then everyone can connect
 to
 the network and get fast builds!

 If you have questions, myself, BenWa, and jeff are probably the ones to
 talk to.


 ___
>>> dev-platform mailing list
>>> dev-platform@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-platform
>>>
>>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Gijs Kruitbosch]

On 04/07/2016 22:06, Benoit Girard wrote:

So to emphasize, if you compile a lot and only have one or two machines on your 
100mps or 1gbps LAN you'll still see big benefits.


I don't understand how this benefits anyone with just one machine 
(that's compatible...) - there's no other machines to delegate compile 
tasks to (or to fetch prebuilt blobs from). Can you clarify? Do you just 
mean "one extra machine"? Am I misunderstanding how this works?


~ Gijs




On Mon, Jul 4, 2016 at 4:39 PM, Gijs Kruitbosch 
wrote:


What about people not lucky enough to (regularly) work in an office,
including but not limited to our large number of volunteers? Do we intend
to set up something public for people to use?

~ Gijs


On 04/07/2016 20:09, Michael Layzell wrote:


If you saw the platform lightning talk by Jeff and Ehsan in London, you
will know that in the Toronto office, we have set up a distributed
compiler
called `icecc`, which allows us to perform a clobber build of
mozilla-central in around 3:45. After some work, we have managed to get it
so that macOS computers can also dispatch cross-compiled jobs to the
network, have streamlined the macOS install process, and have refined the
documentation some more.

If you are in the Toronto office, and running a macOS or Linux machine,
getting started using icecream is as easy as following the instructions on
the wiki:

https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

If you are in another office, then I suggest that your office starts an
icecream cluster! Simply choose one linux desktop in the office, run the
scheduler on it, and put its IP in the Wiki, then everyone can connect to
the network and get fast builds!

If you have questions, myself, BenWa, and jeff are probably the ones to
talk to.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Benoit Girard]

This barely works in a office with 10MB/sec wireless uplink. Ideally you
want machines to be accessible on a gigabit LAN. It's more about bandwidth
throughput than latency AFAIK. i.e. can you *upload* dozens of 2-4MB
compressed pre-processed file faster than you compile it? I'd imagine
unless you can get reliable 50MB/sec upload throughput then you probably
wont benefit from connecting to a remote cluster.

However the good news is you can see a lot of benefits from having a
network of just one machine! In my case my Linux desktop can compile a mac
build faster than my top of the line 2013 macbook pro. and with a network
of 2 machines it's drastically faster. A cluster of 12 machines is nice,
but you're getting diminishing returns on that until the build system gets
better.

I'd imagine distributed object caching will have a similar bandwidth
problem, however users tend to have better download speeds than upload
speeds.

So to emphasize, if you compile a lot and only have one or two machines on
your 100mps or 1gbps LAN you'll still see big benefits.

On Mon, Jul 4, 2016 at 4:39 PM, Gijs Kruitbosch 
wrote:

> What about people not lucky enough to (regularly) work in an office,
> including but not limited to our large number of volunteers? Do we intend
> to set up something public for people to use?
>
> ~ Gijs
>
>
> On 04/07/2016 20:09, Michael Layzell wrote:
>
>> If you saw the platform lightning talk by Jeff and Ehsan in London, you
>> will know that in the Toronto office, we have set up a distributed
>> compiler
>> called `icecc`, which allows us to perform a clobber build of
>> mozilla-central in around 3:45. After some work, we have managed to get it
>> so that macOS computers can also dispatch cross-compiled jobs to the
>> network, have streamlined the macOS install process, and have refined the
>> documentation some more.
>>
>> If you are in the Toronto office, and running a macOS or Linux machine,
>> getting started using icecream is as easy as following the instructions on
>> the wiki:
>>
>> https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream
>>
>> If you are in another office, then I suggest that your office starts an
>> icecream cluster! Simply choose one linux desktop in the office, run the
>> scheduler on it, and put its IP in the Wiki, then everyone can connect to
>> the network and get fast builds!
>>
>> If you have questions, myself, BenWa, and jeff are probably the ones to
>> talk to.
>>
>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[David Burns]

Yes!

Part of the build project work that I regularly email this list[1] we have
it on our roadmap to have the same distributed cache that we use in
automation available for engineers who are working on C++ code. We have
completed our rewrite and will be putting the initial work through try over
the next fortnight to make sure we havent regressed anything. After that we
will be working towards making it available to engineers before the end of
Q3 (at least on one platform).

David


[1]
https://groups.google.com/forum/#!topicsearchin/mozilla.dev.platform/Build$20System$20Project$20

On 4 July 2016 at 21:39, Gijs Kruitbosch  wrote:

> What about people not lucky enough to (regularly) work in an office,
> including but not limited to our large number of volunteers? Do we intend
> to set up something public for people to use?
>
> ~ Gijs
>
>
> On 04/07/2016 20:09, Michael Layzell wrote:
>
>> If you saw the platform lightning talk by Jeff and Ehsan in London, you
>> will know that in the Toronto office, we have set up a distributed
>> compiler
>> called `icecc`, which allows us to perform a clobber build of
>> mozilla-central in around 3:45. After some work, we have managed to get it
>> so that macOS computers can also dispatch cross-compiled jobs to the
>> network, have streamlined the macOS install process, and have refined the
>> documentation some more.
>>
>> If you are in the Toronto office, and running a macOS or Linux machine,
>> getting started using icecream is as easy as following the instructions on
>> the wiki:
>>
>> https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream
>>
>> If you are in another office, then I suggest that your office starts an
>> icecream cluster! Simply choose one linux desktop in the office, run the
>> scheduler on it, and put its IP in the Wiki, then everyone can connect to
>> the network and get fast builds!
>>
>> If you have questions, myself, BenWa, and jeff are probably the ones to
>> talk to.
>>
>>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Ralph Giles]

On Mon, Jul 4, 2016 at 1:39 PM, Gijs Kruitbosch
 wrote:

> What about people not lucky enough to (regularly) work in an office,
> including but not limited to our large number of volunteers? Do we intend to
> set up something public for people to use?

By all accounts, the available distributed compilers aren't very good
at hiding latency. The build server needs to be on the local lan to
help much.

More generally, we have artifact builds for developers who don't need
the change C++ code, and there's some experiments happening to see if
the build can pull smaller pieces from the s3 build cache for those
who do.

 -r
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Faster gecko builds with IceCC on Mac and Linux

[Gijs Kruitbosch]

What about people not lucky enough to (regularly) work in an office, 
including but not limited to our large number of volunteers? Do we 
intend to set up something public for people to use?


~ Gijs

On 04/07/2016 20:09, Michael Layzell wrote:

If you saw the platform lightning talk by Jeff and Ehsan in London, you
will know that in the Toronto office, we have set up a distributed compiler
called `icecc`, which allows us to perform a clobber build of
mozilla-central in around 3:45. After some work, we have managed to get it
so that macOS computers can also dispatch cross-compiled jobs to the
network, have streamlined the macOS install process, and have refined the
documentation some more.

If you are in the Toronto office, and running a macOS or Linux machine,
getting started using icecream is as easy as following the instructions on
the wiki:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

If you are in another office, then I suggest that your office starts an
icecream cluster! Simply choose one linux desktop in the office, run the
scheduler on it, and put its IP in the Wiki, then everyone can connect to
the network and get fast builds!

If you have questions, myself, BenWa, and jeff are probably the ones to
talk to.



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: URL translation map Re: MXR permanently offline, please transition to DXR

[Mike Hoye]

On 2016-06-30 9:50 PM, Karl Dubost wrote:

Gregory,

Le 1 juil. 2016 à 09:33, Gregory Szorc  a écrit :

I want the site to publish a "URL translation map"
for URL patterns so whole URL namespaces can be bulk updated.

Interesting idea.

Probably something to explain in a wiki page somewhere on 
https://wiki.mozilla.org/ with use cases and examples how you would see it 
working.

This sounds like Apache's "Redirect" or "RedirectMatch" config options.

More powerfully, Apache's mod_rewrite can be made to return a 301 and 
redirect you to one of a series of regex-modified rewrites of the 
inbound URL.




- mhoye
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Faster gecko builds with IceCC on Mac and Linux

[Michael Layzell]

If you saw the platform lightning talk by Jeff and Ehsan in London, you
will know that in the Toronto office, we have set up a distributed compiler
called `icecc`, which allows us to perform a clobber build of
mozilla-central in around 3:45. After some work, we have managed to get it
so that macOS computers can also dispatch cross-compiled jobs to the
network, have streamlined the macOS install process, and have refined the
documentation some more.

If you are in the Toronto office, and running a macOS or Linux machine,
getting started using icecream is as easy as following the instructions on
the wiki:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Using_Icecream

If you are in another office, then I suggest that your office starts an
icecream cluster! Simply choose one linux desktop in the office, run the
scheduler on it, and put its IP in the Wiki, then everyone can connect to
the network and get fast builds!

If you have questions, myself, BenWa, and jeff are probably the ones to
talk to.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: The integration/autoland repo

[Henrik Skupin]

Gijs Kruitbosch wrote on 07/01/2016 11:47 PM:
> On 01/07/2016 20:52, Henrik Skupin wrote:
>> I do not see any single merge of autoland to mozilla-central
> 
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-central=61ed5c0d64195c58de57489147046aeaf14252d3
> 
> is a merge from autoland to m-c earlier today, and you are CC'd on the 
> bug...

I was able to sort out the issue with Carsten on IRC. He just posted a
message to this list about the current issues with autoland->mc updates.

-- 
Henrik
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform