Re: Removing tinderbox-builds from archive.mozilla.org
On 5/11/18 11:28 PM, Gregory Szorc wrote: You could trigger dozens of builds on Taskcluster and they would all come back in the time it takes a single build to run. This is doable, and basically corresponds to doing N-ary search for N>2. With some tooling support so you don't have to pick the revisions by hand, could work. Or you codify the thing you are testing for, submit it to bisect-in-the-cloud, and it does everything for you and reports on the results. It still might take dozens of minutes to a few hours. But at least you wouldn't be burdened with the overhead of doing everything manually. Yeah, the typical thing that needs testing during a bisect is some random website, that's a bit hard to fully automate... I agree that for cases we can turn into an automated test we should be able to do way better. The latency is not that big a deal usually if it's truly zero-attention. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removing tinderbox-builds from archive.mozilla.org
On Fri, May 11, 2018 at 7:47 PM, Boris Zbarsky wrote: > On 5/11/18 7:06 PM, Gregory Szorc wrote: > >> Artifact retention and expiration boils down to a >> trade-off between the cost of storage and the convenience of accessing >> something immediately (as opposed to waiting several dozen minutes to >> populate the cache). >> > > Just to be clear, when doing a bisect, one _can_ just deal with local > builds. But the point is that then it takes tens of minutes per build as > you point out. So a bisect task that might otherwise take 10-15 minutes > total (1 minute per downloaded build) ends up taking hours... And this is where bisect-in-the-cloud would come in. You could trigger dozens of builds on Taskcluster and they would all come back in the time it takes a single build to run. Or you codify the thing you are testing for, submit it to bisect-in-the-cloud, and it does everything for you and reports on the results. It still might take dozens of minutes to a few hours. But at least you wouldn't be burdened with the overhead of doing everything manually. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removing tinderbox-builds from archive.mozilla.org
On 5/11/18 7:06 PM, Gregory Szorc wrote: Artifact retention and expiration boils down to a trade-off between the cost of storage and the convenience of accessing something immediately (as opposed to waiting several dozen minutes to populate the cache). Just to be clear, when doing a bisect, one _can_ just deal with local builds. But the point is that then it takes tens of minutes per build as you point out. So a bisect task that might otherwise take 10-15 minutes total (1 minute per downloaded build) ends up taking hours... -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Update on rustc/clang goodness
On Thu, May 10, 2018 at 5:35 PM, Anthony Jones wrote: > You may already know that the Low-Level Tools team support important tools > and code infrastructure. Lately we’ve also been improving our rustc/clang > (LLVM) story and I’d like bring everyone up to date. > > There are a lot of important and interesting things going on: > > * Michael Woerister and Nathan Froyd recently (mid-March) enabled Rust > incremental compilation for Firefox developers > * Michael is experimenting with cross language inlining[1] > (rustc/clang) using LTO > * Preliminary results show compiling LLVM with clang and using LTO > on rustc improves stylo compilation time by around 15% [on-my-machine > measurements] > * LTO requires more work to support Firefox > * Nick Nethercote has started speeding up rustc[2] and has already had > a number of wins > * David Major has got Windows clang builds working with green tests > * LTO is still buggy (but works well enough to run benchmarks) and > PGO won’t run on Windows > * Win32 - clang with LTO w/o PGO is marginally faster than MSVC > with LTO/PGO > * Win64 - clang with LTO w/o PGO is ~5-10% slower than MSVC with > LTO/PGO > * Windows users can build locally with clang[3] > * Mike Hommey is tracking improvements and regressions in the Rust > compiler (performance regression[4], crash reporting[5] and more crash > reporting[6]) > * Tom Tromey is continuing to work on first class Rust support in GDB > and LLDB > > Note: this a summary of things rustc/clang stuff that is happening only. > > Ultimately, I’d like to see us standardise on clang on all platforms > because it makes Rust/C++ integration better as well as making things > simpler for developers and build system maintainers. We’ll get more done if > we make our own lives simpler. > > I have some specific requests for you: > > Let me know if you have specific Firefox related cases where Rust is > slowing you down (thanks Jeff [7]) > Cross language inlining is coming - avoid duplication between Rust and > C++ in the name of performance > Do developer builds with clang > This is all fantastic work! Speaking as a build system maintainer, I would also like to see us standardize on Clang on all platform for various reasons, especially simplicity. Speaking as an open source advocate, I can't wait to be using an open source toolchain on Windows. Speaking as someone who cares about developer ergonomics, I can't wait until we reduce the number of toolchains we support as tier 1 platforms so we all don't have to spend so much effort making things work on N+1 platforms. Reducing the number of toolchains we need to worry about is very much analogous to the security benefits of reducing attack surface area. I do have some concerns about dropping support for non-Clang toolchains and what that will mean for various people who rely on them. But I think "switching toolchains" is a separate discussion from "dropping toolchains" and we can largely defer those discussions. As far as switching to Clang for all builds we ship to users is concerned, I'm all aboard from the build system side. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Removing tinderbox-builds from archive.mozilla.org
On Wed, May 9, 2018 at 11:01 AM, Ted Mielczarek wrote: > On Wed, May 9, 2018, at 1:11 PM, L. David Baron wrote: > > > mozregression won't be able to bisect into inbound branches then, but I > > > believe we've always been expiring build artifacts created from > integration > > > branches after a few months in any case. > > > > > > My impression was that people use mozregression primarily for tracking > down > > > relatively recent regressions. Please correct me if I'm wrong. > > > > It's useful for tracking down regressions no matter how old the > > regression is; I pretty regularly see mozregression finding useful > > data on bugs that regressed multiple years ago. > > To be clear here--we still have an archive of nightly builds dating back > to 2004, so you should be able to bisect to a single day using that. We > haven't ever had a great policy for retaining individual CI builds like > these tinderbox-builds. They're definitely useful, and storage is not that > expensive, but given the number of build configurations we produce nowadays > and the volume of changes being pushed we can't archive everything forever. It's worth noting that once builds are deterministic, a build system is effectively a highly advanced caching mechanism. It follows that cache eviction is therefore a tolerable problem: if the entry isn't in the cache, you just build again! Artifact retention and expiration boils down to a trade-off between the cost of storage and the convenience of accessing something immediately (as opposed to waiting several dozen minutes to populate the cache). The good news is that Linux Firefox builds have been effectively deterministic (modulo PGO and some minor build details like the build time) for several months now (thanks, glandium!). And moving to Clang on all platforms will make it easier to achieve deterministic builds on other platforms. The bad news is we still have many areas of CI that are not hermetic and attempts to retrigger Firefox build tasks in the future have a very high possibility of failing for numerous reasons (e.g. some dependent task of the build hits a 3rd party server that is no longer available or has deleted a file). In other words, our CI results may not be reproducible in the future. So if we delete an artifact, even though the build is deterministic, we may not have all the inputs to reconstruct that result. Making CI hermetic and reproducible far in the future is a hard problem. There are esoteric failure scenarios like "what if we need to fetch content from a server in 2030 but TLS 1.2 has been disabled due to a critical vulnerability and code in the hermetic build task doesn't support TLS 1.3." In order to realistically achieve reproducible builds in the future, we need to store *all* inputs somewhere reliable where they will always be available. Version control is one possibility. A content-indexed service like tooltool is another. (At Google, they check in the source code for Clang, glibc, binutils, Linux, etc into version control so all they need is a version revision and a bootstrap compiler (which I also suspect they check into the monorepo) to rebuild the world from source.) What I'm trying to say is we're making strides towards making builds deterministic and reproducible far in the future. So hopefully in a few years we won't need to be concerned about deleting old data because our answer will be "we can easily reproduce it at any time." ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: PSA: Avoid Visual Studio 2017 15.7.0
On 2018-05-08 6:40 a.m., Ryan VanderMeulen wrote: Yesterday, Microsoft released Visual Studio 2017 15.7.0. Unfortunately, it is currently not usable for building Firefox due to bug 1458247 (internal compiler errors in WebRTC code). The bug was already reported and confirmed upstream during the 15.7 preview cycle, but unfortunately the final release still shipped with the bug present. At this point, there are no workarounds available for this issue, so avoid the update if you want to be able to continue building Firefox. If you need to update to 15.6 but hadn't yet, you can download old versions at https://docs.microsoft.com/en-us/visualstudio/productinfo/installing-an-earlier-release-of-vs2017 - MattN ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Update on rustc/clang goodness
On 5/11/18 12:16 PM, Anthony Jones wrote: On Friday, 11 May 2018 21:14:21 UTC+12, Sylvestre Ledru wrote: Do we have a list of blockers (or meta bug) to make the switch on Linux? We're already using clang for ASAN builds on Linux and passing tests. We'll want to switch to clang on Linux pretty soon after getting it running on Windows, it not at the same time. Once we have cross language inlining (through LTO), we can save a lot of complexity by relying on it for performance. We're making good progress on Windows but I don't have a timeline because of dependencies on third parties. Note that we have at least one dependency on keeping gcc working: the JavaScript GC rooting hazard static analysis. The first layer uses a gcc plugin and is very tied to the internal gcc data structures. Rewriting it for clang should be possible, but is not currently scheduled and would likely be a multiple month project. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Update on rustc/clang goodness
On Friday, 11 May 2018 21:14:21 UTC+12, Sylvestre Ledru wrote: > Do we have a list of blockers (or meta bug) to make the switch on Linux? We're already using clang for ASAN builds on Linux and passing tests. We'll want to switch to clang on Linux pretty soon after getting it running on Windows, it not at the same time. Once we have cross language inlining (through LTO), we can save a lot of complexity by relying on it for performance. We're making good progress on Windows but I don't have a timeline because of dependencies on third parties. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Update on rustc/clang goodness
Hello, Bravo for all this work. Having a single opensource toolchain would be a great gain for us! On 11/05/2018 02:35, Anthony Jones wrote: > You may already know that the Low-Level Tools team support important tools > and code infrastructure. Lately we’ve also been improving our rustc/clang > (LLVM) story and I’d like bring everyone up to date. > > There are a lot of important and interesting things going on: [...] > Note: this a summary of things rustc/clang stuff that is happening only. > > Ultimately, I’d like to see us standardise on clang on all platforms because > it makes Rust/C++ integration better as well as making things simpler for > developers and build system maintainers. We’ll get more done if we make our > own lives simpler. Do we have a list of blockers (or meta bug) to make the switch on Linux? Thanks, Sylvestre ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
