MinGW Target Re-Enabled on TaskCluster
Previous Thread: https://groups.google.com/forum/#!topic/mozilla.dev.platform/r3mYWbb42pM As of a few hours ago, there is a new Tier 2 MinGW build on TaskCluster. It's in the 'Windows MinGW all' line, with the group WMC64 for 'Windows MinGW Clang x64'. The MinGW builds are part of the Tor Uplift project, where we work closely with Tor to upstream their patches and in general move Tor Browser closer and closer to 'Firefox with Addons and Pref Changes' - instead of a fork of Firefox with a bunch of custom patches. Tor is currently using ESR releases: the bump to ESR60 (which occurred last week! It's a huge release for them with a lot of UX improvements: https://blog.torproject.org/new-release-tor-browser-80 ) was a lot smoother - build-wise - than other bumps because we had the MinGW build running in TC. Without keeping the MinGW build working, every ESR they have to go through a potentially colossal amount of work on a short timescale to get the build working under MinGW again. By minimizing their effort in fixing the build, rebasing patches and the like - we can free up their limited resources to continue to research and experiment on privacy technology like First Party Isolation and Fingerprinting Resistance. Now, you might be wondering "I thought we had a MinGW build?" We did. But we had to disable it. Shortly after 60 went to beta, we removed support for --disable-stylo. Stylo requires libclang to build; and getting that working with MinGW was very complicated (see https://bugzilla.mozilla.org/show_bug.cgi?id=1390583) so MinGW fell behind and had to be disabled. However, thanks (again) to the efforts of all the reviewers, build peers, and especially Jacek Caban - we've been able to re-enable a MinGW build. We are now building with clang using the MinGW headers. (Previously it was gcc.) I believe we're the first 'real' project that is building with MinGW-clang, and Tor will be the first major project to ship it (but I could be wrong there.) In configure and moz.build files, CC_TYPE will be 'clang-cl' for our normal Windows builds (which build on Windows) and will be 'clang' for the MinGW-clang builds (which build on Linux). There are still some outstanding issues: I hope to land a x86 build, we need to remove some of the --disable-foo flags, and once ESR60 gets a NSS uplift I intend the backport the jobs there also. We hope to get pdbs generating so we can debug easier - major appreciation to David Major and Bob Owen who both have debugged pretty ugly crashes without symbols. Eventually, I'd like to enable a limited set of tests to catch browser crashes. Because there is no path forward for getting the MinGW gcc builds re-enabled (nor anyone who wants to work on it...) I intend to remove the (disabled) build jobs from the tree also. And finally I need to document how to get a local build environment for it. MinGW is Tier 2, and sometimes breaking it is unavoidable because the fix needs to happen upstream in MinGW. Other times breaking it is avoidable, and one just needs to special-case it. The Tor Uplift team and Tor themselves greatly appreciate all of your efforts to keep the build green. As always, if you have a MinGW question or a build error you can't quite understand - feel free to reach out to me via irc or email. -tom ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: MinGW Target Re-Enabled on TaskCluster
Tom Ritter: > Previous Thread: > https://groups.google.com/forum/#!topic/mozilla.dev.platform/r3mYWbb42pM > > As of a few hours ago, there is a new Tier 2 MinGW build on TaskCluster. > It's in the 'Windows MinGW all' line, with the group WMC64 for 'Windows > MinGW Clang x64'. > Yay, thanks so much for that. I am pretty sure it will help us saving a ton of work once we plan to switch to the mingw-w64/clang toolchain, as your work on integrating the mingw-w64/gcc toolchain into TaskCluster already did. > > The MinGW builds are part of the Tor Uplift project, where we work closely > with Tor to upstream their patches and in general move Tor Browser closer > and closer to 'Firefox with Addons and Pref Changes' - instead of a fork of > Firefox with a bunch of custom patches. > > Tor is currently using ESR releases: the bump to ESR60 (which occurred last > week! It's a huge release for them with a lot of UX improvements: > https://blog.torproject.org/new-release-tor-browser-80 ) was a lot smoother > - build-wise - than other bumps because we had the MinGW build running in > TC. Without keeping the MinGW build working, every ESR they have to go > through a potentially colossal amount of work on a short timescale to get > the build working under MinGW again. By minimizing their effort in fixing > the build, rebasing patches and the like - we can free up their limited > resources to continue to research and experiment on privacy technology like > First Party Isolation and Fingerprinting Resistance. Let me jump in here as the one responsible for coordinating our efforts to switch Tor Browser to ESR 60, and the one working on the mingw-w64-based toolchain upgrades in the past. Having to deal with the mingw-w64 upgrade for a new ESR has been one of the top three pain points in previous years. I usually started weeks before we began with the "official" transition part, to estimate how much is broken and how complex fixes would be by bisecting and filing bugs at bugzilla. Thanks to the help of Jacek Caban we always had the toolchain ready before we needed to switch, but it was pretty close sometimes. All this work has essentially not been needed anymore this time, thanks to the effort of Tom and others, and we are very grateful for that. Given that background I am more than happy to see the results of the continued work in that direction with the arrival of the mingw-w64/clang-based Windows build on Taskcluster. Thanks Mozilla and thanks to those engineers in particular who helped (and still help) with that effort. Georg signature.asc Description: OpenPGP digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: MinGW Target Re-Enabled on TaskCluster
On Wed, Sep 12, 2018, at 1:09 AM, Tom Ritter wrote: > Previous Thread: > https://groups.google.com/forum/#!topic/mozilla.dev.platform/r3mYWbb42pM > > As of a few hours ago, there is a new Tier 2 MinGW build on TaskCluster. > It's in the 'Windows MinGW all' line, with the group WMC64 for 'Windows > MinGW Clang x64'. > > > The MinGW builds are part of the Tor Uplift project, where we work closely > with Tor to upstream their patches and in general move Tor Browser closer > and closer to 'Firefox with Addons and Pref Changes' - instead of a fork of > Firefox with a bunch of custom patches. Big thanks to Tom and everyone else that helped for doing all of this hard work twice: once to stand up the initial mingw build, and again after we broke that one by requiring stylo. The work that the Tor project does is incredibly important, and making it easier for them to ship Tor Browser by not having to spend a bunch of time fixing all the things we broke upstream should have a huge impact. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: MinGW Target Re-Enabled on TaskCluster
On Wed, Sep 12, 2018 at 12:09 AM, Tom Ritter wrote: > However, thanks (again) to the efforts of all the reviewers, build peers, > and especially Jacek Caban - we've been able to re-enable a MinGW build. > We are now building with clang using the MinGW headers. (Previously it was > gcc.) I believe we're the first 'real' project that is building with > MinGW-clang, and Tor will be the first major project to ship it (but I > could be wrong there.) > I forgot to add a couple of really big thanks that are large enough to prompt a followup-email: Georg did a lot of the initial Stylo build investigation; and Martin Storsjö also helped on Stylo and was either the creator of the mingw-clang toolchain or has worked on it enough and done enough fixes to it that I thought he was. -tom ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: MinGW Target Re-Enabled on TaskCluster
Thank you. This will make my development flow much easier. On Wed, Sep 12, 2018 at 12:09:36AM -0500, Tom Ritter wrote: Previous Thread: https://groups.google.com/forum/#!topic/mozilla.dev.platform/r3mYWbb42pM As of a few hours ago, there is a new Tier 2 MinGW build on TaskCluster. It's in the 'Windows MinGW all' line, with the group WMC64 for 'Windows MinGW Clang x64'. The MinGW builds are part of the Tor Uplift project, where we work closely with Tor to upstream their patches and in general move Tor Browser closer and closer to 'Firefox with Addons and Pref Changes' - instead of a fork of Firefox with a bunch of custom patches. Tor is currently using ESR releases: the bump to ESR60 (which occurred last week! It's a huge release for them with a lot of UX improvements: https://blog.torproject.org/new-release-tor-browser-80 ) was a lot smoother - build-wise - than other bumps because we had the MinGW build running in TC. Without keeping the MinGW build working, every ESR they have to go through a potentially colossal amount of work on a short timescale to get the build working under MinGW again. By minimizing their effort in fixing the build, rebasing patches and the like - we can free up their limited resources to continue to research and experiment on privacy technology like First Party Isolation and Fingerprinting Resistance. Now, you might be wondering "I thought we had a MinGW build?" We did. But we had to disable it. Shortly after 60 went to beta, we removed support for --disable-stylo. Stylo requires libclang to build; and getting that working with MinGW was very complicated (see https://bugzilla.mozilla.org/show_bug.cgi?id=1390583) so MinGW fell behind and had to be disabled. However, thanks (again) to the efforts of all the reviewers, build peers, and especially Jacek Caban - we've been able to re-enable a MinGW build. We are now building with clang using the MinGW headers. (Previously it was gcc.) I believe we're the first 'real' project that is building with MinGW-clang, and Tor will be the first major project to ship it (but I could be wrong there.) In configure and moz.build files, CC_TYPE will be 'clang-cl' for our normal Windows builds (which build on Windows) and will be 'clang' for the MinGW-clang builds (which build on Linux). There are still some outstanding issues: I hope to land a x86 build, we need to remove some of the --disable-foo flags, and once ESR60 gets a NSS uplift I intend the backport the jobs there also. We hope to get pdbs generating so we can debug easier - major appreciation to David Major and Bob Owen who both have debugged pretty ugly crashes without symbols. Eventually, I'd like to enable a limited set of tests to catch browser crashes. Because there is no path forward for getting the MinGW gcc builds re-enabled (nor anyone who wants to work on it...) I intend to remove the (disabled) build jobs from the tree also. And finally I need to document how to get a local build environment for it. MinGW is Tier 2, and sometimes breaking it is unavoidable because the fix needs to happen upstream in MinGW. Other times breaking it is avoidable, and one just needs to special-case it. The Tor Uplift team and Tor themselves greatly appreciate all of your efforts to keep the build green. As always, if you have a MinGW question or a build error you can't quite understand - feel free to reach out to me via irc or email. -- It is practically impossible to teach good programming style to students that have had prior exposure to Basic; as potential programmers they are mentally mutilated beyond hope of regeneration. --Edsger W. Dijkstra ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
