Welcome to the Q2 edition of the Engineering Effectiveness Newsletter! The Engineering Effectiveness org makes it easy to develop, test and release Mozilla software at scale. See below for some highlights, then read on for more detailed info! Highlights
- On the 30th April 2025 we successfully migrated Firefox’s source of truth from Mercurial to a new repo on Github <https://github.com/mozilla-firefox/firefox> 🎉 - This was a massive effort that spanned many people and teams. It took meticulous planning and preparation from as early as 2023. - Please see the detailed updates for some of the specific contributions that made this migration possible! - The last vestiges of old-configure were removed <https://bugzilla.mozilla.org/show_bug.cgi?id=1957023>! This marks the end of a 9 year journey <https://bugzilla.mozilla.org/show_bug.cgi?id=1247781> to move build configuration from a shell script to Python. - The Bugzilla ETL <https://bugzilla.mozilla.org/show_bug.cgi?id=1922985> was completed! This provides daily snapshots of Bugzilla data in BigQuery <https://docs.google.com/document/d/1vNdb3Vg2E9hzFzkpW_WYSRHN5f63-Kz21nUNuv4w6u8/>, opening the door to many new and exciting tooling opportunities. - Several improvements to our Python integration in mach: - Bumped the minimum Python version <https://bugzilla.mozilla.org/show_bug.cgi?id=1966763> from 3.8 to 3.9. - Now using uv <https://docs.astral.sh/uv/> for all Python package installation and vendoring across mach and mozharness <https://phabricator.services.mozilla.com/D251223>. This results in huge performance improvements locally and in CI. - Tooling to verify and fix the consistency and correctness <https://phabricator.services.mozilla.com/D250374> of our vendored packages and the commands that use them. - Lots of tooling support for Jujutsu <https://jj-vcs.github.io/jj/latest/>, a modern git compatible vcs with strong roots in Mercurial. - New documentation on using Jujutsu for Firefox development <https://firefox-source-docs.mozilla.org/contributing/jujutsu.html>. - Moz-phab now supports Jujutsu <https://bugzilla.mozilla.org/show_bug.cgi?id=1956345>, which should be released soon with moz-phab 2.0. - A Jujutsu implementation for mozversioncontrol <https://bugzilla.mozilla.org/show_bug.cgi?id=1929372>. This means any tooling that uses mozversioncontrol should in theory support Jujutsu repos for free. - Bootstrap support <https://bugzilla.mozilla.org/show_bug.cgi?id=1964189>via ./mach vcs-setup –vcs=jj Contributors - :anutrix and :jadavila9 have been steadily chipping away at removing usages of six <https://bugzilla.mozilla.org/show_bug.cgi?id=1714690> and making mach use orjson when possible <https://bugzilla.mozilla.org/show_bug.cgi?id=1970521>. Detailed Project UpdatesBugzilla and Bugbug - Dave Lawrence completed the Bugzilla ETL <https://bugzilla.mozilla.org/show_bug.cgi?id=1922985>! This provides daily snapshots of Bugzilla data in Big Query <https://docs.google.com/document/d/1vNdb3Vg2E9hzFzkpW_WYSRHN5f63-Kz21nUNuv4w6u8/> . - Suhaib Mujahid made many improvements to the Review-Helper <https://docs.google.com/document/u/1/d/e/2PACX-1vRuiuBsOxq5VOgpJHSFdOXxBo2yxFC9z0ylpD8CwbGviLcD4Ea3geTx55AQnCCF06tcg2JgmGJm40mG/pub> Firefox addon: - Learn continuously from both approved and rejected comments, which will help avoid suggesting comments similar to those that were rejected. - Approved comments are now posted immediately, without waiting for feedback on all comments. - Each suggested comment now has a justification explaining why it was recommended. - Added a new option in the add-on preferences to control verbosity levels. Build System and Mach Environment - Alex Hochheiden updated mach configure to automatically update Android dependencies <https://bugzilla.mozilla.org/show_bug.cgi?id=1686880> (SDK, NDK, build-tools, emulator, etc.) - Alex Hochheiden added ./mach generate-python-lockfiles to allow for quickly verifying site compatibility after ./mach vendor python or adding PyPi dependencies to <site>.txt files <https://searchfox.org/mozilla-central/source/python/sites>. - Also added the python-sites <https://phabricator.services.mozilla.com/D250374> lint to automatically fix common mistakes. - Alex Hochheiden enabled uv in mozharness <https://phabricator.services.mozilla.com/D251223> for faster virtualenv setup/pip installs in CI. - Alex Hochheiden released MozillaBuild 4.2 <https://groups.google.com/a/mozilla.org/g/dev-platform/c/JI3ZWLINqx4>. - Alex Hochheiden bumped the minimum Python version <https://bugzilla.mozilla.org/show_bug.cgi?id=1966763> for mach to 3.9. - Serge Guelton removed the last vestiges of old-configure <https://bugzilla.mozilla.org/show_bug.cgi?id=1957023> 🎉. This marks the end of a 9 year journey <https://bugzilla.mozilla.org/show_bug.cgi?id=1247781> to move build configuration from a shell script to Python. - Serge Guelton reduced the number of symbols we export on Android <https://bugzilla.mozilla.org/show_bug.cgi?id=1841946> - which resulted in big wins in terms of binary size, startup time, and resident memory. - Serge Guelton added ONNX runtime support <https://bugzilla.mozilla.org/show_bug.cgi?id=1959422>, in support of AI workflows. Crash Management - Alex Franchuk added support <https://bugzilla.mozilla.org/show_bug.cgi?id=1866863> for a Necko-based network transport layer in the crash reporter client, allowing the client to take advantage of user-configured network settings (e.g., proxies). - Suhaib Mujahid improved BugBot to avoid filing bugs on Bugzilla for crashes caused by broken CPUs <https://github.com/mozilla/bugbot/issues/2659>. Firefox-CI, Taskcluster and Treeherder - Matt Boris and Jonathan Moss migrated Firefox Translations training tasks to a modern, dockerized worker image <https://github.com/mozilla/translations/pull/1154>, which improves developer experience and ensures these tasks stay up to date with the latest fixes and improvements. - Andrew Halberstadt created Firefox-CI dashboards tracking average pending duration <https://mozilla.cloud.looker.com/dashboards/2401>, task duration by date <https://mozilla.cloud.looker.com/dashboards/2401> and more <https://mozilla.cloud.looker.com/boards/156>. - Andrew Halberstadt re-wrote the Decision task to use orjson everywhere <https://bugzilla.mozilla.org/show_bug.cgi?id=1969023>, shaving ~15 seconds off of Decision task runtimes. Phabricator , moz-phab, and Lando - Erich Gubler added Jujutsu support to moz-phab <https://bugzilla.mozilla.org/show_bug.cgi?id=1956345>. - Paul Adenot added a moz-phab abandon command to abandon revisions from the command line. - Claudia Pellegrino and Kagami Rosylight removed usages of the pkg_resources package in moz-phab to resolve deprecation warnings during install. - Connor Sheehan improved moz-phab’s base commit detection by looking for the first commit not present on a remote branch. - Connor Sheehan improved Lando’s detection of pre-initialized repos, bringing the worker startup time down from 20 minutes to 10 seconds. - Olivier Mehani enabled the diff inspection checks in Lando, which replaced many of the functions of the hg.mozilla.org server-side hooks. Connor Sheehan later added a flag to disable the hooks in case pushes are incorrectly blocked by the hooks. - Moz-phab had several new releases (1.9.0 <https://discourse.mozilla.org/t/moz-phab-1-9-0-released/142913>, 1.10.0 <https://discourse.mozilla.org/t/moz-phab-1-10-0-released/143926>, 1.11.0 <https://discourse.mozilla.org/t/mozphab-1-11-0-released/144326>) Release Engineering and Release Management - Ben Hearsum migrated CI & release automation that pushes changes to the Gecko repo to use Lando instead of pushing directly <https://bugzilla.mozilla.org/show_bug.cgi?id=1948315>. - Heitor Neiva and Ryan Curran deployed <https://bugzilla.mozilla.org/show_bug.cgi?id=1968561> a new [M2] Mac Signer pool with updated host OS and tooling. - Andrew Halberstadt made several improvements to the System Addon release pipeline: - Ensured folks actually working on the addons are the ones that sign off <https://bugzilla.mozilla.org/show_bug.cgi?id=1951760>. - Added ability to build and ship System Addons directly out of firefox-main <https://bugzilla.mozilla.org/show_bug.cgi?id=1927754>. - Reduced number of phases <https://github.com/mozilla-extensions/xpi-manifest/pull/253> in Shipit for System Addons. - Landed a smattering of minor improvements and performance improvements. Version Control - Steve Fink added support for Jujutsu <https://firefox-source-docs.mozilla.org/contributing/jujutsu.html> in mach via mozversioncontrol.repo.jj.py <https://bugzilla.mozilla.org/show_bug.cgi?id=1929372>. - Alex Hochheiden improved ./mach vcs-setup <https://bugzilla.mozilla.org/show_bug.cgi?id=1963477> for git. The default configuration now enables the built-in fsmonitor (or watchman on Linux, if installed) for improved performance. - Alex Hochheiden added Jujutsu <https://firefox-source-docs.mozilla.org/contributing/jujutsu.html> support to ./mach vcs-setup <https://bugzilla.mozilla.org/show_bug.cgi?id=1964189> via vcs=jj. It sets up a git co-located jj repo with sensible configuration defaults (also sets up watchman) to make it easier to get started. - Alex Hochheiden enabled mozversioncontrol tests for jj to run in CI <https://bugzilla.mozilla.org/show_bug.cgi?id=1962485>. - Connor Sheehan and Julien Cristau created the mozilla-esr140 and comm-esr140 repositories and associated resources on hg.mozilla.org. - Connor Sheehan added references to the upstream Git commit to the changeset view on hg.mozilla.org, and the pushlog JSON API. - Julien Cristau added support for Git-style backouts to hg.mozilla.org’s backout detection. - Julien Cristau added a hook to block pushes that touch multiple Mercurial branches. - Julien Cristau restored the alert for an excessive number of try heads. - Julien Cristau moved the SSH push server to a virtualised node, and upgraded Mercurial to version 6.9. Mercurial to Git project - On the 30th April 2025 we successfully migrated the source of truth for Firefox from Mercurial to Git 🎉 - https://github.com/mozilla-firefox/firefox - glob proposed the migration in 2023, with final approval at the end of September of that year - The core team was Zeid Zabaneh, Connor Sheehan and Olivier Mehani. Liz Henry was the Technical Program Manager. - Major components of this work include: - A new Lando <https://lando.moz.tools/> built with Django <https://bugzilla.mozilla.org/show_bug.cgi?id=1863756> that addressed considerable technical issues with the old system (Zeid Zabaneh and Connor Sheehan) - A service that synchronises Git and Mercurial repositories <https://bugzilla.mozilla.org/show_bug.cgi?id=1940614>, even if the shape of the source and destination repositories are different (Olivier Mehani) - A headless API <https://bugzilla.mozilla.org/show_bug.cgi?id=1887982> allowing automation to queue jobs through Lando as direct pushing is no longer available (Connor Sheehan) - Reimplementation of the Pulse/Push notifications in Lando <https://bugzilla.mozilla.org/show_bug.cgi?id=1940610>, as they are currently generated by Mercurial (Olivier Mehani) - A Lando API <https://bugzilla.mozilla.org/show_bug.cgi?id=1963822> that maps between Git and Mercurial commit SHAs (both directions) (Zeid Zabaneh) - Submission of mach try tasks through Lando <https://bugzilla.mozilla.org/show_bug.cgi?id=1835962>, to abstract away SCM differences (Connor Sheehan) - Any many many more changes - This is the start of our Git journey; there's still a lot of work to be done to fully complete this migration, including but not limited to migrating off and decommissioning hg.mozilla.org Thanks for reading and see you next quarter! -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CAAJAz%2B67fHEebJtPs0Szs6p0q6F-1c4C8yYGyh-zW6N2G9OD_w%40mail.gmail.com.
