On 10/31/14 11:03 AM, Ehsan Akhgari wrote:
On 2014-10-31 12:51 PM, Gregory Szorc wrote:
All problems in computer science can be solved by another level of
indirection. The problem here is that Git users need to push to the
canonical Mercurial repositories (inbound, try, etc). You will be
hearing me say this a lot in the months ahead, but "Autoland changes
everything."
I don't understand how. Doesn't Autoland still require you to push your
changes somewhere? If that somewhere is an hg repo we're back to square
one.
The exact end state of Autoland hasn't been implemented yet. But, yes,
it likely relies on your pushing somewhere or uploading a bundle to a
web service. (The existing approach of attaching state/patches to bugs
will not be the final solution.)
With Autoland in place, there is a level of indirection between the user
and landing: a landing queue. In my utopian end state for Autoland, no
user - whether they use Git or even Mercurial - are pushing directly to
the canonical repositories. Only the Autoland agent performs that role.
FWIW I think that's a bit too eutopian. I have no hopes of eliminating
all of the need for pushing stuff manually. "Land this thing at some
point in the near future" is a very common use case but it's far from
being the only one.
This end state is necessary to provide higher and more resilient SLAs
and backout free repo history. OK, sure, there will always be some
direct landings (chemspills come to mind). But for the majority of us,
pushing to the final repository will not be part of our workflow.
Instead we'll signal intent for something to land and machines will do
the heavy lifting for us.
Suddenly the publishing workflow becomes unidirectional and this whole
syncing problem with Hg<->Git SHA-1 maps becomes abstracted away into
the implementation of Autoland. We can stand up Git servers to allow Git
native ingestion, without burdening Git users with the knowledge,
overhead, or cognitive dissonance of Mercurial.
Not sure if I follow. Are you suggesting that git users push to their
own vanilla git server, and then autoland comes around some time later,
converts that into hg, and tries to transplant the result? Why is that
better than just having a git server that pushes to an hg server before
accepting a push? That seems to both address the use case of pushing to
Autoland's hg repo, and also the use cases that do not involve Autoland.
Part of Autoland is discovery of unpublished heads. I imagine this will
be triggered by pushing to a well-defined repository. But having
Autoland discover things from "user repositories" is certainly in the
realm of possibility.
If Autoland knows the Git<->Hg SHA-1 mapping, there should be little
reason Autoland can't discover landing requests from Git repositories. I
think details such as when the Git -> Hg conversion occurs are an
implementation detail of the Autoland service and shouldn't be relevant
to Git users. Correct me if I'm wrong, but if the Git conversion is
happening on the server and the Git commit SHA-1's are stable, I don't
think the Git users should care about what's actually happening, right?
It also means the server-side infrastructure becomes more complex and
harder to maintain. To achieve the tight cohesion necessary to present a
fully optimized and seamless workflow, all these services likely need to
be running as part of the same logical system, sharing the same
underlying data store, without syncing being a major issue. Do I think
this is the right and inevitable solution? Yes, I do. Ideal, no. (Ideal
would be to support one system - and tell the users of the other to take
a hike. If I had my way, that one system would be Mercurial because it
is easier to extend and can lead to better productivity. But short of a
mandate that makes a lot of people upset and has extremely high costs to
implement, this ideal isn't going to happen.)
Since you keep mentioning this (and *without meaning to start a flame
war*), let this be on the record that I think mercurial's extensibility
etc don't really matter for a user of the VCS, unless they work in a
very specialized environment (such as the big silicon valley tech
companies that have used hg in the past few years). Productivity is
subjective, and not an objective property of either git or mercurial.
The reason I and others go through this pain right now to avoid having
to use hg more is increased productivity. So, I'd appreciate if you
didn't make it sounds like mercurial is all about user productivity and
git isn't. :-)
Mozilla *is* a big silicon valley tech company. Although I concede we
haven't traditionally functioned like one.
Mozilla has version control scaling challenges that are beyond what most
companies experience, at least companies using open source and free
tools. I believe we qualify as a "specialized environment" that would
benefit from extensibility. You start operating at our scale, and any
change, no matter how small, multiplied by a 1000+ developers adds up. I
think it is wasteful for developers to e.g. spend 20s pushing to Try
when we could ask them to install a Mercurial extension and be
guaranteed <1s latencies. Ignoring the extensibility of a tool is a lost
opportunity to move faster. I think we advance The Mission faster by
deploying optimized workflows.
I'm not saying it's impossible to achieve similar wins with Git. But I
think the path forward with Mercurial is considerably less effort. One
only has to look at Facebook's recent decisions in this realm to
reinforce that belief.
Unless a major decision is made, we're trending towards more complexity
on the server to support Git ingestion.
Are we though? Cause for the past recent years, our approach towards
git users has been very close to "take a hike". Is there now a push
towards treating them better? (Genuinely asking, since I haven't heard
anything about an organizational drive towards treating git users better
recently.)
Honestly, I'm not sure. This thread started as me asking the Git crowd
how much the Mercurial workflow could be optimized before the Git users
started rioting. I empathize with your position of wanting to use your
tool of choice. (Don't forget I was a long-time Git user before joining
Mozilla and I initially thought Mercurial was a joke. I even helped
improve hg-git to enable a faster and more robust Git mirror so I could
use Git in my day-to-day Firefox development.)
On one had, I like the idea of letting developers use their tool of
choice. Autoland makes that very realistic. On the other, tragedy of the
commons applies and I kind of wish we could focus on one tool instead of
many. I think putting all our energy behind one tool will enable
everyone to move faster.
1. Are the changes that you're planning to make to the try server
inherently incompatible with git? An example of such an incompatibility
would be for example changing the wire format of the protocol to send in
data that is only available on the client and not transmittable through
git push.
We will devise a mechanism that is compatible with Git users one way or
another. I can't promise it will initially be the most optimal
implementation (`git push`). But then again, Git users are far from
optimal today, so there shouldn't be a net regression.
2. What if anything should we do differently in the mean time?
Reading the replies on this thread, I think the Git tools for working
with Firefox need some love. Workflows appear to be all over the map and
the workflow with the strongest convergence (moz-git-tools) is far from
optimal (e.g. using qimport + qfinish instead of just `hg import`, no
use of hg-git, SHA-1 map only known to server, etc).
Improving the Git workflow is a subset of a larger problem: improving
the overall version control workflow. IMO the average developer is
spending way too much time battling source control instead of "doing
code." We need to improve systems to eliminate things like Try latencies
and overhead when initiating and conducting code review. We need to
improve user eduction so people can use version control more optimally.
It's a hard and deeply-entrenched problem to solve. It will likely take
at least a year to overcome a decade of organizational inertia. I've
done this before at previous companies and I believe the best path
forward is to provide a compelling and superior developer experience. If
you achieve that, winning users over comes easier.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
