Announcing git-cinnabar 0.1.0 (Was: Announcing a new (prototype) git-remote-hg tool)
Hi, Cinnabar is the common natural form in which mercury can be found on Earth. It contains mercury sulfide and its powder is used to make the vermillion pigment. What does that have to do with git? Hint: mercury. Git-cinnabar is a git remote helper to interact with mercurial repositories. It allows to clone, pull and push from/to mercurial remote repositories, using git. If you've already seen the original thread this message is spawned from, this is the same tool, with a new name, and two months worth of additional work and testing. Where git-cinnabar stands out compared to other similar tools is that it doesn't use a local mercurial clone under the hood (unlike all the existing other such tools), and is close to an order of magnitude faster to clone a repository like http://hg.mozilla.org/mozilla-central than the git-remote-hg tool that used to be shipped in contrib/. I won't claim it is exempt of problems and limitations, which is why it's not a 1.0. I'm however confident enough with its state to make the first official release. Code on https://github.com/glandium/git-cinnabar Cheers, Mike -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Announcing a new (prototype) git-remote-hg tool
Hi, I've (re)started work on a longstanding idea of mine of having a git tool talking the mercurial wire protocol directly. I'm now at a stage where the tool can clone and pull/fetch from mercurial. As it is a prototype, there are many things that it doesn't handle (like named branches, bookmarks, phases, obsolescence markers), but it currently transposes a complete mercurial dag to git and maintains metadata about the original mercurial data. Code on https://github.com/glandium/git-remote-hg It doesn't support push, but support for that should come in the coming weeks. More background on http://glandium.org/blog/?p=3382 This is meant to be a prototype, and will stay that way for now. It's a validation that this can actually work. Now that I have pull support I know I can make it push. I'm currently evaluating what the final tool would look like. I'm *very* tempted to implement it in C, based on core git code, because there are many things that this helper does that would be so much easier to do with direct access to git's guts. And that wouldn't require more dependencies than git currently has: it would just need curl and ssh, and git already uses both. If I were to go in that direction, would you consider integrating it in git core? If not, would you rather see git helpers to make this git-remote-hg helper more efficient? Cheers, Mike -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
Mike Hommey wrote: I'm currently evaluating what the final tool would look like. I'm *very* tempted to implement it in C, based on core git code, because there are many things that this helper does that would be so much easier to do with direct access to git's guts. And that wouldn't require more dependencies than git currently has: it would just need curl and ssh, and git already uses both. If I were to go in that direction, would you consider integrating it in git core? Yes --- I would like this a lot. The general trend has been to carry fewer contrib-style tools in-tree, since the problem of discovering tools built on top of git is not as hard as it used to be. What you describe above seems to be a bit of an exception: - libgit.a in its current state evolves too quickly for it to be convenient for out-of-tree tools to use. cgit http://git.zx2c4.com/cgit/ uses git pinned to a particular version as a submodule to get around this, which is fussy and has bad implications for remembering to get security updates. - an in-tree user of libgit.a would be useful as a reference example to use to try to make libgit.a into be a better library internally (and eventually expose e.g. by merging with libgit2 as something outside tools can link to, I hope) - if it makes sense to help people using the current remote helper in contrib to migrate to this, it could be convenient for users In other words, although in the long term I would be happiest if libgit becomes good enough to let this project live in a separate tree and link to it, it's tempting to build this in-tree because we're not there yet. Some other alternatives: - using libgit2 https://libgit2.github.com/ - improving git plumbing (e.g., with new fast-import commands) or exposing a small library with a stable API for the tool's use I haven't thought it through carefully but at the moment I like the in-tree approach best. Thanks, Jonathan -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
From: Mike Hommey m...@glandium.org Hi, I've (re)started work on a longstanding idea of mine of having a git tool talking the mercurial wire protocol directly. I'm now at a stage where the tool can clone and pull/fetch from mercurial. As it is a prototype, there are many things that it doesn't handle (like named branches, bookmarks, phases, obsolescence markers), but it currently transposes a complete mercurial dag to git and maintains metadata about the original mercurial data. Code on https://github.com/glandium/git-remote-hg It doesn't support push, but support for that should come in the coming weeks. More background on http://glandium.org/blog/?p=3382 This is meant to be a prototype, and will stay that way for now. It's a validation that this can actually work. Now that I have pull support I know I can make it push. I'm currently evaluating what the final tool would look like. I'm *very* tempted to implement it in C, based on core git code, because there are many things that this helper does that would be so much easier to do with direct access to git's guts. And that wouldn't require more dependencies than git currently has: it would just need curl and ssh, and git already uses both. If I were to go in that direction, would you consider integrating it in git core? If not, would you rather see git helpers to make this git-remote-hg helper more efficient? You may also be interested in https://felipec.wordpress.com/2013/08/26/whats-new-in-git-v1-8-4-remote-hg/ and https://github.com/felipec/git-remote-hg. Though Filipe's unique work style hasn't found favour locally. see also https://github.com/buchuki/gitifyhg/wiki/List-of-git-hg-bridges Philip -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
On Fri, Dec 05, 2014 at 02:13:19PM -0800, Jonathan Nieder wrote: Mike Hommey wrote: I'm currently evaluating what the final tool would look like. I'm *very* tempted to implement it in C, based on core git code, because there are many things that this helper does that would be so much easier to do with direct access to git's guts. And that wouldn't require more dependencies than git currently has: it would just need curl and ssh, and git already uses both. If I were to go in that direction, would you consider integrating it in git core? Yes --- I would like this a lot. I'm concerned that this tool will have drawbacks that Felipe's remote-hg does not. And I can well imagine that it may, as that tool builds on Mercurial's API, which will probably handle some corner cases differently. This isn't to disparage Mike's attempt; it will probably have some upsides, too. But given that the approaches are so different, it does not seem obvious to me that one will always be better than the other. One of the nice things about spinning remote-hg out of the core repo is that it means we do not have to endorse a particular implementation, and they can compete with each other on their merits. I would very much hate to see Felipe's remote-hg project wither and die just because another implementation becomes the de facto standard by being included in git.git. It's a proven tool, and this new thing is not yet. It's a shame that both squat on the name remote-hg, because it makes it difficult to tell the two apart. But of course that is the only way to make git clone hg::... work. Maybe we need a layer of indirection? :) - libgit.a in its current state evolves too quickly for it to be convenient for out-of-tree tools to use. cgit http://git.zx2c4.com/cgit/ uses git pinned to a particular version as a submodule to get around this, which is fussy and has bad implications for remembering to get security updates. I'm not sure that this approach is any better than carrying something in contrib/ in git.git. If I refactor a function in libgit.a, I notice breakage in the callers because it no longer compiles, or because I am thorough and look at the implications to git callers. I do _not_ want to be responsible for making sure that contrib/* still builds. That is the problem of the maintainer of the contrib/ project in question. That may sound a little selfish, but I think that is what it means to be in contrib, and not in the regular tree. So once you realize that is the burden of the contrib/ author to fix breakages, then the process is: git pull cd contrib/c-remote-hg make # oops, it broke fix fix fix That is not any different than: git submodule add git... git submodule update make # oops, it broke fix fix fix The hard part is not how you pull changes from the new git into your tree. It is the fact that upstream may be breaking the interface behind your back. And your best bet is to aggressively merge with upstream, rather than trying to track only occasional release versions. Of course, if you meant to _really_ carry it in-tree, not in contrib/, then none of that applies. But then I worry doubly about the endorsement issue. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
Jeff King wrote: One of the nice things about spinning remote-hg out of the core repo is that it means we do not have to endorse a particular implementation, and they can compete with each other on their merits. True. [...] It's a shame that both squat on the name remote-hg, because it makes it difficult to tell the two apart. But of course that is the only way to make git clone hg::... work. Maybe we need a layer of indirection? :) If the helpers are roughly interchangeable (that is, if you can switch between fetching using each one into the same on-disk git repository), then picking one to symlink as git-remote-hg in your $PATH should be enough. If they don't have that level of interoperability, then there's an argument to be made that the URLs shouldn't be the same. Unfortunately url.*.insteadof rules are resolved at fetch time instead of being resolved once and the result recorded in .git/config. So yes, it seems like a way to have abbreviations for URLs (e.g., hg:: meaning hg+mh:: or hg+fc::) that get resolved at clone time would be useful. It's a layer of indirection we don't provide. :/ Thanks, Jonathan -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
On Fri, Dec 05, 2014 at 05:59:30PM -0500, Jeff King wrote: On Fri, Dec 05, 2014 at 02:13:19PM -0800, Jonathan Nieder wrote: Mike Hommey wrote: I'm currently evaluating what the final tool would look like. I'm *very* tempted to implement it in C, based on core git code, because there are many things that this helper does that would be so much easier to do with direct access to git's guts. And that wouldn't require more dependencies than git currently has: it would just need curl and ssh, and git already uses both. If I were to go in that direction, would you consider integrating it in git core? Yes --- I would like this a lot. I'm concerned that this tool will have drawbacks that Felipe's remote-hg does not. And I can well imagine that it may, as that tool builds on Mercurial's API, which will probably handle some corner cases differently. FWIW, my tool only uses the mercurial code for the wire protocol. This can (and if I go the C route, will) be implemented without using mercurial code, it's really not a hard problem. This isn't to disparage Mike's attempt; it will probably have some upsides, too. But given that the approaches are so different, it does not seem obvious to me that one will always be better than the other. One of the nice things about spinning remote-hg out of the core repo is that it means we do not have to endorse a particular implementation, and they can compete with each other on their merits. I would very much hate to see Felipe's remote-hg project wither and die just because another implementation becomes the de facto standard by being included in git.git. It's a proven tool, and this new thing is not yet. Note that I'm only talking about an hypothetical long term goal. If there's not even a slim chance that this may end up in git core, or in the git.git repo, I'm not sure it's worth writing the tool in C at all, considering the burden for users. IOW, I'm only trying to assess if I should follow my temptation or not. But I can probably reassess after I actually get my prototype to do more than it does now. But maybe there are ways to make it work for users outside of git.git even if it's in C. I don't know. It's a shame that both squat on the name remote-hg, because it makes it difficult to tell the two apart. But of course that is the only way to make git clone hg::... work. Maybe we need a layer of indirection? :) Yeah, that's an unfortunate consequence of how remote helpers work. There are already two different git-remote-hgs (there's felipe's, and there's another one using hg-git under the hood) that I know of. I'm adding a third one. For what it's worth, none of the existing one is satisfying on repos the size of Mozilla's, and apparently noone at Mozilla uses them because of that. Add to that the disk space inefficiency of actually keeping a copy of the mercurial repo locally. The existing tools can likely be improved to scale better, but that wouldn't change the disk space problem. Mike -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
On Fri, Dec 05, 2014 at 03:13:30PM -0800, Jonathan Nieder wrote: Jeff King wrote: One of the nice things about spinning remote-hg out of the core repo is that it means we do not have to endorse a particular implementation, and they can compete with each other on their merits. True. [...] It's a shame that both squat on the name remote-hg, because it makes it difficult to tell the two apart. But of course that is the only way to make git clone hg::... work. Maybe we need a layer of indirection? :) If the helpers are roughly interchangeable (that is, if you can switch between fetching using each one into the same on-disk git repository), then picking one to symlink as git-remote-hg in your $PATH should be enough. If they don't have that level of interoperability, then there's an argument to be made that the URLs shouldn't be the same. I don't think Felipe's and the one that uses hg-git under the hood are already interoperable. Mine is also different from both. They should all produce the same git trees. They don't produce the same commits. They also don't share the same metadata. Mike -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Announcing a new (prototype) git-remote-hg tool
On Fri, Dec 05, 2014 at 03:13:30PM -0800, Jonathan Nieder wrote: It's a shame that both squat on the name remote-hg, because it makes it difficult to tell the two apart. But of course that is the only way to make git clone hg::... work. Maybe we need a layer of indirection? :) If the helpers are roughly interchangeable (that is, if you can switch between fetching using each one into the same on-disk git repository), then picking one to symlink as git-remote-hg in your $PATH should be enough. That may be enough. For the most part you do not need to agree with other members of the project on which implementation to use. My experience with import tools has been that either: 1. you are using them personally (because you do not like the upstream's choice of VCS and would prefer to transparently work in your favorite tool), or 2. there is a group of developers who want to use git, but somebody provides an unofficial git mirror. They do not have to agree on the tool, because they just use git directly from the mirror. So it is mostly a personal choice. But the two confusions I'd still anticipate are: a. It's difficult to even _talk_ about the tools, because the names are the same (so searching for tips on the tool, reporting bugs, etc, are harder than necessary). b. You may want different tools for different projects. If one tool is much more efficient, you may need it for a large repo (e.g., mozilla). But another tool may provide other features, and you would prefer it for smaller repos. This is largely speculation, though, and I do not actively use the tools myself. So I'd be happy to push off dealing with it until it itches enough for somebody to scratch. -Peff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
