Re: Re: Publishing filtered branch repositories - workflow / recommendations?
On Wed, Dec 11, 2013 at 03:16:24PM -0800, Junio C Hamano wrote: Jens Lehmann jens.lehm...@web.de writes: I think this is closely related to Martin's list of wishes we earlier saw in the thread: remind the user to push necessary submodule tip before the top-level commit that needs that commit in the submodule is pushed out. Giving projects a way to implement such a policy decision would be good, and having a default policy, if we can find one that would be reasonable for any submodule users, would be even better. Would adding a generic pre-push hook at the top-level sufficient for that kind of thing, I have to wonder. That could call git push --dry-run --recurse-submodules=check to deny the push if the submodule commit isn't on a remote branch. that would only work for a single hardcoded remote, as the remote itself does not seem to be passed to the pre-push hook. So me thinks adding a configuration option for the --recurse-submodule option of push is the best way to achieve that. This could be set to check ... Yes, that uses only a single hard-coded decision, and making the branch name or remote name customizable is not enough, as you are still hardcoding if ... isn't on part. It is not far-fetched to imagine a project wants to add more restrictions to what commit in the submodule history can be bound to a tree of a published commit in the top-level project (e.g. must be a tagged release point, must be older at least by more than two weeks, must be signed by one of these developers' keys, etc.). So I am not yet convinced that a simple option that supplies a few parameters to a single hard-coded policy is sufficient in the long run. Well, for the implementation of --recurse-submodules=check on push we tried to be as dumb as possible and just try to find out whether the commit is on any remote. The reason is that when someone works on a branch and pushes it for demonstration/backup/before holiday the most common mistake is that he forgets to push the submodule when pushing the superproject. So some remote seems to be the most common denominator for push here. At least its better than no check at all. For extended rules that answer questions like: When am I allowed to merge in master? we need some kind of workflow definition from which we can deduct such rules. A pre-push hook would allow to define rules but how about an approach that is easier for the user and can maybe guide him to the correct workflow. I find workflow guidance is a general problem in git and even though we use hooks at $dayjob they are not always sufficient. E.g. we enforce an all commits have to be reviewed first before they enter master policy which is correct in most situations. But for more lightweight projects I would like to loosen this rule. The problem relies in the automatic distribution of hooks or options for them by project instead of by installation. One idea: We could distribute a default set of enabled hooks that implement several typical workflow options which can then be enabled by simply choosing one option. The configuration should be distributeable via the project e.g. a .githookconfig or .gitworkflowconfig ? The git tools could then adapt their defaults depending on such a workflow definition. But it also means we would first have to collect and define some typical workflows. What do others think? There probably have been other ideas about workflow definitions already, no? Cheers Heiko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Am 10.12.2013 00:56, schrieb Junio C Hamano: Heiko Voigt hvo...@hvoigt.net writes: On Fri, Dec 06, 2013 at 02:40:15PM -0500, Martin Langhoff wrote: On Fri, Dec 6, 2013 at 3:48 AM, Jens Lehmann jens.lehm...@web.de wrote: Right you are, we need tutorials for the most prominent use cases. In the meantime, are there any hints? Emails on this list showing a current smart workflow? Blog posts? Notes on a wiki? None that I know of mainly because we have not yet reached the goal we are aiming at. Maybe we should write something, A few points from $dayjob that come to my mind: * A submodule commit is only allowed to be merged into master in a superproject commit if it is merged into master (or a stable branch) in the submodule. That way you ensure that any submodules commits that are tracked in a superproject are contained in each other and can be cleanly merged. (no rewinds, one commit contains the other) I think this is closely related to Martin's list of wishes we earlier saw in the thread: remind the user to push necessary submodule tip before the top-level commit that needs that commit in the submodule is pushed out. Giving projects a way to implement such a policy decision would be good, and having a default policy, if we can find one that would be reasonable for any submodule users, would be even better. Would adding a generic pre-push hook at the top-level sufficient for that kind of thing, I have to wonder. That could call git push --dry-run --recurse-submodules=check to deny the push if the submodule commit isn't on a remote branch. But that would only work for a single hardcoded remote, as the remote itself does not seem to be passed to the pre-push hook. So me thinks adding a configuration option for the --recurse-submodule option of push is the best way to achieve that. This could be set to check for those who want to push the submodule manually and to on-demand for those who would like to push automatically. And with that option the user could configure push just like he already can do for fetch and pull. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Jens Lehmann jens.lehm...@web.de writes: I think this is closely related to Martin's list of wishes we earlier saw in the thread: remind the user to push necessary submodule tip before the top-level commit that needs that commit in the submodule is pushed out. Giving projects a way to implement such a policy decision would be good, and having a default policy, if we can find one that would be reasonable for any submodule users, would be even better. Would adding a generic pre-push hook at the top-level sufficient for that kind of thing, I have to wonder. That could call git push --dry-run --recurse-submodules=check to deny the push if the submodule commit isn't on a remote branch. that would only work for a single hardcoded remote, as the remote itself does not seem to be passed to the pre-push hook. So me thinks adding a configuration option for the --recurse-submodule option of push is the best way to achieve that. This could be set to check ... Yes, that uses only a single hard-coded decision, and making the branch name or remote name customizable is not enough, as you are still hardcoding if ... isn't on part. It is not far-fetched to imagine a project wants to add more restrictions to what commit in the submodule history can be bound to a tree of a published commit in the top-level project (e.g. must be a tagged release point, must be older at least by more than two weeks, must be signed by one of these developers' keys, etc.). So I am not yet convinced that a simple option that supplies a few parameters to a single hard-coded policy is sufficient in the long run. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Re: Publishing filtered branch repositories - workflow / recommendations?
On Fri, Dec 06, 2013 at 02:40:15PM -0500, Martin Langhoff wrote: On Fri, Dec 6, 2013 at 3:48 AM, Jens Lehmann jens.lehm...@web.de wrote: Right you are, we need tutorials for the most prominent use cases. In the meantime, are there any hints? Emails on this list showing a current smart workflow? Blog posts? Notes on a wiki? None that I know of mainly because we have not yet reached the goal we are aiming at. Maybe we should write something, A few points from $dayjob that come to my mind: * A submodule commit is only allowed to be merged into master in a superproject commit if it is merged into master (or a stable branch) in the submodule. That way you ensure that any submodules commits that are tracked in a superproject are contained in each other and can be cleanly merged. (no rewinds, one commit contains the other) * Submodule should be selfcontained (i.e. if its a library have tests that use the code you implement). That way changes in the submodule can be made independent from the superproject * If a submodule needs another submodule have them side by side instead of one inside another. See the next point for explanation. * Only one depth of recursion for submodules. Even though intuition tell you that if some submodule needs another it should contain the other its IMO not wise to do so. There will be times when some other submodule needs the same submodule that is contained in the other and then you end up with two copies of the same code. My suggestion: Let the superproject bundle all the dependencies between modules. * Submodules are a good solution for shared code where the dependency goes superproject needs submodule. If you divide code into submodules because of access control and the dependency is actually that the submodule needs the superproject it works but is less than optimal. Thats what I can quickly suggest and probably far from complete. Cheers Heiko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Heiko Voigt hvo...@hvoigt.net writes: On Fri, Dec 06, 2013 at 02:40:15PM -0500, Martin Langhoff wrote: On Fri, Dec 6, 2013 at 3:48 AM, Jens Lehmann jens.lehm...@web.de wrote: Right you are, we need tutorials for the most prominent use cases. In the meantime, are there any hints? Emails on this list showing a current smart workflow? Blog posts? Notes on a wiki? None that I know of mainly because we have not yet reached the goal we are aiming at. Maybe we should write something, A few points from $dayjob that come to my mind: * A submodule commit is only allowed to be merged into master in a superproject commit if it is merged into master (or a stable branch) in the submodule. That way you ensure that any submodules commits that are tracked in a superproject are contained in each other and can be cleanly merged. (no rewinds, one commit contains the other) I think this is closely related to Martin's list of wishes we earlier saw in the thread: remind the user to push necessary submodule tip before the top-level commit that needs that commit in the submodule is pushed out. Giving projects a way to implement such a policy decision would be good, and having a default policy, if we can find one that would be reasonable for any submodule users, would be even better. Would adding a generic pre-push hook at the top-level sufficient for that kind of thing, I have to wonder. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Am 05.12.2013 23:06, schrieb Martin Langhoff: On Thu, Dec 5, 2013 at 2:54 PM, Jens Lehmann jens.lehm...@web.de wrote: Am 05.12.2013 20:27, schrieb Martin Langhoff: On Thu, Dec 5, 2013 at 2:18 PM, Jens Lehmann jens.lehm...@web.de wrote: Without knowing more I can't think of a reason why submodules should not suit your use case (but you'd have to script branching and tagging yourself until these commands learn to recurse into submodules too). The submodules feature is way too fiddly and has abundant gotchas. Care to explain what bothers you the most? Being one of the people improving submodules I'm really interested in hearing more about that. Very glad to hear submodules is getting TLC! I have other scenarios at $dayjob where I may need submodules, so happy happy. I may be unaware of recent improvements, here's my (perhaps outdated) list Thanks a lot for your feedback! - git clone does not clone existing submodules by default. An ideal workflow assumes that the user wants a fully usable checkout. You get that when using git clone --recurse-submodules, but there is no configuration option yet to switch that on by default (but we are planning to add one). - git pull does not fetchupdate all submodules (assuming a trivial tracking repos scenario) Current pull does fetch them (when changes to them are found in the fetched commits of the superproject), but it does not yet update them (there is the recursive update work in progress to do that). - git push does not warn if you forgot to push commits in the submodule We do have a command line option (--recurse-submodules=check is what you want here), but no configuration option yet. there's possibly a few others that I've forgotten. The main issue is that things that are conceptually simple (clone, git pull with no local changes) are very fiddly. Our new developers, testers and support folks hurt themselves with it plenty. Seems like we already solved some of those, and your feedback shows me that we are moving in the right direction. I keep a list of open issues we are aware of at: https://github.com/jlehmann/git-submod-enhancements/wiki Feel free to point out missing topics. I don't mind complex scenarios being complex to handle. If you hit a nasty merge conflict in your submodule, and that's gnarly to resolve, that's not a showstopper. Good to hear that! Solving them automatically is hard, only fast forwards are currently resolved without user intervention. While writing this email, I reviewed Documentation/git-submodule.txt in git master, and it does seem to have grown some new options. I wonder if there is a tutorial with an example workflow anywhere showing the current level of usability. My hope is actually for some bits of automagic default behaviors to help things along (rather than new options)... Right you are, we need tutorials for the most prominent use cases. Early git was very pedantic, and over time it learned some DWIMery. You're giving me hope that similar smarts might have grown in around submodule support ... That's what we are aiming at :-) -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
On Fri, Dec 6, 2013 at 3:48 AM, Jens Lehmann jens.lehm...@web.de wrote: Right you are, we need tutorials for the most prominent use cases. In the meantime, are there any hints? Emails on this list showing a current smart workflow? Blog posts? Notes on a wiki? Early git was very pedantic, and over time it learned some DWIMery. You're giving me hope that similar smarts might have grown in around submodule support ... That's what we are aiming at :-) That is fantastic! Thank you. m -- martin.langh...@gmail.com - ask interesting questions - don't get distracted with shiny stuff - working code first ~ http://docs.moodle.org/en/User:Martin_Langhoff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
On Wed, Dec 4, 2013 at 6:01 PM, Martin Langhoff martin.langh...@gmail.com wrote: Is there a reasonable approach to scripting this? Found my answers. The 'subtree' merge strategy is smart enough to mostly help here. However, it does not handle new files created in the subdirectory. My workflow is this one. It is similar to the recipes for the subtree merge strategies, but invoking git mv to pull files out of the git merge -s ours --no-commit upstream/branch git read-tree --prefix= -u upstream/branch git mv mysubdir/* ./ ### read-tree can't do this git commit ... time passes git merge -s subtree --no-commit upstream/branch if [ -d mysubdir ]; then # handle new files git mv mysubdir/* ./ fi git commit glad that I ended up reading a lot about subtree. cheers, m -- martin.langh...@gmail.com - ask interesting questions - don't get distracted with shiny stuff - working code first ~ http://docs.moodle.org/en/User:Martin_Langhoff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Am 05.12.2013 00:01, schrieb Martin Langhoff:
Hi folks.
currently working on a project based on Moodle (the LMS that got me
into git in the first place). This is a highly modular software, and I
would like to maintain a bunch of out of tree modules in a single
repository, and be able to publish them in per-module repositories.
So I would like to maintain a tree with looking like
auth/foomatic/{code}
mod/foomatic/{code}
where I can develop, branch and tag all the foomatic code together.
Yet, at release time I want to _also_ publish two repos
auth-foomatic.git
mod-foomatic.git
each of them with matching tags and code at the root of the git
tree, and ideally with a truthful history (i.e.: similar to having run
git filter-branch --subdirectory-filter, but able to update that
filtered history incrementally).
Is there a reasonable approach to scripting this?
Alternatively, has git submodule been improved so that it's usable by
mere mortals (i.e.: my team), or are there strong alternatives to git
submodule?
Without knowing more I can't think of a reason why submodules should
not suit your use case (but you'd have to script branching and tagging
yourself until these commands learn to recurse into submodules too).
But maybe you'll be happier with git subtree, dunno.
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
On Thu, Dec 5, 2013 at 2:18 PM, Jens Lehmann jens.lehm...@web.de wrote: Without knowing more I can't think of a reason why submodules should not suit your use case (but you'd have to script branching and tagging yourself until these commands learn to recurse into submodules too). The submodules feature is way too fiddly and has abundant gotchas. I am diving into subtrees, and finding it a lot more workable. cheers, m -- martin.langh...@gmail.com - ask interesting questions - don't get distracted with shiny stuff - working code first ~ http://docs.moodle.org/en/User:Martin_Langhoff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
Am 05.12.2013 20:27, schrieb Martin Langhoff: On Thu, Dec 5, 2013 at 2:18 PM, Jens Lehmann jens.lehm...@web.de wrote: Without knowing more I can't think of a reason why submodules should not suit your use case (but you'd have to script branching and tagging yourself until these commands learn to recurse into submodules too). The submodules feature is way too fiddly and has abundant gotchas. Care to explain what bothers you the most? Being one of the people improving submodules I'm really interested in hearing more about that. I am diving into subtrees, and finding it a lot more workable. Yep, for exporting parts of your repo as separate repos one way subtree might be just what you want. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Publishing filtered branch repositories - workflow / recommendations?
On Thu, Dec 5, 2013 at 2:54 PM, Jens Lehmann jens.lehm...@web.de wrote: Am 05.12.2013 20:27, schrieb Martin Langhoff: On Thu, Dec 5, 2013 at 2:18 PM, Jens Lehmann jens.lehm...@web.de wrote: Without knowing more I can't think of a reason why submodules should not suit your use case (but you'd have to script branching and tagging yourself until these commands learn to recurse into submodules too). The submodules feature is way too fiddly and has abundant gotchas. Care to explain what bothers you the most? Being one of the people improving submodules I'm really interested in hearing more about that. Very glad to hear submodules is getting TLC! I have other scenarios at $dayjob where I may need submodules, so happy happy. I may be unaware of recent improvements, here's my (perhaps outdated) list - git clone does not clone existing submodules by default. An ideal workflow assumes that the user wants a fully usable checkout. - git pull does not fetchupdate all submodules (assuming a trivial tracking repos scenario) - git push does not warn if you forgot to push commits in the submodule there's possibly a few others that I've forgotten. The main issue is that things that are conceptually simple (clone, git pull with no local changes) are very fiddly. Our new developers, testers and support folks hurt themselves with it plenty. I don't mind complex scenarios being complex to handle. If you hit a nasty merge conflict in your submodule, and that's gnarly to resolve, that's not a showstopper. While writing this email, I reviewed Documentation/git-submodule.txt in git master, and it does seem to have grown some new options. I wonder if there is a tutorial with an example workflow anywhere showing the current level of usability. My hope is actually for some bits of automagic default behaviors to help things along (rather than new options)... Early git was very pedantic, and over time it learned some DWIMery. You're giving me hope that similar smarts might have grown in around submodule support ... cheers, m -- martin.langh...@gmail.com - ask interesting questions - don't get distracted with shiny stuff - working code first ~ http://docs.moodle.org/en/User:Martin_Langhoff -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Publishing filtered branch repositories - workflow / recommendations?
Hi folks.
currently working on a project based on Moodle (the LMS that got me
into git in the first place). This is a highly modular software, and I
would like to maintain a bunch of out of tree modules in a single
repository, and be able to publish them in per-module repositories.
So I would like to maintain a tree with looking like
auth/foomatic/{code}
mod/foomatic/{code}
where I can develop, branch and tag all the foomatic code together.
Yet, at release time I want to _also_ publish two repos
auth-foomatic.git
mod-foomatic.git
each of them with matching tags and code at the root of the git
tree, and ideally with a truthful history (i.e.: similar to having run
git filter-branch --subdirectory-filter, but able to update that
filtered history incrementally).
Is there a reasonable approach to scripting this?
Alternatively, has git submodule been improved so that it's usable by
mere mortals (i.e.: my team), or are there strong alternatives to git
submodule?
cheers,
m
--
martin.langh...@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
