Re: FW: Query on git submodules
Frawley, Sarah sarah.frawley at intel.com writes: Correct when I referred to 10+ layers I meant nested repositories which make up a large hierarchy. Some repositories are repeated across the hierarchy. We check-out submodules to tag versions (as opposed to master branch). If we need to roll out a particular change across a hierarchy (e.g. 60 repos) then every level in the hierarchy needs to pick up new submodule tags. The main 2 issues that we see are: 1. Enforcement of absolute paths in git submodules - I am currently trialing using a pre-commit hook to highlight this issue to users so that they can fix their submodule urls to relative paths. We enforced the config file instead (containing the host, port and username) because we use Gerrit. So, our urls are just ssh:\\server:repo 2. Slowness Integrating updates to submodule hierarchy -I have been looking at ways of automating such a roll out - this can be useful for new project setups or for rolling out an update to all repos and quickly integrating it into the submodule hierarchy. The link below shows something similar however it checks out a master branch of each submodule in its hierarchy. https://chrisjean.com/recursively-updating-git-submodules/ I haven't been able to check performance at our site yet. Thanks, Sarah I am planning on using submodules in a way, where I can specify a configuration of repos with tags, and rebuild the submodule tree, when selecting a configuration on the top level. But, yet I have found no easy way to select the tag in a subrepo. I would like the .gitmodules to accept tag instead of branch. I think there is very little difference between specifying a branch or a tag, but it is more convenient to have the release version as a tag. So, if a product A, consist of subcomponents a1, a2 and a3, the version v1.1 of product A will have a configuration saying: a1, tag=v0.4; a2, tag=v1.0; a3, tag=v2.0 Subcomponent can be both a pre-built binary or a shared/common source base, or a complete 3rd party repository. Is this comparable to your situation? I am currently working on editing the git-submodules to add -t (as in tag). I need to find an easy way to switch between tracking a branch and locking to tag, though. BR /Mattias
Re: FW: Query on git submodules
On Wed, May 27, 2015 at 01:20:11PM +, Frawley, Sarah wrote: Thanks Heiko for getting back to me. Correct when I referred to 10+ layers I meant nested repositories which make up a large hierarchy. Some repositories are repeated across the hierarchy. We check-out submodules to tag versions (as opposed to master branch). If we need to roll out a particular change across a hierarchy (e.g. 60 repos) then every level in the hierarchy needs to pick up new submodule tags. The main 2 issues that we see are: 1. Enforcement of absolute paths in git submodules - I am currently trialing using a pre-commit hook to highlight this issue to users so that they can fix their submodule urls to relative paths. So you mean, enforcement of relative paths. Right? It might be a good idea to put some code in 'git submodule add' that would automatically use a relative path instead of absolute path if it can calculate one from the superproject. I think that would help your users even more as they do not get an error from the pre-commit hook, but just having everything right after doing the add. If you want to go that route it should not be that much different from the hook, since the submodule subcommand code is all bash. 2. Slowness Integrating updates to submodule hierarchy -I have been looking at ways of automating such a roll out - this can be useful for new project setups or for rolling out an update to all repos and quickly integrating it into the submodule hierarchy. The link below shows something similar however it checks out a master branch of each submodule in its hierarchy. https://chrisjean.com/recursively-updating-git-submodules/ How about git submodule update --init --recursive ? That should do the same as the script written in that blog post. Or am I missing something? Cheers Heiko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: FW: Query on git submodules
Heiko Voigt hvoigt at hvoigt.net writes: What exactly is the feature of using a tag instead of a commit id like it is currently the default? A tag should be nailed to a commit anyway or does it change in your situation? In that case I would say it is a branch. Cheers Heiko Essentially, it is a commit, but an annotated tag is easier to read, than the SHA-1. And, I don't want gerrit to track a tag, but I want gerrit to track a branch. BR /Mattias -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: FW: Query on git submodules
Thanks for getting back to me Heiko To comment on the below. Adding some automated way of setting a relative path via git submodule add would be really useful. Is this something that is possible in a new git release? As regards using the command below. git submodule update --init --recursive We use this already however it just refreshes any submodule changes that have been pushed to git. What I am looking for is a way to re-align submodule tags to new tags across a large hierarchy. See example below. It show a simple submodule hierarchy. You can see that the 'c_block' occurs underneath both the 'e' and the 'd' blocks and they are both set to the same tag (tag_1). If I decide to create a new tag for 'c_block' called tag_2 in order for both d_block and e_block to pick it up I will need to make a update to both blocks where I check out the tag_2 for 'd/e_blocks' and commit, push and tag and make another change to 'a_block' so that it checks out the new tags for 'c_block','d_block' and 'e_block'. This all happens manually in my team currently. Sometimes there is a need to roll out the same change to all repos and for the hierarchy to be up rev'd to the latest tag for every submodule so a script for this case would be useful. It would be nice to be able to have a script read in a list of tags for update across the hierarchy. a_block - c_block (tag_1)- d_block(tag_4) d_block (tag_4) - c_block(tag_1) e_block (tag_8) - d_block(tag_4) c_block(tag_1) Commandline use to update e_block to pick up new tag for c_block: cd e_block/submodules/c_block git checkout tag_2 cd ../.. git commit git push git tag git submodule update --init --recursive Thanks, Sarah Frawley Quark DA To raise a DA help ticket please file using the url below: https://hsdes.intel.com/home/default.html#article?subject=supporttenant=quark -Original Message- From: Heiko Voigt [mailto:hvo...@hvoigt.net] Sent: Tuesday, June 2, 2015 12:12 PM To: Frawley, Sarah Cc: git@vger.kernel.org Subject: Re: FW: Query on git submodules On Wed, May 27, 2015 at 01:20:11PM +, Frawley, Sarah wrote: Thanks Heiko for getting back to me. Correct when I referred to 10+ layers I meant nested repositories which make up a large hierarchy. Some repositories are repeated across the hierarchy. We check-out submodules to tag versions (as opposed to master branch). If we need to roll out a particular change across a hierarchy (e.g. 60 repos) then every level in the hierarchy needs to pick up new submodule tags. The main 2 issues that we see are: 1. Enforcement of absolute paths in git submodules - I am currently trialing using a pre-commit hook to highlight this issue to users so that they can fix their submodule urls to relative paths. So you mean, enforcement of relative paths. Right? It might be a good idea to put some code in 'git submodule add' that would automatically use a relative path instead of absolute path if it can calculate one from the superproject. I think that would help your users even more as they do not get an error from the pre-commit hook, but just having everything right after doing the add. If you want to go that route it should not be that much different from the hook, since the submodule subcommand code is all bash. 2. Slowness Integrating updates to submodule hierarchy -I have been looking at ways of automating such a roll out - this can be useful for new project setups or for rolling out an update to all repos and quickly integrating it into the submodule hierarchy. The link below shows something similar however it checks out a master branch of each submodule in its hierarchy. https://chrisjean.com/recursively-updating-git-submodules/ How about git submodule update --init --recursive ? That should do the same as the script written in that blog post. Or am I missing something? Cheers Heiko - Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: FW: Query on git submodules
On Tue, Jun 02, 2015 at 09:45:39AM +, Mattias Vannergård wrote: I am planning on using submodules in a way, where I can specify a configuration of repos with tags, and rebuild the submodule tree, when selecting a configuration on the top level. But, yet I have found no easy way to select the tag in a subrepo. I would like the .gitmodules to accept tag instead of branch. I think there is very little difference between specifying a branch or a tag, but it is more convenient to have the release version as a tag. So, if a product A, consist of subcomponents a1, a2 and a3, the version v1.1 of product A will have a configuration saying: a1, tag=v0.4; a2, tag=v1.0; a3, tag=v2.0 Subcomponent can be both a pre-built binary or a shared/common source base, or a complete 3rd party repository. Is this comparable to your situation? I am currently working on editing the git-submodules to add -t (as in tag). I need to find an easy way to switch between tracking a branch and locking to tag, though. What exactly is the feature of using a tag instead of a commit id like it is currently the default? A tag should be nailed to a commit anyway or does it change in your situation? In that case I would say it is a branch. Cheers Heiko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
FW: Query on git submodules
Thanks Heiko for getting back to me. Correct when I referred to 10+ layers I meant nested repositories which make up a large hierarchy. Some repositories are repeated across the hierarchy. We check-out submodules to tag versions (as opposed to master branch). If we need to roll out a particular change across a hierarchy (e.g. 60 repos) then every level in the hierarchy needs to pick up new submodule tags. The main 2 issues that we see are: 1. Enforcement of absolute paths in git submodules - I am currently trialing using a pre-commit hook to highlight this issue to users so that they can fix their submodule urls to relative paths. 2. Slowness Integrating updates to submodule hierarchy -I have been looking at ways of automating such a roll out - this can be useful for new project setups or for rolling out an update to all repos and quickly integrating it into the submodule hierarchy. The link below shows something similar however it checks out a master branch of each submodule in its hierarchy. https://chrisjean.com/recursively-updating-git-submodules/ Thanks, Sarah -Original Message- From: Heiko Voigt [mailto:hvo...@hvoigt.net] Sent: Tuesday, May 26, 2015 6:01 PM To: Frawley, Sarah Cc: git@vger.kernel.org Subject: Re: Query on git submodules Hi, On Fri, May 22, 2015 at 01:46:24PM +, Frawley, Sarah wrote: I am a design automation engineer supporting 200+ designers who use git for hardware design. We also use the submodule feature where we can have quite complex hierarchy's with 10+ layers. What does this 10+ layers mean? Nested submodule repositories 10 recursion steps deep? We have experience issues with re-use of design projects was we move from one derivative to another due to the complexity of the hierarchy along with lack of discipline (using absolute paths in .gitmodule files). To enforce more discipline I am currently working on a pre-commit hook to check the integrity of .gitmodule files. I would be interested in seeing how other users in the community find submodules for large scale projects and if there are any best known methods for using them. I do not have anything to share here since our projects did not have such problems (not that large scale). It would be interesting to see what you come up with. Maybe we can add some of that into core git to support such large scale projects better. So maybe you can share exactly what problems you have (only absolute paths?) or the pre-commit hooks you will use. Cheers Heiko - Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Query on git submodules
Hi, On Fri, May 22, 2015 at 01:46:24PM +, Frawley, Sarah wrote: I am a design automation engineer supporting 200+ designers who use git for hardware design. We also use the submodule feature where we can have quite complex hierarchy's with 10+ layers. What does this 10+ layers mean? Nested submodule repositories 10 recursion steps deep? We have experience issues with re-use of design projects was we move from one derivative to another due to the complexity of the hierarchy along with lack of discipline (using absolute paths in .gitmodule files). To enforce more discipline I am currently working on a pre-commit hook to check the integrity of .gitmodule files. I would be interested in seeing how other users in the community find submodules for large scale projects and if there are any best known methods for using them. I do not have anything to share here since our projects did not have such problems (not that large scale). It would be interesting to see what you come up with. Maybe we can add some of that into core git to support such large scale projects better. So maybe you can share exactly what problems you have (only absolute paths?) or the pre-commit hooks you will use. Cheers Heiko -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Query on git submodules
Hi there I am a design automation engineer supporting 200+ designers who use git for hardware design. We also use the submodule feature where we can have quite complex hierarchy's with 10+ layers. We have experience issues with re-use of design projects was we move from one derivative to another due to the complexity of the hierarchy along with lack of discipline (using absolute paths in .gitmodule files). To enforce more discipline I am currently working on a pre-commit hook to check the integrity of .gitmodule files. I would be interested in seeing how other users in the community find submodules for large scale projects and if there are any best known methods for using them. Operating system (specifically which version) - Suse Linux (SLES11SP2) Git version (git --version) - 1.7.12.2 Thanks, Sarah - Intel Ireland Limited (Branch) Collinstown Industrial Park, Leixlip, County Kildare, Ireland Registered Number: E902934 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
