Philip, I just wanted to close out this thread, and once again offer my thanks for the feedback! I also wanted to quickly note the conclusion, just in case someone else finds themselves in a similarly ugly place.
Hugh, yes, one repo would be a dream! But that's outside of my control. I'm going with the date based approach (as you suggested) and will consider adding more complexity if the need arises. I'm using the "--branches=master" filter for now, we'll see if that causes issues. if it does, it's all in a script, so I can just tweak it and commit it back and the change will propagate. The bundle export (create) script automatically probes the local environment to determine where it is, it prompts the user for the necessary information about what to export (1 months, 1 week, custom date), and makes sure the generated bundle file uses the proper naming convention. The import script will decode relevant information out of the bundle filename, check to make sure the local repos is up-to-date with the remote repository, check that the working tree is clean, and import the new bundle's content into the appropriate branch. So far it all seems to be working well. I will say that storing the bundle import/export scripts within the repository being synchronized is a mixed bag. On the one hand it's really nice to have everything in one place, and not to have to deal with synchronizing it separately. But on the other hand, the first few times back and forth can be a bit difficult. It's the classic chicken/egg problem. For example, if you uncover an issue with the import script, you can commit it and put in the next bundle. However, you won't be able to use it on the far side until after your done the import, and which point it don't need it anymore because you've already done it manually. Whoops. I think that long term this will much less of an issue as things stabilize. I did take a stab at trying to introduce some "tracking", but instead of using tags (which have issues when you try to move them) I'm just using special purpose "branches" (special purposed by convention only). So now I can ask more environmentally-aware questions, like: what changes were made on my local master branch since I last synced with upstream?" Of course I can't get a "real-time" answer to that (without network connectivity), but it's better than nothing. (I was unable to do any of this before, because the last-export tag wasn't specific to an environment, and it wasn't ever pushed to the central repo (GitHub enterprise), it always just lived in a local repository.) I came up with a very simple "sync/ENVIRONMENT-NAME" branch naming convention, and I push around all the "sync/*" refs in both the import and export scripts. I also put the ref for the current environment in each bundle. Long term I may attempt to use these "Tracking" branches (general english meaning, not necessary the various possible git-specific meanings) for the bundle creation process, but for now I'm trying to keep it simple. Thanks again, Lowell On Monday, February 6, 2017 at 5:04:51 PM UTC-5, Philip Oakley wrote: > > Hi Hugh, > > I think Lowell would love to have a networked system with a golden bare > repo serving all the machines. > > However, I strongly suspect this set-up is in a security setting > (goverment vetting and all that), so that some parts cannot be connected to > other parts, etc etc (see also stuff about TEMPEST security regarding the > concerns that folk have..) > > (My case was test assets which had to be kept separate from IT corporate > assets to avoid 'cross contamination'). > > Bundles are like RFC 1149 / 2549, with Nike enhancements;-) > > Philip > > ----- Original Message ----- > *From:* Hugh Gleaves <javascript:> > *To:* Git for human beings <javascript:> > *Sent:* Monday, February 06, 2017 8:07 PM > *Subject:* [git-users] Re: Synchronizing air gapped git repositories > using bundles > > I'm questioning the overall wisdom of what you describe Lowell. > > Why not just have a single repository? Doing that eliminates all of the > fussing you describe. > > If the "other" repos are little more than holding pens for stuff to > eventually get merged into your primary repo, then why? this is what > branches are for. > > It sounds as if you're not really taking full advantage of all that Git > offers. > > > > On Wednesday, February 1, 2017 at 2:58:59 PM UTC-7, Lowell Alleman wrote: >> >> I have 3 separate air-gapped git repositories (hosted on local GitHub >> enterprise) that I'm trying to keep in sync. Currently, I'm using "git >> bundle" to push revisions back and forth, which worked fairly well with >> just 2 repositories, but I'm struggling a bit since the 3rd (and final) >> repository has been added to the mix. I was using a single tag to track >> the point of last export as noted in the "git bundle" docs, but I'm >> struggling to make that scale with 2+ total repositories. >> >> In terms of information flow, we've deemed one of the repositories as >> "primary" and the other two as "secondary" repositories. So in a sense we >> are using the "primary" repository like a development and merging area so >> that all changes go through the primary repository and trickle down to the >> secondary repositories. Changes are always pushed upstream to primary, and >> then synced down to the other secondary repository. >> >> Please note that our use of git is more like a "versioned file system" >> than the typical developer use case. I go on to explain that a bit more >> later, but wanted to get to my main question before everyone gives up on >> reading this really long and complicated explanation of the mess I made. >> >> *Q: Does anyone know of any existing scripts, documented methods, or >> best practices to follow when syncing a branch between multiple air-gapped >> repositories?* >> >> *How we are using git:* As noted above, this is NOT a typical >> development-centered use-case. Branching is very infrequent, and most work >> is done on the "master" branch in each repository. Unlike typical >> developer-centric approaches, each clone (working copy) ends up tied to a >> specific server, rather than a single developer. So multiple users end up >> working in the same working copy and committing code from one place. The >> team is small and the changes are infrequent enough that this works for us, >> despite the atypical and less-than-ideal use case. >> >> *How we are using branches:* We treat each repository as if it has >> just one branch, a single "master". However, because of the >> synchronization requirements, we create special purpose branches in each >> repository that essentially mirror the master branches of the other >> repositories. So the primary repository has 2 mirrored branches, one for >> each of the secondary repositories. And each secondary repository has a >> single mirrored branch that represents the primary (upstream) repository. >> (By convention, we have agreed never to synchronize revisions directly >> between the two secondary repositories.) Local changes are never applied >> to a mirrored repository branch, so that it should match the "master" >> branch of the mirrored repository exactly. (That is, the only changes to >> these mirrored branches are fast-forward only "pull"s made from bundle >> files exported from the mirrored repository.) The process of merging >> changes between branches is manual, and I think I want to keep it that way >> for the foreseeable future. (Perhaps one day I'll make fast-forward merges >> apply automatically, but in general I want a human to be responsible for >> this step.) So while each repositories' "master" branch may diverge, or at >> least have a slightly different history, in the end, they should all end up >> with the same content. Well, at least that's the ultimate goal. >> >> *File transfer: *Transferring bundle files between air-gapped >> environments involve multiple human steps including content review, >> approval, and some safety checks for compliance. Therefore, there's no way >> to automatically schedule synchronization, which is a bummer. That being >> said, I'd like to make this as painless as possible within the realm of >> what I can control. I'm looking to create import and export scripts (or >> find existing ones to borrow from) that handle bundle creation and the >> import process. >> >> I'm looking for a little help designing an appropriate synchronization >> solution, and would appreciate any feedback you may have. The combination >> of using git bundle and our non-traditional use case has made it difficult >> to find relevant resources. If there is anything I've missed, please point >> me in the right direction. >> >> -- > You received this message because you are subscribed to the Google Groups > "Git for human beings" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
