On 1 September 2011 05:59, Christian Robottom Reis <k...@canonical.com> wrote: > On Tue, Aug 30, 2011 at 11:13:08AM -0500, Zach Pfeffer wrote: >> > Any other issues that I've missed? Where should we come down in this case? >> >> I say we stay the course and fix the issue of sha's disappearing. I >> feel this way because its: >> >> 1. Technically feasible in the short term. >> 2. Saves the substantial burden of redirecting to the user. >> 3. Supports the only way that you can guarantee 100% build fidelity. > > Can you restate "fix the issue of sha's [sic] disappearing" in a way > which I can understand? As far as I know, the actual problem is that > repo isn't fetching all tags. > > Or are you describing the symptom as seen from the build system? Because > my suggestion (ship a forked repo temporarily) would address that.
Here's the symptom. Android uses a tool called repo to fetch all the source code in a project. repo reads a manifest file that lists each git it needs to clone. The syntax used to specify the git includes a feature that allows you to fetch a branch, tag or specific revision. An example of an entry of this manifest file that fetches a particular branch is: <remote name="linaro-android" fetch="git://android.git.linaro.org/" review="review.android.git.linaro.org"/> <project path="device/linaro/beagleboard" name="device/linaro/beagleboard" revision="linaro_android_2.3.5" remote="linaro-android"/> We use this for our tip builds. An example that fetches a particular revision is: <remote fetch="git://android.git.linaro.org/" name="linaro-android" review="review.android.git.linaro.org"/> <project name="platform/build" path="build" remote="linaro-android" revision="45f7969d21930a1ffa8776cd643e1637a88ce632"> We use entries like this for our release builds. We do this because the sha1 allows for the exact state of the git to be checked out. Its better than a tag or a branch because both are moveable and the tags tend to pileup. Its also very easy to create a "pinned-manifest" that just lists each sha; you can do it in automation, and you don't have to write the tags back to the subgits which you may not control. The pinned-manifest.xml allows a user to create a build that what we built exactly; this is useful if someone is trying to bisect a bug on the exact version of a build or they want to perform an experiment on the exact version. Developers who just want to work on tip, which is most developers, can just pull the manifest that tracks a particular branch. Developers are encouraged to just use tip since the pinned-manifests may not reflect the current build state. When I sync a build using the pinned-manifest.xml then I essentially do a for each git in manifest x operation. If the sha doesn't exist the repo sync fails and the user has to start over from scratch. Why would a sha disappear? git-gc handles clean up of unreachable objects. git-gc (http://www.kernel.org/pub/software/scm/git/docs/v1.7.6.1/git-gc.html) Commits that are reachable from (taken from man): Objects referenced by your current set of branches and tags Objects referenced by the index Remote-tracking branches Refs saved by git filter-branch in refs/original/ Reflogs (which may reference commits in branches that were later amended or rewound). So then what happens in a typical repo init, repo sync People who want to use a pinned-manifest.xml will run something like: repo init -u git://git.linaro.org/android/platform/manifests.git -b linaro-android-11.08-release -m LEB-panda.xml repo sync Prefix both with REPO_TRACE=1 to see exactly what git commands get executed. We create release builds using the pinned-manifest.xml since we want to reproduce the builds exactly. Again we use the sha's because we may not be able to lay a tag on a git we don't own. under the covers repo sync is doing a few things (see REPO_TRACE=1 for the whole list), but it basically comes down to: For each git: git init git fetch git pack-refs git update-ref (sha should be valid, points HEAD to branch head or sha) For each git git gc, to save space For each git git read-tree ..., to bring the tree up-to-date So repo is really doing a good job. Its saving space by getting rid of things, limiting disk-space usage, etc...and is really just using some low level git commands to accomplish its job. Anyhow, we have a few ways to solve this issue. I would say that as long as we reference trees whose shas are reachable from the branches in the remote git, then we should be fine. From a manifest perspective, we like to sync a tip branch, once the build is completed the sha will be the tip of that tip branch, as long as that sha sticks around in another branch in the repo everything should work. > -- > Christian Robottom Reis | [+55 16] 3376 0125 | http://launchpad.net/~kiko > Canonical Ltd. | [+55 16] 9112 6430 | http://async.com.br/~kiko > _______________________________________________ linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
