On 26-Nov-2015, Barry Warsaw wrote: > +1 for adding DPMT to Maintainers or Uploaders, but yes, it would > have to go into DPMT git as a source-full repo.
Okay, I've been struggling mightily with Git. Both in the “working together” sense, and in the “fighting against” sense. Can you help? I'm quite loath to lose the simplicity of a Debian packaging repository. There is a deliberate layer of separation that has served very well in the VCS to date for this package: the separation of VCS repos matches the separation of concerns. Debian's packaging is of necessity applied on top of whatever upstream does with their source. In order to have the history of packaging work retroactively include upstream's sources – in order for every past revision in the VCS be accompanied in the VCS by whatever upstream source is referenced by that packaging work – I assume we need to import and merge the upstream sources at each of those states, from the upstream tarballs. This I have tried to do, with only moderate success. The failures only confirm me more deeply in my opposition to combining Debian packaging work with upstream source in the same VCS. The current work-in-progress Git VCS repository: * Uses the ‘debian/*’, ‘upstream/*’ conventional tags. * Includes, in the ‘master’ branch, all Debian packaging history. These all have the correct author and timestamp, imported from Bazaar. * Has a Git-compatible tag for each Debian release. * Includes, in the ‘upstream’ branch, a commit of upstream source for each new upstream version referenced in the Debian packaging. These ‘pristine-tar’ branch contains data for the ‘pristine-tar’ tool to regenerate each tarball. The commits on ‘upstream’ and ‘pristine-tar’ each have a timestamp matching the timestamp on the upstream tarball that was imported. * Merges each upstream version, in sequence, into ‘master’. I've attached a document that shows the sequence of commands I recorded as a recipe for what I've achieved so far. The part that has me despairing is how to get the merges of upstream source to appear in ‘master’ at the *correct* place for the subsequent Debian packaging work referencing that upstream version. At the moment, all the merges of each historical upstream version appears *after* all the historical Debian packaging work. What's needed, of course, is to interleave them at the appropriate points. I can't even get this to happen manually, let alone automatically. I've tried all manner of rebasing, grafting, and other obscure Git concepts. Git seems incapable of dealing with merge history in a coherent way; merge commits keep disappearing, as though Git really doesn't want them to exist. That's not good enough; merges, complete with all the commits they include and the timestamps of those commmits, are valuable historical information that needs to be preserved through the transition. Nothing I can do will simultaneously preserve the data painstakingly translated from Bazaar, while getting the merge sequence correct. That's all prior to even beginning to think about teaching ‘git-dpm’ about the patches from Debian packaging history (in sufficient detail that I can record that, too, in the transition document). I need to get just the history of upstream and Debian work coherent, first. I've been at this for days now, trashing and re-creating and importing and manipulating the repo over and over to try to get a reliable recipe, and I'm out of puff. Can you help? The work-in-progress (hence, subject to upheval) Git repository is at <URL:ssh://git.debian.org/git/collab-maint/python-lockfile/trunk-wip.git>. -- \ “I bet one legend that keeps recurring throughout history, in | `\ every culture, is the story of Popeye.” —Jack Handey | _o__) | Ben Finney <b...@benfinney.id.au>
Create Git-compatible tags in Bazaar VCS
========================================
Bazaar happily allows a tag to be named with any valid Debian release
string.
Git has baroque reference syntax that reserves a bunch of punctuation
characters, which severely restricts the set of Git tag names. We need
to translate the names to something Git can use.
Get the releases from the Debian changelog, as tuples of
(full_version, bzr_tag, git_tag)::
releases=$(
python3 <<_EOT_ | uniq
import debian.changelog
with open("debian/changelog") as infile:
changelog = debian.changelog.Changelog(infile, allow_empty_author=True)
for version in sorted(set(changelog.versions)):
bzr_tag = "release {0.full_version}".format(version)
git_compatible_version = version.full_version.translate(
str.maketrans({':': "%", '~': "_"}))
git_tag = "release/{0}".format(git_compatible_version)
print("{0.full_version}\t{1}\t{2}'".format(
version, bzr_tag, git_tag))
_EOT_
)
Create a corresponding Git-compatible tag for each release::
echo "${releases}" \
| while IFS="$(printf "\t\n")" read full_version bzr_tag git_tag
do
bzr tag --revision "tag:${bzr_tag}" "${git_tag}"
done
Create Git repository
=====================
Make the repository and configure it::
cd ~/Projects/debian/python-lockfile/
rm -rf trunk/
git init trunk/
cd trunk/
git config user.name "$DEBFULLNAME"
git config user.email "$DEBEMAIL"
git config user.signingkey "$DEBSIGN_KEYID"
git config branch.master.description "Debian mainline development."
Commit an empty revision tagged “initial” that we can later use as
common root::
git commit --allow-empty --message="Initial empty commit."
git tag "initial"
Import Debian packaging work from Bazaar VCS
============================================
Use the ‘git-remote-bzr’ extension to connect to the Bazaar repository
as a remote::
git remote add alioth-bzr
bzr::bzr+ssh://bzr.debian.org/bzr/collab-maint/pkg-python-lockfile/trunk/
git config remote.alioth-bzr.description "Debian packaging work in Bazaar
VCS at Alioth."
Fetch all the Debian packaging history to the ‘master’ branch::
git checkout master
git fetch alioth-bzr
git pull --rebase alioth-bzr master
Connect the imported history to our common empty root commit::
for root in $(git rev-list --max-parents=0 HEAD) ; do
git replace --graft "$root" "initial"
done
Import each upstream tarball
============================
Get the releases from the Debian changelog, as tuples of
(upstream_version, debian_upstream_version)::
upstream_versions=$(
python3 <<_EOT_ | uniq
import debian.changelog
with open("debian/changelog") as infile:
changelog = debian.changelog.Changelog(infile, allow_empty_author=True)
for version in sorted(set(changelog.versions)):
version.debian_upstream_version = "".join([
("{0.epoch}:" if version.epoch else "").format(version),
version.upstream_version])
print("{0.upstream_version}
{0.debian_upstream_version}".format(version))
_EOT_
)
Make the branches we'll need for upstream tarballs, and ‘pristine-tar’
metadata::
git branch upstream "initial"
git config branch.upstream.description "Upstream mainline development."
git branch pristine-tar "initial"
git config branch.pristine-tar.description "Data to recreate pristine
upstream tarballs."
Import the source corresponding to each release that needs a new
upstream tarball, and then merge that source into ‘master’::
echo "${upstream_versions}" \
| while read upstream_version debian_upstream_version
do
tarfile_name="python-lockfile_${upstream_version}.orig.tar.gz"
tarfile_path="../tarballs/$tarfile_name"
tarfile_timestamp="$(date --reference "$tarfile_path")"
git checkout upstream
rm -r $(ls -1 --almost-all | grep -v '^\.git$')
tar --strip-components=1 -xf "$tarfile_path"
git add .
git commit \
--date="$tarfile_timestamp" \
--message="Switch to upstream source from tarball ‘$tarfile_name’."
git_alioth_tag="upstream/alioth-bzr/${debian_upstream_version/:/%}"
git tag "${git_alioth_tag}"
pristine-tar commit "$tarfile_path" upstream
git checkout pristine-tar
git commit --amend --no-edit --date="${tarfile_timestamp}"
git checkout master
git merge --no-ff --no-commit "${git_alioth_tag}"
git commit --reuse-message "${git_alioth_tag}"
git commit --amend \
--message="Merge Debian upstream version
“$debian_upstream_version”."
git_upstream_tag="upstream/${upstream_version}"
git tag "${git_upstream_tag}"
done
Reorganise sequence of upstream versions and Debian releases
============================================================
This part doesn't work; I can't convince ‘git rebase’ to move merges
from ‘upstream’ around in history while preserving the full
information.
Git's ‘rebase’ keeps presenting conflicts, that are surely bogus. The
Debian packaging work on ‘master’ solely touches the ‘debian/’
directory, and nothing in ‘upstream’ ever touches anything in that
directory. The two are quite distinct on the filesystem, so there
should never be a conflict by moving its merges around. And yet, they
appear.
Or, it keeps flattening the history so that all the merges disappear.
That's unacceptable, and blatantly contradicts the ‘--preserve-merges’
option.
So either I've unwittingly made the game un-winnable by some earlier
move, or Git's godawful UI is obscuring the magic incantation needed.
For now, I give up.
::
git rebase --interactive --committer-date-is-author-date --preserve-merges
--keep-empty
git tag --delete "initial"
..
Local variables:
coding: utf-8
mode: text
mode: rst
End:
vim: fileencoding=utf-8 filetype=rst :
signature.asc
Description: PGP signature
