Filippo Giunchedi has submitted this change and it was merged. Change subject: swift-ring: manage swift rings via git ......................................................................
swift-ring: manage swift rings via git first attempt at being able to code-review changes to the swift rings, currently this taps into our normal workflow of distributing the ring files via the puppet master. That might change in the future and we can consider using trebuchet for that task too in addition to this. TODO: * the puppet master name is hard coded to palladium Change-Id: I69ce0852586b364e456cca7a6035db8cbf55c2f9 --- A .gitattributes A .gitignore A Makefile A README.md 4 files changed, 70 insertions(+), 0 deletions(-) Approvals: Ori.livneh: Looks good to me, but someone else must approve Filippo Giunchedi: Verified; Looks good to me, approved diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..9fd6084 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +*.ring.gz filter=fat -crlf +*.builder filter=fat -crlf diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4c10b4d --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +*/backups diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..b0e38e2 --- /dev/null +++ b/Makefile @@ -0,0 +1,29 @@ +DESTHOST=palladium.eqiad.wmnet +TARGETS=eqiad-prod esams-prod + +BUILDER_FILES=$(foreach dir,$(TARGETS),$(wildcard $(dir)/*.builder)) +RING_FILES=$(foreach builder,$(BUILDER_FILES),$(subst .builder,.ring.gz,$(builder))) +DUMP_FILES=$(foreach builder,$(BUILDER_FILES),$(subst .builder,.dump,$(builder))) + +rebuild: $(DUMP_FILES) $(RING_FILES) + +%.dump: %.builder + swift-ring-builder $^ > $@ + +%.ring.gz: %.builder + # don't fail when rebalance exits 1 (nothing has been done) + swift-ring-builder $^ rebalance ; [ $$? -le 1 ] || exit 2 + swift-ring-builder $^ write_ring + +# TODO(fgiunchedi): rsync HEAD, not the working tree +deploy: + rsync --progress --verbose --archive --compress --relative \ + $(RING_FILES) $(BUILDER_FILES) $(DESTHOST):swift-ring + ssh $(DESTHOST) "sudo rsync --verbose --backup --recursive \ + --links --times --chmod=a=r,Da+x \ + swift-ring/ /var/lib/puppet/volatile/swift/" + +clean: + -rm */backups/* + +.PHONY: deploy clean diff --git a/README.md b/README.md new file mode 100644 index 0000000..fac67d8 --- /dev/null +++ b/README.md @@ -0,0 +1,38 @@ +swift ring management via git +============================= + +This repository allows the management of swift rings via git. The main +reason to do this is peer reviews before pushing and better tracking and +auditing of changes. + +Swift ring files are composed by a `.builder` file that describes the ring and +a `.ring.gz` file which is the actual ring generated from the `.builder` file +by `swift-ring-builder` however both files come in binary form, making code +review tools useless and git unhappy. Thus, the solution is to dump a text +version of the ring (the `.dump` files) upon changing and review that alongside +the binary files. + + +Usage +----- + +* Make sure you have `swift-ring-builder` tool (`swift` debian package) +* Make the appropriate changes to the relevant swift cluster and ring (most + likely `object.builder`) +* Rebuild the rings that have been changed: + + make + +* Commit the result (`.ring.gz` `.builder` and `.dump` files) and send it for + review +* Once merged, deploy the rings to the puppet master: + + make deploy + + +Testing +------- +To test a deploy the DESTHOST make variable can be overridden from the command +line, e.g: + + make deploy DESTHOST=testhost.eqiad.wmflabs -- To view, visit https://gerrit.wikimedia.org/r/153584 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I69ce0852586b364e456cca7a6035db8cbf55c2f9 Gerrit-PatchSet: 5 Gerrit-Project: operations/software/swift-ring Gerrit-Branch: master Gerrit-Owner: Filippo Giunchedi <fgiunch...@wikimedia.org> Gerrit-Reviewer: Alexandros Kosiaris <akosia...@wikimedia.org> Gerrit-Reviewer: Andrew Bogott <abog...@wikimedia.org> Gerrit-Reviewer: Faidon Liambotis <fai...@wikimedia.org> Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org> Gerrit-Reviewer: Ori.livneh <o...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
