[Twisted-Python] 64 bit windows machine
I see from the Trac page that you guys are (or were) hunting a 64-bit windows machine. I also see there are no 64-bit Twisted packages on pypi. Are the two related? Do you still want a windows 64-bit machine? When you ask for a Visual Studio license, is not VC Express sufficient? This is possibly something we can provide for you. Cheers, Phil ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted Project Jobs Volunteer
On Mon, Nov 14, 2011 at 01:33:45PM -0500, Glyph wrote: Really the most important thing here though is just to get the automatic mirroring initially set up, not the never-ending ambassadorial work. That way git users wouldn't _need_ elaborate instructions as to what to clone and how; if we just say get twisted from github and have that automatically updated it would be easier for everyone. I've done some experimentation, and (as best I can tell) it's impossible to use git-svn to create a sensible SVN → Git conversion of the Twisted repository. This is because of the various, inconsistent branch-naming schemes that have been used over Twisted's long history. As best I can tell, git-svn can import branches that exist at some specific depth in the tree... so if your branches look like this: root | +- branches | +- add-a-feature-1234 +- fix-a-bug-2345 +- release-v1.0.x +- release-v1.1.x ...you can say git svn clone --branches='branches/*' and they'll be imported as Git branches. Likewise, if you group your branches in some way: root | +- branches | +- topic-branches | | | +- add-a-feature-1234 | +- fix-a-bug-2345 | +- release-branches | +- v1.0.x +- v1.1.x ...then you can say git svn clone --branches='branches/*/*' and they'll all be imported cleanly. Unfortunately, the Twisted repository's branches directory contains subdirectories representing branches *and* subdirectories representing groups of branches... and *those* contain both branch directories and branch-group directories. Here are the branch-group directories I've discovered so far: branches/ branches/releases/ branches/releases/conch/ branches/releases/mail/ branches/releases/names/ branches/releases/words/ When I import a range of revisions that happens to include a change to a branch in branches/releases (I've been using the range 33049:33062 for testing), git-svn winds up importing the entire releases subtree as a branch, which is... suboptimal. If I use the --ignore-paths option to ignore everything under branches/releases, then it still generates the branch and faithfully records all the commits affecting it... but ignores all the files, so when you check out that branch it deletes everything in your working directory. What's really annoying about this is that git itself doesn't care about branch naming, and it's perfectly happy to have add-a-feature-1234 and releases/v1.0.x as branch names. It's just the git-svn tool not being flexible enough. A quick Google doesn't reveal much in the way of other people who have dealt with repository layouts like this, except for one report which basically went use 'svnadmin dump' to export the repository, use sed to rewrite all the paths, load the dump into a new svn repository and import from there. That sounds like a lot of work to me, and not the sort of thing that could be reliably repeated to keep the two repositories in sync. Perhaps someone should file a bug with the git-svn maintainer, if only they had a bug-tracker. So, would a repository with a huge, bogus releases branch be acceptable in an Official Mirror? It's not strictly needed for developing patches, but it *will* create lots of puzzled frowns and annoyance, perhaps from people who want a Git mirror to integrate with their automatically download new releases code. Also in the spirit of writing things down so future contributors can figure things out, I note the Git FAQ on the Git Wiki has a How do I mirror a SVN repository to git? question: https://git.wiki.kernel.org/articles/g/i/t/GitFaq_ebc3.html ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted Project Jobs Volunteer
On 16/11/11 13:13, Tim Allen wrote: On Mon, Nov 14, 2011 at 01:33:45PM -0500, Glyph wrote: Really the most important thing here though is just to get the automatic mirroring initially set up, not the never-ending ambassadorial work. That way git users wouldn't _need_ elaborate instructions as to what to clone and how; if we just say get twisted from github and have that automatically updated it would be easier for everyone. I've done some experimentation, and (as best I can tell) it's impossible to use git-svn to create a sensible SVN → Git conversion of the Twisted repository. There's a tool called svn-all-fast-export [1] that in my experience is much faster and is much more flexible. It has some issues: * the documentation is scarce, but after some googling it's possible to get the hang of it * it's a one-off conversion (!) * AFAIR it requires access to the actual SVN repository, not just a checkout Perhaps it would be feasible for you to get a tarball of the SVN repo, write a config file for svn-all-fast-export that will make all the required branches and tags be created and then pick up with git-svn from that (I'm also not 100% sure that picking up will be possible). Anyway, just wanted to point out a tool that's been used by for instance Gnome to move from SVN to git and that I had some positive experience with personally. Cheers, Jan [1] http://repo.or.cz/w/svn-all-fast-export.git ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted Project Jobs Volunteer
On Wed, Nov 16, 2011 at 11:13:50PM +1100, Tim Allen wrote: On Mon, Nov 14, 2011 at 01:33:45PM -0500, Glyph wrote: Really the most important thing here though is just to get the automatic mirroring initially set up, not the never-ending ambassadorial work. That way git users wouldn't _need_ elaborate instructions as to what to clone and how; if we just say get twisted from github and have that automatically updated it would be easier for everyone. I've done some experimentation, and (as best I can tell) it's impossible to use git-svn to create a sensible SVN → Git conversion of the Twisted repository. This is because of the various, inconsistent branch-naming schemes that have been used over Twisted's long history. Actually, maybe git-svn by itself is not smart enough to handle all the possible cases and weird repository schemes like Twisted's one. However, I think it should be possible with a second pass script to have a pretty good, or at least better, conversion. If you init the repository with --prefix=svn/ --trunk=trunk, I think you will got a working copy with trunk, and all the branches and tags as references in the svn/ namespace. Then, with a script smart enough, which looks through the references in svn/branches/ and svn/tags/, it can create the right branches and tags like --stdlayout would do if the scheme was compatible. The difficulty I think lies into the smart enough, but if we can't find a way with this, I don't know how git-svn would be capable of. As an example of what I mean, if we want to import the tags from the Subversion repository, the job will consist in: 1. list all of the svn/tags/releases/release-* references 2. strip the release- prefix to keep only the x.y.z scheme 3. for each tag, if it doesn't already exit, git tag x.y.z svn/tags/releases/release-x.y.z Maybe it needs to be even smarter for branches, but the final step will only be git branch BRANCH_NAME SVN_REFERENCE, which might end up with a cleaner Git repository than the original Subversion repository... If some help is needed, I'm willing to help too. I'm not sure to fully understand where and how this conversion is heading to. My 2 cents, Jonathan ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted Project Jobs Volunteer
On Wed, Nov 16, 2011 at 6:41 AM, Jonathan Ballet j...@multani.info wrote: On Wed, Nov 16, 2011 at 11:13:50PM +1100, Tim Allen wrote: I've done some experimentation, and (as best I can tell) it's impossible to use git-svn to create a sensible SVN → Git conversion of the Twisted repository. This is because of the various, inconsistent branch-naming schemes that have been used over Twisted's long history. Actually, maybe git-svn by itself is not smart enough to handle all the possible cases and weird repository schemes like Twisted's one. Yes, git-svn by itself cannot handle Twisted's repository both because of the repository layout that is different from what it expects, but also because if git-svn tries to mirror a repository with a large number of branches all at once it slows to a molasses-in-January crawl. However, I think it should be possible with a second pass script to have a pretty good, or at least better, conversion. The solution that I've come up with is to use git-svn to create a separate Git repository for each branch, and then git-push that branch into a central Git repository. This does burn *a lot* of disk space, but only one person needs to do it - everyone else can clone the central repository. The initial setup takes a while, but subsequent runs should be relatively fast. You can see the converted repository I'm building at: https://github.com/jcollie/twisted It's still in the process of building up so not all the branches are there yet. I've put the script that I'm using here: https://github.com/jcollie/twisted-svn-convert One feature that I'd like to add before calling this final is converting SVN usernames to proper names/email addresses like Git prefers to use. I use a similar script to mirror the Asterisk SVN repository with some good results: https://github.com/jcollie/asterisk -- Jeff Ollie ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] 64 bit windows machine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/16/2011 12:33 PM, Phil Mayers wrote: I see from the Trac page that you guys are (or were) hunting a 64-bit windows machine. I also see there are no 64-bit Twisted packages on pypi. Thanks for the offer but I think there's already a 64-bit python 2.7 buildslave in use so it seems the wiki is out of date.. http://buildbot.twistedmatrix.com/buildslaves/bot-glyph-6 I'm not sure about the 64-bit packages, but its likely these builders were added after the 11.0 release and the upcoming 11.1 will have those packages. Thijs -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAk7D99cACgkQBRbMGagbo3nlfgCeKufnUeANVc0eGMk7adDiK7Ho 50EAn3JZQZmwKgrk7IZkYIZ/Wl1I2b1T =zsz/ -END PGP SIGNATURE- ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] 64 bit windows machine
On 11/16/2011 05:50 PM, Thijs Triemstra wrote: Thanks for the offer but I think there's already a 64-bit python 2.7 buildslave in use so it seems the wiki is out of date.. http://buildbot.twistedmatrix.com/buildslaves/bot-glyph-6 I'm not sure about the 64-bit packages, but its likely these builders were added after the 11.0 release and the upcoming 11.1 will have those packages. Cool. Good to hear. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Log output formatting (was Re: logging question)
On 11/15/2011 8:57 AM, Christopher Armstrong wrote: If you pass a 'system' kwarg to log.msg then it will go in that field. log.msg(hi, system=stuff) results in date [stuff] hi This triggered me to look further into what LogPublisher.msg does, since I've long wanted to customize the format of the output. I found what I wanted in http://twistedmatrix.com/documents/current/api/twisted.python.log.ILogObserver.html : In msg(), the message gets added to the kwargs dictionary (along with a timestamp called 'time'), and that dict is passed to LogObserver.emit, which uses textFromEventDict() to create what's actually output. This dictionary has the following keys recognized by textFromEventDict: * |message|: A |tuple| of |str| containing messages to be logged. * |system|: A |str| which indicates the system which is generating this event. * |isError|: A |bool| indicating whether this event represents an error. * |failure|: A |failure.Failure| http://twistedmatrix.com/documents/current/api/twisted.python.failure.Failure.html instance, required if the event is an error. * |why|: Used as header of the traceback in case of errors. * |format|: A string format used in place of |message| to customize the event. The intent is for the observer to format a message by doing something like |format % eventDict|. So, for example, you could do log.msg(format='', foo='stuff', bar='things') I suggest that this be added to the documentation string of the msg() method and http://twistedmatrix.com/documents/current/core/howto/logging.html. -- Don Dwiggins Advanced Publishing Technology ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] 64 bit windows machine
On Nov 16, 2011, at 3:16 PM, Phil Mayers wrote: On 11/16/2011 05:50 PM, Thijs Triemstra wrote: Thanks for the offer but I think there's already a 64-bit python 2.7 buildslave in use so it seems the wiki is out of date.. http://buildbot.twistedmatrix.com/buildslaves/bot-glyph-6 I'm not sure about the 64-bit packages, but its likely these builders were added after the 11.0 release and the upcoming 11.1 will have those packages. Cool. Good to hear. It's always helpful to get more hardware for the build farm though; there are more configurations that we want to test than we can possibly buy enough hardware for :). It would also be very helpful if you could contribute some time to help administer our existing Windows buildslaves; Windows expertise (especially administrator expertise) is fairly thin among the core developer community. What were you thinking of offering? -glyph ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] gnupg protocol implementation, does one exist?
There is a bit of an example in http://twistedmatrix.com/documents/current/core/howto/process.html but I'm sure somebody out there must have fleshed this out into a more complete implementation. Can anybody point me to such a thing, before we write out own? We are currently using GnuPGInterface.py, but its non-twisted child process handling is causing flakyness, especially around the waitpid() call. Thanks, Sam ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted Project Jobs Volunteer
On Wed, Nov 16, 2011 at 08:17:51AM -0600, Jeffrey Ollie wrote: The solution that I've come up with is to use git-svn to create a separate Git repository for each branch, and then git-push that branch into a central Git repository. This does burn *a lot* of disk space, but only one person needs to do it - everyone else can clone the central repository. The initial setup takes a while, but subsequent runs should be relatively fast. You can see the converted repository I'm building at: https://github.com/jcollie/twisted It's still in the process of building up so not all the branches are there yet. Oooh. That's looking pretty good. As you say, not all the branches are there, and none of the tags, but I can see you've already successfully imported some of the tricky release branches. I've put the script that I'm using here: https://github.com/jcollie/twisted-svn-convert One feature that I'd like to add before calling this final is converting SVN usernames to proper names/email addresses like Git prefers to use. I see your script repository already has the beginnings of such a mapping file; unfortunately you've already filled in most of the names I recognise, and I don't know if there's any other authoritative list of such information (and of course, it would need to be updated as Twisted adds new committers). The how to mirror Git to SVN section of the Git FAQ includes information on how to take a cloned git repository and add git-svn magic so you can commit to the original upstream repository; I guess that wouldn't really work with your custom import script. I think this would be a much better base for an Official Twisted Git Mirror than anything I've come up with so far. How difficult would it be for somebody unfamiliar with Git (such as, say, the Twisted core devs) to keep running? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
