Re: [PATCH v3] nmbug: Add an 'init' command

[W. Trevor King]

On Sat, Nov 29, 2014 at 12:09:17PM -0800, W. Trevor King wrote:
 On Sat, Nov 29, 2014 at 10:40:01AM +0100, Michal Sojka wrote:
  On Út, říj 28 2014, W. Trevor King wrote:
   +_spawn(
   +args=['git', 'init', '--separate-git-dir', NMBGIT, workdir],
   +wait=True)
   +_git(args=['config', '--unset', 'core.worktree'], wait=True)
   +_git(args=['config', 'core.bare', 'true'], wait=True)
  
  Why do you create a non-bare repository and then make it bare?
 
 Bare repositories don't usually have upstream tracking branches.  See
 the commit message for c2001674 (nmbug: Add 'clone' and replace
 FETCH_HEAD with @{upstream}, 2014-03-09) for details [1].  I can
 resubmit this patch with a commit message that mentions the
 explanation in c2001674 if folks want the extra clarity here.

Ah, it seems that the lack of remote-tracking branches is unique to
'clone --bare'.  When I compare 'init --bare' with the above (using
Git 2.1.0), the only difference in the resulting repository is that
the the 'init --bare' form doesn't set core.logallrefupdates, while
the form above sets core.logallrefupdates to true.  I'll submit v4
using 'init --bare' and an explicit core.logallrefupdates config.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Mail archives in Git using ssoma

[W. Trevor King]

Hello everyone :),

I like Git, so when folks suggest storing things in Git, I'm usually
excited ;).  Eric Wong has been working on some tools to store email
in a Git repository, and his client-side code is ssoma [1].  I wanted
a bit more metadata than the stock ssoma-mda [2], and ended up just
writing a ssoma-mda in Python [3].  It needs Python ?3.4 and pygit2.
I had pygit2 already installed for Python 3.3 (which gave me a local
libgit2), so I used pip to install it for 3.4:

  $ python3.4 -m ensurepip --user
  $ pip3.4 install --user pygit2

Then I grabbed the archives, and pulled them into Git:

  $ wget http://notmuchmail.org/archives/notmuch.mbox
  $ git init --bare notmuch-archives.git
  $ cd notmuch-archives.git
  $ python3.4
  >>> import email.utils
  >>> import mailbox
  >>> import ssoma_mda
  >>> mbox = mailbox.mbox('../notmuch.mbox', factory=None, create=False)
  >>> messages = sorted(mbox, key=lambda m: 
email.utils.mktime_tz(email.utils.parsedate_tz(m['date'])))
  >>> for message in messages:
  ... if ((message['message-id'] == '' and
  ... message['X-List-Received-Date'] == 'Sat, 26 Feb 2011 14:23:34 
-') or
  ...   (message['message-id'] == '<4EDF728E.3050204 at gmail.com>' and
  ... message['X-List-Received-Date'] == 'Wed, 07 Dec 2011 14:05:16 
-') or
  ...   (message['message-id'] == <4FE369F2.5080804 at gmail.com>' and
  ... message['X-List-Received-Date'] == 'Thu, 21 Jun 2012 18:38:07 
-') or
  ...   (message['message-id'] == '<5122353D.4060601 at gmail.com>' and
  ... message['X-List-Received-Date'] == 'Mon, 18 Feb 2013 14:06:12 
-') or
  ...   (message['message-id'] == 
'' and
  ... message['X-List-Received-Date'] == 'Wed, 24 Apr 2013 18:09:55 
-') or
  ...   (message['message-id'] == '<527B9E8C.501 at krugs.de>' and
  ... message['X-List-Received-Date'] == 'Thu, 07 Nov 2013 14:07:32 
-') or
  ...   (message['message-id'] == 
'<1399645162-8653-1-git-send-email-wael.nasreddine at gmail.com>' and
  ... message['X-List-Received-Date'] == 'Fri, 09 May 2014 14:19:36 
-') or
  ...   (message['message-id'] == '' and
  ... message['X-List-Received-Date'] == 'Thu, 18 Sep 2014 10:27:35 
-') or
  ...   (message['message-id'] == '' and
  ... message['X-List-Received-Date'] != 'Mon, 22 Sep 2014 09:54:16 
-')):
  ... continue
  ... ssoma_mda.deliver(message=message, once=True)
  >>> ^D

On my 1.1GHz Intel Celeron 847 Sandy Bridge netbook, that took about
half an hour.  The initial repository was large:

  $ du -hs .
  394M.

But packing it up made it small:

  $ git gc --aggressive
  du -hs .
  51M .

With a few less images than the mbox:

  $ git log --oneline | wc -l
  19650

Compared with 19660 messages in the mbox at 107 MB (160 MB for the
associated Maildir).

The messages I dropped removed duplicate Message-IDs:

* id:m2k4gmyjer.fsf at ecocode.net had different received dates:

-X-List-Received-Date: Sat, 26 Feb 2011 14:12:20 -
+X-List-Received-Date: Sat, 26 Feb 2011 14:23:34 -

  but no significant differences.

* id:4EDF728E.3050204 at gmail.com had a real address in the
  first-to-arrive version:

-X-List-Received-Date: Wed, 07 Dec 2011 14:10:13 -
-> <4winter at informatik.uni-hamburg.de>

  an an obfuscated one in the second-to-arrive version:

+X-List-Received-Date: Wed, 07 Dec 2011 14:05:16 -
+> <4winter-jNDFPZUTrfQBEfOqpokbeYV0Y/DQsy6Ps0AfqQuZ5sE at public.gmane.org>

* id:4FE369F2.5080804 at gmail.com had the same:

-X-List-Received-Date: Thu, 21 Jun 2012 18:37:54 -
->  > wrote:

* id:5122353D.4060601 at gmail.com had different received dates:

-X-List-Received-Date: Mon, 18 Feb 2013 14:06:05 -
+X-List-Received-Date: Mon, 18 Feb 2013 14:06:12 -

  but no significant differences.

* id:CA+eQo_1hMsTD4+6ifqgEQXW0_qYXGOdfkO6tBuGQKV+W7OSaKA at mail.gmail.com
  had different MIME boundaries:

-Content-Type: multipart/alternative; boundary=f46d043be11ac45a0904db1f3428
-X-List-Received-Date: Wed, 24 Apr 2013 18:09:46 -

+Content-Type: multipart/alternative; boundary=e89a8f646ff3faa11d04db1f3294
+X-List-Received-Date: Wed, 24 Apr 2013 18:09:55 -

  but no significant differences.

* id:527B9E8C.501 at krugs.de had obfuscated addresses:

-X-List-Received-Date: Thu, 07 Nov 2013 14:07:33 -
-> Rainer M Krug  writes:

+X-List-Received-Date: Thu, 07 Nov 2013 14:07:32 -
+> Rainer M Krug  writes:

* id:1399645162-8653-1-git-send-email-wael.nasreddine at gmail.com had
  additional content in the later submission:

-Subject: [PATCH] Add Travis-CI config file.
-Date: Fri,  9 May 2014 07:19:22 -0700
-X-List-Received-Date: Fri, 09 May 2014 

Mail archives in Git using ssoma

[W. Trevor King]

Hello everyone :),

I like Git, so when folks suggest storing things in Git, I'm usually
excited ;).  Eric Wong has been working on some tools to store email
in a Git repository, and his client-side code is ssoma [1].  I wanted
a bit more metadata than the stock ssoma-mda [2], and ended up just
writing a ssoma-mda in Python [3].  It needs Python ≥3.4 and pygit2.
I had pygit2 already installed for Python 3.3 (which gave me a local
libgit2), so I used pip to install it for 3.4:

  $ python3.4 -m ensurepip --user
  $ pip3.4 install --user pygit2

Then I grabbed the archives, and pulled them into Git:

  $ wget http://notmuchmail.org/archives/notmuch.mbox
  $ git init --bare notmuch-archives.git
  $ cd notmuch-archives.git
  $ python3.4
   import email.utils
   import mailbox
   import ssoma_mda
   mbox = mailbox.mbox('../notmuch.mbox', factory=None, create=False)
   messages = sorted(mbox, key=lambda m: 
email.utils.mktime_tz(email.utils.parsedate_tz(m['date'])))
   for message in messages:
  ... if ((message['message-id'] == 'm2k4gmyjer@ecocode.net' and
  ... message['X-List-Received-Date'] == 'Sat, 26 Feb 2011 14:23:34 
-') or
  ...   (message['message-id'] == '4edf728e.3050...@gmail.com' and
  ... message['X-List-Received-Date'] == 'Wed, 07 Dec 2011 14:05:16 
-') or
  ...   (message['message-id'] == 4fe369f2.5080...@gmail.com' and
  ... message['X-List-Received-Date'] == 'Thu, 21 Jun 2012 18:38:07 
-') or
  ...   (message['message-id'] == '5122353d.4060...@gmail.com' and
  ... message['X-List-Received-Date'] == 'Mon, 18 Feb 2013 14:06:12 
-') or
  ...   (message['message-id'] == 
'ca+eqo_1hmstd4+6ifqgeqxw0_qyxgodfko6tbugqkv+w7os...@mail.gmail.com' and
  ... message['X-List-Received-Date'] == 'Wed, 24 Apr 2013 18:09:55 
-') or
  ...   (message['message-id'] == '527b9e8c.5000...@krugs.de' and
  ... message['X-List-Received-Date'] == 'Thu, 07 Nov 2013 14:07:32 
-') or
  ...   (message['message-id'] == 
'1399645162-8653-1-git-send-email-wael.nasredd...@gmail.com' and
  ... message['X-List-Received-Date'] == 'Fri, 09 May 2014 14:19:36 
-') or
  ...   (message['message-id'] == 'm2mw9xkyvg@krugs.de' and
  ... message['X-List-Received-Date'] == 'Thu, 18 Sep 2014 10:27:35 
-') or
  ...   (message['message-id'] == 
'cover.1411379395.git.j...@nikula.org' and
  ... message['X-List-Received-Date'] != 'Mon, 22 Sep 2014 09:54:16 
-')):
  ... continue
  ... ssoma_mda.deliver(message=message, once=True)
   ^D

On my 1.1GHz Intel Celeron 847 Sandy Bridge netbook, that took about
half an hour.  The initial repository was large:

  $ du -hs .
  394M.

But packing it up made it small:

  $ git gc --aggressive
  du -hs .
  51M .

With a few less images than the mbox:

  $ git log --oneline | wc -l
  19650

Compared with 19660 messages in the mbox at 107 MB (160 MB for the
associated Maildir).

The messages I dropped removed duplicate Message-IDs:

* id:m2k4gmyjer@ecocode.net had different received dates:

-X-List-Received-Date: Sat, 26 Feb 2011 14:12:20 -
+X-List-Received-Date: Sat, 26 Feb 2011 14:23:34 -

  but no significant differences.

* id:4edf728e.3050...@gmail.com had a real address in the
  first-to-arrive version:

-X-List-Received-Date: Wed, 07 Dec 2011 14:10:13 -
- 4win...@informatik.uni-hamburg.de

  an an obfuscated one in the second-to-arrive version:

+X-List-Received-Date: Wed, 07 Dec 2011 14:05:16 -
+ 4winter-jNDFPZUTrfQBEfOqpokbeYV0Y/dqsy6ps0afqquz...@public.gmane.org

* id:4fe369f2.5080...@gmail.com had the same:

-X-List-Received-Date: Thu, 21 Jun 2012 18:37:54 -
- r.m.k...@gmail.com

+X-List-Received-Date: Thu, 21 Jun 2012 18:38:07 -
- mailto:r.m.k...@gmail.com wrote:

* id:5122353d.4060...@gmail.com had different received dates:

-X-List-Received-Date: Mon, 18 Feb 2013 14:06:05 -
+X-List-Received-Date: Mon, 18 Feb 2013 14:06:12 -

  but no significant differences.

* id:ca+eqo_1hmstd4+6ifqgeqxw0_qyxgodfko6tbugqkv+w7os...@mail.gmail.com
  had different MIME boundaries:

-Content-Type: multipart/alternative; boundary=f46d043be11ac45a0904db1f3428
-X-List-Received-Date: Wed, 24 Apr 2013 18:09:46 -

+Content-Type: multipart/alternative; boundary=e89a8f646ff3faa11d04db1f3294
+X-List-Received-Date: Wed, 24 Apr 2013 18:09:55 -

  but no significant differences.

* id:527b9e8c.5000...@krugs.de had obfuscated addresses:

-X-List-Received-Date: Thu, 07 Nov 2013 14:07:33 -
- Rainer M Krug rai...@krugs.de writes:

+X-List-Received-Date: Thu, 07 Nov 2013 14:07:32 -
+ Rainer M Krug Rainer-vfylz/ys...@public.gmane.org writes:

* id:1399645162-8653-1-git-send-email-wael.nasredd...@gmail.com had
  additional content in the later submission:

-Subject: 

[PATCH] test: Make gen-threads work with python3

[W. Trevor King]

On Fri, Oct 31, 2014 at 02:04:50PM -0400, Jesse Rosenthal wrote:
> W. Trevor King writes:
> > On Fri, Oct 31, 2014 at 01:33:25PM -0400, Jesse Rosenthal wrote:
> >> We instead initalize the dictionary using the dict comprehension
> >> and then update it with the values from the tree. This will work
> >> with both python2 and python3.
> >
> > Dict comprehensions are new in 2.7 [1,2], so this drops support
> > for systems where ?python? means ?python2.6?.  Personally, I'm
> > fine with that, but I thought I'd point it out in case 2.6 users
> > wanted to push back ;).
> 
> The comprehension was already in the previous version, so I figured that
> people were already cool with 2.7+.

Ah, good point :).  Besides looking good to me, I can confirm that I
see a TypeError (and a lot of CPU usage ;) from T260-thread-order.sh
before this patch which is fixed by this patch.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20141031/4d4eaff3/attachment-0001.pgp>

[PATCH] test: Make gen-threads work with python3

[W. Trevor King]

On Fri, Oct 31, 2014 at 01:33:25PM -0400, Jesse Rosenthal wrote:
> We instead initalize the dictionary using the dict comprehension and
> then update it with the values from the tree. This will work with
> both python2 and python3.

Dict comprehensions are new in 2.7 [1,2], so this drops support for
systems where ?python? means ?python2.6?.  Personally, I'm fine with
that, but I thought I'd point it out in case 2.6 users wanted to push
back ;).

> diff --git a/test/gen-threads.py b/test/gen-threads.py
> index 9fbb847..70fb1f6 100644
> --- a/test/gen-threads.py
> +++ b/test/gen-threads.py
> @@ -2,7 +2,6 @@
>  # argv[1].  Each output line is a thread structure, where the n'th
>  # field is either a number giving the parent of message n or "None"
>  # for the root.
> -
>  import sys

Why remove this blank line?

>  from itertools import chain, combinations
>  
> @@ -28,6 +27,7 @@ while queue:
>  else:
>  # Expand node to_expand[0] with each possible set of children
>  for children in subsets(free):
> -ntree = dict(tree, **{child: to_expand[0] for child in children})
> +ntree = {child: to_expand[0] for child in children}
> +ntree.update(tree)

This looks good to me.

Cheers,
Trevor

[1]: https://docs.python.org/3/whatsnew/2.7.html#other-language-changes
[2]: http://legacy.python.org/dev/peps/pep-0274/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

Re: [PATCH] test: Make gen-threads work with python3

[W. Trevor King]

On Fri, Oct 31, 2014 at 01:33:25PM -0400, Jesse Rosenthal wrote:
 We instead initalize the dictionary using the dict comprehension and
 then update it with the values from the tree. This will work with
 both python2 and python3.

Dict comprehensions are new in 2.7 [1,2], so this drops support for
systems where ‘python’ means ‘python2.6’.  Personally, I'm fine with
that, but I thought I'd point it out in case 2.6 users wanted to push
back ;).

 diff --git a/test/gen-threads.py b/test/gen-threads.py
 index 9fbb847..70fb1f6 100644
 --- a/test/gen-threads.py
 +++ b/test/gen-threads.py
 @@ -2,7 +2,6 @@
  # argv[1].  Each output line is a thread structure, where the n'th
  # field is either a number giving the parent of message n or None
  # for the root.
 -
  import sys

Why remove this blank line?

  from itertools import chain, combinations
  
 @@ -28,6 +27,7 @@ while queue:
  else:
  # Expand node to_expand[0] with each possible set of children
  for children in subsets(free):
 -ntree = dict(tree, **{child: to_expand[0] for child in children})
 +ntree = {child: to_expand[0] for child in children}
 +ntree.update(tree)

This looks good to me.

Cheers,
Trevor

[1]: https://docs.python.org/3/whatsnew/2.7.html#other-language-changes
[2]: http://legacy.python.org/dev/peps/pep-0274/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: Make gen-threads work with python3

[W. Trevor King]

On Fri, Oct 31, 2014 at 02:04:50PM -0400, Jesse Rosenthal wrote:
 W. Trevor King writes:
  On Fri, Oct 31, 2014 at 01:33:25PM -0400, Jesse Rosenthal wrote:
  We instead initalize the dictionary using the dict comprehension
  and then update it with the values from the tree. This will work
  with both python2 and python3.
 
  Dict comprehensions are new in 2.7 [1,2], so this drops support
  for systems where ‘python’ means ‘python2.6’.  Personally, I'm
  fine with that, but I thought I'd point it out in case 2.6 users
  wanted to push back ;).
 
 The comprehension was already in the previous version, so I figured that
 people were already cool with 2.7+.

Ah, good point :).  Besides looking good to me, I can confirm that I
see a TypeError (and a lot of CPU usage ;) from T260-thread-order.sh
before this patch which is fixed by this patch.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] nmbug: Add an 'init' command

[W. Trevor King]

For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.

The empty-blob hash-object call avoids errors like:

  $ nmbug commit
  error: invalid object 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 for
'tags/...'
  fatal: git-write-tree: error building trees
  'git HASH(0x9ef3eb8) write-tree' exited with nonzero value

David Bremner suggested [1]:

  $ git hash-object -w /dev/null

instead of my Python version of:

  $ git hash-object -w --stdin <&-

but I expect that closing stdin is more portable than the /dev/null
path (which doesn't exist on Windows, for example).

[1]: id:87y4vu6uvf.fsf at maritornes.cs.unb.ca
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18720
---
The only change since v2 [1] is a commit-message tweak:

* Mention Windows as an OS with stdin but no /dev/null [2].

Cheers,
Trevor

[1]: id:eaa9cf1cb3c00c591dc675c0f314ca31909ff74c.1412965476.git.wking at 
tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19289
[2]: id:20141011071000.GB10926 at odin.tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19294

 devel/nmbug/nmbug | 24 
 1 file changed, 24 insertions(+)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..23bac5c 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -373,6 +373,29 @@ def fetch(remote=None):
 _git(args=args, wait=True)


+def init(remote=None):
+"""
+Create an empty nmbug repository.
+
+This wraps 'git init' with a few extra steps to support subsequent
+status and commit commands.
+"""
+with _tempfile.TemporaryDirectory(prefix='nmbug-init.') as workdir:
+_spawn(
+args=['git', 'init', '--separate-git-dir', NMBGIT, workdir],
+wait=True)
+_git(args=['config', '--unset', 'core.worktree'], wait=True)
+_git(args=['config', 'core.bare', 'true'], wait=True)
+# create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
+_git(args=['hash-object', '-w', '--stdin'], input='', wait=True)
+_git(
+args=[
+'commit', '--allow-empty', '-m', 'Start a new nmbug repository'
+],
+additional_env={'GIT_WORK_TREE': workdir},
+wait=True)
+
+
 def checkout():
 """
 Update the notmuch database from Git.
@@ -703,6 +726,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'init',
 'log',
 'merge',
 'pull',
-- 
2.1.0.60.g85f0837

[PATCH v3] nmbug: Add an 'init' command

[W. Trevor King]

For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.

The empty-blob hash-object call avoids errors like:

  $ nmbug commit
  error: invalid object 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 for
'tags/...'
  fatal: git-write-tree: error building trees
  'git HASH(0x9ef3eb8) write-tree' exited with nonzero value

David Bremner suggested [1]:

  $ git hash-object -w /dev/null

instead of my Python version of:

  $ git hash-object -w --stdin -

but I expect that closing stdin is more portable than the /dev/null
path (which doesn't exist on Windows, for example).

[1]: id:87y4vu6uvf@maritornes.cs.unb.ca
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18720
---
The only change since v2 [1] is a commit-message tweak:

* Mention Windows as an OS with stdin but no /dev/null [2].

Cheers,
Trevor

[1]: id:eaa9cf1cb3c00c591dc675c0f314ca31909ff74c.1412965476.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19289
[2]: id:20141011071000.gb10...@odin.tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19294

 devel/nmbug/nmbug | 24 
 1 file changed, 24 insertions(+)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..23bac5c 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -373,6 +373,29 @@ def fetch(remote=None):
 _git(args=args, wait=True)
 
 
+def init(remote=None):
+
+Create an empty nmbug repository.
+
+This wraps 'git init' with a few extra steps to support subsequent
+status and commit commands.
+
+with _tempfile.TemporaryDirectory(prefix='nmbug-init.') as workdir:
+_spawn(
+args=['git', 'init', '--separate-git-dir', NMBGIT, workdir],
+wait=True)
+_git(args=['config', '--unset', 'core.worktree'], wait=True)
+_git(args=['config', 'core.bare', 'true'], wait=True)
+# create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
+_git(args=['hash-object', '-w', '--stdin'], input='', wait=True)
+_git(
+args=[
+'commit', '--allow-empty', '-m', 'Start a new nmbug repository'
+],
+additional_env={'GIT_WORK_TREE': workdir},
+wait=True)
+
+
 def checkout():
 
 Update the notmuch database from Git.
@@ -703,6 +726,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'init',
 'log',
 'merge',
 'pull',
-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

On Sat, Oct 11, 2014 at 09:02:20AM +0200, David Bremner wrote:
> W. Trevor King writes:
> > On Sat, Oct 11, 2014 at 06:53:11AM +0200, David Bremner wrote:
> >> W. Trevor King writes:
> >> > but I expect that closing stdin is more portable than the
> >> > /dev/null path.
> >> 
> >> /dev/null is part of POSIX
> >
> > Maybe folks want to use nmbug on Windows or some other crazy
> > non-POSIX OS?  I don't know how Windows-compatible the rest of
> > notmuch is (it looks like Xapian can be built with MSYS+mingw or
> > MSVC [1,2]), and I don't think supporting non-POSIX OSes is worth
> > a lot of effort, but using stdin instead here is easy ;).
> 
> I have no objection to the code, but I think the comment about
> portability just causes confusion. As witnessed by this discussion.

I wanted to explain why I wasn't using /dev/null, especially since
that's what the Perl version used and that phrasing is preserved in
the current comment:

  # magic hash for Git (git hash-object -t blob /dev/null)  
  
  _EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'

So how should I more clearly explain why I prefer stdin to /dev/null?
Maybe ?? is more portable than the /dev/null path (which doesn't exist
on Windows, for example).??

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20141011/53b52e07/attachment.pgp>

Re: [PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

On Sat, Oct 11, 2014 at 09:02:20AM +0200, David Bremner wrote:
 W. Trevor King writes:
  On Sat, Oct 11, 2014 at 06:53:11AM +0200, David Bremner wrote:
  W. Trevor King writes:
   but I expect that closing stdin is more portable than the
   /dev/null path.
  
  /dev/null is part of POSIX
 
  Maybe folks want to use nmbug on Windows or some other crazy
  non-POSIX OS?  I don't know how Windows-compatible the rest of
  notmuch is (it looks like Xapian can be built with MSYS+mingw or
  MSVC [1,2]), and I don't think supporting non-POSIX OSes is worth
  a lot of effort, but using stdin instead here is easy ;).
 
 I have no objection to the code, but I think the comment about
 portability just causes confusion. As witnessed by this discussion.

I wanted to explain why I wasn't using /dev/null, especially since
that's what the Perl version used and that phrasing is preserved in
the current comment:

  # magic hash for Git (git hash-object -t blob /dev/null)  
  
  _EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'

So how should I more clearly explain why I prefer stdin to /dev/null?
Maybe “… is more portable than the /dev/null path (which doesn't exist
on Windows, for example).”?

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

On Sat, Oct 11, 2014 at 06:53:11AM +0200, David Bremner wrote:
> W. Trevor King writes:
> > but I expect that closing stdin is more portable than the /dev/null
> > path.
> 
> /dev/null is part of POSIX

Maybe folks want to use nmbug on Windows or some other crazy non-POSIX
OS?  I don't know how Windows-compatible the rest of notmuch is (it
looks like Xapian can be built with MSYS+mingw or MSVC [1,2]), and I
don't think supporting non-POSIX OSes is worth a lot of effort, but
using stdin instead here is easy ;).

Cheers,
Trevor

[1]: http://xapian.org/download (this seems to be down now, so see
 here [2] instead)
[2]: https://web.archive.org/web/20140902022239/http://xapian.org/download

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20141010/b97c3311/attachment.pgp>

[PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.

The empty-blob hash-object call avoids errors like:

  $ nmbug commit
  error: invalid object 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 for
'tags/...'
  fatal: git-write-tree: error building trees
  'git HASH(0x9ef3eb8) write-tree' exited with nonzero value

David Bremner suggested [1]:

  $ git hash-object -w /dev/null

instead of my Python version of:

  $ git hash-object -w --stdin <&-

but I expect that closing stdin is more portable than the /dev/null
path.

[1]: id:87y4vu6uvf.fsf at maritornes.cs.unb.ca
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18720
---
This is just like v1 [1], but rebased into the new Python nmbug.  I'd
initially supported the /dev/null approach [2], but have since gone
back to my original --stdin approach for the empty blob object (as
explained in the commit message).  Sorry for waffling ;).

Cheers,
Trevor

[1]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wking at 
tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18630
[2]: id:20140716001239.GH30232 at odin
 http://article.gmane.org/gmane.mail.notmuch.general/18722
 devel/nmbug/nmbug | 24 
 1 file changed, 24 insertions(+)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..23bac5c 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -373,6 +373,29 @@ def fetch(remote=None):
 _git(args=args, wait=True)


+def init(remote=None):
+"""
+Create an empty nmbug repository.
+
+This wraps 'git init' with a few extra steps to support subsequent
+status and commit commands.
+"""
+with _tempfile.TemporaryDirectory(prefix='nmbug-init.') as workdir:
+_spawn(
+args=['git', 'init', '--separate-git-dir', NMBGIT, workdir],
+wait=True)
+_git(args=['config', '--unset', 'core.worktree'], wait=True)
+_git(args=['config', 'core.bare', 'true'], wait=True)
+# create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
+_git(args=['hash-object', '-w', '--stdin'], input='', wait=True)
+_git(
+args=[
+'commit', '--allow-empty', '-m', 'Start a new nmbug repository'
+],
+additional_env={'GIT_WORK_TREE': workdir},
+wait=True)
+
+
 def checkout():
 """
 Update the notmuch database from Git.
@@ -703,6 +726,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'init',
 'log',
 'merge',
 'pull',
-- 
2.1.0.60.g85f0837

[PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.

The empty-blob hash-object call avoids errors like:

  $ nmbug commit
  error: invalid object 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 for
'tags/...'
  fatal: git-write-tree: error building trees
  'git HASH(0x9ef3eb8) write-tree' exited with nonzero value

David Bremner suggested [1]:

  $ git hash-object -w /dev/null

instead of my Python version of:

  $ git hash-object -w --stdin -

but I expect that closing stdin is more portable than the /dev/null
path.

[1]: id:87y4vu6uvf@maritornes.cs.unb.ca
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18720
---
This is just like v1 [1], but rebased into the new Python nmbug.  I'd
initially supported the /dev/null approach [2], but have since gone
back to my original --stdin approach for the empty blob object (as
explained in the commit message).  Sorry for waffling ;).

Cheers,
Trevor

[1]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18630
[2]: id:20140716001239.GH30232@odin
 http://article.gmane.org/gmane.mail.notmuch.general/18722
 devel/nmbug/nmbug | 24 
 1 file changed, 24 insertions(+)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..23bac5c 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -373,6 +373,29 @@ def fetch(remote=None):
 _git(args=args, wait=True)
 
 
+def init(remote=None):
+
+Create an empty nmbug repository.
+
+This wraps 'git init' with a few extra steps to support subsequent
+status and commit commands.
+
+with _tempfile.TemporaryDirectory(prefix='nmbug-init.') as workdir:
+_spawn(
+args=['git', 'init', '--separate-git-dir', NMBGIT, workdir],
+wait=True)
+_git(args=['config', '--unset', 'core.worktree'], wait=True)
+_git(args=['config', 'core.bare', 'true'], wait=True)
+# create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
+_git(args=['hash-object', '-w', '--stdin'], input='', wait=True)
+_git(
+args=[
+'commit', '--allow-empty', '-m', 'Start a new nmbug repository'
+],
+additional_env={'GIT_WORK_TREE': workdir},
+wait=True)
+
+
 def checkout():
 
 Update the notmuch database from Git.
@@ -703,6 +726,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'init',
 'log',
 'merge',
 'pull',
-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] nmbug: Add an 'init' command

[W. Trevor King]

On Sat, Oct 11, 2014 at 06:53:11AM +0200, David Bremner wrote:
 W. Trevor King writes:
  but I expect that closing stdin is more portable than the /dev/null
  path.
 
 /dev/null is part of POSIX

Maybe folks want to use nmbug on Windows or some other crazy non-POSIX
OS?  I don't know how Windows-compatible the rest of notmuch is (it
looks like Xapian can be built with MSYS+mingw or MSVC [1,2]), and I
don't think supporting non-POSIX OSes is worth a lot of effort, but
using stdin instead here is easy ;).

Cheers,
Trevor

[1]: http://xapian.org/download (this seems to be down now, so see
 here [2] instead)
[2]: https://web.archive.org/web/20140902022239/http://xapian.org/download

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] NEWS: Document "nmbug: Translate to Python"

[W. Trevor King]

For more details, see the commit message for 7f2cb3be (nmbug:
Translate to Python, 2014-10-03).  I realized while writing this that
the 7f2cb3be commit message has:

  * 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.

But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007
---
Changes since v1 [1]:

* Trimmed down to a list of altered commands, and suggest people run
  'nmbug COMMAND --help' for details [2].

[1]: id:30987ac1951b6703839d1ec8546c91f2e94935ac.1412523971.git.wking at 
tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19225
[2]: id:87tx3fajqu.fsf at maritornes.cs.unb.ca

 NEWS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/NEWS b/NEWS
index fa57e5d..2efb49d 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,16 @@ Library changes
 Add return status to notmuch_database_close and
 notmuch_database_destroy

+nmbug
+-
+
+The Perl script has been translated to Python; you'll need Python 2.7
+or anything from the 3.x line.  Most of the user-facing interface is
+the same, but `nmbug help` is not `nmbug --help`, and the following nmbug
+commands have slightly different interfaces: `archive`, `commit`,
+`fetch`, `log`, `pull`, `push`, and `status`.  For details on the
+new interface for a given command, run `nmbug COMMAND --help`.
+
 nmbug-status
 

-- 
2.1.0.60.g85f0837

[PATCH] NEWS: Document "nmbug: Translate to Python"

[W. Trevor King]

On Tue, Oct 07, 2014 at 08:30:34AM +0200, David Bremner wrote:
> W. Trevor King writes:
> > This is mostly culled from the commit message for 7f2cb3be (nmbug:
> > Translate to Python, 2014-10-03).  I realized while writing it
> > that the 7f2cb3be commit message has:
> 
> This seems a bit long for a NEWS item, especially for a change that
> impacts only a minority of current notmuch users. Typically even
> fairly major changes (from a user point of view) have less a 10-20
> line NEWS item and refer to some external documentation.

Ok, so maybe:

  ?'nmbug help' is now 'nmbug --help', and the following nmbug
  commands have slightly different interfaces: 'archive', 'commit',
  'fetch', 'log', 'pull', 'push', and 'status'.  For details on the
  new interface, run 'nmbug COMMAND --help'.?

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information,
see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20141007/a9d6b920/attachment.pgp>

Re: [PATCH] NEWS: Document nmbug: Translate to Python

[W. Trevor King]

On Tue, Oct 07, 2014 at 08:30:34AM +0200, David Bremner wrote:
 W. Trevor King writes:
  This is mostly culled from the commit message for 7f2cb3be (nmbug:
  Translate to Python, 2014-10-03).  I realized while writing it
  that the 7f2cb3be commit message has:
 
 This seems a bit long for a NEWS item, especially for a change that
 impacts only a minority of current notmuch users. Typically even
 fairly major changes (from a user point of view) have less a 10-20
 line NEWS item and refer to some external documentation.

Ok, so maybe:

  “'nmbug help' is now 'nmbug --help', and the following nmbug
  commands have slightly different interfaces: 'archive', 'commit',
  'fetch', 'log', 'pull', 'push', and 'status'.  For details on the
  new interface, run 'nmbug COMMAND --help'.”

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information,
see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] NEWS: Document nmbug: Translate to Python

[W. Trevor King]

For more details, see the commit message for 7f2cb3be (nmbug:
Translate to Python, 2014-10-03).  I realized while writing this that
the 7f2cb3be commit message has:

  * 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.

But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007
---
Changes since v1 [1]:

* Trimmed down to a list of altered commands, and suggest people run
  'nmbug COMMAND --help' for details [2].

[1]: id:30987ac1951b6703839d1ec8546c91f2e94935ac.1412523971.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19225
[2]: id:87tx3fajqu@maritornes.cs.unb.ca

 NEWS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/NEWS b/NEWS
index fa57e5d..2efb49d 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,16 @@ Library changes
 Add return status to notmuch_database_close and
 notmuch_database_destroy
 
+nmbug
+-
+
+The Perl script has been translated to Python; you'll need Python 2.7
+or anything from the 3.x line.  Most of the user-facing interface is
+the same, but `nmbug help` is not `nmbug --help`, and the following nmbug
+commands have slightly different interfaces: `archive`, `commit`,
+`fetch`, `log`, `pull`, `push`, and `status`.  For details on the
+new interface for a given command, run `nmbug COMMAND --help`.
+
 nmbug-status
 
 
-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] NEWS: Document "nmbug: Translate to Python"

[W. Trevor King]

This is mostly culled from the commit message for 7f2cb3be (nmbug:
Translate to Python, 2014-10-03).  I realized while writing it that
the 7f2cb3be commit message has:

  * 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.

But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007
---
 NEWS | 48 
 1 file changed, 48 insertions(+)

diff --git a/NEWS b/NEWS
index fa57e5d..c111dd0 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,54 @@ Library changes
 Add return status to notmuch_database_close and
 notmuch_database_destroy

+nmbug
+-
+
+The Perl script has been translated to Python; you'll need Python 2.7
+or anything from the 3.x line.  This gives us better control over
+subprocesses without resorting to third-party modules.  Most of the
+user-facing interface is the same, but there are a few changes, where
+reproducing the original interface was too difficult or we saw a
+change to make the underlying Git interface accessible:
+
+* 'nmbug help' has been split between the general 'nmbug --help' and
+  the command-specific 'nmbug COMMAND --help'.
+
+* Commands are no longer split into "most common", "other useful", and
+  "less common" sets.
+
+* 'nmbug commit' now only uses a single argument for the optional
+  commit-message text.
+
+* The default repository for 'nmbug push' and 'nmbug fetch' is now the
+  current branch's upstream (branch..remote) instead of
+  'origin'.  When we have to, we extract this remote by hand, but
+  where possible we just call the Git command without a repository
+  argument, and leave it to Git to figure out the default.
+
+* 'nmbug push' accepts multiple refspecs if you want to explicitly
+  specify what to push.  Otherwise, the refspec(s) pushed depend on
+  push.default.  The Perl version hardcoded 'master' as the pushed
+  refspec.
+
+* 'nmbug pull' defaults to the current branch's upstream
+  (branch..remote and branch..merge) instead of hardcoding
+  'origin' and 'master'.  It also supports multiple refspecs if for
+  some crazy reason you need an octopus merge (but mostly to avoid
+  breaking consistency with 'git pull').
+
+* 'nmbug status' now catches stderr, and doesn't print errors like:
+
+No upstream configured for branch 'master'
+
+  The Perl implementation had just learned to avoid crashing on that
+  case, but wasn't yet catching the dying subprocess's stderr.
+
+* 'nmbug archive' now accepts positional arguments for the tree-ish
+  and additional 'git archive' options.  For example, you can run:
+
+$ nmbug archive HEAD -- --format tar.gz
+
 nmbug-status
 

-- 
2.1.0.60.g85f0837

[PATCH] NEWS: Document nmbug: Translate to Python

[W. Trevor King]

This is mostly culled from the commit message for 7f2cb3be (nmbug:
Translate to Python, 2014-10-03).  I realized while writing it that
the 7f2cb3be commit message has:

  * 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.

But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007
---
 NEWS | 48 
 1 file changed, 48 insertions(+)

diff --git a/NEWS b/NEWS
index fa57e5d..c111dd0 100644
--- a/NEWS
+++ b/NEWS
@@ -22,6 +22,54 @@ Library changes
 Add return status to notmuch_database_close and
 notmuch_database_destroy
 
+nmbug
+-
+
+The Perl script has been translated to Python; you'll need Python 2.7
+or anything from the 3.x line.  This gives us better control over
+subprocesses without resorting to third-party modules.  Most of the
+user-facing interface is the same, but there are a few changes, where
+reproducing the original interface was too difficult or we saw a
+change to make the underlying Git interface accessible:
+
+* 'nmbug help' has been split between the general 'nmbug --help' and
+  the command-specific 'nmbug COMMAND --help'.
+
+* Commands are no longer split into most common, other useful, and
+  less common sets.
+
+* 'nmbug commit' now only uses a single argument for the optional
+  commit-message text.
+
+* The default repository for 'nmbug push' and 'nmbug fetch' is now the
+  current branch's upstream (branch.name.remote) instead of
+  'origin'.  When we have to, we extract this remote by hand, but
+  where possible we just call the Git command without a repository
+  argument, and leave it to Git to figure out the default.
+
+* 'nmbug push' accepts multiple refspecs if you want to explicitly
+  specify what to push.  Otherwise, the refspec(s) pushed depend on
+  push.default.  The Perl version hardcoded 'master' as the pushed
+  refspec.
+
+* 'nmbug pull' defaults to the current branch's upstream
+  (branch.name.remote and branch.name.merge) instead of hardcoding
+  'origin' and 'master'.  It also supports multiple refspecs if for
+  some crazy reason you need an octopus merge (but mostly to avoid
+  breaking consistency with 'git pull').
+
+* 'nmbug status' now catches stderr, and doesn't print errors like:
+
+No upstream configured for branch 'master'
+
+  The Perl implementation had just learned to avoid crashing on that
+  case, but wasn't yet catching the dying subprocess's stderr.
+
+* 'nmbug archive' now accepts positional arguments for the tree-ish
+  and additional 'git archive' options.  For example, you can run:
+
+$ nmbug archive HEAD -- --format tar.gz
+
 nmbug-status
 
 
-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v6 2/2] nmbug: Add a 'help' command for folks who don't like --help

[W. Trevor King]

The 'if args.func == help' block at the end avoids:

AttributeError: 'functools.partial' object has no attribute '__code__'
---
 devel/nmbug/nmbug | 31 ++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..932ec12 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -32,6 +32,7 @@ from __future__ import unicode_literals

 import codecs as _codecs
 import collections as _collections
+import functools as _functools
 import inspect as _inspect
 import locale as _locale
 import logging as _logging
@@ -677,6 +678,24 @@ def _unpack_diff_lines(stream):
 yield (id, tag)


+def _help(parser, command=None):
+"""
+Show help for an nmbug command.
+
+Because some folks prefer:
+
+  $ nmbug help COMMAND
+
+to
+
+  $ nmbug COMMAND --help
+"""
+if command:
+parser.parse_args([command, '--help'])
+else:
+parser.parse_args(['--help'])
+
+
 if __name__ == '__main__':
 import argparse

@@ -692,6 +711,8 @@ if __name__ == '__main__':
 help='Log verbosity.  Defaults to {!r}.'.format(
 _logging.getLevelName(_LOG.level).lower()))

+help = _functools.partial(_help, parser=parser)
+help.__doc__ = _help.__doc__
 subparsers = parser.add_subparsers(
 title='commands',
 description=(
@@ -703,6 +724,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'help',
 'log',
 'merge',
 'pull',
@@ -746,6 +768,10 @@ if __name__ == '__main__':
 'Override the default configured in branch..remote '
 'to fetch from a particular remote repository (e.g. '
 "'origin')."))
+elif command == 'help':
+subparser.add_argument(
+'command', metavar='COMMAND', nargs='?',
+help='The command to show help for.')
 elif command == 'log':
 subparser.add_argument(
 'args', metavar='ARG', nargs='*',
@@ -796,7 +822,10 @@ if __name__ == '__main__':
 parser.print_usage()
 _sys.exit(1)

-(arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__)
+if args.func == help:
+arg_names = ['command']
+else:
+(arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__)
 kwargs = {key: getattr(args, key) for key in arg_names if key in args}
 try:
 args.func(**kwargs)
-- 
2.1.0.60.g85f0837

[PATCH v6 1/2] nmbug: Translate to Python

[W. Trevor King]

in local git.
-
-
-=item B
-
-Tag is present in local git, but not in remote git.
-
-
-=back
-
-=head1 DUMP FORMAT
-
-Each tag $tag for message with Message-Id $id is written to
-an empty file
-
-   tags/encode($id)/encode($tag)
-
-The encoding preserves alphanumerics, and the characters "+-_@=.:,"
-(not the quotes).  All other octets are replaced with '%' followed by
-a two digit hex number.
-
-=head1 ENVIRONMENT
-
-B specifies the location of the git repository used by nmbug.
-If not specified $HOME/.nmbug is used.
-
-B specifies the prefix in the notmuch database for tags of
-interest to nmbug. If not specified 'notmuch::' is used.
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner 
+# W. Trevor King 
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+"""
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+"""
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import locale as _locale
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::')
+_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}')
+_TAG_FILE_REGEX = _re.compile('tags/(?P[^/]*)/(?P[^/]*)')
+
+# magic hash for Git (git hash-object -t blob /dev/null)
+_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'
+
+
+try:
+getattr(_tempfile, 'TemporaryDirectory')
+except AttributeError:  # Python < 3.2
+class _TemporaryDirectory(object):
+"""
+Fallback context manager for Python < 3.2
+
+See PEP 343 for details on context managers [1].
+
+[1]: http://legacy.python.org/dev/peps/pep-0343/
+"""
+def __init__(self, **kwargs):
+self.name = _tempfile.mkdtemp(**kwargs)
+
+def __enter__(self):
+return self.name
+
+def __exit__(self, type, value, traceback):
+_shutil.rmtree(self.name)
+
+
+_tempfile.TemporaryDirectory = _TemporaryDirectory
+
+
+def _hex_quote(string, safe='+@=:,'):
+"""
+quote('abc def') -> 'abc%20def'.
+
+Wrap urllib.parse.quote with additional safe characters (in
+addition to letters, digits, and '_.-') and lowercase hex digits
+(e.g. '%3a' instead of '%3A').
+"""
+uppercase_escapes = _quote(string, safe)
+return _HEX_ESCAPE_REGEX.sub(
+lambda match: match.group(0).lower(),
+uppercase_escapes)
+
+
+_ENCODED_TAG_PREFIX = _hex_quote(TAG_PREFIX, safe='+@=,')  # quote ':'
+
+
+def _xapian_quote(string):
+"""
+Quote a string for Xapian's QueryParser.
+
+Xapian uses double-quotes for quoting strings.  You can escape
+internal quotes by repeating them [1,2,3].
+
+[1]: http://trac.xapian.org/ticket/128#comment:2
+[2]: http://trac.xapian.org/ticket/128#comment:17
+[3]: http://trac.xapian.org/changeset/13823/svn
+"""
+return '"{0}"'.format(string.replace('"', '""'))
+
+
+def _xapian_unquote(string):
+"""
+Unquote a Xapian-quoted string.
+"""
+if string.startswith('"') and string.endswith('"'):
+return string[1:-1].replace('""', '"')
+return string
+
+
+class SubprocessError(RuntimeError):

[PATCH v6 0/2] nmbug: Translate to Python

[W. Trevor King]

Hopefully the last round :).  Changes since v5 [1]:

* Dropped 2.6 compatibility claim from the first patch's commit
  message, since Tomi pointed out that at least the '{}'.format()
  syntax is just since 2.7.
* Added a new 'help' command as a separate patch at David's request.

[1]: id:d44bb6ad59ee0a30ac4a8d2e9fe50e3b98d1c408.1411572592.git.wking at 
tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19108

W. Trevor King (2):
  nmbug: Translate to Python
  nmbug: Add a 'help' command for folks who don't like --help

 devel/nmbug/nmbug | 1544 +
 1 file changed, 836 insertions(+), 708 deletions(-)

-- 
2.1.0.60.g85f0837

[PATCH v6 2/2] nmbug: Add a 'help' command for folks who don't like --help

[W. Trevor King]

The 'if args.func == help' block at the end avoids:

AttributeError: 'functools.partial' object has no attribute '__code__'
---
 devel/nmbug/nmbug | 31 ++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 9402ead..932ec12 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -32,6 +32,7 @@ from __future__ import unicode_literals
 
 import codecs as _codecs
 import collections as _collections
+import functools as _functools
 import inspect as _inspect
 import locale as _locale
 import logging as _logging
@@ -677,6 +678,24 @@ def _unpack_diff_lines(stream):
 yield (id, tag)
 
 
+def _help(parser, command=None):
+
+Show help for an nmbug command.
+
+Because some folks prefer:
+
+  $ nmbug help COMMAND
+
+to
+
+  $ nmbug COMMAND --help
+
+if command:
+parser.parse_args([command, '--help'])
+else:
+parser.parse_args(['--help'])
+
+
 if __name__ == '__main__':
 import argparse
 
@@ -692,6 +711,8 @@ if __name__ == '__main__':
 help='Log verbosity.  Defaults to {!r}.'.format(
 _logging.getLevelName(_LOG.level).lower()))
 
+help = _functools.partial(_help, parser=parser)
+help.__doc__ = _help.__doc__
 subparsers = parser.add_subparsers(
 title='commands',
 description=(
@@ -703,6 +724,7 @@ if __name__ == '__main__':
 'clone',
 'commit',
 'fetch',
+'help',
 'log',
 'merge',
 'pull',
@@ -746,6 +768,10 @@ if __name__ == '__main__':
 'Override the default configured in branch.name.remote '
 'to fetch from a particular remote repository (e.g. '
 'origin').))
+elif command == 'help':
+subparser.add_argument(
+'command', metavar='COMMAND', nargs='?',
+help='The command to show help for.')
 elif command == 'log':
 subparser.add_argument(
 'args', metavar='ARG', nargs='*',
@@ -796,7 +822,10 @@ if __name__ == '__main__':
 parser.print_usage()
 _sys.exit(1)
 
-(arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__)
+if args.func == help:
+arg_names = ['command']
+else:
+(arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__)
 kwargs = {key: getattr(args, key) for key in arg_names if key in args}
 try:
 args.func(**kwargs)
-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v6 0/2] nmbug: Translate to Python

[W. Trevor King]

Hopefully the last round :).  Changes since v5 [1]:

* Dropped 2.6 compatibility claim from the first patch's commit
  message, since Tomi pointed out that at least the '{}'.format()
  syntax is just since 2.7.
* Added a new 'help' command as a separate patch at David's request.

[1]: id:d44bb6ad59ee0a30ac4a8d2e9fe50e3b98d1c408.1411572592.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19108

W. Trevor King (2):
  nmbug: Translate to Python
  nmbug: Add a 'help' command for folks who don't like --help

 devel/nmbug/nmbug | 1544 +
 1 file changed, 836 insertions(+), 708 deletions(-)

-- 
2.1.0.60.g85f0837

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v6 1/2] nmbug: Translate to Python

[W. Trevor King]

/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner da...@tethera.net
+# W. Trevor King wk...@tremily.us
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import locale as _locale
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::')
+_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}')
+_TAG_FILE_REGEX = _re.compile('tags/(?Pid[^/]*)/(?Ptag[^/]*)')
+
+# magic hash for Git (git hash-object -t blob /dev/null)
+_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'
+
+
+try:
+getattr(_tempfile, 'TemporaryDirectory')
+except AttributeError:  # Python  3.2
+class _TemporaryDirectory(object):
+
+Fallback context manager for Python  3.2
+
+See PEP 343 for details on context managers [1].
+
+[1]: http://legacy.python.org/dev/peps/pep-0343/
+
+def __init__(self, **kwargs):
+self.name = _tempfile.mkdtemp(**kwargs)
+
+def __enter__(self):
+return self.name
+
+def __exit__(self, type, value, traceback):
+_shutil.rmtree(self.name)
+
+
+_tempfile.TemporaryDirectory = _TemporaryDirectory
+
+
+def _hex_quote(string, safe='+@=:,'):
+
+quote('abc def') - 'abc%20def'.
+
+Wrap urllib.parse.quote with additional safe characters (in
+addition to letters, digits, and '_.-') and lowercase hex digits
+(e.g. '%3a' instead of '%3A').
+
+uppercase_escapes = _quote(string, safe)
+return _HEX_ESCAPE_REGEX.sub(
+lambda match: match.group(0).lower(),
+uppercase_escapes)
+
+
+_ENCODED_TAG_PREFIX = _hex_quote(TAG_PREFIX, safe='+@=,')  # quote ':'
+
+
+def _xapian_quote(string):
+
+Quote a string for Xapian's QueryParser.
+
+Xapian uses double-quotes for quoting strings.  You can escape
+internal quotes by repeating them [1,2,3].
+
+[1]: http://trac.xapian.org/ticket/128#comment:2
+[2]: http://trac.xapian.org/ticket/128#comment:17
+[3]: http://trac.xapian.org/changeset/13823/svn
+
+return '{0}'.format(string.replace('', ''))
+
+
+def _xapian_unquote(string):
+
+Unquote a Xapian-quoted string.
+
+if string.startswith('') and string.endswith(''):
+return string[1:-1].replace('', '')
+return string
+
+
+class SubprocessError(RuntimeError):
+A subprocess exited with a nonzero status
+def __init__(self, args, status, stdout=None, stderr=None):
+self.status = status
+self.stdout = stdout
+self.stderr = stderr
+msg = '{args} exited with {status}'.format(args=args, status=status)
+if stderr:
+msg = '{msg}: {stderr}'.format(msg=msg, stderr=stderr)
+super(SubprocessError, self).__init__(msg)
+
+
+class _SubprocessContextManager(object):
+
+PEP 343 context manager for subprocesses.
+
+'expect' holds a tuple of acceptable exit codes, otherwise we'll
+raise a SubprocessError in __exit__.
+
+def __init__(self, process, args, expect=(0,)):
+self._process = process
+self._args = args
+self._expect = expect
+
+def __enter__(self):
+return self._process
+
+def __exit__(self, type, value

[PATCH v4] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Thu, Oct 02, 2014 at 04:39:41PM -0400, Austin Clements wrote:
> On Thu, 02 Oct 2014, W. Trevor King wrote:
> > On Thu, Oct 02, 2014 at 03:19:08PM -0400, Austin Clements wrote:
> >> This patch simplifies notmuch_database_close to explicitly abort
> >> any outstanding transaction and then just call Database::close.
> >> This works for both read-only and read/write databases, takes
> >> care of committing changes, unifies the exception handling path,
> >> and codifies aborting outstanding transactions.
> >
> > I don't expect atomic blocks are particularly useful for read-only
> > connections.  If they aren't, I'd quibble with the ?This works for
> > both read-only?? wording above.  If they are, I'd drop the
> > read/write
> 
> It's true that atomic sections aren't very useful on a read-only
> database, but we do allow them for symmetry.

Heh, and they're no-ops since the beginning in 957f1ba3 (lib: Add
notmuch_database_{begin,end}_atomic, 2011-01-29).  So both the commit
message and read/write check make sense.  Quibble retracted.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20141002/e2fe9c19/attachment.pgp>

[PATCH v4] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Thu, Oct 02, 2014 at 03:19:08PM -0400, Austin Clements wrote:
> This patch simplifies notmuch_database_close to explicitly abort any
> outstanding transaction and then just call Database::close.  This
> works for both read-only and read/write databases, takes care of
> committing changes, unifies the exception handling path, and codifies
> aborting outstanding transactions.

I don't expect atomic blocks are particularly useful for read-only
connections.  If they aren't, I'd quibble with the ?This works for
both read-only?? wording above.  If they are, I'd drop the read/write
check below:

> + /* If there's an outstanding transaction, it's unclear if
> +  * closing the Xapian database commits everything up to
> +  * that transaction, or may discard committed (but
> +  * unflushed) transactions.  To be certain, explicitly
> +  * cancel any outstanding transaction before closing. */
> + if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE &&
> + notmuch->atomic_nesting)
> + (static_cast  (notmuch->xapian_db))
> + ->cancel_transaction ();

Thats a very minor quibble though, and I'd be glad to see this patch
land as is.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

Re: [PATCH v4] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Thu, Oct 02, 2014 at 03:19:08PM -0400, Austin Clements wrote:
 This patch simplifies notmuch_database_close to explicitly abort any
 outstanding transaction and then just call Database::close.  This
 works for both read-only and read/write databases, takes care of
 committing changes, unifies the exception handling path, and codifies
 aborting outstanding transactions.

I don't expect atomic blocks are particularly useful for read-only
connections.  If they aren't, I'd quibble with the “This works for
both read-only…” wording above.  If they are, I'd drop the read/write
check below:

 + /* If there's an outstanding transaction, it's unclear if
 +  * closing the Xapian database commits everything up to
 +  * that transaction, or may discard committed (but
 +  * unflushed) transactions.  To be certain, explicitly
 +  * cancel any outstanding transaction before closing. */
 + if (notmuch-mode == NOTMUCH_DATABASE_MODE_READ_WRITE 
 + notmuch-atomic_nesting)
 + (static_cast Xapian::WritableDatabase * (notmuch-xapian_db))
 + -cancel_transaction ();

Thats a very minor quibble though, and I'd be glad to see this patch
land as is.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v4] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Thu, Oct 02, 2014 at 04:39:41PM -0400, Austin Clements wrote:
 On Thu, 02 Oct 2014, W. Trevor King wrote:
  On Thu, Oct 02, 2014 at 03:19:08PM -0400, Austin Clements wrote:
  This patch simplifies notmuch_database_close to explicitly abort
  any outstanding transaction and then just call Database::close.
  This works for both read-only and read/write databases, takes
  care of committing changes, unifies the exception handling path,
  and codifies aborting outstanding transactions.
 
  I don't expect atomic blocks are particularly useful for read-only
  connections.  If they aren't, I'd quibble with the “This works for
  both read-only…” wording above.  If they are, I'd drop the
  read/write
 
 It's true that atomic sections aren't very useful on a read-only
 database, but we do allow them for symmetry.

Heh, and they're no-ops since the beginning in 957f1ba3 (lib: Add
notmuch_database_{begin,end}_atomic, 2011-01-29).  So both the commit
message and read/write check make sense.  Quibble retracted.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 05:32:50PM -0400, Austin Clements wrote:
> + * If the caller is currently in an atomic section (there was a
> + * notmuch_database_begin_atomic without a matching
> + * notmuch_database_end_atomic), this will abort the atomic section,
> + * discarding any modifications made in the atomic section.  All
> + * changes up to this will be committed.

I still think Xapian's wording is more readable [1]:

  For a WritableDatabase, if a transaction is active it will be
  aborted, while if no transaction is active commit() will be
  implicitly called.

How about:

  For a writable database, if a transaction is active (there was a
  notmuch_database_begin_atomic without a matching
  notmuch_database_end_atomic) it will be aborted, while if no
  transaction is active any pending changes will be committed.

Cheers,
Trevor

[1]: 
http://xapian.org/docs/apidoc/html/classXapian_1_1Database.html#a59f5f8b137723dcaaabdbdccbc0cf1eb

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH v2] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 05:20:23PM -0400, Austin Clements wrote:
> diff --git a/lib/notmuch.h b/lib/notmuch.h
> index fe2340b..5c40c67 100644
> --- a/lib/notmuch.h
> +++ b/lib/notmuch.h
> @@ -292,6 +292,11 @@ notmuch_database_open (const char *path,
>   * notmuch_database_close can be called multiple times.  Later calls
>   * have no effect.
>   *
> + * If the caller is currently in an atomic section (there was a
> + * notmuch_database_begin_atomic without a matching
> + * notmuch_database_end_atomic), this will abort the atomic section,
> + * discarding any modifications made in the atomic section.
> + *
>   * Return value:
>   *
>   * NOTMUCH_STATUS_SUCCESS: Successfully closed the database.

Still no mention of ?commit? or ?if you're not in an atomic section?
;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 10:13:31PM +0200, David Bremner wrote:
> W. Trevor King writes:
> I think it would be better to write our own, not because of licensing
> issues, but because the user of the notmuch API won't know what a xapian
> commit is.

Between version control and databases, I feel like most folks using
libnotmuch will know what a commit is ;).  But I don't really mind, so
long as the new docs mention it for non-atomic closes.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140924/2a98db8b/attachment.pgp>

[PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 09:25:20PM +0200, David Bremner wrote:
> I think the fact that you have to close the notmuch database (when
> not using begin/end atomic) to get a commit is surprising for many
> people, so it would be nice to make that clearer somehow.

It looks like Xapian is GPLv2+, so we should just be able to
copy/paste/edit the line I quoted from the Xapian docs.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 08:09:27PM +0200, David Bremner wrote:
> W. Trevor King writes:
> > Ah, I thought the implicit flush/commit was just in our code.
> > Since it's also in the underlying Xapian close, then this patch
> > looks pretty good to me.  I'd mention Xapian's explicit close in
> > the notmuch.h message.  Xapain's docs say [1]:
> >
> >   For a WritableDatabase, if a transaction is active it will be
> >   aborted, while if no transaction is active commit() will be
> >   implicitly called.
> 
> I'm not sure what you're asking for here by "explicit close". Isn't
> what you quote a restatement of
> 
> + * If the caller is currently in an atomic section (there was a
> + * notmuch_database_begin_atomic without a matching
> + * notmuch_database_end_atomic), this will abort the atomic section,
> + * discarding any modifications made in the atomic section.
> 
> in terms of underyling Xapian mechanics?

Sorry, I didn't phrase that very well.  The notmuch docs (as of this
patch) explain that we don't commit if we're in an atomic block.  The
Xapian docs also say that, *and* they say that if we're not in atomic
block the close *does* try to commit.  I think that's worth mentioning
in our close docs.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140924/90c2af76/attachment.pgp>

[PATCH v5] nmbug: Translate to Python

[W. Trevor King]

h or git repo. See below for more
-information about the output format.
-
-=back
-
-=head2 Less common commands
-
-=over 8
-
-=item B
-
-Dump a tar archive (using git archive) of the current nmbug tag set.
-
-=back
-
-=head1 STATUS FORMAT
-
-B prints lines of the form
-
-   ng Message-Id tag
-
-where n is a single character representing notmuch database status
-
-=over 8
-
-=item B
-
-Tag is present in notmuch database, but not committed to nmbug
-(equivalently, tag has been deleted in nmbug repo, e.g. by a pull, but
-not restored to notmuch database).
-
-=item B
-
-Tag is present in nmbug repo, but not restored to notmuch database
-(equivalently, tag has been deleted in notmuch)
-
-=item B
-
-Message is unknown (missing from local notmuch database)
-
-=back
-
-The second character (if present) represents a difference between remote
-git and local. Typically C needs to be run to update this.
-
-=over 8
-
-
-=item B
-
-Tag is present in remote, but not in local git.
-
-
-=item B
-
-Tag is present in local git, but not in remote git.
-
-
-=back
-
-=head1 DUMP FORMAT
-
-Each tag $tag for message with Message-Id $id is written to
-an empty file
-
-   tags/encode($id)/encode($tag)
-
-The encoding preserves alphanumerics, and the characters "+-_@=.:,"
-(not the quotes).  All other octets are replaced with '%' followed by
-a two digit hex number.
-
-=head1 ENVIRONMENT
-
-B specifies the location of the git repository used by nmbug.
-If not specified $HOME/.nmbug is used.
-
-B specifies the prefix in the notmuch database for tags of
-interest to nmbug. If not specified 'notmuch::' is used.
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner 
+# W. Trevor King 
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+"""
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+"""
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import locale as _locale
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::')
+_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}')
+_TAG_FILE_REGEX = _re.compile('tags/(?P[^/]*)/(?P[^/]*)')
+
+# magic hash for Git (git hash-object -t blob /dev/null)
+_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'
+
+
+try:
+getattr(_tempfile, 'TemporaryDirectory')
+except AttributeError:  # Python < 3.2
+class _TemporaryDirectory(object):
+"""
+Fallback context manager for Python < 3.2
+
+See PEP 343 for details on context managers [1].
+
+[1]: http://legacy.python.org/dev/peps/pep-0343/
+"""
+def __init__(self, **kwargs):
+self.name = _tempfile.mkdtemp(**kwargs)
+
+def __enter__(self):
+return self.name
+
+def __exit__(self, type, value, traceback):
+_shutil.rmtree(self.name)
+
+
+_tempfile.TemporaryDirectory = _TemporaryDirectory
+
+
+def _hex_quote(string, safe='+@=:,'):
+"""
+quote('abc def') -> 'abc%20def'.
+
+Wrap urllib.parse.quote with additional safe characters (in
+addition to letters, digits, and '_.-') and lowercase hex digits
+(e.g. '%3a' instead of '%3A').
+"""
+uppercase_escapes = _quote(string, safe)
+return _HE

[PATCH v5] nmbug: Translate to Python

[W. Trevor King]

 in notmuch)
-
-=item BU
-
-Message is unknown (missing from local notmuch database)
-
-=back
-
-The second character (if present) represents a difference between remote
-git and local. Typically Cnmbug fetch needs to be run to update this.
-
-=over 8
-
-
-=item Ba
-
-Tag is present in remote, but not in local git.
-
-
-=item Bd
-
-Tag is present in local git, but not in remote git.
-
-
-=back
-
-=head1 DUMP FORMAT
-
-Each tag $tag for message with Message-Id $id is written to
-an empty file
-
-   tags/encode($id)/encode($tag)
-
-The encoding preserves alphanumerics, and the characters +-_@=.:,
-(not the quotes).  All other octets are replaced with '%' followed by
-a two digit hex number.
-
-=head1 ENVIRONMENT
-
-BNMBGIT specifies the location of the git repository used by nmbug.
-If not specified $HOME/.nmbug is used.
-
-BNMBPREFIX specifies the prefix in the notmuch database for tags of
-interest to nmbug. If not specified 'notmuch::' is used.
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner da...@tethera.net
+# W. Trevor King wk...@tremily.us
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import locale as _locale
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::')
+_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}')
+_TAG_FILE_REGEX = _re.compile('tags/(?Pid[^/]*)/(?Ptag[^/]*)')
+
+# magic hash for Git (git hash-object -t blob /dev/null)
+_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'
+
+
+try:
+getattr(_tempfile, 'TemporaryDirectory')
+except AttributeError:  # Python  3.2
+class _TemporaryDirectory(object):
+
+Fallback context manager for Python  3.2
+
+See PEP 343 for details on context managers [1].
+
+[1]: http://legacy.python.org/dev/peps/pep-0343/
+
+def __init__(self, **kwargs):
+self.name = _tempfile.mkdtemp(**kwargs)
+
+def __enter__(self):
+return self.name
+
+def __exit__(self, type, value, traceback):
+_shutil.rmtree(self.name)
+
+
+_tempfile.TemporaryDirectory = _TemporaryDirectory
+
+
+def _hex_quote(string, safe='+@=:,'):
+
+quote('abc def') - 'abc%20def'.
+
+Wrap urllib.parse.quote with additional safe characters (in
+addition to letters, digits, and '_.-') and lowercase hex digits
+(e.g. '%3a' instead of '%3A').
+
+uppercase_escapes = _quote(string, safe)
+return _HEX_ESCAPE_REGEX.sub(
+lambda match: match.group(0).lower(),
+uppercase_escapes)
+
+
+_ENCODED_TAG_PREFIX = _hex_quote(TAG_PREFIX, safe='+@=,')  # quote ':'
+
+
+def _xapian_quote(string):
+
+Quote a string for Xapian's QueryParser.
+
+Xapian uses double-quotes for quoting strings.  You can escape
+internal quotes by repeating them [1,2,3].
+
+[1]: http://trac.xapian.org/ticket/128#comment:2
+[2]: http://trac.xapian.org/ticket/128#comment:17
+[3]: http://trac.xapian.org/changeset/13823/svn
+
+return '{0}'.format(string.replace('', ''))
+
+
+def _xapian_unquote(string):
+
+Unquote a Xapian-quoted string.
+
+if string.startswith('') and string.endswith(''):
+return

Re: [PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 09:25:20PM +0200, David Bremner wrote:
 I think the fact that you have to close the notmuch database (when
 not using begin/end atomic) to get a commit is surprising for many
 people, so it would be nice to make that clearer somehow.

It looks like Xapian is GPLv2+, so we should just be able to
copy/paste/edit the line I quoted from the Xapian docs.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 10:13:31PM +0200, David Bremner wrote:
 W. Trevor King writes:
 I think it would be better to write our own, not because of licensing
 issues, but because the user of the notmuch API won't know what a xapian
 commit is.

Between version control and databases, I feel like most folks using
libnotmuch will know what a commit is ;).  But I don't really mind, so
long as the new docs mention it for non-atomic closes.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Wed, Sep 24, 2014 at 05:20:23PM -0400, Austin Clements wrote:
 diff --git a/lib/notmuch.h b/lib/notmuch.h
 index fe2340b..5c40c67 100644
 --- a/lib/notmuch.h
 +++ b/lib/notmuch.h
 @@ -292,6 +292,11 @@ notmuch_database_open (const char *path,
   * notmuch_database_close can be called multiple times.  Later calls
   * have no effect.
   *
 + * If the caller is currently in an atomic section (there was a
 + * notmuch_database_begin_atomic without a matching
 + * notmuch_database_end_atomic), this will abort the atomic section,
 + * discarding any modifications made in the atomic section.
 + *
   * Return value:
   *
   * NOTMUCH_STATUS_SUCCESS: Successfully closed the database.

Still no mention of “commit” or “if you're not in an atomic section”
;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Mon, Sep 22, 2014 at 06:50:50PM +, Austin Clements wrote:
> Quoth W. Trevor King on Sep 22 at  9:59 am:
> > On Mon, Sep 22, 2014 at 11:43:35AM -0400, Austin Clements wrote:
> > > This patch simplifies notmuch_database_close to just call
> > > Database::close.  This works for both read-only and read/write
> > > databases, takes care of committing changes, unifies the
> > > exception handling path, and codifies aborting outstanding
> > > transactions.
> > 
> > If we're dropping the flush call here, where will it be triggered
> > instead?  We'll need to flush/commit our changes to the database
> > at some point before closing.  Do clients now need an explicit
> > flush/commit command (explicit, client-initiated flushes sound
> > like a good idea to me).
> 
> The call to Database::close implicitly flushes/commits, as mentioned
> in the comment in the patch, so there's no need for any new APIs or
> client changes.  The call to Database::flush in
> notmuch_database_close was entirely redundant with the call to
> Database::close.

Ah, I thought the implicit flush/commit was just in our code.  Since
it's also in the underlying Xapian close, then this patch looks pretty
good to me.  I'd mention Xapian's explicit close in the notmuch.h
message.  Xapain's docs say [1]:

  For a WritableDatabase, if a transaction is active it will be
  aborted, while if no transaction is active commit() will be
  implicitly called.

Cheers,
Trevor

[1]: 
http://xapian.org/docs/apidoc/html/classXapian_1_1Database.html#a59f5f8b137723dcaaabdbdccbc0cf1eb

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140922/4cb115a4/attachment.pgp>

[announce] nmhive v0.1.0, a bookmarklet/server for nmbug tags

[W. Trevor King]

On Mon, Sep 22, 2014 at 10:19:35AM -0700, W. Trevor King wrote:
> I like nmbug's distributed tag maintenance, but not everyone has
> notmuch/nmbug installed locally (yet ;).  However, everyone that I
> know does have a browser and a mail client.  They can submit
> messages with their mail client already, but we've been missing a
> way for them to help tag messages.

Ah, and the other piece to this workflow is the existing nmbug-status,
which collects the results of canned searches so folks without a local
notmuch can use the tags [1].  Folks using a local nmhive will
probably want to run their own status-genertion via a post-commit hook
in their nmhive repository.  Then their users will have their
search-results updated after each web-initiated change.  If you also
wanted them to see updates from changes to tethera's nmbug repository,
you'd probably also want a cron job that tried to fetch and merge
tethera's changes with the nmhive changes:

  -o---o---oo  tethera/master
\\
 \o  nmhive/status  (auto-generated merge for nmbug-status)
  \  /
   o---oo   nmhive/master (with web-initiated changes)

You'd want to resolve conflicts somehow, but any resolution strategy
is probably fine, since it's unlikely that we get conflicts very
often.

Cheers,
Trevor

[1]: http://nmbug.tethera.net/status/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140922/de3e518b/attachment.pgp>

[announce] nmhive v0.1.0

[W. Trevor King]

I like nmbug's distributed tag maintenance, but not everyone has
notmuch/nmbug installed locally (yet ;).  However, everyone that I
know does have a browser and a mail client.  They can submit messages
with their mail client already, but we've been missing a way for them
to help tag messages.  Nmhive is a lightweight server that allows
clients to read and write notmuch tags using a JSON API [1].  It uses
nmbug locally to commit after each write, and the admin can then pull
tag changes made by nmhive and push them into the native nmbug
ecosystem:

  web client ? nmhive ? nmhive's nmbug ? admin's nmbug ? tethera's nmbug

To make it easy for folks to drive nmhive, the repository also
contains a bookmarklet [2] that you can use to interactively manage
tags from a message's Gmane page (e.g., [3,4]).  The setup currently
uses my ported-to-Python nmbug [5] if you want to play with it
locally.  I haven't added user authentication yet, so it's probably
best to just run your own nmhive for now.  If anyone has preferences
for authentication, send a patch :).  Or at least let me know, and
I'll see what I can do ;).  I'd appreciate any other feedback folks
have as well.

Cheers,
Trevor

[1]: https://github.com/wking/nmhive
[2]: https://github.com/wking/nmhive/blob/v0.1.0/nmbug.js
[3]: http://thread.gmane.org/gmane.mail.notmuch.general/19091/focus=19092
[4]: http://article.gmane.org/gmane.mail.notmuch.general/19092
[5]: http://thread.gmane.org/gmane.mail.notmuch.general/19007
 id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking at 
tremily.us

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Mon, Sep 22, 2014 at 11:43:35AM -0400, Austin Clements wrote:
> This patch simplifies notmuch_database_close to just call
> Database::close.  This works for both read-only and read/write
> databases, takes care of committing changes, unifies the exception
> handling path, and codifies aborting outstanding transactions.

If we're dropping the flush call here, where will it be triggered
instead?  We'll need to flush/commit our changes to the database at
some point before closing.  Do clients now need an explicit
flush/commit command (explicit, client-initiated flushes sound like a
good idea to me).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

Re: [PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Mon, Sep 22, 2014 at 11:43:35AM -0400, Austin Clements wrote:
 This patch simplifies notmuch_database_close to just call
 Database::close.  This works for both read-only and read/write
 databases, takes care of committing changes, unifies the exception
 handling path, and codifies aborting outstanding transactions.

If we're dropping the flush call here, where will it be triggered
instead?  We'll need to flush/commit our changes to the database at
some point before closing.  Do clients now need an explicit
flush/commit command (explicit, client-initiated flushes sound like a
good idea to me).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [announce] nmhive v0.1.0, a bookmarklet/server for nmbug tags

[W. Trevor King]

On Mon, Sep 22, 2014 at 10:19:35AM -0700, W. Trevor King wrote:
 I like nmbug's distributed tag maintenance, but not everyone has
 notmuch/nmbug installed locally (yet ;).  However, everyone that I
 know does have a browser and a mail client.  They can submit
 messages with their mail client already, but we've been missing a
 way for them to help tag messages.

Ah, and the other piece to this workflow is the existing nmbug-status,
which collects the results of canned searches so folks without a local
notmuch can use the tags [1].  Folks using a local nmhive will
probably want to run their own status-genertion via a post-commit hook
in their nmhive repository.  Then their users will have their
search-results updated after each web-initiated change.  If you also
wanted them to see updates from changes to tethera's nmbug repository,
you'd probably also want a cron job that tried to fetch and merge
tethera's changes with the nmhive changes:

  -o---o---oo  tethera/master
\\
 \o  nmhive/status  (auto-generated merge for nmbug-status)
  \  /
   o---oo   nmhive/master (with web-initiated changes)

You'd want to resolve conflicts somehow, but any resolution strategy
is probably fine, since it's unlikely that we get conflicts very
often.

Cheers,
Trevor

[1]: http://nmbug.tethera.net/status/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: Simplify close and codify aborting atomic section

[W. Trevor King]

On Mon, Sep 22, 2014 at 06:50:50PM +, Austin Clements wrote:
 Quoth W. Trevor King on Sep 22 at  9:59 am:
  On Mon, Sep 22, 2014 at 11:43:35AM -0400, Austin Clements wrote:
   This patch simplifies notmuch_database_close to just call
   Database::close.  This works for both read-only and read/write
   databases, takes care of committing changes, unifies the
   exception handling path, and codifies aborting outstanding
   transactions.
  
  If we're dropping the flush call here, where will it be triggered
  instead?  We'll need to flush/commit our changes to the database
  at some point before closing.  Do clients now need an explicit
  flush/commit command (explicit, client-initiated flushes sound
  like a good idea to me).
 
 The call to Database::close implicitly flushes/commits, as mentioned
 in the comment in the patch, so there's no need for any new APIs or
 client changes.  The call to Database::flush in
 notmuch_database_close was entirely redundant with the call to
 Database::close.

Ah, I thought the implicit flush/commit was just in our code.  Since
it's also in the underlying Xapian close, then this patch looks pretty
good to me.  I'd mention Xapian's explicit close in the notmuch.h
message.  Xapain's docs say [1]:

  For a WritableDatabase, if a transaction is active it will be
  aborted, while if no transaction is active commit() will be
  implicitly called.

Cheers,
Trevor

[1]: 
http://xapian.org/docs/apidoc/html/classXapian_1_1Database.html#a59f5f8b137723dcaaabdbdccbc0cf1eb

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 08:09:11PM +0200, David Bremner wrote:
> W. Trevor King writes:
> > Give it a spin to kick the tires, and reply to the v4 patch saying
> > that none of the pieces fell off while you were kicking it ;).
> > Basically, it's just waiting for review and a consensus that it's
> > a solid change.
> 
> v3 crashed when merging quite often for me.

I didn't actually change anything there, because I was unable to
reproduce the crash (as I point out in my v4 message, ?I've also left
off the merge/pull checkouts, since I haven't been able to reproduce
David's error locally?? [1]).  Any advice on reproducing the merge
errors would be helpful :).

Cheers,
Trevor

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140915/c8a3f894/attachment.pgp>

[PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 05:32:59PM +0100, David Edmondson wrote:
> On Mon, Sep 15 2014, W. Trevor King wrote:
> > On Mon, Sep 15, 2014 at 05:13:50PM +0100, David Edmondson wrote:
> >> On Sun, Jul 06 2014, W. Trevor King wrote:
> >> > For folks that want to start versioning a new tag-space, instead
> >> > of cloning one that someone else has already started.
> >> 
> >> I tried this patch, and it (appeared) to work for me. Given that the
> >> procedure for creating a new tag repository is arcane, could this
> >> patch (or a version of it) be pushed?
> >
> > Rewriting this patch is the first thing on my list after nmbug's
> > Python translation lands [1].
> 
> What is required to make that happen?

Give it a spin to kick the tires, and reply to the v4 patch saying
that none of the pieces fell off while you were kicking it ;).
Basically, it's just waiting for review and a consensus that it's a
solid change.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140915/9ed11e5c/attachment.pgp>

[PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 05:13:50PM +0100, David Edmondson wrote:
> On Sun, Jul 06 2014, W. Trevor King wrote:
> > For folks that want to start versioning a new tag-space, instead
> > of cloning one that someone else has already started.
> 
> I tried this patch, and it (appeared) to work for me. Given that the
> procedure for creating a new tag repository is arcane, could this
> patch (or a version of it) be pushed?

Rewriting this patch is the first thing on my list after nmbug's
Python translation lands [1].  Interestingly, there have been versions
of 'nmbug init' kicking around in the wings for some time [2].

Cheers,
Trevor

[1]: 
id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking%40tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19007
[2]: id:87obf66k7x.fsf at zancas.localnet
 http://thread.gmane.org/gmane.mail.notmuch.general/14474/focus=14493

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140915/dbd21e96/attachment-0001.pgp>

multiple machine tagging

[W. Trevor King]

On Mon, Sep 15, 2014 at 03:26:26PM +0100, David Edmondson wrote:
> On Mon, Sep 15 2014, W. Trevor King wrote:
> > Does nmbug not do this for you?  This is exactly what it was designed
> > to do (sync tags).
> 
> Perhaps. Is anyone using it for personal tag sync?

I will once my Python translation lands [1], which will let me get
back to 'nmbug init' [2] ;).  Once it's easy to initialize a new
repository for all your tags (or some other subset), I think the nmbug
route will be pretty straightforward.

Cheers,
Trevor

[1]: 
id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking%40tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19007
[2]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wking at 
tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18630

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140915/b9c1912d/attachment.pgp>

multiple machine tagging

[W. Trevor King]

On Mon, Sep 15, 2014 at 02:49:16PM +0100, David Edmondson wrote:
> On Thu, May 20 2010, David Edmondson wrote:
> (Wow, was it really more than four years ago?)
> 
> > What's the current state of the art in merging tags from multiple
> > machines?
> >
> > In my own case the contents of the mail store can be considered
> > identical on the different machines. Automated tagging is mostly fine
> > - it can just happen on each of the machines. Any hand-added tags are
> > a problem, though.
> 
> Is there any significant change to this picture?

Does nmbug not do this for you?  This is exactly what it was designed
to do (sync tags).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

Re: multiple machine tagging

[W. Trevor King]

On Mon, Sep 15, 2014 at 02:49:16PM +0100, David Edmondson wrote:
 On Thu, May 20 2010, David Edmondson wrote:
 (Wow, was it really more than four years ago?)
 
  What's the current state of the art in merging tags from multiple
  machines?
 
  In my own case the contents of the mail store can be considered
  identical on the different machines. Automated tagging is mostly fine
  - it can just happen on each of the machines. Any hand-added tags are
  a problem, though.
 
 Is there any significant change to this picture?

Does nmbug not do this for you?  This is exactly what it was designed
to do (sync tags).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: multiple machine tagging

[W. Trevor King]

On Mon, Sep 15, 2014 at 03:26:26PM +0100, David Edmondson wrote:
 On Mon, Sep 15 2014, W. Trevor King wrote:
  Does nmbug not do this for you?  This is exactly what it was designed
  to do (sync tags).
 
 Perhaps. Is anyone using it for personal tag sync?

I will once my Python translation lands [1], which will let me get
back to 'nmbug init' [2] ;).  Once it's easy to initialize a new
repository for all your tags (or some other subset), I think the nmbug
route will be pretty straightforward.

Cheers,
Trevor

[1]: 
id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking%40tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19007
[2]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18626/focus=18630

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 05:13:50PM +0100, David Edmondson wrote:
 On Sun, Jul 06 2014, W. Trevor King wrote:
  For folks that want to start versioning a new tag-space, instead
  of cloning one that someone else has already started.
 
 I tried this patch, and it (appeared) to work for me. Given that the
 procedure for creating a new tag repository is arcane, could this
 patch (or a version of it) be pushed?

Rewriting this patch is the first thing on my list after nmbug's
Python translation lands [1].  Interestingly, there have been versions
of 'nmbug init' kicking around in the wings for some time [2].

Cheers,
Trevor

[1]: 
id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wking%40tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/19007
[2]: id:87obf66k7x.fsf@zancas.localnet
 http://thread.gmane.org/gmane.mail.notmuch.general/14474/focus=14493

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 05:32:59PM +0100, David Edmondson wrote:
 On Mon, Sep 15 2014, W. Trevor King wrote:
  On Mon, Sep 15, 2014 at 05:13:50PM +0100, David Edmondson wrote:
  On Sun, Jul 06 2014, W. Trevor King wrote:
   For folks that want to start versioning a new tag-space, instead
   of cloning one that someone else has already started.
  
  I tried this patch, and it (appeared) to work for me. Given that the
  procedure for creating a new tag repository is arcane, could this
  patch (or a version of it) be pushed?
 
  Rewriting this patch is the first thing on my list after nmbug's
  Python translation lands [1].
 
 What is required to make that happen?

Give it a spin to kick the tires, and reply to the v4 patch saying
that none of the pieces fell off while you were kicking it ;).
Basically, it's just waiting for review and a consensus that it's a
solid change.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Mon, Sep 15, 2014 at 08:09:11PM +0200, David Bremner wrote:
 W. Trevor King writes:
  Give it a spin to kick the tires, and reply to the v4 patch saying
  that none of the pieces fell off while you were kicking it ;).
  Basically, it's just waiting for review and a consensus that it's
  a solid change.
 
 v3 crashed when merging quite often for me.

I didn't actually change anything there, because I was unable to
reproduce the crash (as I point out in my v4 message, “I've also left
off the merge/pull checkouts, since I haven't been able to reproduce
David's error locally…” [1]).  Any advice on reproducing the merge
errors would be helpful :).

Cheers,
Trevor

[1]: id:e630b6763e9d0771718afee41ea15b29bb4a1de8.1409935538.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/19007

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4] nmbug: Translate to Python

[W. Trevor King]

bcommand [options]
-
-B for more help
-
-=head1 OPTIONS
-
-=head2 Most common commands
-
-=over 8
-
-=item B [message]
-
-Commit appropriately prefixed tags from the notmuch database to
-git. Any extra arguments are used (one per line) as a commit message.
-
-=item  B [remote]
-
-push local nmbug git state to remote repo
-
-=item  B [remote] [branch]
-
-pull (merge) remote repo changes to notmuch. B is equivalent to
-B followed by B.  The default remote is C, and
-the default branch is C.
-
-=back
-
-=head2 Other Useful Commands
-
-=over 8
-
-=item B repository
-
-Create a local nmbug repository from a remote source.  This wraps
-C, adding some options to avoid creating a working tree
-while preserving remote-tracking branches and upstreams.
-
-=item B
-
-Update the notmuch database from git. This is mainly useful to discard
-your changes in notmuch relative to git.
-
-=item B [remote]
-
-Fetch changes from the remote repo (see merge to bring those changes
-into notmuch).
-
-=item B [subcommand]
-
-print help [for subcommand]
-
-=item B [parameters]
-
-A simple wrapper for git log. After running C, you can
-inspect the changes with C
-
-=item B [commit]
-
-Merge changes from C into HEAD, and load the result into
-notmuch.  The default commit is C<@{upstream}>.
-
-=item  B
-
-Show pending updates in notmuch or git repo. See below for more
-information about the output format.
-
-=back
-
-=head2 Less common commands
-
-=over 8
-
-=item B
-
-Dump a tar archive (using git archive) of the current nmbug tag set.
-
-=back
-
-=head1 STATUS FORMAT
-
-B prints lines of the form
-
-   ng Message-Id tag
-
-where n is a single character representing notmuch database status
-
-=over 8
-
-=item B
-
-Tag is present in notmuch database, but not committed to nmbug
-(equivalently, tag has been deleted in nmbug repo, e.g. by a pull, but
-not restored to notmuch database).
-
-=item B
-
-Tag is present in nmbug repo, but not restored to notmuch database
-(equivalently, tag has been deleted in notmuch)
-
-=item B
-
-Message is unknown (missing from local notmuch database)
-
-=back
-
-The second character (if present) represents a difference between remote
-git and local. Typically C needs to be run to update this.
-
-=over 8
-
-
-=item B
-
-Tag is present in remote, but not in local git.
-
-
-=item B
-
-Tag is present in local git, but not in remote git.
-
-
-=back
-
-=head1 DUMP FORMAT
-
-Each tag $tag for message with Message-Id $id is written to
-an empty file
-
-   tags/encode($id)/encode($tag)
-
-The encoding preserves alphanumerics, and the characters "+-_@=.:,"
-(not the quotes).  All other octets are replaced with '%' followed by
-a two digit hex number.
-
-=head1 ENVIRONMENT
-
-B specifies the location of the git repository used by nmbug.
-If not specified $HOME/.nmbug is used.
-
-B specifies the prefix in the notmuch database for tags of
-interest to nmbug. If not specified 'notmuch::' is used.
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner 
+# W. Trevor King 
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+"""
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+"""
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX =

[PATCH v4] nmbug: Translate to Python

[W. Trevor King]

 database from git. This is mainly useful to discard
-your changes in notmuch relative to git.
-
-=item Bfetch [remote]
-
-Fetch changes from the remote repo (see merge to bring those changes
-into notmuch).
-
-=item Bhelp [subcommand]
-
-print help [for subcommand]
-
-=item Blog [parameters]
-
-A simple wrapper for git log. After running Cnmbug fetch, you can
-inspect the changes with Cnmbug log HEAD..@{upstream}
-
-=item Bmerge [commit]
-
-Merge changes from Ccommit into HEAD, and load the result into
-notmuch.  The default commit is C@{upstream}.
-
-=item  Bstatus
-
-Show pending updates in notmuch or git repo. See below for more
-information about the output format.
-
-=back
-
-=head2 Less common commands
-
-=over 8
-
-=item Barchive
-
-Dump a tar archive (using git archive) of the current nmbug tag set.
-
-=back
-
-=head1 STATUS FORMAT
-
-Bnmbug status prints lines of the form
-
-   ng Message-Id tag
-
-where n is a single character representing notmuch database status
-
-=over 8
-
-=item BA
-
-Tag is present in notmuch database, but not committed to nmbug
-(equivalently, tag has been deleted in nmbug repo, e.g. by a pull, but
-not restored to notmuch database).
-
-=item BD
-
-Tag is present in nmbug repo, but not restored to notmuch database
-(equivalently, tag has been deleted in notmuch)
-
-=item BU
-
-Message is unknown (missing from local notmuch database)
-
-=back
-
-The second character (if present) represents a difference between remote
-git and local. Typically Cnmbug fetch needs to be run to update this.
-
-=over 8
-
-
-=item Ba
-
-Tag is present in remote, but not in local git.
-
-
-=item Bd
-
-Tag is present in local git, but not in remote git.
-
-
-=back
-
-=head1 DUMP FORMAT
-
-Each tag $tag for message with Message-Id $id is written to
-an empty file
-
-   tags/encode($id)/encode($tag)
-
-The encoding preserves alphanumerics, and the characters +-_@=.:,
-(not the quotes).  All other octets are replaced with '%' followed by
-a two digit hex number.
-
-=head1 ENVIRONMENT
-
-BNMBGIT specifies the location of the git repository used by nmbug.
-If not specified $HOME/.nmbug is used.
-
-BNMBPREFIX specifies the prefix in the notmuch database for tags of
-interest to nmbug. If not specified 'notmuch::' is used.
+#!/usr/bin/env python
+#
+# Copyright (c) 2011-2014 David Bremner da...@tethera.net
+# W. Trevor King wk...@tremily.us
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see http://www.gnu.org/licenses/ .
+
+
+Manage notmuch tags with Git
+
+Environment variables:
+
+* NMBGIT specifies the location of the git repository used by nmbug.
+  If not specified $HOME/.nmbug is used.
+* NMBPREFIX specifies the prefix in the notmuch database for tags of
+  interest to nmbug. If not specified 'notmuch::' is used.
+
+
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import codecs as _codecs
+import collections as _collections
+import inspect as _inspect
+import logging as _logging
+import os as _os
+import re as _re
+import shutil as _shutil
+import subprocess as _subprocess
+import sys as _sys
+import tempfile as _tempfile
+import textwrap as _textwrap
+try:  # Python 3
+from urllib.parse import quote as _quote
+from urllib.parse import unquote as _unquote
+except ImportError:  # Python 2
+from urllib import quote as _quote
+from urllib import unquote as _unquote
+
+
+__version__ = '0.2'
+
+_LOG = _logging.getLogger('nmbug')
+_LOG.setLevel(_logging.ERROR)
+_LOG.addHandler(_logging.StreamHandler())
+
+NMBGIT = _os.path.expanduser(
+_os.getenv('NMBGIT', _os.path.join('~', '.nmbug')))
+_NMBGIT = _os.path.join(NMBGIT, '.git')
+if _os.path.isdir(_NMBGIT):
+NMBGIT = _NMBGIT
+
+TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::')
+_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}')
+_TAG_FILE_REGEX = _re.compile('tags/(?Pid[^/]*)/(?Ptag[^/]*)')
+
+# magic hash for Git (git hash-object -t blob /dev/null)
+_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391'
+
+
+try:
+getattr(_tempfile, 'TemporaryDirectory')
+except AttributeError:  # Python  3.2
+class _TemporaryDirectory(object):
+
+Fallback context manager for Python  3.2
+
+See PEP 343 for details on context managers [1].
+
+[1]: http://legacy.python.org/dev/peps/pep-0343/
+
+def __init__(self, **kwargs):
+self.name = _tempfile.mkdtemp(**kwargs

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Tue, Aug 05, 2014 at 10:24:10PM -0300, David Bremner wrote:
> I have a local commit that deletes a couple tags; when I attempt to
> merge I get complaints about local changes to files.
> 
> error: Your local changes to the following files would be overwritten by 
> merge:
>   tags/1406859003-11561-2-git-send-email-amdragon at mit.edu/needs-review
>   tags/1406859003-11561-3-git-send-email-amdragon at mit.edu/needs-review
> Please, commit your changes or stash them before you can merge.
> ?
> Calling the perl version of nmbug successfully creates a little 
> diamond merge

The Perl version has:

  git ( { GIT_WORK_TREE => $tempwork }, 'checkout', '-f', 'HEAD');
  git ( { GIT_WORK_TREE => $tempwork }, 'merge', $commit);

But the Python version only has:

  _git(
  args=['merge', reference],
  additional_env={'GIT_WORK_TREE': workdir},
  wait=True)

I suppose we need the checkout to populate the working directory, but
I'm not sure we want to force the checkout.  Do we expect to have
unmerged entries in the index?

I'll add an unforced checkout here in v4.  I'll add it to pull() too,
now that it's decoupled from nmbug's merge implementation.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Tue, Aug 05, 2014 at 10:16:10PM -0300, David Bremner wrote:
> David Bremner writes:
> > I did notice that merging was noticably noisier than I remembered. 
> 
> rejected pushes also seem noisier than before; I'm not sure the 
> python backtrace adds anything here.
> 
> To nmbug at nmbug.tethera.net:nmbug-tags
>  ! [rejected]master -> master (fetch first)
> error: failed to push some refs to 'nmbug at nmbug.tethera.net:nmbug-tags'
> hint: Updates were rejected because the remote contains work that you do
> hint: not have locally. This is usually caused by another repository pushing
> hint: to the same ref. You may want to first integrate the remote changes
> hint: (e.g., 'git pull ...') before pushing again.
> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
> Traceback (most recent call last):
>   File "/home/bremner/config/scripts/nmbug", line 766, in 
> args.func(**kwargs)
>   File "/home/bremner/config/scripts/nmbug", line 463, in push
> _git(args=args, wait=True)
>   File "/home/bremner/config/scripts/nmbug", line 210, in _git
> return _spawn(args=args, **kwargs)
>   File "/home/bremner/config/scripts/nmbug", line 193, in _spawn
> args=args, status=status, stdout=stdout, stderr=stderr)
> __main__.SubprocessError: ['git', '--git-dir', '/home/bremner/.nmbug', 
> 'push'] exited with 1

I can drop the backtrace and just print the SubprocessError, and just
show the traceback if the logging is set to ?debug?.  If that sounds
reasonable, I'll to it in v4.

I'm not sure what the current nmbug shows in this case.  Do you want
me to prune the ?hint? lines too?  I usually find those to be pretty
informative, but the're not directly applicable to folks using nmbug
who prefer to ignore the underlying Git layer.  Ideally, we'd
translate them to apply to nmbug (??(e.g. nmbug pull ...) before
pushing??), but I can't think of a maintainable way to do that.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Mon, Aug 04, 2014 at 09:14:46PM -0300, David Bremner wrote:
> W. Trevor King writes:
> > * Commands are no longer split into "most common", "other useful",
> >   and "less common" sets.  If we need something like this, I'd
> >   prefer workflow examples highlighting common commands in the
> >   module docstring (available with 'nmbug --help').
> 
> I don't feel strongly about this, but I remember implementing it by
> request in the first version. OTOH, I think you shortened up the
> main help string when you split it.

I didn't intentionally remove any information.  I think it's just
shorter because command-specific details are now in the
command-specific docstring/help.

> We may want to think about a seperate man page as a follow project.

Works for me, but I think it would look a lot like the wiki page [1].

> One thing I did notice is that there is no hint to call nmbug
> {command} --help in the main docstring.

I'll add that in v4.

> > +#!/usr/bin/env python
> > +# Copyright (c) 2011 David Bremner
> > +# License: same as notmuch
> 
> You should add your self, update the date, and probably explicitly
> state the license, as in Carl's patch for nmbug-status.

Will do in v4.

> > +__version__ = '0.2'
> 
> Do we need/want a version distinct from that of notmuch?

nmbug is very loosely bound to the notmuch core.  To me it feels like
a separate project that happens to share the same version control
repository.  I'm happy to synchronize versions, but then we have to
remember to bump the nmbug version for each notmuch release.

> > +def _hex_quote(string, safe='+@=:,'):
> 
> I'm not sure I really understand what makes a function/variable
> "private" and hence prefixed with _ in your translation.

The public interface is what I thought was reasonably stable for folks
who want to call nmbug as a Python library.  It's basically the
command-line functionality, with a few other helpers that seemed
important enough to be worth preserving.  If we keep a separate nmbug
version, I'd cut major releases for anything that broke compatibility
on a public function.  I don't need to use nmbug as a library myself,
so this might all be over-engineered ;).

> > +status, tree, stderr = _git(
> 
> as a non-native speaker of python, I find this a bit hard to read.
> How about adding some parens to make the multiple return more clear,
> so
> 
> (status, tree, stderr) = _git(

That's legal, but I rarely see the parenthesized version in the wild.
For examples showing the unparenthesized version, see [2,3].
Parentheses are optional for Python tuples [4], so you'd only want
them if ?=? had a higher precedence than ?,?.  That's my argument for
the unparenthesized version, but feel free to overrule me ;).

> I did notice that merging was noticably noisier than I remembered. 
> 
> > +output = _collections.defaultdict(
> > +lambda : _collections.defaultdict( # {tag: status_string}
> > +lambda : ' '))  # default local status
> 
> The initial comment is confusing (to me) because it looks like code.
> The two layers of defaultdict are a bit confusing too; it would help
> to mention the key of the outer dictionary.

In v4, I'll use a leading comment for the whole structure, instead of
interleaving the comments.  I'll also mention explicitly that the
outer layer is keyed by message id.

> I guess it makes sense to go for a relatively straight translation
> as a first step; I did wonder about whether using the python
> bindings to notmuch would speed things up.  Any ideas about how to
> even figure out where the bottlenecks are?

You could profile the Python script [5].  With my usual workflow, the
existing implementation isn't generating too many subprocesses.
Loading the database is probably slow though, so I'd expect reasonable
gains everywhere we call notmuch more than once.  I expect checkout
out working directories to also be slow, for the few command that do
that on behalf of Git.

Cheers,
Trevor


[1]: http://notmuchmail.org/nmbug/
[2]: 
https://docs.python.org/3.4/tutorial/introduction.html#first-steps-towards-programming
[3]: http://legacy.python.org/dev/peps/pep-3132/
[4]: https://docs.python.org/3.4/reference/expressions.html#parenthesized-forms
[5]: https://docs.python.org/3.4/library/profile.html

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140806/ff36993e/attachment.pgp>

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Mon, Aug 04, 2014 at 09:14:46PM -0300, David Bremner wrote:
 W. Trevor King writes:
  * Commands are no longer split into most common, other useful,
and less common sets.  If we need something like this, I'd
prefer workflow examples highlighting common commands in the
module docstring (available with 'nmbug --help').
 
 I don't feel strongly about this, but I remember implementing it by
 request in the first version. OTOH, I think you shortened up the
 main help string when you split it.

I didn't intentionally remove any information.  I think it's just
shorter because command-specific details are now in the
command-specific docstring/help.

 We may want to think about a seperate man page as a follow project.

Works for me, but I think it would look a lot like the wiki page [1].

 One thing I did notice is that there is no hint to call nmbug
 {command} --help in the main docstring.

I'll add that in v4.

  +#!/usr/bin/env python
  +# Copyright (c) 2011 David Bremner
  +# License: same as notmuch
 
 You should add your self, update the date, and probably explicitly
 state the license, as in Carl's patch for nmbug-status.

Will do in v4.

  +__version__ = '0.2'
 
 Do we need/want a version distinct from that of notmuch?

nmbug is very loosely bound to the notmuch core.  To me it feels like
a separate project that happens to share the same version control
repository.  I'm happy to synchronize versions, but then we have to
remember to bump the nmbug version for each notmuch release.

  +def _hex_quote(string, safe='+@=:,'):
 
 I'm not sure I really understand what makes a function/variable
 private and hence prefixed with _ in your translation.

The public interface is what I thought was reasonably stable for folks
who want to call nmbug as a Python library.  It's basically the
command-line functionality, with a few other helpers that seemed
important enough to be worth preserving.  If we keep a separate nmbug
version, I'd cut major releases for anything that broke compatibility
on a public function.  I don't need to use nmbug as a library myself,
so this might all be over-engineered ;).

  +status, tree, stderr = _git(
 
 as a non-native speaker of python, I find this a bit hard to read.
 How about adding some parens to make the multiple return more clear,
 so
 
 (status, tree, stderr) = _git(

That's legal, but I rarely see the parenthesized version in the wild.
For examples showing the unparenthesized version, see [2,3].
Parentheses are optional for Python tuples [4], so you'd only want
them if ‘=’ had a higher precedence than ‘,’.  That's my argument for
the unparenthesized version, but feel free to overrule me ;).

 I did notice that merging was noticably noisier than I remembered. 
 
  +output = _collections.defaultdict(
  +lambda : _collections.defaultdict( # {tag: status_string}
  +lambda : ' '))  # default local status
 
 The initial comment is confusing (to me) because it looks like code.
 The two layers of defaultdict are a bit confusing too; it would help
 to mention the key of the outer dictionary.

In v4, I'll use a leading comment for the whole structure, instead of
interleaving the comments.  I'll also mention explicitly that the
outer layer is keyed by message id.

 I guess it makes sense to go for a relatively straight translation
 as a first step; I did wonder about whether using the python
 bindings to notmuch would speed things up.  Any ideas about how to
 even figure out where the bottlenecks are?

You could profile the Python script [5].  With my usual workflow, the
existing implementation isn't generating too many subprocesses.
Loading the database is probably slow though, so I'd expect reasonable
gains everywhere we call notmuch more than once.  I expect checkout
out working directories to also be slow, for the few command that do
that on behalf of Git.

Cheers,
Trevor


[1]: http://notmuchmail.org/nmbug/
[2]: 
https://docs.python.org/3.4/tutorial/introduction.html#first-steps-towards-programming
[3]: http://legacy.python.org/dev/peps/pep-3132/
[4]: https://docs.python.org/3.4/reference/expressions.html#parenthesized-forms
[5]: https://docs.python.org/3.4/library/profile.html

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Tue, Aug 05, 2014 at 10:16:10PM -0300, David Bremner wrote:
 David Bremner writes:
  I did notice that merging was noticably noisier than I remembered. 
 
 rejected pushes also seem noisier than before; I'm not sure the 
 python backtrace adds anything here.
 
 To nm...@nmbug.tethera.net:nmbug-tags
  ! [rejected]master - master (fetch first)
 error: failed to push some refs to 'nm...@nmbug.tethera.net:nmbug-tags'
 hint: Updates were rejected because the remote contains work that you do
 hint: not have locally. This is usually caused by another repository pushing
 hint: to the same ref. You may want to first integrate the remote changes
 hint: (e.g., 'git pull ...') before pushing again.
 hint: See the 'Note about fast-forwards' in 'git push --help' for details.
 Traceback (most recent call last):
   File /home/bremner/config/scripts/nmbug, line 766, in module
 args.func(**kwargs)
   File /home/bremner/config/scripts/nmbug, line 463, in push
 _git(args=args, wait=True)
   File /home/bremner/config/scripts/nmbug, line 210, in _git
 return _spawn(args=args, **kwargs)
   File /home/bremner/config/scripts/nmbug, line 193, in _spawn
 args=args, status=status, stdout=stdout, stderr=stderr)
 __main__.SubprocessError: ['git', '--git-dir', '/home/bremner/.nmbug', 
 'push'] exited with 1

I can drop the backtrace and just print the SubprocessError, and just
show the traceback if the logging is set to ‘debug’.  If that sounds
reasonable, I'll to it in v4.

I'm not sure what the current nmbug shows in this case.  Do you want
me to prune the ‘hint’ lines too?  I usually find those to be pretty
informative, but the're not directly applicable to folks using nmbug
who prefer to ignore the underlying Git layer.  Ideally, we'd
translate them to apply to nmbug (“…(e.g. nmbug pull ...) before
pushing…”), but I can't think of a maintainable way to do that.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Tue, Aug 05, 2014 at 10:24:10PM -0300, David Bremner wrote:
 I have a local commit that deletes a couple tags; when I attempt to
 merge I get complaints about local changes to files.
 
 error: Your local changes to the following files would be overwritten by 
 merge:
   tags/1406859003-11561-2-git-send-email-amdra...@mit.edu/needs-review
   tags/1406859003-11561-3-git-send-email-amdra...@mit.edu/needs-review
 Please, commit your changes or stash them before you can merge.
 …
 Calling the perl version of nmbug successfully creates a little 
 diamond merge

The Perl version has:

  git ( { GIT_WORK_TREE = $tempwork }, 'checkout', '-f', 'HEAD');
  git ( { GIT_WORK_TREE = $tempwork }, 'merge', $commit);

But the Python version only has:

  _git(
  args=['merge', reference],
  additional_env={'GIT_WORK_TREE': workdir},
  wait=True)

I suppose we need the checkout to populate the working directory, but
I'm not sure we want to force the checkout.  Do we expect to have
unmerged entries in the index?

I'll add an unforced checkout here in v4.  I'll add it to pull() too,
now that it's decoupled from nmbug's merge implementation.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

I should also import print_function and unicode_literals from
__future__ for Python 2.x compatibility, since I use print() once and
never use bytes.  I hadn't turned up any problems with 2.x without the
__future__ imports, but it's nice to be explicit ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Sun, Jul 20, 2014 at 03:59:49PM -0700, W. Trevor King wrote:
> +def pull(repository=None, refspecs=None):
> +"""
> +Pull (merge) remote repository changes to notmuch.
> +
> +'pull' is equivalent to 'fetch' followed by 'merge'.  We use the
> +Git-configured repository for your current branch
> +(branch..repository, likely 'origin', and
> +branch..merge, likely 'master').
> +"""
> +_insist_committed()
> +if refspecs and not repository:
> +repository = _get_remote()
> +args = ['pull']
> +if repository:
> +args.append(repository)
> +if refspecs:
> +args.extend(refspecs)
> +with _tempfile.TemporaryDirectory(prefix='nmbug-merge.') as workdir:
> +_git(args=args, additional_env={'GIT_WORK_TREE': workdir}, wait=True)
> +checkout()

The TemporaryDirectory prefix should probably be 'nmbug-pull.'.
Queued for v4.

> +def log(args=()):
> +"""
> +A simple wrapper for 'git log'.
> +
> +After running 'nmbug fetch', you can inspect the changes with
> +'nmbug log HEAD..@{upstream}'.
> +"""
> +# we don't want output trapping here, because we want the pager.
> +args = ['git', '--git-dir', NMBGIT, 'log', '--name-status'] + list(args)
> +_LOG.debug('exec {args}'.format(args=args))
> +_os.execvp('git', args)

I don't exec any other commands.  Maybe we want '_git(args=args,
wait=True)' here (with the appropriate args adjustments)?

> +def _diff_index(index, filter):
> +"""Get an {id: {tag, ...}} dict for a given filter.
> +
> +For example, use 'A' to find added tags, and 'D' to find deleted tags.
> +"""

I'll shift the summary onto the next line here to match the pattern
set by the command functions (e.g. archive()).  They *need* the
summary to be on the line after the opening triple-quote to support
the textwrap.dedent() help used for the argument parser.

There were also a few docstrings missing the trailing period
recommended by PEP 257 [1] (for _hex_quote, get_tags, _read_tree,
fetch, _index_tags, and _unpack_diff_lines).  I'll add those periods
in v4.

Cheers,
Trevor

[1]: http://legacy.python.org/dev/peps/pep-0257/#one-line-docstrings

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140720/f7ed883e/attachment.pgp>

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Sun, Jul 20, 2014 at 03:59:49PM -0700, W. Trevor King wrote:
> Most of the user-facing interface is the same, but there are a few
> changes, where reproducing the original interface was too difficult
> or I saw a change to make the underlying Git UI accessible:

It's not listed in the commit message, but another change is that
'nmbug commit' now only uses a single argument for the optional
commit-message text.  I'll document this in the v4 commit message,
after v3 has cooked for a bit.  I wanted to expose more of the
underlying 'git commit' UI, since I personally like to write my commit
messages in an editor with the notes added by 'git commit -v' to jog
my memory.  Unfortunately, we're using 'git commit-tree' instead of
'git commit', and commit-tree is too low-level for editor-launching.
I'd be interested in rewriting commit() to use 'git commit', but that
seemed like it was outside the scope of this rewrite.  So I'm not
supporting all of Git's commit syntax in this patch, but I can at
least match 'git commit -m MESSAGE' in requiring command-line commit
messages to be a single argument.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140720/901cc989/attachment.pgp>

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into "most common", "other useful", and
  "less common" sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch..remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch..remote and branch..merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
Changes since v2 [1]:

* Use four spaces (instead of two) to indent the _is_committed() body.
* Finish the dangling sentence for the _insist_committed() docstring.

Sorry for the noisy v1/v2/v3 submisson.  Obviously this patch is too
long for me to proof-read accurately so soon ;).  I'd hold off merging
it for at least a week to give me time to go over it again with
clearer eyes.  Of course, anyone else who wants to chip in reviewing
it is welcome to :).

Cheers,
Trevor

[1]: id:57f22f6cd86b390969851e7805c9499ba99d2489.1405896148.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18758

 devel/nmbug/nmbug | 1474 -
 1 file changed, 766 insertions(+), 708 deletions(-)
 rewrite devel/nmbug/nmbug (97%)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
dissimilarity index 97%
index 998ee6b..57948d3 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,766 @@
-#!/usr/bin/env perl
-# Copyright (c) 2011 David Bremner
-# License: same as notmuch
-
-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive=> \_archive,
-checkout   => \_checkout,
-clone  => \_clone,
-commit => \_commit,
-fetch  => \_fetch,
-help   => \_help,
-log=> \_log,
-merge  => \_merge,
-pull   => \_pull,
-push   => \_push,
-status => \_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash for git
-my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
-
-&{$command{$subcommand}}(@ARGV);
-
-sub git_pipe {
-  my $envref = (ref $_[0] eq 

[PATCH v2] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into "most common", "other useful", and
  "less common" sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch..remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch..remote and branch..merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
Fixes since v1 [1]:

* args.append('fetch') -> args.append(remote) in fetch().
* Don't bother with default refspecs in push().  Git has configs for
  the implicit refspec case, and vanilla nmbug users will be fine with
  those.
* args.push(...) -> args.append(...) in push().
* Add 'wait=True' to _git() calls in fetch() and push(), because we
  want to check the exit codes and complain if the call failed.

This patch is also formatted with --break-rewrites [2].

Cheers,
Trevor

[1]: id:6fed92570268096111794740f4ef09d53d94115f.1405892434.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18756
[2]: id:CAB+hUn9vCn-7SW=_=tDzeH4eUGAF4QYX4_BvGGzdkpkVkZXM4A at mail.gmail.com
 http://article.gmane.org/gmane.mail.notmuch.general/18757

 devel/nmbug/nmbug | 1474 -
 1 file changed, 766 insertions(+), 708 deletions(-)
 rewrite devel/nmbug/nmbug (97%)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
dissimilarity index 97%
index 998ee6b..9dbad87 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,766 @@
-#!/usr/bin/env perl
-# Copyright (c) 2011 David Bremner
-# License: same as notmuch
-
-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive=> \_archive,
-checkout   => \_checkout,
-clone  => \_clone,
-commit => \_commit,
-fetch  => \_fetch,
-help   => \_help,
-log=> \_log,
-merge  => \_merge,
-pull   => \_pull,
-push   => \_push,
-status => \_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash for git

[PATCH] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into "most common", "other useful", and
  "less common" sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch..remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch..remote and branch..merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
As discussed here [1], this may also make it easier for others to
contribute.  A future patches can slot in the notmuch Python bindings,
which will likely be faster than spawning zillions of subprocesses to
check for message-id existence in get_status when there are many
maybe-deleted tags.

Once (if?) this lands, I'll submit an additional patch with the 'init'
command [2].

Cheers,
Trevor

[1]: id:87y4vt8vrw.fsf at maritornes.cs.unb.ca
 http://article.gmane.org/gmane.mail.notmuch.general/18729
[2]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wking at 
tremily.us
 http://http://article.gmane.org/gmane.mail.notmuch.general/18630

 devel/nmbug/nmbug | 1450 -
 1 file changed, 755 insertions(+), 695 deletions(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 998ee6b..208f893 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,768 @@
-#!/usr/bin/env perl
+#!/usr/bin/env python
 # Copyright (c) 2011 David Bremner
 # License: same as notmuch

-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive=> \_archive,
-checkout   => \_checkout,
-clone  => \_clone,
-commit => \_commit,
-fetch  => \_fetch,
-help   => \_help,
-log=> \_log,
-merge  => \_merge,
-pull   => \_pull,
-push   => \_push,
-status => \_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash for git
-my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
-
-&{$command{$subcommand}}(@ARGV);
-
-sub git_pipe {
-  my $envref = (ref $_[0] eq 

[PATCH] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into most common, other useful, and
  less common sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch.name.remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch.name.remote and branch.name.merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
As discussed here [1], this may also make it easier for others to
contribute.  A future patches can slot in the notmuch Python bindings,
which will likely be faster than spawning zillions of subprocesses to
check for message-id existence in get_status when there are many
maybe-deleted tags.

Once (if?) this lands, I'll submit an additional patch with the 'init'
command [2].

Cheers,
Trevor

[1]: id:87y4vt8vrw@maritornes.cs.unb.ca
 http://article.gmane.org/gmane.mail.notmuch.general/18729
[2]: id:05ccd672f55444f74da62250e2305fb84fdc6c42.1404678709.git.wk...@tremily.us
 http://http://article.gmane.org/gmane.mail.notmuch.general/18630

 devel/nmbug/nmbug | 1450 -
 1 file changed, 755 insertions(+), 695 deletions(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 998ee6b..208f893 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,768 @@
-#!/usr/bin/env perl
+#!/usr/bin/env python
 # Copyright (c) 2011 David Bremner
 # License: same as notmuch
 
-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive= \do_archive,
-checkout   = \do_checkout,
-clone  = \do_clone,
-commit = \do_commit,
-fetch  = \do_fetch,
-help   = \do_help,
-log= \do_log,
-merge  = \do_merge,
-pull   = \do_pull,
-push   = \do_push,
-status = \do_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash for git
-my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
-
-{$command{$subcommand}}(@ARGV);
-
-sub git_pipe {
-  my $envref = (ref $_[0] 

[PATCH v2] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into most common, other useful, and
  less common sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch.name.remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch.name.remote and branch.name.merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
Fixes since v1 [1]:

* args.append('fetch') - args.append(remote) in fetch().
* Don't bother with default refspecs in push().  Git has configs for
  the implicit refspec case, and vanilla nmbug users will be fine with
  those.
* args.push(...) - args.append(...) in push().
* Add 'wait=True' to _git() calls in fetch() and push(), because we
  want to check the exit codes and complain if the call failed.

This patch is also formatted with --break-rewrites [2].

Cheers,
Trevor

[1]: id:6fed92570268096111794740f4ef09d53d94115f.1405892434.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18756
[2]: id:CAB+hUn9vCn-7SW=_=tdzeh4eugaf4qyx4_bvggzdkpkvkzx...@mail.gmail.com
 http://article.gmane.org/gmane.mail.notmuch.general/18757

 devel/nmbug/nmbug | 1474 -
 1 file changed, 766 insertions(+), 708 deletions(-)
 rewrite devel/nmbug/nmbug (97%)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
dissimilarity index 97%
index 998ee6b..9dbad87 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,766 @@
-#!/usr/bin/env perl
-# Copyright (c) 2011 David Bremner
-# License: same as notmuch
-
-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive= \do_archive,
-checkout   = \do_checkout,
-clone  = \do_clone,
-commit = \do_commit,
-fetch  = \do_fetch,
-help   = \do_help,
-log= \do_log,
-merge  = \do_merge,
-pull   = \do_pull,
-push   = \do_push,
-status = \do_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash 

[PATCH v3] nmbug: Translate to Python

[W. Trevor King]

This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages.  It should be compatible with Python 2.6 and later
(including the 3.x series), although with 2.6 you'll need the external
argparse package.

Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:

* 'nmbug help' has been split between the general 'nmbug --help' and
  the command-specific 'nmbug COMMAND --help'.

* Commands are no longer split into most common, other useful, and
  less common sets.  If we need something like this, I'd prefer
  workflow examples highlighting common commands in the module
  docstring (available with 'nmbug --help').

* The default repository for 'nmbug push' and 'nmbug fetch' is now the
  current branch's upstream (branch.name.remote) instead of
  'origin'.  When we have to, we extract this remote by hand, but
  where possible we just call the Git command without a repository
  argument, and leave it to Git to figure out the default.

* 'nmbug push' accepts multiple refspecs if you want to explicitly
  specify what to push.  Otherwise, the refspec(s) pushed depend on
  push.default.  The Perl version hardcoded 'master' as the pushed
  refspec.

* 'nmbug pull' defaults to the current branch's upstream
  (branch.name.remote and branch.name.merge) instead of hardcoding
  'origin' and 'master'.  It also supports multiple refspecs if for
  some crazy reason you need an octopus merge (but mostly to avoid
  breaking consistency with 'git pull').

* 'nmbug log' now execs 'git log', as there's no need to keep the
  Python process around once we've launched Git there.

* 'nmbug status' now catches stderr, and doesn't print errors like:

No upstream configured for branch 'master'

  The Perl implementation had just learned to avoid crashing on that
  case, but wasn't yet catching the dying subprocess's stderr.

* 'nmbug archive' now accepts positional arguments for the tree-ish
  and additional 'git archive' options.  For example, you can run:

$ nmbug archive HEAD -- --format tar.gz

  I wish I could have preserved the argument order from 'git archive'
  (with the tree-ish at the end), but I'm not sure how to make
  argparse accept arbitrary possitional arguments (some of which take
  arguments).  Flipping the order to put the tree-ish first seemed
  easiest.
---
Changes since v2 [1]:

* Use four spaces (instead of two) to indent the _is_committed() body.
* Finish the dangling sentence for the _insist_committed() docstring.

Sorry for the noisy v1/v2/v3 submisson.  Obviously this patch is too
long for me to proof-read accurately so soon ;).  I'd hold off merging
it for at least a week to give me time to go over it again with
clearer eyes.  Of course, anyone else who wants to chip in reviewing
it is welcome to :).

Cheers,
Trevor

[1]: id:57f22f6cd86b390969851e7805c9499ba99d2489.1405896148.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18758

 devel/nmbug/nmbug | 1474 -
 1 file changed, 766 insertions(+), 708 deletions(-)
 rewrite devel/nmbug/nmbug (97%)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
dissimilarity index 97%
index 998ee6b..57948d3 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -1,708 +1,766 @@
-#!/usr/bin/env perl
-# Copyright (c) 2011 David Bremner
-# License: same as notmuch
-
-use strict;
-use warnings;
-use File::Temp qw(tempdir);
-use Pod::Usage;
-
-no encoding;
-
-my $NMBGIT = $ENV{NMBGIT} || $ENV{HOME}.'/.nmbug';
-
-$NMBGIT .= '/.git' if (-d $NMBGIT.'/.git');
-
-my $TAGPREFIX = defined($ENV{NMBPREFIX}) ? $ENV{NMBPREFIX} : 'notmuch::';
-
-# for encoding
-
-my $ESCAPE_CHAR =  '%';
-my $NO_ESCAPE =
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'.
-   '0123456789+-_@=.:,';
-my $MUST_ENCODE =  qr{[^\Q$NO_ESCAPE\E]};
-my $ESCAPED_RX =   qr{$ESCAPE_CHAR([A-Fa-f0-9]{2})};
-
-my %command = (
-archive= \do_archive,
-checkout   = \do_checkout,
-clone  = \do_clone,
-commit = \do_commit,
-fetch  = \do_fetch,
-help   = \do_help,
-log= \do_log,
-merge  = \do_merge,
-pull   = \do_pull,
-push   = \do_push,
-status = \do_status,
-);
-
-# Convert prefix into form suitable for literal matching against
-# notmuch dump --format=batch-tag output.
-my $ENCPREFIX = encode_for_fs ($TAGPREFIX);
-$ENCPREFIX =~ s/:/%3a/g;
-
-my $subcommand = shift || usage ();
-
-if (!exists $command{$subcommand}) {
-  usage ();
-}
-
-# magic hash for git
-my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
-
-{$command{$subcommand}}(@ARGV);
-
-sub git_pipe {
-  my $envref = (ref 

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Sun, Jul 20, 2014 at 03:59:49PM -0700, W. Trevor King wrote:
 Most of the user-facing interface is the same, but there are a few
 changes, where reproducing the original interface was too difficult
 or I saw a change to make the underlying Git UI accessible:

It's not listed in the commit message, but another change is that
'nmbug commit' now only uses a single argument for the optional
commit-message text.  I'll document this in the v4 commit message,
after v3 has cooked for a bit.  I wanted to expose more of the
underlying 'git commit' UI, since I personally like to write my commit
messages in an editor with the notes added by 'git commit -v' to jog
my memory.  Unfortunately, we're using 'git commit-tree' instead of
'git commit', and commit-tree is too low-level for editor-launching.
I'd be interested in rewriting commit() to use 'git commit', but that
seemed like it was outside the scope of this rewrite.  So I'm not
supporting all of Git's commit syntax in this patch, but I can at
least match 'git commit -m MESSAGE' in requiring command-line commit
messages to be a single argument.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

On Sun, Jul 20, 2014 at 03:59:49PM -0700, W. Trevor King wrote:
 +def pull(repository=None, refspecs=None):
 +
 +Pull (merge) remote repository changes to notmuch.
 +
 +'pull' is equivalent to 'fetch' followed by 'merge'.  We use the
 +Git-configured repository for your current branch
 +(branch.name.repository, likely 'origin', and
 +branch.name.merge, likely 'master').
 +
 +_insist_committed()
 +if refspecs and not repository:
 +repository = _get_remote()
 +args = ['pull']
 +if repository:
 +args.append(repository)
 +if refspecs:
 +args.extend(refspecs)
 +with _tempfile.TemporaryDirectory(prefix='nmbug-merge.') as workdir:
 +_git(args=args, additional_env={'GIT_WORK_TREE': workdir}, wait=True)
 +checkout()

The TemporaryDirectory prefix should probably be 'nmbug-pull.'.
Queued for v4.

 +def log(args=()):
 +
 +A simple wrapper for 'git log'.
 +
 +After running 'nmbug fetch', you can inspect the changes with
 +'nmbug log HEAD..@{upstream}'.
 +
 +# we don't want output trapping here, because we want the pager.
 +args = ['git', '--git-dir', NMBGIT, 'log', '--name-status'] + list(args)
 +_LOG.debug('exec {args}'.format(args=args))
 +_os.execvp('git', args)

I don't exec any other commands.  Maybe we want '_git(args=args,
wait=True)' here (with the appropriate args adjustments)?

 +def _diff_index(index, filter):
 +Get an {id: {tag, ...}} dict for a given filter.
 +
 +For example, use 'A' to find added tags, and 'D' to find deleted tags.
 +

I'll shift the summary onto the next line here to match the pattern
set by the command functions (e.g. archive()).  They *need* the
summary to be on the line after the opening triple-quote to support
the textwrap.dedent() help used for the argument parser.

There were also a few docstrings missing the trailing period
recommended by PEP 257 [1] (for _hex_quote, get_tags, _read_tree,
fetch, _index_tags, and _unpack_diff_lines).  I'll add those periods
in v4.

Cheers,
Trevor

[1]: http://legacy.python.org/dev/peps/pep-0257/#one-line-docstrings

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] nmbug: Translate to Python

[W. Trevor King]

I should also import print_function and unicode_literals from
__future__ for Python 2.x compatibility, since I use print() once and
never use bytes.  I hadn't turned up any problems with 2.x without the
__future__ imports, but it's nice to be explicit ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/4] nmbug: Handle missing @upstream in is_unmerged

[W. Trevor King]

On Thu, Jul 17, 2014 at 06:28:55AM -0300, David Bremner wrote:
 W. Trevor King writes:
  On Wed, Jul 16, 2014 at 07:36:10PM -0300, David Bremner wrote:
  W. Trevor King writes:
   If we don't have an upstream, there is nothing to merge, so
   nothing is unmerged.  This avoids errors like:
  
  pushed this one patch.
 
  Without the stderr-catching of something like patch 3, this means
  folks without an upstream are going to see a distracting:
 
error: No upstream configured for branch 'master'
 
  if they run 'nmbug status' without an @upstream.
 
  I'm working through the Python translation now ;).
 
 Perhaps the best thing is to revert that patch then?

Sure.

Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/4] nmbug: Handle missing @upstream in is_unmerged

[W. Trevor King]

On Wed, Jul 16, 2014 at 07:36:10PM -0300, David Bremner wrote:
 W. Trevor King writes:
  If we don't have an upstream, there is nothing to merge, so
  nothing is unmerged.  This avoids errors like:
 
 pushed this one patch.

Without the stderr-catching of something like patch 3, this means
folks without an upstream are going to see a distracting:

  error: No upstream configured for branch 'master'

if they run 'nmbug status' without an @upstream.

I'm working through the Python translation now ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/4] nmbug: Catch stderr in is_unmerged

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:49:58PM -0300, David Bremner wrote:
> "W. Trevor King"  writes:
> 
> > +  if ($dir eq '-2|') {
> > +$dir = '-|';
> > +  }
> > +
> 
> I think I'd prefer an extra flag, rather than making new syntax.
> The existing syntax is not pretty, but it is standard perl

There should be standard Perl syntax for capturing both streams ;).  I
can add a separate parameter instead, but Perl doesn't seem to have
keyword-arguments.  At least, nmbug relies on assumptions about
argument values:

  sub git_pipe {
my $envref = (ref $_[0] eq 'HASH') ? shift : {};
my $ioref  = (ref $_[0] eq 'ARRAY') ? shift : undef;
my $dir = ($_[0] eq '-|' or $_[0] eq '|-') ? shift : undef;
?
  }

to map positional arguments back into particular semantic meanings.
That seemed ugly enough (can we only have a single boolean argument
with this pattern?) that I'd prefer piggy-backing on Perl's
stream-direction syntax.  But let me know what you like best, and I'll
go that way with v2.  I'd be happy to rewrite nmbug in Python too,
which has (to my eyes) much saner kwargs and subprocess handling.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140715/5641a03a/attachment.pgp>

[PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:54:28PM -0300, David Bremner wrote:
> "W. Trevor King"  writes:
> 
> > +sub do_init {
> > +  my $tempwork = tempdir ('/tmp/nmbug-init.XX', CLEANUP => 1);
> > +  system ('git', 'init', '--separate-git-dir', $NMBGIT, $tempwork) == 0
> > +or die "'git init' exited with nonzero value\n";
> > +  git ('config', '--unset', 'core.worktree');
> > +  git ('config', 'core.bare', 'true');
> > +  # create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
> > +  git ('hash-object', '-w', '--stdin');
> > +  git ( { GIT_WORK_TREE => $tempwork }, 'commit', '--allow-empty',
> > +'-m', 'Start a new nmbug repository' );
> > +}
> > +
> 
> 
> Shouldn't this empty blob already be created by the following line:
> 
> my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
> 
> Or is the key point to write it into the database?  Anyway I like my
> hack slightly better than yours ;).

We need to write it to the database.  I'll use /dev/null in v2,
though.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140715/c652b5c9/attachment.pgp>

[PATCH 2/4] nmbug: Handle missing @upstream in is_unmerged

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:44:53PM -0300, David Bremner wrote:
> W. Trevor King writes:
> > -  my $fetch_head = git ('rev-parse', $commit);
> > +  my ($fetch_head, $status) = git_with_status ('rev-parse', $commit);
> > +  if ($status) {
> > +return 0;
> > +  }
> 
> Could there be other errors here, other than @{upstream} not
> existing?  At first glance it seems like there is potential to hide
> errors here.

Possible errors that I can find:

* fatal: Not a git repository: ? (with a poor GIT_DIR config)
* fatal: ambiguous argument ? (with an invalid/missing revision name)
* fatal: No upstream configured for branch ? (when
  branch..remote or branch..merge aren't set)

All of which return 128 as of Git v1.9.1.  We're only interested in
the last.  I'm fine looking for ?No upstream configured for branch?
after we capture stderr with ?nmbug: Catch stderr in is_unmerged? [1],
but I don't expect the other two error cases to happen very often.  At
least, I doubt you could get this far into do_status with a broken
GIT_DIR, and we're hard-coding '@{upstream}' here.

Cheers,
Trevor

[1]: id:d55cf02465c5f2d83f2dd0bc666831ee524b0fb7.1404678709.git.wking at 
tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18627

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140715/023b1ed3/attachment-0001.pgp>

Re: [PATCH 2/4] nmbug: Handle missing @upstream in is_unmerged

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:44:53PM -0300, David Bremner wrote:
 W. Trevor King writes:
  -  my $fetch_head = git ('rev-parse', $commit);
  +  my ($fetch_head, $status) = git_with_status ('rev-parse', $commit);
  +  if ($status) {
  +return 0;
  +  }
 
 Could there be other errors here, other than @{upstream} not
 existing?  At first glance it seems like there is potential to hide
 errors here.

Possible errors that I can find:

* fatal: Not a git repository: … (with a poor GIT_DIR config)
* fatal: ambiguous argument … (with an invalid/missing revision name)
* fatal: No upstream configured for branch … (when
  branch.name.remote or branch.name.merge aren't set)

All of which return 128 as of Git v1.9.1.  We're only interested in
the last.  I'm fine looking for “No upstream configured for branch”
after we capture stderr with “nmbug: Catch stderr in is_unmerged” [1],
but I don't expect the other two error cases to happen very often.  At
least, I doubt you could get this far into do_status with a broken
GIT_DIR, and we're hard-coding '@{upstream}' here.

Cheers,
Trevor

[1]: id:d55cf02465c5f2d83f2dd0bc666831ee524b0fb7.1404678709.git.wk...@tremily.us
 http://article.gmane.org/gmane.mail.notmuch.general/18627

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] nmbug: Add an 'init' command

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:54:28PM -0300, David Bremner wrote:
 W. Trevor King wk...@tremily.us writes:
 
  +sub do_init {
  +  my $tempwork = tempdir ('/tmp/nmbug-init.XX', CLEANUP = 1);
  +  system ('git', 'init', '--separate-git-dir', $NMBGIT, $tempwork) == 0
  +or die 'git init' exited with nonzero value\n;
  +  git ('config', '--unset', 'core.worktree');
  +  git ('config', 'core.bare', 'true');
  +  # create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
  +  git ('hash-object', '-w', '--stdin');
  +  git ( { GIT_WORK_TREE = $tempwork }, 'commit', '--allow-empty',
  +'-m', 'Start a new nmbug repository' );
  +}
  +
 
 
 Shouldn't this empty blob already be created by the following line:
 
 my $EMPTYBLOB = git (qw{hash-object -t blob /dev/null});
 
 Or is the key point to write it into the database?  Anyway I like my
 hack slightly better than yours ;).

We need to write it to the database.  I'll use /dev/null in v2,
though.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/4] nmbug: Catch stderr in is_unmerged

[W. Trevor King]

On Tue, Jul 15, 2014 at 08:49:58PM -0300, David Bremner wrote:
 W. Trevor King wk...@tremily.us writes:
 
  +  if ($dir eq '-2|') {
  +$dir = '-|';
  +  }
  +
 
 I think I'd prefer an extra flag, rather than making new syntax.
 The existing syntax is not pretty, but it is standard perl

There should be standard Perl syntax for capturing both streams ;).  I
can add a separate parameter instead, but Perl doesn't seem to have
keyword-arguments.  At least, nmbug relies on assumptions about
argument values:

  sub git_pipe {
my $envref = (ref $_[0] eq 'HASH') ? shift : {};
my $ioref  = (ref $_[0] eq 'ARRAY') ? shift : undef;
my $dir = ($_[0] eq '-|' or $_[0] eq '|-') ? shift : undef;
…
  }

to map positional arguments back into particular semantic meanings.
That seemed ugly enough (can we only have a single boolean argument
with this pattern?) that I'd prefer piggy-backing on Perl's
stream-direction syntax.  But let me know what you like best, and I'll
go that way with v2.  I'd be happy to rewrite nmbug in Python too,
which has (to my eyes) much saner kwargs and subprocess handling.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] nmbug-status: Use 'show-ref --heads' for loading configs

[W. Trevor King]

On Mon, Jul 14, 2014 at 06:51:22AM -0300, David Bremner wrote:
> W. Trevor King writes:
> > On Sun, Jul 13, 2014 at 09:30:56AM -0300, David Bremner wrote:
> >> I consider it a useful feature that it works without the user
> >> configuring a local branch.  I agree that in more complex setups
> >> this ambiguity is not as nice, but I'd rather it was only the
> >> minority of users with unusual setups (e.g. multiple remotes)
> >> have to do configuration.
> >
> > I could work up a patch that tried ?git show-ref -s config? first,
> > and only fell back to ?show-ref -s --heads? if there were multiple
> > matches.  That way folks with only origin/config wouldn't need a
> > local branch, but folks with multiple config-carrying remotes (or
> > a single config-carrying remote and a local branch) would have to
> > have a local config to break the tie.  That's possible, and not
> > *too* complicated, but I personally prefer the consistency of just
> > requiring a local config branch.
>
> It might be simpler to implement (and understand) to try first the
> local config branch and then fall back to "origin/config".

I'd rather avoid hard-coding an upstream name here.  We do hard-code
?origin? as the default remote for ?fetch?, ?pull?, and ?push?.  I'd
rather drop those in favor of the configured Git defaults (for
example, running a bare ?git fetch? if the user doesn't specify a
remote).  For most configurations, falling back to ?Is ?git show-ref
-s config? a unique hash?? should be equivalent to hard-coding ?git
show-ref -s origin/config?.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140714/2d19e3d2/attachment.pgp>

Re: [PATCH 2/2] nmbug-status: Use 'show-ref --heads' for loading configs

[W. Trevor King]

On Mon, Jul 14, 2014 at 06:51:22AM -0300, David Bremner wrote:
 W. Trevor King writes:
  On Sun, Jul 13, 2014 at 09:30:56AM -0300, David Bremner wrote:
  I consider it a useful feature that it works without the user
  configuring a local branch.  I agree that in more complex setups
  this ambiguity is not as nice, but I'd rather it was only the
  minority of users with unusual setups (e.g. multiple remotes)
  have to do configuration.
 
  I could work up a patch that tried ‘git show-ref -s config’ first,
  and only fell back to ‘show-ref -s --heads’ if there were multiple
  matches.  That way folks with only origin/config wouldn't need a
  local branch, but folks with multiple config-carrying remotes (or
  a single config-carrying remote and a local branch) would have to
  have a local config to break the tie.  That's possible, and not
  *too* complicated, but I personally prefer the consistency of just
  requiring a local config branch.

 It might be simpler to implement (and understand) to try first the
 local config branch and then fall back to origin/config.

I'd rather avoid hard-coding an upstream name here.  We do hard-code
‘origin’ as the default remote for ‘fetch’, ‘pull’, and ‘push’.  I'd
rather drop those in favor of the configured Git defaults (for
example, running a bare ‘git fetch’ if the user doesn't specify a
remote).  For most configurations, falling back to “Is ‘git show-ref
-s config’ a unique hash?” should be equivalent to hard-coding ‘git
show-ref -s origin/config’.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] nmbug-status: Use 'show-ref --heads' for loading configs

[W. Trevor King]

On Sun, Jul 13, 2014 at 09:30:56AM -0300, David Bremner wrote:
> I consider it a useful feature that it works without the user
> configuring a local branch.  I agree that in more complex setups
> this ambiguity is not as nice, but I'd rather it was only the
> minority of users with unusual setups (e.g. multiple remotes) have
> to do configuration.

I could work up a patch that tried ?git show-ref -s config? first, and
only fell back to ?show-ref -s --heads? if there were multiple
matches.  That way folks with only origin/config wouldn't need a local
branch, but folks with multiple config-carrying remotes (or a single
config-carrying remote and a local branch) would have to have a local
config to break the tie.  That's possible, and not *too* complicated,
but I personally prefer the consistency of just requiring a local
config branch.

There's probably a project-size threshold after which you want better
access controls than a shared public repository like nmbug.tethera.net
gives you.  At that point, most folks are going to want their own,
personal public repository.  Once you have that, the folks using the
?config? branch (maybe not very many?) are going to need to keep their
own local branches.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

[PATCH v3 0/5] rst2man.py support and doc-build cleanups

[W. Trevor King]

On Sun, Jul 13, 2014 at 09:05:41AM +0300, Tomi Ollila wrote:
> I am satisfied with rst-man2any.py, but as being normal picky me I
> wonder whether the command prefix 'rst-' is being too generic
> i.e. is invading that "namespace". If no one else has the same
> feeling (or the feeling is just wrong (or insignificant)) then this
> can be forgotten :D

For what it's worth, I don't have any ?rst-*? commands on my system.
I do have ?rst2*? commands, and an ?rstpep2html.py?.  I'm happy to
rename to whatever, but rst-man2any.py was the best that I could think
of following the texi2any pattern.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: 

Re: [PATCH v3 0/5] rst2man.py support and doc-build cleanups

[W. Trevor King]

On Sun, Jul 13, 2014 at 09:05:41AM +0300, Tomi Ollila wrote:
 I am satisfied with rst-man2any.py, but as being normal picky me I
 wonder whether the command prefix 'rst-' is being too generic
 i.e. is invading that namespace. If no one else has the same
 feeling (or the feeling is just wrong (or insignificant)) then this
 can be forgotten :D

For what it's worth, I don't have any ‘rst-*’ commands on my system.
I do have ‘rst2*’ commands, and an ‘rstpep2html.py’.  I'm happy to
rename to whatever, but rst-man2any.py was the best that I could think
of following the texi2any pattern.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/2] nmbug-status: Use 'show-ref --heads' for loading configs

[W. Trevor King]

On Sun, Jul 13, 2014 at 09:30:56AM -0300, David Bremner wrote:
 I consider it a useful feature that it works without the user
 configuring a local branch.  I agree that in more complex setups
 this ambiguity is not as nice, but I'd rather it was only the
 minority of users with unusual setups (e.g. multiple remotes) have
 to do configuration.

I could work up a patch that tried ‘git show-ref -s config’ first, and
only fell back to ‘show-ref -s --heads’ if there were multiple
matches.  That way folks with only origin/config wouldn't need a local
branch, but folks with multiple config-carrying remotes (or a single
config-carrying remote and a local branch) would have to have a local
config to break the tie.  That's possible, and not *too* complicated,
but I personally prefer the consistency of just requiring a local
config branch.

There's probably a project-size threshold after which you want better
access controls than a shared public repository like nmbug.tethera.net
gives you.  At that point, most folks are going to want their own,
personal public repository.  Once you have that, the folks using the
‘config’ branch (maybe not very many?) are going to need to keep their
own local branches.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 5/5] doc: Add rst2html support for building HTML docs

[W. Trevor King]

We already fall back to rst2man if Sphinx isn't available for building
the man pages.  With this commit we'll fall back to rst2html for
building the HTML docs too.  The only tricky bit here is that
HAVE_SPHINX explicitly checks for sphinx.writers.manpage.  I'm just
assuming sphinx.writers.html exists, to avoid adding a separate
SPHINX_HTML variable.  However, the HTML builder is Sphinx's default,
so it's hard to imagine it not being installed :p.  Perhaps it would
be better to drop the HAVE_* settings and SPHINXBUILD to use:

* SPHINX2MAN empty for not-available, otherwise path to script that
  can handle '-b manpage'
* SPHINX2HTML empty for not-available, otherwise path to script that
  can handle '-b html'
* RST2MAN empty for not-available, otherwise path to script
* RST2HTML empty for not-available, otherwise path to script

I'm sticking with the less intrusive change for now, but I'm happy
with either approach.
---
 configure  | 25 -
 doc/INSTALL|  7 +--
 doc/Makefile.local |  2 ++
 3 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 20a2d5f..446439f 100755
--- a/configure
+++ b/configure
@@ -414,8 +414,10 @@ if hash sphinx-build > /dev/null 2>&1 && python -m 
sphinx.writers.manpage > /dev
 have_sphinx=1
 have_rst2man=0
 RST2MAN=
+have_rst2html=0
+RST2HTML=
 else
-printf "No (falling back to rst2man).\n"
+printf "No (falling back to Docutils).\n"
 have_sphinx=0

 printf "Checking if rst2man is available... "
@@ -431,6 +433,20 @@ else
have_rst2man=0
printf "No (so will not install man pages).\n"
 fi
+
+printf "Checking if rst2html is available... "
+for RST2HTML in rst2html rst2html.py; do
+   if "${RST2HTML}" -V > /dev/null 2>&1; then
+   have_rst2html=1
+   printf "Yes (${RST2HTML}).\n"
+   break
+   fi
+   RST2HTML=
+done
+if [ -z "${RST2HTML}" ]; then
+   have_rst2html=0
+   printf "No (so can not build HTML docs).\n"
+fi
 fi

 libdir_in_ldconfig=0
@@ -830,6 +846,13 @@ HAVE_RST2MAN=${have_rst2man}
 # an empty string if no such program is available.
 RST2MAN=${RST2MAN}

+# Whether there's a rst2html binary available for building documentation
+HAVE_RST2HTML=${have_rst2html}
+
+# The path to the rst2html program for building documentation.  Set to
+# an empty string if no such program is available.
+RST2HTML=${RST2HTML}
+
 # The directory to which desktop files should be installed
 desktop_dir = \$(prefix)/share/applications

diff --git a/doc/INSTALL b/doc/INSTALL
index 8352f7b..bd00dcf 100644
--- a/doc/INSTALL
+++ b/doc/INSTALL
@@ -20,8 +20,8 @@ Building with Docutils

 If you don't have Sphinx installed, you can use Docutils_ with the
 same targets outlined above for Sphinx.  The Docutils converters
-(rst2man, rst2html, etc.) are used instead of Sphinx.  Only the man
-targets are currently supported.
+(rst2man, rst2html, etc.) are used instead of Sphinx.  Only the HTML
+and man targets are currently supported.

 Configuring the builder
 ---
@@ -36,5 +36,8 @@ your ``PATH`` and sets an appropriate ``RST2MAN`` in
 ``Makefile.config``.  If neither ``HAVE_SPHINX`` nor ``HAVE_RST2MAN``
 is 1, attempting to build the documentation will fail with an error.

+A parallel set of variables (``HAVE_RST2HTML`` and ``RST2HTML``) is
+used when building HTML docs with Docutills.
+
 .. _Sphinx: http://sphinx-doc.org/
 .. _Docutils: http://docutils.sourceforge.net/
diff --git a/doc/Makefile.local b/doc/Makefile.local
index 9b99c19..e0a6963 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -23,6 +23,8 @@ ALLSPHINXOPTS   := -d $(DOCBUILDDIR)/doctrees $(SPHINXOPTS) 
$(srcdir)/$(dir)
 build-html:
 ifeq ($(HAVE_SPHINX),1)
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(DOCBUILDDIR)/html
+else ifeq ($(HAVE_RST2HTML),1)
+   $(rstman2any) --converter "$(RST2HTML)" --input $(srcdir)/doc --output 
$(DOCBUILDDIR)/html --extension html
 else
@echo "fatal: no Sphinx, cannot build HTML docs"
@false
-- 
1.9.1.353.gc66d89d

[PATCH v3 4/5] doc: Consolidate Makefile targets around {build|install}-{format}

[W. Trevor King]

The rst2man target was removed in 9d9a700 (doc: build man pages at
build time; introduce HAVE_SPHINX, HAVE_RST2MAN, 2014-03-13), but a
reference in the install docs slipped through.  While I was removing
that reference, I also:

* Converted doc/INSTALL to reStructuredText, so I can link to Sphinx
  and Docutils directly.  Not everyone has access to Debian's
  python-docutils, so it's better to be genric here.
* Converted from an unordered list to paragraphs, because I think it
  flows better.
* Dropped the rst2man no-automatic-install caveat.  I don't think this
  applies to the current code, although I haven't tried to track down
  a commit that adds the automatic-install support.  Anyhow,

$ make HAVE_SPHINX=0 HAVE_RST2MAN=1 RST2MAN=rst2man.py DESTDIR=/tmp/ 
install-man

  works for me.
* Restructured the Makefile to use build- and install- prefixes
  consistently, regardless of Sphinx/rst2x backend.
* Instead of disabling the build-man and install-man targets if
  HAVE_SPHINX and HAVE_RST2MAN are both empty, just pull build-man and
  install-man out of the default build tree.  That way:

$ make HAVE_SPHINX=0 HAVE_RST2MAN=0 build-man

  will fail like it should, instead of successfully doing nothing.  If
  packagers *do* want to force building the docs, despite any config
  settings, you can override the BUILD_MAN and INSTALL_MAN variables:

$ make BUILD_MAN=build-man all
---
 Makefile.local | 14 --
 doc/INSTALL| 44 ++--
 doc/Makefile.local | 42 --
 3 files changed, 66 insertions(+), 34 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index fa07d81..83984fd 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -43,6 +43,15 @@ GPG_FILE=$(SHA1_FILE).asc

 PV_FILE=bindings/python/notmuch/version.py

+# Disable docs if we don't have the builders
+ifeq ($(HAVE_SPHINX)$(HAVE_RST2MAN),00)
+BUILD_MAN =
+INSTALL_MAN =
+else
+BUILD_MAN = build-man
+INSTALL_MAN = install-man
+endif
+
 # Smash together user's values with our extra values
 FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(CPPFLAGS) $(CFLAGS) 
$(WARN_CFLAGS) $(extra_cflags) $(CONFIGURE_CFLAGS)
 FINAL_CXXFLAGS = $(CPPFLAGS) $(CXXFLAGS) $(WARN_CXXFLAGS) $(extra_cflags) 
$(extra_cxxflags) $(CONFIGURE_CXXFLAGS)
@@ -58,7 +67,7 @@ endif
 FINAL_LIBNOTMUCH_LDFLAGS = $(LDFLAGS) $(AS_NEEDED_LDFLAGS) $(CONFIGURE_LDFLAGS)

 .PHONY: all
-all: notmuch notmuch-shared build-man
+all: notmuch notmuch-shared $(BUILD_MAN)
 ifeq ($(MAKECMDGOALS),)
 ifeq ($(shell cat .first-build-message 2>/dev/null),)
@NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all
@@ -299,7 +308,8 @@ notmuch-shared: $(notmuch_client_modules) lib/$(LINKER_NAME)
$(call quiet,$(FINAL_NOTMUCH_LINKER) $(CFLAGS)) 
$(notmuch_client_modules) $(FINAL_NOTMUCH_LDFLAGS) -o $@

 .PHONY: install
-install: all install-man
+
+install: all $(INSTALL_MAN)
mkdir -p "$(DESTDIR)$(prefix)/bin/"
install notmuch-shared "$(DESTDIR)$(prefix)/bin/notmuch"
 ifeq ($(MAKECMDGOALS), install)
diff --git a/doc/INSTALL b/doc/INSTALL
index e37c2b9..8352f7b 100644
--- a/doc/INSTALL
+++ b/doc/INSTALL
@@ -1,24 +1,40 @@
+.. -*- rst -*-
+
 This file contains some more detailed information about building and
 installing the documentation.

-Building with sphinx.
--
+Building with Sphinx
+

-- You need sphinx at least version 1.0.
+With Sphinx_ version 1.0 or greater, you can build HTML, man, and info
+versions of the documentation with::

-- You can build build and install man pages with 'make install-man'
+  make build-{html|info|man}

-- You can build man, info, html, and pdf versions of the docs
-  (currently only the man pages) with
+You can build and install the documentation with::

- 'make install-{man|info|html|pdf}'
+  make install-{info|man}

-Building the man pages
+Building with Docutils
 --

-- You can build the man pages with rst2man (from python-docutils) with
-  'make rst2man'.
-
-- Currently there is no support to automagically install the resulting
-  nroff files, but it should work to modify the target install-man
-  in doc/Makefile.local.
+If you don't have Sphinx installed, you can use Docutils_ with the
+same targets outlined above for Sphinx.  The Docutils converters
+(rst2man, rst2html, etc.) are used instead of Sphinx.  Only the man
+targets are currently supported.
+
+Configuring the builder
+---
+
+The builder (Sphinx or rst2man) used for compilation is chosen based
+on variables set in ``Makefile.config`` by the ``configure`` script.
+If ``HAVE_SPHINX`` is 1, ``SPHINXBUILD`` (``sphinx-build`` by default)
+is used to build the documentation.  If ``HAVE_SPHINX`` is not 1, and
+``HAVE_RST2MAN`` is 1, ``RST2MAN`` is used to build the documentation.
+The ``configure`` script autodetects ``rst2man`` or ``rst2man.py`` in
+your ``PATH`` and sets an appropriate ``RST2MAN`` in

[PATCH v3 3/5] doc/rst-man2any.py: Adjust to handle any output format, not just man pages

[W. Trevor King]

For example, with these changes we can build HTML output using:

  $ rst-man2any.py -c rst2html -i ${SRCDIR} -o ${OUTDIR} -e html

The extension adjustment ensures that the output filenames from the
above command match what we currently generate with sphinx-html.

Adding argparse handling at the top of the script tipped this over
into "complicated enough to refactor" for me, so I've shifted the
single-file-conversion logic into a new convert() function and
streamlined things a bit.
---
 doc/Makefile.local |   4 +-
 doc/prerst2man.py  |  66 -
 doc/rst-man2any.py | 105 +
 3 files changed, 107 insertions(+), 68 deletions(-)
 delete mode 100644 doc/prerst2man.py
 create mode 100755 doc/rst-man2any.py

diff --git a/doc/Makefile.local b/doc/Makefile.local
index d96cdd5..045f823 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -7,7 +7,7 @@ SPHINXOPTS:= -q
 SPHINXBUILD   = sphinx-build
 DOCBUILDDIR  := $(dir)/_build

-prerst2man := python $(srcdir)/$(dir)/prerst2man.py
+rstman2any := python $(srcdir)/$(dir)/rst-man2any.py
 mkdocdeps := python $(srcdir)/$(dir)/mkdocdeps.py

 # Internal variables.
@@ -49,7 +49,7 @@ ifeq ($(HAVE_SPHINX),1)
mv $(DOCBUILDDIR)/man/*.$${section} 
$(DOCBUILDDIR)/man/man$${section}; \
done
 else ifeq ($(HAVE_RST2MAN),1)
-   $(prerst2man) "$(RST2MAN)" $(srcdir)/doc $(DOCBUILDDIR)/man
+   $(rstman2any) --converter "$(RST2MAN)" --input $(srcdir)/doc --output 
$(DOCBUILDDIR)/man
 else
@echo "Fatal: build dependency fail."
@false
diff --git a/doc/prerst2man.py b/doc/prerst2man.py
deleted file mode 100644
index 7d78e9b..000
--- a/doc/prerst2man.py
+++ /dev/null
@@ -1,66 +0,0 @@
-import sys
-from datetime import date
-from os.path import dirname, isdir
-from os import makedirs, system
-import re
-
-rst2man = sys.argv[1]
-sourcedir = sys.argv[2]
-outdir = sys.argv[3]
-
-sys.path.insert(0, sourcedir)
-import conf
-
-
-if not isdir(outdir):
-makedirs(outdir, 0o755)
-
-
-def header(file, startdocname, command, description, authors, section):
-file.write("""
-{0:s}
-{1:s}
-{2:s}
-
-:Date:   {3:s}
-:Version: {4:s}
-:Manual section: {5:d}
-:Manual group: {6:s}
-
-""".format(
-'-' * len(description),
-description,
-'-' * len(description),
-date.today().isoformat(), conf.release, section, conf.project))
-
-blankre = re.compile("^\s*$")
-for page in conf.man_pages:
-outdirname = outdir + '/' + dirname(page[0])
-if not isdir(outdirname):
-makedirs(outdirname, 0o755)
-filename = outdir + '/' + page[0] + '.rst'
-outfile = open(filename, 'w')
-infile = open(sourcedir + '/' + page[0] + '.rst', 'r')
-
-# this is a crude hack. We look for the first blank line, and
-# insert the rst2man header there.
-#
-# XXX consider really parsing input
-
-count = 0
-lines = infile.readlines()
-for line in lines:
-outfile.write(line)
-if (blankre.match(line)):
-break
-count = count + 1
-
-del lines[0:count + 1]
-
-header(outfile, *page)
-
-outfile.write("".join(lines))
-outfile.close()
-
-system('set -x; {0} {1} {2}/{3}.{4}'
-   .format(rst2man, filename, outdir, page[0], page[4]))
diff --git a/doc/rst-man2any.py b/doc/rst-man2any.py
new file mode 100755
index 000..52a4f9e
--- /dev/null
+++ b/doc/rst-man2any.py
@@ -0,0 +1,105 @@
+#!/usr/bin/python
+
+import argparse
+import datetime
+import os
+import sys
+
+
+def header(stream, description, authors, section, group, version):
+"Write a man-page's ReST headers to 'stream'."
+if len(authors) != 1:
+raise NotImplementedError(
+'cannot handle {count} authors ({authors}'.format(
+count=len(authors), authors=authors))
+stream.write('\n'.join([
+'{description_rule}',
+'{description}',
+'{description_rule}',
+'',
+':Author: {author}',
+':Date: {date}',
+':Version: {version}',
+':Manual section: {section:d}',
+':Manual group: {group}',
+'',
+'']).format(
+description=description,
+description_rule='-' * len(description),
+author=authors[0],
+date=datetime.date.today().isoformat(),
+version=version,
+section=section,
+group=group))
+
+
+def convert(startdocname, description, authors, section,
+group, version, source_dir, output_dir, command, extension=None):
+"""Convert the ReST source at path to the target format in 'output_dir'.
+
+The initial arguments (startdocname through section) follow
+Sphinx's man_pages option [1].
+
+[1]: http://sphinx-doc.org/config.html#confval-man_pages
+"""
+source_path = os.path.join(source_dir, startdocname) + '.rst'
+output_base = os.path.join(output_dir, startdocname)
+temp_path = output_base + '.rst'
+if 

[PATCH v3 2/5] doc/prerst2man.py: Convert execfile to import

[W. Trevor King]

excefile is gone in Python 3 [1].  Instead of exec-ing the
configuration, it's easier to insert the source directory in Python's
path [2], and just import the configuration.  With this change,
prerst2man.py is compatible with both Python 2 and 3.

[1]: https://docs.python.org/3.0/whatsnew/3.0.html#builtins
[2]: https://docs.python.org/3/library/sys.html#sys.path
---
 doc/prerst2man.py | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/doc/prerst2man.py b/doc/prerst2man.py
index 81ce817..7d78e9b 100644
--- a/doc/prerst2man.py
+++ b/doc/prerst2man.py
@@ -1,18 +1,20 @@
-from sys import argv
+import sys
 from datetime import date
 from os.path import dirname, isdir
 from os import makedirs, system
 import re

-rst2man = argv[1]
-sourcedir = argv[2]
-outdir = argv[3]
+rst2man = sys.argv[1]
+sourcedir = sys.argv[2]
+outdir = sys.argv[3]
+
+sys.path.insert(0, sourcedir)
+import conf
+

 if not isdir(outdir):
 makedirs(outdir, 0o755)

-execfile(sourcedir + "/conf.py")
-

 def header(file, startdocname, command, description, authors, section):
 file.write("""
@@ -29,10 +31,10 @@ def header(file, startdocname, command, description, 
authors, section):
 '-' * len(description),
 description,
 '-' * len(description),
-date.today().isoformat(), release, section, project))
+date.today().isoformat(), conf.release, section, conf.project))

 blankre = re.compile("^\s*$")
-for page in man_pages:
+for page in conf.man_pages:
 outdirname = outdir + '/' + dirname(page[0])
 if not isdir(outdirname):
 makedirs(outdirname, 0o755)
-- 
1.9.1.353.gc66d89d

[PATCH v3 1/5] doc: Allow rst2man.py as an alternative to rst2man

[W. Trevor King]

Gentoo's dev-python/docutils-0.10 installs Docutils scripts with a
*.py extension, so I have /usr/bin/rst2man.py and no rst2man script.
This patch supports users with both types of systems by checking for
rst2man, falling back on rst2man.py, and giving up only if neither is
found.  Users can also set the new RST2MAN path variable explicitly
when they call Make:

  make RST2MAN=/my/custom/rst_to_man_converter build-man

We pass the configured RST2MAN path through to prerst2man.py to use in
its system call.

We can use a non-empty RST2MAN to check for the availability of an
rst2man program, so there's no need for a separate HAVE_RST2MAN.
However, we keep the existing HAVE_RST2MAN for consistency with
HAVE_SPHINX.
---
 configure  | 20 +++-
 doc/Makefile.local |  2 +-
 doc/prerst2man.py  |  9 +
 3 files changed, 21 insertions(+), 10 deletions(-)

diff --git a/configure b/configure
index 9bde2eb..20a2d5f 100755
--- a/configure
+++ b/configure
@@ -413,17 +413,23 @@ if hash sphinx-build > /dev/null 2>&1 && python -m 
sphinx.writers.manpage > /dev
 printf "Yes.\n"
 have_sphinx=1
 have_rst2man=0
+RST2MAN=
 else
 printf "No (falling back to rst2man).\n"
 have_sphinx=0

 printf "Checking if rst2man is available... "
-if rst2man -V > /dev/null 2>&1; then
-   printf "Yes.\n"
-   have_rst2man=1
-else
-   printf "No (so will not install man pages).\n"
+for RST2MAN in rst2man rst2man.py; do
+   if "${RST2MAN}" -V > /dev/null 2>&1; then
+   have_rst2man=1
+   printf "Yes (${RST2MAN}).\n"
+   break
+   fi
+   RST2MAN=
+done
+if [ -z "${RST2MAN}" ]; then
have_rst2man=0
+   printf "No (so will not install man pages).\n"
 fi
 fi

@@ -820,6 +826,10 @@ HAVE_SPHINX=${have_sphinx}
 # Whether there's a rst2man binary available for building documentation
 HAVE_RST2MAN=${have_rst2man}

+# The path to the rst2man program for building documentation.  Set to
+# an empty string if no such program is available.
+RST2MAN=${RST2MAN}
+
 # The directory to which desktop files should be installed
 desktop_dir = \$(prefix)/share/applications

diff --git a/doc/Makefile.local b/doc/Makefile.local
index bbd4610..d96cdd5 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -49,7 +49,7 @@ ifeq ($(HAVE_SPHINX),1)
mv $(DOCBUILDDIR)/man/*.$${section} 
$(DOCBUILDDIR)/man/man$${section}; \
done
 else ifeq ($(HAVE_RST2MAN),1)
-   $(prerst2man) $(srcdir)/doc $(DOCBUILDDIR)/man
+   $(prerst2man) "$(RST2MAN)" $(srcdir)/doc $(DOCBUILDDIR)/man
 else
@echo "Fatal: build dependency fail."
@false
diff --git a/doc/prerst2man.py b/doc/prerst2man.py
index 437dea9..81ce817 100644
--- a/doc/prerst2man.py
+++ b/doc/prerst2man.py
@@ -4,8 +4,9 @@ from os.path import dirname, isdir
 from os import makedirs, system
 import re

-sourcedir = argv[1]
-outdir = argv[2]
+rst2man = argv[1]
+sourcedir = argv[2]
+outdir = argv[3]

 if not isdir(outdir):
 makedirs(outdir, 0o755)
@@ -59,5 +60,5 @@ for page in man_pages:
 outfile.write("".join(lines))
 outfile.close()

-system('set -x; rst2man {0} {1}/{2}.{3}'
-   .format(filename, outdir, page[0], page[4]))
+system('set -x; {0} {1} {2}/{3}.{4}'
+   .format(rst2man, filename, outdir, page[0], page[4]))
-- 
1.9.1.353.gc66d89d

[PATCH v3 0/5] rst2man.py support and doc-build cleanups

[W. Trevor King]

Changes since v2 [1]:

* In patches 1/5 and 5/5, use for loops to check for rst2man[.py] and
  rst2html[.py].
* In patches 1/5 and 5/5, store the command names, not the full paths
  (command -v ?) [2].
* In patch 3/5, I've added argparse handling to the newly-renamed
  rst-man2any.py.  Tomi suggested modeling the name and UI on texi2any
  [3], but texi2any is for converting a single file, while
  rst-man2any.py is for converting a whole tree (and it takes an
  arbitrary ReST-to-$x converter to do the real work).  I've basically
  just cleaned things up, but if folks aren't satisfied I'm going to
  need more concrete suggestions ;).

Cheers,
Trevor

[1]: id:cover.1399740604.git.wking at tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18291
[2]: id:m2ion3dn0r.fsf at guru.guru-group.fi
 http://article.gmane.org/gmane.mail.notmuch.general/18652
[3]: id:m2lhrzj8kb.fsf at guru.guru-group.fi
 http://article.gmane.org/gmane.mail.notmuch.general/18653

W. Trevor King (5):
  doc: Allow rst2man.py as an alternative to rst2man
  doc/prerst2man.py: Convert execfile to import
  doc/rst-man2any.py: Adjust to handle any output format, not just man
pages
  doc: Consolidate Makefile targets around {build|install}-{format}
  doc: Add rst2html support for building HTML docs

 Makefile.local |  14 ++-
 configure  |  45 ---
 doc/INSTALL|  47 +---
 doc/Makefile.local |  48 ++--
 doc/prerst2man.py  |  63 
 doc/rst-man2any.py | 105 +
 6 files changed, 217 insertions(+), 105 deletions(-)
 delete mode 100644 doc/prerst2man.py
 create mode 100755 doc/rst-man2any.py

-- 
1.9.1.353.gc66d89d

[PATCH v3 4/5] doc: Consolidate Makefile targets around {build|install}-{format}

[W. Trevor King]

The rst2man target was removed in 9d9a700 (doc: build man pages at
build time; introduce HAVE_SPHINX, HAVE_RST2MAN, 2014-03-13), but a
reference in the install docs slipped through.  While I was removing
that reference, I also:

* Converted doc/INSTALL to reStructuredText, so I can link to Sphinx
  and Docutils directly.  Not everyone has access to Debian's
  python-docutils, so it's better to be genric here.
* Converted from an unordered list to paragraphs, because I think it
  flows better.
* Dropped the rst2man no-automatic-install caveat.  I don't think this
  applies to the current code, although I haven't tried to track down
  a commit that adds the automatic-install support.  Anyhow,

$ make HAVE_SPHINX=0 HAVE_RST2MAN=1 RST2MAN=rst2man.py DESTDIR=/tmp/ 
install-man

  works for me.
* Restructured the Makefile to use build- and install- prefixes
  consistently, regardless of Sphinx/rst2x backend.
* Instead of disabling the build-man and install-man targets if
  HAVE_SPHINX and HAVE_RST2MAN are both empty, just pull build-man and
  install-man out of the default build tree.  That way:

$ make HAVE_SPHINX=0 HAVE_RST2MAN=0 build-man

  will fail like it should, instead of successfully doing nothing.  If
  packagers *do* want to force building the docs, despite any config
  settings, you can override the BUILD_MAN and INSTALL_MAN variables:

$ make BUILD_MAN=build-man all
---
 Makefile.local | 14 --
 doc/INSTALL| 44 ++--
 doc/Makefile.local | 42 --
 3 files changed, 66 insertions(+), 34 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index fa07d81..83984fd 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -43,6 +43,15 @@ GPG_FILE=$(SHA1_FILE).asc
 
 PV_FILE=bindings/python/notmuch/version.py
 
+# Disable docs if we don't have the builders
+ifeq ($(HAVE_SPHINX)$(HAVE_RST2MAN),00)
+BUILD_MAN =
+INSTALL_MAN =
+else
+BUILD_MAN = build-man
+INSTALL_MAN = install-man
+endif
+
 # Smash together user's values with our extra values
 FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(CPPFLAGS) $(CFLAGS) 
$(WARN_CFLAGS) $(extra_cflags) $(CONFIGURE_CFLAGS)
 FINAL_CXXFLAGS = $(CPPFLAGS) $(CXXFLAGS) $(WARN_CXXFLAGS) $(extra_cflags) 
$(extra_cxxflags) $(CONFIGURE_CXXFLAGS)
@@ -58,7 +67,7 @@ endif
 FINAL_LIBNOTMUCH_LDFLAGS = $(LDFLAGS) $(AS_NEEDED_LDFLAGS) $(CONFIGURE_LDFLAGS)
 
 .PHONY: all
-all: notmuch notmuch-shared build-man
+all: notmuch notmuch-shared $(BUILD_MAN)
 ifeq ($(MAKECMDGOALS),)
 ifeq ($(shell cat .first-build-message 2/dev/null),)
@NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all
@@ -299,7 +308,8 @@ notmuch-shared: $(notmuch_client_modules) lib/$(LINKER_NAME)
$(call quiet,$(FINAL_NOTMUCH_LINKER) $(CFLAGS)) 
$(notmuch_client_modules) $(FINAL_NOTMUCH_LDFLAGS) -o $@
 
 .PHONY: install
-install: all install-man
+
+install: all $(INSTALL_MAN)
mkdir -p $(DESTDIR)$(prefix)/bin/
install notmuch-shared $(DESTDIR)$(prefix)/bin/notmuch
 ifeq ($(MAKECMDGOALS), install)
diff --git a/doc/INSTALL b/doc/INSTALL
index e37c2b9..8352f7b 100644
--- a/doc/INSTALL
+++ b/doc/INSTALL
@@ -1,24 +1,40 @@
+.. -*- rst -*-
+
 This file contains some more detailed information about building and
 installing the documentation.
 
-Building with sphinx.
--
+Building with Sphinx
+
 
-- You need sphinx at least version 1.0.
+With Sphinx_ version 1.0 or greater, you can build HTML, man, and info
+versions of the documentation with::
 
-- You can build build and install man pages with 'make install-man'
+  make build-{html|info|man}
 
-- You can build man, info, html, and pdf versions of the docs
-  (currently only the man pages) with
+You can build and install the documentation with::
 
- 'make install-{man|info|html|pdf}'
+  make install-{info|man}
 
-Building the man pages
+Building with Docutils
 --
 
-- You can build the man pages with rst2man (from python-docutils) with
-  'make rst2man'.
-
-- Currently there is no support to automagically install the resulting
-  nroff files, but it should work to modify the target install-man
-  in doc/Makefile.local.
+If you don't have Sphinx installed, you can use Docutils_ with the
+same targets outlined above for Sphinx.  The Docutils converters
+(rst2man, rst2html, etc.) are used instead of Sphinx.  Only the man
+targets are currently supported.
+
+Configuring the builder
+---
+
+The builder (Sphinx or rst2man) used for compilation is chosen based
+on variables set in ``Makefile.config`` by the ``configure`` script.
+If ``HAVE_SPHINX`` is 1, ``SPHINXBUILD`` (``sphinx-build`` by default)
+is used to build the documentation.  If ``HAVE_SPHINX`` is not 1, and
+``HAVE_RST2MAN`` is 1, ``RST2MAN`` is used to build the documentation.
+The ``configure`` script autodetects ``rst2man`` or ``rst2man.py`` in
+your ``PATH`` and sets an appropriate ``RST2MAN`` in

[PATCH v3 0/5] rst2man.py support and doc-build cleanups

[W. Trevor King]

Changes since v2 [1]:

* In patches 1/5 and 5/5, use for loops to check for rst2man[.py] and
  rst2html[.py].
* In patches 1/5 and 5/5, store the command names, not the full paths
  (command -v …) [2].
* In patch 3/5, I've added argparse handling to the newly-renamed
  rst-man2any.py.  Tomi suggested modeling the name and UI on texi2any
  [3], but texi2any is for converting a single file, while
  rst-man2any.py is for converting a whole tree (and it takes an
  arbitrary ReST-to-$x converter to do the real work).  I've basically
  just cleaned things up, but if folks aren't satisfied I'm going to
  need more concrete suggestions ;).

Cheers,
Trevor

[1]: id:cover.1399740604.git.wk...@tremily.us
 http://thread.gmane.org/gmane.mail.notmuch.general/18291
[2]: id:m2ion3dn0r@guru.guru-group.fi
 http://article.gmane.org/gmane.mail.notmuch.general/18652
[3]: id:m2lhrzj8kb@guru.guru-group.fi
 http://article.gmane.org/gmane.mail.notmuch.general/18653

W. Trevor King (5):
  doc: Allow rst2man.py as an alternative to rst2man
  doc/prerst2man.py: Convert execfile to import
  doc/rst-man2any.py: Adjust to handle any output format, not just man
pages
  doc: Consolidate Makefile targets around {build|install}-{format}
  doc: Add rst2html support for building HTML docs

 Makefile.local |  14 ++-
 configure  |  45 ---
 doc/INSTALL|  47 +---
 doc/Makefile.local |  48 ++--
 doc/prerst2man.py  |  63 
 doc/rst-man2any.py | 105 +
 6 files changed, 217 insertions(+), 105 deletions(-)
 delete mode 100644 doc/prerst2man.py
 create mode 100755 doc/rst-man2any.py

-- 
1.9.1.353.gc66d89d

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch