Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 03:55:10PM -0400, Konstantin Ryabitsev wrote:
> On Tue, Sep 14, 2021 at 07:35:28PM +, Eric Wong wrote:
> > > I found an interesting problem using lei with imaps:// folders. I'm trying
> > > things out with migadu, and the folder paths use '/' separators, so a full
> > > IMAPS folder path for a folder "lore/mentions" is
> > > imaps://imap.migadu.com/lore/mentions. However, if I configure lei-q to 
> > > use
> > > that remote path, everything actually ends up in the folder
> > > imap.migadu.com/lore (not the "mentions" subfolder).
> > 
> > Oops, I think the patch below should fix it.
> 
> Yep, that worked. Thanks!

I think I found a couple of other bugs while testing this with migadu. E.g.:

$ export MFOLDER=imaps://imap.migadu.com/lore/b4
$ lei q -o $MFOLDER -I https://lore.kernel.org/all/ '(s:b4 OR nq:b4 OR 
dfn:b4) AND rt:1.week.ago..'
# /usr/bin/curl -Sf -s -d '' 
https://lore.kernel.org/all/?q=(s%3Ab4+OR+nq%3Ab4+OR+dfn%3Ab4)+AND+rt%3A1631066349..=m
# /home/user/.local/share/lei/store 54/54

So far so good, but then:

2021-09-14T20:59:12Z 20428 20428 die: BUG: 
imaps://imap.migadu.com/lore/b4;UIDVALIDITY=1621977334 has no UIDVALIDITY at 
/usr/local/share/perl/5.32.1/PublicInbox/LeiStore.pm line 313.
 (from nowait set_sync_info)
# https://lore.kernel.org/all/ 19/?
# https://lore.kernel.org/all/ 25/?
# https://lore.kernel.org/all/ 51/?
# https://lore.kernel.org/all/ 54/54
# 54 written to imaps://imap.migadu.com/lore/b4 (108 matches)

However, it doesn't show up in ls-search:

$ lei ls-search
/home/user/work/temp/lei/lockdown
/home/user/work/temp/lei/mentions

That would appear to be due to them being saved in the lore/ subdir:

$ find .local/share/lei/saved-searches/ -type d
.local/share/lei/saved-searches/

.local/share/lei/saved-searches/lockdown-1804cfad691a409f55598a8528566d5f1539b2632e1db7e206cb147396582631

.local/share/lei/saved-searches/mentions-f467d0a01dfdc3e42523b5d0d090773269e199a6a109b0713dc48142f0e30526
.local/share/lei/saved-searches/lore

.local/share/lei/saved-searches/lore/mentions-e9ca065affe84b4e4637620c72b64b09970a02b83171ba75c86afff95489d392

.local/share/lei/saved-searches/lore/b4-4811ca1722c2c2817e8cdc6a8d390f63a3b723c3c991f0267425d380aa1c8add

Cheers,
-K



Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 08:12:16PM +, Eric Wong wrote:
> Ah, I forgot to update the docs again :x
> 
> My main concern with .netrc was actually inadvertantly sending
> FTP auth info to an IMAP server just because they share the same
> host.

No big deal -- folks can always just use the "store" credential helper to
pretty much the same effect.

> Not sure if plaintext is a real problem on encrypted block
> devices/filesystems.  Ordinary users can't mlock(2) to prevent
> in-memory passwords from hitting swap (thus I always use
> encrypted swap).

Right, plus most of them probably have their .gitconfig with
sendemail.smtppass configured anyway. :)

-K



Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Eric Wong
Konstantin Ryabitsev  wrote:
> On Tue, Sep 14, 2021 at 07:35:28PM +, Eric Wong wrote:
> > Oops, I think the patch below should fix it.
> 
> Yep, that worked. Thanks!

Cool.

> Good to know, thanks. Quick follow-up -- documentation says that .netrc should
> work, but I've found that even though I have the following entries in
> ~/.netrc, I still get prompted for credentials:
> 
> machine imap.migadu.com
>   login konstantin.ryabit...@linux.dev
>   password [...]
> 
> The credential helper works after the initial "lei up" but I'm curious why
> .netrc isn't happy. Not a huge deal, seeing as that requires storing passwords
> in plaintext.

Ah, I forgot to update the docs again :x

My main concern with .netrc was actually inadvertantly sending
FTP auth info to an IMAP server just because they share the same
host.

Not sure if plaintext is a real problem on encrypted block
devices/filesystems.  Ordinary users can't mlock(2) to prevent
in-memory passwords from hitting swap (thus I always use
encrypted swap).

---8<-
Subject: [PATCH] doc: update authentication notes for lei

~/.netrc isn't used by default any more, and I'm not sure it's
worthwhile to document the --netrc switch since it's rare for
non-FTP clients to support.

Followup-to: 9d11ed460ce113dd ("lei: do not read ~/.netrc by default")
Reported-by: Konstantin Ryabitsev 
---
 Documentation/lei-convert.pod| 2 +-
 Documentation/lei-import.pod | 2 +-
 Documentation/lei-q.pod  | 2 +-
 Documentation/public-inbox-watch.pod | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Documentation/lei-convert.pod b/Documentation/lei-convert.pod
index 7f372327..750ba54f 100644
--- a/Documentation/lei-convert.pod
+++ b/Documentation/lei-convert.pod
@@ -13,7 +13,7 @@ lei convert [OPTIONS] (--stdin|-)
 Convert messages to another format.  C is a source of
 messages: a directory (Maildir), a file, or a URL (C,
 C, C, or C).  URLs requiring
-authentication must use L and/or L to
+authentication use L to
 fill in the username and password.
 
 For a regular file, the location must have a CformatE:>
diff --git a/Documentation/lei-import.pod b/Documentation/lei-import.pod
index c29a085b..9322dfa8 100644
--- a/Documentation/lei-import.pod
+++ b/Documentation/lei-import.pod
@@ -13,7 +13,7 @@ lei import [OPTIONS] (--stdin|-)
 Import messages into the local storage of L.  C is a
 source of messages: a directory (Maildir), a file, or a URL
 (C, C, C, or C).  URLs requiring
-authentication must use L and/or L to
+authentication use L to
 fill in the username and password.
 
 For a regular file, the location must have a CformatE:>
diff --git a/Documentation/lei-q.pod b/Documentation/lei-q.pod
index 1d9e66cd..2823ced8 100644
--- a/Documentation/lei-q.pod
+++ b/Documentation/lei-q.pod
@@ -43,7 +43,7 @@ Destination for results (e.g., C,
 C, or
 C).  The prefix may be a supported protocol:
 C or C.  URLs requiring
-authentication must use L and/or L to
+authentication use L to
 fill in the username and password.
 
 A prefix can specify the format of the output: C,
diff --git a/Documentation/public-inbox-watch.pod 
b/Documentation/public-inbox-watch.pod
index 5fa966be..e8f97c80 100644
--- a/Documentation/public-inbox-watch.pod
+++ b/Documentation/public-inbox-watch.pod
@@ -82,7 +82,7 @@ C and C URLs:
 
 This may be specified multiple times to combine several mailboxes
 into a single public-inbox.  URLs requiring authentication
-will require L and/or L to fill
+will require L and/or L (preferred) to fill
 in the username and password.
 
 Default: none
--
unsubscribe: one-click, see List-Unsubscribe header
archive: https://public-inbox.org/meta/



Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 07:35:28PM +, Eric Wong wrote:
> > I found an interesting problem using lei with imaps:// folders. I'm trying
> > things out with migadu, and the folder paths use '/' separators, so a full
> > IMAPS folder path for a folder "lore/mentions" is
> > imaps://imap.migadu.com/lore/mentions. However, if I configure lei-q to use
> > that remote path, everything actually ends up in the folder
> > imap.migadu.com/lore (not the "mentions" subfolder).
> 
> Oops, I think the patch below should fix it.

Yep, that worked. Thanks!

> Btw, if you encounter more IMAP problems, I've found adding
> "-c imap.debug -c imap.compress=0" to the command-line useful
> (Mail::IMAPClient dumps the raw compressed traffic, so I need to
> disable compression).

Good to know, thanks. Quick follow-up -- documentation says that .netrc should
work, but I've found that even though I have the following entries in
~/.netrc, I still get prompted for credentials:

machine imap.migadu.com
  login konstantin.ryabit...@linux.dev
  password [...]

The credential helper works after the initial "lei up" but I'm curious why
.netrc isn't happy. Not a huge deal, seeing as that requires storing passwords
in plaintext.

-K