Re: filtering stable patches in lore queries

2024-05-08 Thread Konstantin Ryabitsev
On Wed, May 08, 2024 at 11:33:14AM GMT, Eric Wong wrote: > > I'm whole-heartedly for this! This ties nicely to my b4 work where I'd > > like to be able to identify code-review trailers sent for a specific > > patch, even if that patch itself is not on lore. For example, this could > > be a

Re: filtering stable patches in lore queries

2024-04-29 Thread Konstantin Ryabitsev
On Sat, Apr 27, 2024 at 07:19:21AM GMT, Eric Wong wrote: > Correct, public-inbox currently won't index every header due to > cost, false positives, and otherwise lack of usefulness (general > gibberish from DKIM sigs, various UUIDs, etc). > > So it doesn't currently know about "X-stable:" > > I

Re: downloading t.mbox.gz messages are not sorted in expected order

2024-04-11 Thread Konstantin Ryabitsev
On Thu, Apr 11, 2024 at 03:32:43PM -0700, Jacob Keller wrote: > I sometimes download patch series off of public inbox hosted servers to > apply with git-am. Occasionally I have found that these do not apply > cleanly because the thread is not sorted in patch order. It's more than just the order

Re: sample robots.txt to reduce WWW load

2024-04-03 Thread Konstantin Ryabitsev
On Mon, Apr 01, 2024 at 01:21:45PM +, Eric Wong wrote: > Performance is still slow, and crawler traffic patterns tend to > do bad things with caches at all levels, so I've regretfully had > to experiment with robots.txt to mitigate performance problems. This has been the source of grief for

Re: [PATCH] lei: support reading MH for convert+import+index

2023-12-16 Thread Konstantin Ryabitsev
On Sat, Dec 16, 2023 at 01:09:32PM +, Eric Wong wrote: > The MH format is widely-supported and used by various MUAs such > as mutt and sylpheed, and a MH-like format is used by mlmmj for > archives, as well. Locking implementations for writes are > inconsistent, so this commit doesn't support

Re: extra search flags and params? (ispatch, replycount, ...)

2023-11-28 Thread Konstantin Ryabitsev
On Tue, Nov 28, 2023 at 06:20:03PM +, Eric Wong wrote: > > Ah. I think here is enough to just say "s:* AND NOT s:PATCH" without > > introducing additional xapian indexing parameters. Though, perhaps the web > > interface can also gain a "collapse threads" view? > > topics_new.html /

Re: extra search flags and params? (ispatch, replycount, ...)

2023-11-28 Thread Konstantin Ryabitsev
On Tue, Nov 28, 2023 at 05:35:09PM +, Eric Wong wrote: > > I understand the reasoning, but I'm not sure we should be trying too hard to > > make public-inbox a patch tracking platform. What makes lei great is ability > > to automatically find and retrieve entire threads -- I feel like we

Re: extra search flags and params? (ispatch, replycount, ...)

2023-11-28 Thread Konstantin Ryabitsev
On Tue, Nov 28, 2023 at 12:10:28AM +, Eric Wong wrote: > Would they be useful? > > It's not currently possible to quickly search for whether or not > a term (e.g. patchid:) is present in a Xapian document. Having > the ability to do so would make it easier to find non-patch messages, > or

Re: [PATCH] cindex: fix test when missing time(1) executable

2023-11-15 Thread Konstantin Ryabitsev
On Wed, Nov 15, 2023 at 05:55:49AM +, Eric Wong wrote: > Eric Wong wrote: > > +++ b/t/cindex.t > > @@ -210,7 +210,7 @@ EOM > > my $cmd = [ qw(-cindex -u --all --associate -d), "$tmp/ext", > > '-I', $basic->{inboxdir} ]; > > $cidx_out = $cidx_err = ''; > > -

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-15 Thread Konstantin Ryabitsev
On Wed, Nov 15, 2023 at 03:09:28AM +, Eric Wong wrote: > > t/imapd.t 2/? Bailout called. Further testing > > stopped: FETCH socket closed while reading data from server > > FAILED--Further testing stopped: FETCH socket closed while reading data > > from server > >

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
On Wed, Nov 15, 2023 at 01:06:42AM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > Looks like the last time I am able to successfully run "make test" is before > > this commit: > > > > b231d91f42d791becf7b6861e723833d71e73237 is the first

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 06:51:00PM -0500, Konstantin Ryabitsev wrote: > On Tue, Nov 14, 2023 at 11:46:20PM +, Eric Wong wrote: > > My, that's a lot of pipes... > > > > I should've told you to try this debug patch earlier, but this > > might help... (and our test

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 11:46:20PM +, Eric Wong wrote: > My, that's a lot of pipes... > > I should've told you to try this debug patch earlier, but this > might help... (and our test suite should really be able to > watch messages like this while capturing) Will try this shorty -- currently

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 10:46:57PM +, Eric Wong wrote: > > I can't do +E because that's not available to me under CentOS7 (I can't wait > > until we move on, but just when we think the yak is fully shaved, we find > > more > > clumps of thick fur we hadn't considered). Is the output of the

Re: t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 10:16:53PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > └─-cindex -u --al,4432 > > ├─cidx shard[0],4646 > > └─cidx shard[1],4647 > > > > Anything I can do to figure out why this is happe

t/cindex.t "associate w/o search" test hangs for me

2023-11-14 Thread Konstantin Ryabitsev
Eric: I'm trying to have tests pass on CentOS7 with the current master and I'm apparently not able to get past the "associate w/o search" test. When I run `prove -bvw t/cindex.t` I get to: ok 76 - xcpdb compact and then it just sits there. If I look at the process table, I can see that the

[PATCH] TestCommon: older strace does not have --version

2023-11-14 Thread Konstantin Ryabitsev
The tests will check for strace >= 4.16, but version 4.24 that I have does not accept --version, only -V. This works for both older and newer strace, so switch to using "strace -V" for the check. Signed-off-by: Konstantin Ryabitsev --- lib/PublicInbox/TestCommon.pm | 4 ++-- 1 fi

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 04:36:29PM +, Eric Wong wrote: > In any case, kernel.org folks should be able to import missing > messages from GNU.org idempotently into lore/qemu-devel without > having to resend: > > https://lists.gnu.org/archive/mbox/qemu-devel/2023-10 >

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 04:11:41PM +, Salil Mehta wrote: > It is not just me there other people (from other organizations CC'ed in this > mail) > I requested to check below set of links and they experienced the same behavior > i.e. except first link no other link opens (ends up in Not Found)

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-14 Thread Konstantin Ryabitsev
On Tue, Nov 14, 2023 at 11:03:38AM +, Salil Mehta wrote: > I have cross confirmed the behavior with other people across companies and > all of them are having issues in viewing above links. Surprising part is > these were present at the first instance when the review comments were floated > by

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-13 Thread Konstantin Ryabitsev
On Mon, Nov 13, 2023 at 07:39:21PM +, Salil Mehta wrote: > > When you go to "git.kernel.org" on your company network, what city do you > > have in the header ("dallas", "amsterdam", etc). > > I am seeing below in the header: > > Git repositories hosted at kernel.org (amsterdam) This is

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-13 Thread Konstantin Ryabitsev
On Mon, Nov 13, 2023 at 07:27:34PM +, Salil Mehta wrote: > I am able to open [Patch V6 1/9] from outside the company network (as you > can see the by [2] [3]) but strangely not from inside the company network. > It purges below error from company network. I am totally stumped. When you go to

Re: [RFC v2] www: add topics_(new|active).(html|atom) endpoints

2023-11-10 Thread Konstantin Ryabitsev
On Fri, Nov 10, 2023 at 03:09:59AM +, Eric Wong wrote: > > Yes, actually thinking about this some more, perhaps it makes sense to > > expose > > this as an RSS feed feature (maybe even exclusively as an RSS feed > > feature?). > > I assume Atom is OK? I don't know of any widely-used feed

Re: [RFC] www: add topics.html endpoint [was: Query to see all new "topics"]

2023-11-09 Thread Konstantin Ryabitsev
On Thu, Nov 09, 2023 at 02:45:08AM +, Eric Wong wrote: > This seems like a easy (but WWW-specific) way to get recent > topics as suggested by Konstantin. Perhaps an Atom endpoint > will also be useful. Yes, actually thinking about this some more, perhaps it makes sense to expose this as an

Query to see all new "topics"

2023-11-07 Thread Konstantin Ryabitsev
Hello: Following the discussion on the ksummit list [1], I wanted to give someone a query they could use to keep an eye on any new threads. Is there a xapian query that can be used to effectively say "return just top-level messages and exclude any follow-ups"? It's not quite as simple as "s:*

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-07 Thread Konstantin Ryabitsev
On Tue, Nov 07, 2023 at 09:57:55AM +, Salil Mehta wrote: > > On Mon, Nov 06, 2023 at 11:49:02AM -0500, Michael S. Tsirkin wrote: > > > > 2023-10-13 10:51 ` [PATCH V6 3/9] hw/acpi: Add ACPI CPU hotplug init > > > > stub Salil Mehta via > > > > [not found] > > > >

Re: [Question] review links are disappearing from the qemu-devel mailing-list

2023-11-06 Thread Konstantin Ryabitsev
On Mon, Nov 06, 2023 at 11:49:02AM -0500, Michael S. Tsirkin wrote: > > 2023-10-13 10:51 ` [PATCH V6 3/9] hw/acpi: Add ACPI CPU hotplug init stub > > Salil Mehta via > > [not found] ` > > <20231027150536.3c481...@imammedo.users.ipa.redhat.com>---> why is this? Unhelpfully, because the

pop3: does uuid need to be unique for all mailboxes?

2023-09-25 Thread Konstantin Ryabitsev
Hopefully an easy question: If I'm subscribing to two different mailboxes via pop3, does the uuid part need to be unique for each mailbox? For example, will the following cause any problems because the uuid is the same? -

Re: [PATCH] pop3: support initial_limit parameter in mailbox name

2023-09-22 Thread Konstantin Ryabitsev
On Fri, Sep 22, 2023 at 02:18:17AM +, Eric Wong wrote: > Subject: [PATCH] pop3: support initial_limit parameter in mailbox name That looks good in my tests. Thanks! Tested-by: Konstantin Ryabitsev -K

Re: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name

2023-09-19 Thread Konstantin Ryabitsev
On Mon, Sep 18, 2023 at 09:14:22PM +, Eric Wong wrote: > > Oh, I did notice what is probably unintentional behaviour -- passing > > ?limit=XXX affects all mailbox access, not just the initial retrieval. > > > > E.g. if I configured pop3 with ?limit=128, then leave for the weekend and > >

Re: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name

2023-09-18 Thread Konstantin Ryabitsev
On Fri, Sep 15, 2023 at 08:41:10PM +, Eric Wong wrote: > Thanks, pushed the series as > a37e3ab3740c24c3 (pop3: limit default mailbox to 1K messages, 2023-09-14) > 392d251f97d46579 (pop3: support `?limit=$NUM' parameter in mailbox name, > 2023-09-12) Oh, I did notice what is probably

RFC: lei searches managed by users in git

2023-09-15 Thread Konstantin Ryabitsev
Hello: I am curious what is the best approach to have a centrally managed set of lei searches, for example via config files tracked in git. For example, the file could look like this: mricon.toml: [search.torvalds] # All mail sent by torvalds q =

Re: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name

2023-09-15 Thread Konstantin Ryabitsev
in both Claws-mail and Thunderbird Tested-by: Konstantin Ryabitsev Thanks! -K

Re: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name

2023-09-15 Thread Konstantin Ryabitsev
On Wed, Sep 13, 2023 at 10:03:26PM +, Eric Wong wrote: > > What if we move the uuid into the password field -- it seems it belongs > > there > > anyway, as it's tied to the user cookie. > > I've thought about that, too; but it can get tricky since passwords > aren't visible in most UIs.

Re: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name

2023-09-13 Thread Konstantin Ryabitsev
On Tue, Sep 12, 2023 at 10:40:34PM +, Eric Wong wrote: > Perhaps 50K is too much? I figured clients would have a way to > limit that, but I don't really pay attention to POP3 clients... The few clients I looked at didn't give any option to specify how many remote messages I want to retrieve,

pop3 usability thoughts

2023-09-12 Thread Konstantin Ryabitsev
Hello: I've been playing around with pop3, and I'm wondering if we can improve its usability by adding a "last NNN messages" pseudo-folder. Currently, if someone wants to access the git mailing list archive via pop3, they have to do the following: - know that the username should be

Re: [PATCH] www: use correct threadid for per-thread search

2023-06-21 Thread Konstantin Ryabitsev
On Fri, Jun 16, 2023 at 11:13:01PM +, Eric Wong wrote: > > Reviving this old thread for some clarification. I noticed that this only > > works for /all/, but not for individual inboxes. E.g.: > > > > $ curl -d '' -sSf \ > > https://lore.kernel.org/all/"$MSGID/?x=m=rt:2023-03-29..; \

Re: Cheap way to check for new messages in a thread

2023-06-16 Thread Konstantin Ryabitsev
On Thu, Mar 30, 2023 at 11:29:51AM +, Eric Wong wrote: > This implements the mbox.gz retrieval. I didn't want to deal > with HTML nor figuring out how to expose more elements, > yet; but I figure mbox.gz is the most important. > > Now deployed on 80x24.org/lore: > >

Re: Indicating the mirror's origin

2023-06-15 Thread Konstantin Ryabitsev
On Wed, Jun 14, 2023 at 11:50:15PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > Good day: > > > > We've had a few requests to mirror public-inbox archives that originate on > > other systems so they can also be searchable and viewable via > > lore.k

Re: Indicating the mirror's origin

2023-06-14 Thread Konstantin Ryabitsev
On Wed, Jun 14, 2023 at 10:18:57PM +0200, Uwe Kleine-König wrote: > Hello Konstantin, > > On Wed, Jun 14, 2023 at 02:42:15PM -0400, Konstantin Ryabitsev wrote: > > We've had a few requests to mirror public-inbox archives that originate on > > other systems so they c

Indicating the mirror's origin

2023-06-14 Thread Konstantin Ryabitsev
Good day: We've had a few requests to mirror public-inbox archives that originate on other systems so they can also be searchable and viewable via lore.kernel.org. I've been dragging my feet on these requests, because they are a potential liability in terms of GDPR compliance. If we are merely

Re: https://lore.kernel.org/linux-mm/

2023-06-08 Thread Konstantin Ryabitsev
On Thu, Jun 08, 2023 at 12:40:52AM -0600, Jonathan Corbet wrote: > > This is the reason why there's a gap from May 31 to June 6. If you would > > like > > to contribute the missing messages, I'll be happy to feed them into the > > archive. > > The LWN archive -

Re: https://lore.kernel.org/linux-mm/

2023-06-07 Thread Konstantin Ryabitsev
On Wed, Jun 07, 2023 at 10:41:18AM +0800, Yin, Fengwei wrote: > > Messages on lore.kernel.org show up because an email address > > they control receives messages from the linux-mm list. Since > > (I assume) kvack.org is controlled by someone else, kernel.org > > needs to subscribe to the linux-mm

Re: Threaded responses to queries (x86)

2023-05-31 Thread Konstantin Ryabitsev
On Wed, May 31, 2023 at 07:50:53AM +, Eric Wong wrote: > I wonder if lore can expose x...@kernel.org as its own inbox even > if it's technically not a subscribable mailing list. That would > be much faster (but less space-efficient) than issuing a Xapian > query to get x...@kernel.org mails.

Re: search by whole thread?

2023-04-12 Thread Konstantin Ryabitsev
On Wed, Apr 12, 2023 at 12:06:53AM +, Eric Wong wrote: > I think the reason it's rare in MUAs is that it's potentially > very expensive. But I think the `thread:{subquery}' feature > from notmuch I discussed with Konstantin the other week[1] can > do what you want it to do. > > Keep in mind,

Re: Cheap way to check for new messages in a thread

2023-03-30 Thread Konstantin Ryabitsev
On Thu, Mar 30, 2023 at 11:29:51AM +, Eric Wong wrote: > > Per-thread search is something I've wanted for a while, anyways, > > so I think I'll do /$MSGID/?q= in between ongoing work for > > This implements the mbox.gz retrieval. I didn't want to deal > with HTML nor figuring out how to

Re: Cheap way to check for new messages in a thread

2023-03-28 Thread Konstantin Ryabitsev
On Tue, Mar 28, 2023 at 10:08:30PM +, Eric Wong wrote: > > I think this is a workable approach, but would require a reindex, right? > > Yes, it requires a reindex to take effect, which takes ~2 days > on my lore mirror. The biggest problem is MUAs are likely to > cull References: when

Re: Cheap way to check for new messages in a thread

2023-03-28 Thread Konstantin Ryabitsev
On Tue, Mar 28, 2023 at 07:45:49PM +, Eric Wong wrote: > C) index References:/In-Reply-To: so searching `ref:$MSGID' >can work. This doesn't work for some MUAs and deep >threads, though. I think this is a workable approach, but would require a reindex, right? -K

Re: Cheap way to check for new messages in a thread

2023-03-28 Thread Konstantin Ryabitsev
On Mon, Mar 27, 2023 at 09:38:49PM +, Eric Wong wrote: > I thought about that, too; but I'm worried about having one-off > stuff that ends up needing to be supported indefinitely. > > JMAP for this would take more time, but I'd be more comfortable > carrying it long-term. > > I don't expect

Re: Cheap way to check for new messages in a thread

2023-03-27 Thread Konstantin Ryabitsev
On Mon, Mar 27, 2023 at 07:10:49PM +, Eric Wong wrote: > > For the bugzilla integration work I'm doing, I need a way to check if there > > were any updates to a thread since the last check. Right now, I'm just > > grabbing the full thread, parsing it and seeing if there are any new > >

Cheap way to check for new messages in a thread

2023-03-27 Thread Konstantin Ryabitsev
Hello: For the bugzilla integration work I'm doing, I need a way to check if there were any updates to a thread since the last check. Right now, I'm just grabbing the full thread, parsing it and seeing if there are any new message-IDs that we don't know about, but it's very wasteful. Any way to

Re: IMAP users: how useful is server-side search to you?

2023-03-15 Thread Konstantin Ryabitsev
On Tue, Mar 14, 2023 at 09:55:02PM +, Eric Wong wrote: > I've always used local search (lei nowadays, mairix in the past). > > I'm considering an option to disable it, or make it available to > AUTH=ANONYMOUS users only, since there seems to be a lot of > scrapers trying to look for private

Re: [PATCH] lei_mirror: unlink FETCH_HEAD when fetching forkgroups

2023-03-08 Thread Konstantin Ryabitsev
On Wed, Mar 08, 2023 at 11:02:58AM +, Eric Wong wrote: > Apparently, --no-write-fetch-head is broken in current git[1]. > It also wasn't in older git, at all. So just unlink FETCH_HEAD > as we see it, but keep using --no-write-fetch-head to avoid the > syscall and I/O overhead when we can.

Re: ActivityPub <=> email bridge?

2023-03-07 Thread Konstantin Ryabitsev
On Tue, Mar 07, 2023 at 10:12:10PM +, Eric Wong wrote: > > Something tells me that if ActivityPub reaches high-enough > > adoption levels; it'll have to deal with a spam problem that > > email folks have been dealing with for decades, too. > > > > So ActivityPub seems like a duplicated effort

Re: POP3 adoption (was: [PATCH 1/2] doc|www: flesh out POP3 documentation for servers) and users

2023-02-06 Thread Konstantin Ryabitsev
On Mon, Feb 06, 2023 at 06:03:38AM +, Eric Wong wrote: > > +The password is: anonymous > > +The username is: \$(uuidgen)\@$ctx->{ibx}->{newsgroup} > > +where \$(uuidgen) in the output of the `uuidgen' command on your system. > > +The UUID in the username functions as a private cookie (don't

Re: [PATCH] eml: header_raw converts octets to Perl UTF-8

2022-11-25 Thread Konstantin Ryabitsev
On Thu, Nov 24, 2022 at 09:31:55PM +, Eric Wong wrote: > The below case generalizes it to all HTML displays and removes > the special case. It looks good to me in some cursory tests, thank you! -K

handling unquoted utf8 in the headers

2022-11-24 Thread Konstantin Ryabitsev
Hello: There's a bit of inconsistency handling messages with utf8 content in the headers: https://lore.kernel.org/b4-sent/20221122-gud-shadow-plane-v1-0-9de3afa33...@tronnes.org/ You can see that the name in the From: line is mangled, but in the thread overview it is displayed correctly. I

Re: [PATCH 0/4] CentOS 7 fixes + fix Gcf2 everywhere

2022-09-29 Thread Konstantin Ryabitsev
On Thu, Sep 29, 2022 at 05:48:27PM +, Eric Wong wrote: > A bunch of CentOS 7.x fixes noted by Konstantin > I also just noticed I broke Gcf2 everywhere :x The tests pass now, thanks! I still noticed this error (master + this series): 1 at

Re: "make test" for 1.9.0 on centos-7

2022-09-28 Thread Konstantin Ryabitsev
On Wed, Sep 28, 2022 at 07:59:39PM +, Eric Wong wrote: > > I'm starting to work on upgrading lore.kernel.org to 1.9.0. For a number of > > yak-shavey reasons we are still on centos-7, though hopefully we'll be able > > to > > move on to something newer soon. Right now, I'm having difficulty

"make test" for 1.9.0 on centos-7

2022-09-28 Thread Konstantin Ryabitsev
Hello: I'm starting to work on upgrading lore.kernel.org to 1.9.0. For a number of yak-shavey reasons we are still on centos-7, though hopefully we'll be able to move on to something newer soon. Right now, I'm having difficulty running "make test": ... [skipping OK tests] ...

Re: extindex for git? [was: an even bigger git show than before...]

2022-08-26 Thread Konstantin Ryabitsev
On Thu, Aug 25, 2022 at 09:34:42PM +, Eric Wong wrote: > > I wanted to add search to git repos ages ago, but it was silly > > expensive in terms of space. That was before extindex... > > > > extindex ought to be able to offer space savings across forks > > and similar documents (commits vs

Re: APOP-only POP3 clients?

2022-05-27 Thread Konstantin Ryabitsev
On Fri, May 27, 2022 at 10:53:02AM +, Eric Wong wrote: > Which may be an extremely long username... Now I'm thinking > it's safe for UUID_1 and UUID_2 to be the same, to save storage > space on the server and to save users from dealing with > excessively long, compression-unfriendly field

Re: RFC: hooks in public-inbox-watch

2022-05-12 Thread Konstantin Ryabitsev
On Thu, May 12, 2022 at 07:38:59PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > Hi, all: > > > > What do you think about a mechanism to run hooks at the stage right before > > public-inbox-watch adds a new message to the archive? One feature that wou

RFC: hooks in public-inbox-watch

2022-05-12 Thread Konstantin Ryabitsev
Hi, all: What do you think about a mechanism to run hooks at the stage right before public-inbox-watch adds a new message to the archive? One feature that would be neat is to search archives for all instances of the same patch using its $(git-patch-id --stable) and adding a header, e.g.:

Re: Trouble running lei

2022-05-03 Thread Konstantin Ryabitsev
On Tue, May 03, 2022 at 01:50:52PM +0100, Filipe Manana wrote: > > Perhaps it's already running lei-daemon as an older version? > > "lei daemon-kill" should kill it and it'll restart on the next > > command, unless something else got wedged. > > Ah, running "lei daemon-kill" fixed it. > I don't

Re: Issue with lore.kernel.org links

2022-03-07 Thread Konstantin Ryabitsev
On Mon, Mar 07, 2022 at 09:44:06AM -0600, Tom Lendacky wrote: > Hi Konstantin, > > Boris Petkov suggested I email you about an issue I'm having with some of > the links on lore.kernel.org. For example, I performed a search and was > presented with a result page: > >

RFC: should lei inject its own "Received:" header?

2021-11-19 Thread Konstantin Ryabitsev
Hello: I wonder if lei should inject its own "received"-like header on writing to a maildir/imap target -- to indicate where the copy of the email came from. I don't think it should use the actual Received: header, as this may cause some weird SPF/DMARC issues, but perhaps something like:

Re: lei: incorrect quoting on saved searches (was Re: lore+lei: getting started)

2021-11-08 Thread Konstantin Ryabitsev
On Mon, Nov 08, 2021 at 09:48:36PM +, Eric Wong wrote: > > Hmm... I noticed that when I `lei edit-search` the initial query that was > > causing quoting issues, I get the following: > > > > [lei] > > q = (dfn:drivers OR dfn:arch OR dfn:Documentation OR > > dfn:include OR

Re: lei: incorrect quoting on saved searches (was Re: lore+lei: getting started)

2021-11-08 Thread Konstantin Ryabitsev
On Mon, Nov 08, 2021 at 08:49:23PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > On Mon, Nov 08, 2021 at 01:49:07PM -0600, Rob Herring wrote: > > > > Moving this to meta. > > I don't think workflows should've been dropped, though. > > > > >

lei: incorrect quoting on saved searches (was Re: lore+lei: getting started)

2021-11-08 Thread Konstantin Ryabitsev
On Mon, Nov 08, 2021 at 01:49:07PM -0600, Rob Herring wrote: Moving this to meta. > > lei q -I https://lore.kernel.org/all/ -o ~/Mail/floppy \ > > --threads --dedupe=mid \ > > '(dfn:drivers/block/floppy.c OR dfhh:floppy_* OR s:floppy \ > > OR ((nq:bug OR nq:regression) AND

test: 'seen set from rename' potentially racy

2021-11-04 Thread Konstantin Ryabitsev
Hello: I was having some trouble using "make check" on the Fedora build system: # Failed test 'seen set from rename' # at t/lei-watch.t line 61. # '/tmp/pi-lei-watch-11080-FYz2/lei-daemon/md2/cur/9bf1002c49eb075df47247b74d69bcd555e23422=99:2,' # doesn't match

Re: [ANNOUNCE] public-inbox 1.7.0

2021-11-04 Thread Konstantin Ryabitsev
On Thu, Nov 04, 2021 at 07:52:00AM +, Eric Wong wrote: > Another big release focused on multi-inbox search and scalability. Congratulations on the release, Eric! Happy to be one of the frontline users, and I'm sure a lot more kernel devs will jump on it now that lei is becoming more available

Initial Fedora packaging for lei

2021-11-02 Thread Konstantin Ryabitsev
Hi, all: I did some initial work to package lei for Fedora 34 and 35 (out today). The lei parts should be ready to use, though I'll continue to work on the server parts (only needed if you're running httpd/nntpd/imapd daemons). For now, you'll need to enable my copr repository to use it:

Re: Troubleshooting threads missing from /all/

2021-10-26 Thread Konstantin Ryabitsev
On Sun, Oct 24, 2021 at 12:03:17AM +, Eric Wong wrote: > > I used the default flags for --reindex --all --fast, so it can perhaps be > > sped > > up with larger memory use, but this is good enough for daily runs already. > > Cool. Just wondering if all is well on your end with daily runs.

Re: Troubleshooting threads missing from /all/

2021-10-18 Thread Konstantin Ryabitsev
On Mon, Oct 18, 2021 at 05:25:26AM +, Eric Wong wrote: > > Btw, I'm chasing a separate bug in v2 which causes recycled > > Message-IDs to go missing sometimes from a v2 over.sqlite3; > > which then causes -extindex to lose a message... > > I just pushed out commit 325fbe26c3e7731e > (v2:

Re: Troubleshooting threads missing from /all/

2021-10-18 Thread Konstantin Ryabitsev
On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote: > With --fast, --reindex takes around 20 minutes for me with > "--batch-size=20m --no-fsync". The first run may take longer > if it has stuff to do. But running it repeatedly should not > cause it to complain about

Re: Troubleshooting threads missing from /all/

2021-10-17 Thread Konstantin Ryabitsev
On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote: > Eric Wong wrote: > > Yes. Though given the current situation with missing messages > > from /all/, I'd wait until a reindex recovers the missing > > messages (and probably a fast fsck checker). > > I think "public-inbox-extindex

Re: Troubleshooting threads missing from /all/

2021-10-08 Thread Konstantin Ryabitsev
On Thu, Oct 07, 2021 at 09:33:07PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > [publicinbox "regressions"] > > address = regressi...@lists.linux.dev > > url = regressions > > inboxdir = /srv/public-inbox/lore.kernel.org/regressions > >

Re: Troubleshooting threads missing from /all/

2021-10-07 Thread Konstantin Ryabitsev
On Thu, Oct 07, 2021 at 08:36:52AM +, Eric Wong wrote: > Also, did you capture any error messages to stderr? > I suppose you would've told us if you did. Yeah, I looked through any place that would have logged an error and I didn't really see anything. I expect this would have happened during

Re: Troubleshooting threads missing from /all/

2021-10-05 Thread Konstantin Ryabitsev
On Tue, Oct 05, 2021 at 04:39:54AM +, Eric Wong wrote: > Eric Wong wrote: > > b) just reindex in place (it /should/ work...) > > I reindexing live on yhbt/lore and it didn't break... > > Btw, did you see my other questions about whether or not boost > was in use? Yes, but I was attending

Re: Troubleshooting threads missing from /all/

2021-10-01 Thread Konstantin Ryabitsev
On Fri, Oct 01, 2021 at 10:25:09PM +, Eric Wong wrote: > export HOME=/tmp/trash # fresh lei/store instance > M=87czop5j33@tynnyri.adurom.net > lei import https://yhbt.net/lore/all/$M/t.mbox.gz > lei q z:0.. | wc -l # should have all (11) msgs > lei q m:$M -t | wc -l # should have

Re: Troubleshooting threads missing from /all/

2021-10-01 Thread Konstantin Ryabitsev
On Fri, Oct 01, 2021 at 08:54:42PM +, Eric Wong wrote: > Oops, inspect doesn't work well w/o initialization (it should). > Running "lei init" first should workaround it, for now. Yes, that fixes the problem, but it still doesn't return much: { "mid" :

Re: Troubleshooting threads missing from /all/

2021-10-01 Thread Konstantin Ryabitsev
On Fri, Oct 01, 2021 at 08:41:31PM +, Eric Wong wrote: > Curious, what is the output of: > >lei inspect --dir /path/to/all mid:87czop5j33@tynnyri.adurom.net > > for you? Not much. :) lei inspect --dir /srv/public-inbox/extindex mid:87czop5j33@tynnyri.adurom.net

Re: Troubleshooting threads missing from /all/

2021-10-01 Thread Konstantin Ryabitsev
On Fri, Oct 01, 2021 at 09:05:27AM -0400, Konstantin Ryabitsev wrote: > I was told about the following problem today: > > The following thread: > https://lore.kernel.org/regressions/87czop5j33@tynnyri.adurom.net/ > > Doesn't appear to show up in /all/: > https:

Re: Troubleshooting threads missing from /all/

2021-10-01 Thread Konstantin Ryabitsev
On Fri, Oct 01, 2021 at 07:58:11PM +, Eric Wong wrote: > Are you running "public-inbox-extindex --all"? > Or relying on "public-inbox-index -E ..."? (which is automatic > for /all/?). After each repository update, we run: public-inbox-index --no-update-extindex And at the end of each

Re: [PATCH 0/3] fixes for odd/old/missing dependencies

2021-09-27 Thread Konstantin Ryabitsev
On Mon, Sep 27, 2021 at 04:05:42PM -0500, Eric Wong wrote: > On Mon, 27 Sep 2021 15:45:20 -0400, Konstantin Ryabitsev wrote: > > On Mon, Sep 27, 2021 at 07:33:46PM +, Eric Wong wrote: > > > The partial fetching would need some work to support working as > > > ro

Re: -fetch failures [was: latest make test failures on CentOS-7]

2021-09-27 Thread Konstantin Ryabitsev
On Mon, Sep 27, 2021 at 07:33:46PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > t/v2mirror.t . 71/? W: > > /tmp/pi-v2mirror-39373-Dl1N/m/git/3.git missing remote.origin.url > > fatal: not a git repository: '/tmp/pi-v2mirror-39373-Dl1N/m/g

Re: [PATCH] t/cmd_ipc: allow extra errors and add diagnostics

2021-09-27 Thread Konstantin Ryabitsev
On Mon, Sep 27, 2021 at 01:35:36PM -0500, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > t/cmd_ipc.t .. 29/? > > # Failed test 'got EMSGSIZE' > > # at t/cmd_ipc.t line 108. > > # Looks like you failed 1 test

latest make test failures on CentOS-7

2021-09-27 Thread Konstantin Ryabitsev
Hello: I wanted to try the searchable /all/ from www_index, but it looks like I'm unable to get a clean make test. Below are a few failures that I can see: t/cmd_ipc.t .. 29/? # Failed test 'got EMSGSIZE' # at t/cmd_ipc.t line 108. # Looks like you failed 1

Re: Holding on to deleted packfiles

2021-09-21 Thread Konstantin Ryabitsev
On Tue, Sep 21, 2021 at 07:06:53PM +, Eric Wong wrote: > Was this from /all/ (ALL.git using batch-file) or Gcf2? I believe this was from Gcf2, though I can't go back and check, unfortunately. > The old stuff has timers to do periodic cleanup, but the new > stuff is trickier as the cost of a

Holding on to deleted packfiles

2021-09-21 Thread Konstantin Ryabitsev
Hello: A large git repack job that ran over the weekend revealed a minor problem -- public-inbox daemon processes will hold on to deleted pack files until they are restarted. Is there any way to gracefully recognize and handle this condition? It's not quite benign, as this ended up keeping 40GB+

Re: make menuconfig interface for lei / grok-pull

2021-09-16 Thread Konstantin Ryabitsev
On Wed, Sep 15, 2021 at 11:06:05PM +, Eric Wong wrote: > Does lore.kernel.org run public-inbox-imapd? I'm still not convinced it's useful for huge collections, especially considering how chatty IMAP is. Is there any benefit to enable it for lei use? -K

Re: make menuconfig interface for lei / grok-pull

2021-09-16 Thread Konstantin Ryabitsev
On Wed, Sep 15, 2021 at 02:34:40PM -0700, Luis Chamberlain wrote: > My use case is I'm subscribed to a few kernel mailign lists and I use > mutt with Maildir. I had configured recently pi-piper and grokmirror > so that I get only the last 1 year of email from a few set of mailing > lists. For this

Re: mbox support in other software

2021-09-16 Thread Konstantin Ryabitsev
On Thu, Sep 16, 2021 at 07:34:37AM +, Eric Wong wrote: > Since I've written the lei-mail-formats manpage, I've been > curious what other software differentiates between the various > mbox formats and supports several/all of them? > > AFAIK, none of the Perl Mail::* stuff does, nor does

Re: [PATCH] uri_imap: fix ->uidvalidity and ->uid w/ `/' separator

2021-09-15 Thread Konstantin Ryabitsev
On Wed, Sep 15, 2021 at 05:43:41PM +, Eric Wong wrote: > > The error is gone, but the saved search still doesn't show up when I run > > "lei ls-search". Looking in .local/share/lei/saved-searches, I see that they > > still get created as lore/foldername-${checksum}, which is probably why > >

Re: [PATCH] uri_imap: fix ->uidvalidity and ->uid w/ `/' separator

2021-09-15 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 10:10:36PM +, Eric Wong wrote: > Konstantin Ryabitsev wrote: > > 2021-09-14T20:59:12Z 20428 20428 die: BUG: > > imaps://imap.migadu.com/lore/b4;UIDVALIDITY=1621977334 has no UIDVALIDITY > > at /usr/local/share/perl/5.32.1/PublicInbox

Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 03:55:10PM -0400, Konstantin Ryabitsev wrote: > On Tue, Sep 14, 2021 at 07:35:28PM +, Eric Wong wrote: > > > I found an interesting problem using lei with imaps:// folders. I'm trying > > > things out with migadu, and the folder paths use '/'

RFC: lei-daemon and auto-up

2021-09-14 Thread Konstantin Ryabitsev
Hello: Since lei-daemon is already up and running, would it be possible to tell it to automatically "lei up" things at certain intervals? Maybe something like: [lei] q = [...] [lei "q"] output = [...] include = https://lore.kernel.org/all/ external = 1

Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 08:12:16PM +, Eric Wong wrote: > Ah, I forgot to update the docs again :x > > My main concern with .netrc was actually inadvertantly sending > FTP auth info to an IMAP server just because they share the same > host. No big deal -- folks can always just use the "store"

Re: [PATCH] uri_imap: handle '/' as an IMAP hierarchy separator

2021-09-14 Thread Konstantin Ryabitsev
On Tue, Sep 14, 2021 at 07:35:28PM +, Eric Wong wrote: > > I found an interesting problem using lei with imaps:// folders. I'm trying > > things out with migadu, and the folder paths use '/' separators, so a full > > IMAPS folder path for a folder "lore/mentions" is > >

  1   2   3   >