Konstantin Ryabitsev wrote:
> On Sun, Oct 24, 2021 at 12:03:17AM +, Eric Wong wrote:
> > I'm thinking about cutting a new release soonish and just
> > putting giant warnings around the not-yet-ready parts of
> > lei for now...
>
> I think this may be a good plan, considering that lei is now
On Sun, Oct 24, 2021 at 12:03:17AM +, Eric Wong wrote:
> > I used the default flags for --reindex --all --fast, so it can perhaps be
> > sped
> > up with larger memory use, but this is good enough for daily runs already.
>
> Cool. Just wondering if all is well on your end with daily runs.
Konstantin Ryabitsev wrote:
> On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote:
> > With --fast, --reindex takes around 20 minutes for me with
> > "--batch-size=20m --no-fsync". The first run may take longer
> > if it has stuff to do. But running it repeatedly should not
> > cause it
On Mon, Oct 18, 2021 at 05:25:26AM +, Eric Wong wrote:
> > Btw, I'm chasing a separate bug in v2 which causes recycled
> > Message-IDs to go missing sometimes from a v2 over.sqlite3;
> > which then causes -extindex to lose a message...
>
> I just pushed out commit 325fbe26c3e7731e
> (v2:
On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote:
> With --fast, --reindex takes around 20 minutes for me with
> "--batch-size=20m --no-fsync". The first run may take longer
> if it has stuff to do. But running it repeatedly should not
> cause it to complain about
Eric Wong wrote:
> Konstantin Ryabitsev wrote:
> > On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote:
> > > Eric Wong wrote:
> > > > Yes. Though given the current situation with missing messages
> > > > from /all/, I'd wait until a reindex recovers the missing
> > > > messages (and
Konstantin Ryabitsev wrote:
> On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote:
> > Eric Wong wrote:
> > > Yes. Though given the current situation with missing messages
> > > from /all/, I'd wait until a reindex recovers the missing
> > > messages (and probably a fast fsck checker).
>
On Sat, Oct 16, 2021 at 09:43:24AM +, Eric Wong wrote:
> Eric Wong wrote:
> > Yes. Though given the current situation with missing messages
> > from /all/, I'd wait until a reindex recovers the missing
> > messages (and probably a fast fsck checker).
>
> I think "public-inbox-extindex
Eric Wong wrote:
> Yes. Though given the current situation with missing messages
> from /all/, I'd wait until a reindex recovers the missing
> messages (and probably a fast fsck checker).
I think "public-inbox-extindex --reindex --all --fast" is
reasonably ready as an fsck checker. I've been
Konstantin Ryabitsev wrote:
> On Thu, Oct 07, 2021 at 09:33:07PM +, Eric Wong wrote:
> > Konstantin Ryabitsev wrote:
> > > [publicinbox "regressions"]
> > > address = regressi...@lists.linux.dev
> > > url = regressions
> > > inboxdir = /srv/public-inbox/lore.kernel.org/regressions
> >
On Thu, Oct 07, 2021 at 09:33:07PM +, Eric Wong wrote:
> Konstantin Ryabitsev wrote:
> > [publicinbox "regressions"]
> > address = regressi...@lists.linux.dev
> > url = regressions
> > inboxdir = /srv/public-inbox/lore.kernel.org/regressions
> > indexlevel = full
>
> Btw, "indexlevel
Konstantin Ryabitsev wrote:
> On Thu, Oct 07, 2021 at 08:36:52AM +, Eric Wong wrote:
> > Also, did you capture any error messages to stderr?
> > I suppose you would've told us if you did.
>
> Yeah, I looked through any place that would have logged an error and I didn't
> really see anything.
Also, did you capture any error messages to stderr?
I suppose you would've told us if you did.
(resend, MTA dropped this part:)
In particular, I just posted a patch to fix
"Can't bless non-reference value" messages could've been causing
some messages to fail indexing completely.
(resend, screwed up something with my MTA :x)
OK. I tried reproducing the problem even with f28fdcd6d8d6
(content_hash: normalize whitespace before hashing addresses, 2021-10-02)
reverted, but haven't been able to...
So far I've found some gc and dedupe bugs, but something's still
eluding me.
On Thu, Oct 07, 2021 at 08:36:52AM +, Eric Wong wrote:
> Also, did you capture any error messages to stderr?
> I suppose you would've told us if you did.
Yeah, I looked through any place that would have logged an error and I didn't
really see anything. I expect this would have happened during
Also, did you capture any error messages to stderr?
I suppose you would've told us if you did.
--
unsubscribe: one-click, see List-Unsubscribe header
archive: https://public-inbox.org/meta/
Konstantin Ryabitsev wrote:
> On Tue, Oct 05, 2021 at 04:39:54AM +, Eric Wong wrote:
> > Eric Wong wrote:
> > > b) just reindex in place (it /should/ work...)
> >
> > I reindexing live on yhbt/lore and it didn't break...
> >
> > Btw, did you see my other questions about whether or not
On Tue, Oct 05, 2021 at 04:39:54AM +, Eric Wong wrote:
> Eric Wong wrote:
> > b) just reindex in place (it /should/ work...)
>
> I reindexing live on yhbt/lore and it didn't break...
>
> Btw, did you see my other questions about whether or not boost
> was in use?
Yes, but I was attending
Eric Wong wrote:
> b) just reindex in place (it /should/ work...)
I reindexing live on yhbt/lore and it didn't break...
Btw, did you see my other questions about whether or not boost
was in use?
I need to work on some read-only "fsck"-type functionality
and limited reindexing.
--
unsubscribe:
Konstantin Ryabitsev wrote:
> HTTP/1.1 300 Multiple Choices
>
> However, this seems to happen for all unknown message-ids (e.g.
> lore.kernel.org/all/bogus@bogus), so probably entirely unrelated.
Yeah, unrelated, it's always returned 300 since I wanted to highlight
the fact there's many hosts
Eric Wong wrote:
> If you have extra space somewhere to:
>
> a) copy the old extindex somewhere and do extindex --reindex on it
> b) just reindex in place (it /should/ work...)
> c) start a new extindex from scratch...
I might have to try c) myself. Though it might be time/batch-related;
Konstantin Ryabitsev wrote:
> On Fri, Oct 01, 2021 at 10:25:09PM +, Eric Wong wrote:
> > export HOME=/tmp/trash # fresh lei/store instance
> > M=87czop5j33@tynnyri.adurom.net
> > lei import https://yhbt.net/lore/all/$M/t.mbox.gz
> > lei q z:0.. | wc -l # should have all (11) msgs
On Fri, Oct 01, 2021 at 10:25:09PM +, Eric Wong wrote:
> export HOME=/tmp/trash # fresh lei/store instance
> M=87czop5j33@tynnyri.adurom.net
> lei import https://yhbt.net/lore/all/$M/t.mbox.gz
> lei q z:0.. | wc -l # should have all (11) msgs
> lei q m:$M -t | wc -l # should have
Eric Wong wrote:
> Can you confirm the above gives all 11 msgs for you?
erm, that's 10 msgs, I forget the JSON output always ends with a
"null" line
--
unsubscribe: one-click, see List-Unsubscribe header
archive: https://public-inbox.org/meta/
OK, so I'm also trying this on a CentOS 7.x VM, and
the lei part works as expected, so it seems specific to
extindex and not a problem with package differences in CentOS.
export HOME=/tmp/trash # fresh lei/store instance
M=87czop5j33@tynnyri.adurom.net
lei import
On Fri, Oct 01, 2021 at 08:54:42PM +, Eric Wong wrote:
> Oops, inspect doesn't work well w/o initialization (it should).
> Running "lei init" first should workaround it, for now.
Yes, that fixes the problem, but it still doesn't return much:
{
"mid" :
Konstantin Ryabitsev wrote:
> On Fri, Oct 01, 2021 at 08:41:31PM +, Eric Wong wrote:
> > Curious, what is the output of:
> >
> >lei inspect --dir /path/to/all mid:87czop5j33@tynnyri.adurom.net
> >
> > for you?
>
> Not much. :)
>
> lei inspect --dir /srv/public-inbox/extindex
On Fri, Oct 01, 2021 at 08:41:31PM +, Eric Wong wrote:
> Curious, what is the output of:
>
>lei inspect --dir /path/to/all mid:87czop5j33@tynnyri.adurom.net
>
> for you?
Not much. :)
lei inspect --dir /srv/public-inbox/extindex
mid:87czop5j33@tynnyri.adurom.net
Konstantin Ryabitsev wrote:
> On Fri, Oct 01, 2021 at 07:58:11PM +, Eric Wong wrote:
> > Are you running "public-inbox-extindex --all"?
> > Or relying on "public-inbox-index -E ..."? (which is automatic
> > for /all/?).
> public-inbox-extindex --all
>
> I believe this is what you
On Fri, Oct 01, 2021 at 09:05:27AM -0400, Konstantin Ryabitsev wrote:
> I was told about the following problem today:
>
> The following thread:
> https://lore.kernel.org/regressions/87czop5j33@tynnyri.adurom.net/
>
> Doesn't appear to show up in /all/:
>
On Fri, Oct 01, 2021 at 07:58:11PM +, Eric Wong wrote:
> Are you running "public-inbox-extindex --all"?
> Or relying on "public-inbox-index -E ..."? (which is automatic
> for /all/?).
After each repository update, we run:
public-inbox-index --no-update-extindex
And at the end of each
Konstantin Ryabitsev wrote:
> Hello:
>
> I was told about the following problem today:
>
> The following thread:
> https://lore.kernel.org/regressions/87czop5j33@tynnyri.adurom.net/
>
> Doesn't appear to show up in /all/:
> https://lore.kernel.org/all/87czop5j33@tynnyri.adurom.net/
>
32 matches
Mail list logo
