<< Why does one even have to run a periodic indexer? Aren't there guarantees in
the system that updates are seen through to the index in realtime, do bulk
updates not trigger a refresh of updated records selectively? Reading the code
seems to suggest that updates are queued until processed, does AS need a more
durable queue? >>
In theory, if your index is “up to date” (according to the indexer_state
directory) the periodic indexer should have no work to do. I think this is part
of a class of problems that arise when for some reason the periodic indexer
cannot get through its workload and therefore tries and tries again. That is
what happens, for example, when a MySQL database contains a record with a bad
timestamp due to DST. If someone could file a JIRA issue with as much info as
possible for recreating the problem (and maybe someone who could be contacted
to supply a database copy) then it could probably be prioritized and addressed.
From: archivesspace_users_group-boun...@lyralists.lyrasis.org
<archivesspace_users_group-boun...@lyralists.lyrasis.org> on behalf of Peter
Heiner <ph...@cam.ac.uk>
Date: Saturday, January 28, 2023 at 3:51 AM
To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] External Solr - Memory Allocation?
Joshua D. Shaw wrote on 2023-01-26 23:02:10:
> Thanks, Blake!
>
> I'm running default config values for the AS log levels so they are all set
> to 'debug'. I took a closer look, and the timeout message happens exactly
> after the timeout amount I set (as you'd expect). Interestingly, Solr is in
> the middle of deleting documents when it goes silent
>
> I, [2023-01-26T09:18:40.357101 #78764] INFO -- : Thread-3384: Deleted 100
> documents: #<Net::HTTPOK:0x72b3d9e>
>
> .... 40 minutes pass with all the other AS log chatter ...
>
> E, [2023-01-26T09:58:40.400971 #78764] ERROR -- : Thread-3384:
> SolrIndexerError when deleting records: Timeout error with POST {....}
> I, [2023-01-26T09:58:40.410522 #78764] INFO -- : Thread-3384: Deleted 100
> documents: #<Net::HTTPOK:0x4ab44e31>
>
> This continuing delete phase goes on for a bit until it stops logging batch
> deletes.
>
> I, [2023-01-26T09:59:11.734200 #78764] INFO -- : Thread-3384: Deleted 9
> documents: #<Net::HTTPOK:0x1be6c3e9>
>
> .... 40 minutes pass with all the other AS log chatter ... And then the
> commit error pops up
>
> E, [2023-01-26T10:39:11.746166 #78764] ERROR -- : Thread-3384:
> SolrIndexerError when committing:
> Timeout error with POST {"commit":{"softCommit":false}}.
>
> Then after some more time
>
> I, [2023-01-26T11:06:35.678926 #78764] INFO -- : Thread-3384: Deleted 186992
> documents: #<Net::HTTPOK:0x7e298af9>
>
> .... This all seems to indicate to me that the commit phase is taking an
> inordinate amount of time (almost 2 hours - maybe that's what I need to set
> the timeout to?). After that, the indexer starts the 2nd repo
We're experiencing the exact same issue at least in the largest of our 30-odd
repositories:
I, [2023-01-28T07:24:48.015356 #2036632] INFO -- : Thread-2006: Staff Indexer
[2023-01-28 07:24:48 +0000] ~~~ Indexed 536300 of 587664 archival_object
records in repository CUL
E, [2023-01-28T07:25:47.217953 #2036632] ERROR -- : Thread-2016:
SolrIndexerError when committing:
Timeout error with POST {"commit":{"softCommit":false}}.
We've had this problem from the start but were unable to dig deeper because
our in-house monitoring wasn't granular or even capable enough. Our crude
solution was using an external Solr since 2.5 and an external indexer since
around 2.7, and periodic restarts out of hours, but we've started getting
problems despite that. Our Solr is allocated 6GB of memory and timeout is set
to 1200 seconds, but the problem is that searches in AS fail during the wait
and that makes AS unusable, so we're reluctant to increase that.
Why does one even have to run a periodic indexer? Aren't there guarantees in
the system that updates are seen through to the index in realtime, do bulk
updates not trigger a refresh of updated records selectively? Reading the code
seems to suggest that updates are queued until processed, does AS need a more
durable queue?
Thanks,
p
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
_______________________________________________
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
