https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44024
Bug ID: 44024
Summary: es_indexer_daemon.pl: keep the batch and retry while
Elasticsearch is unreachable, jobs are lost with STOMP
Initiative type: ---
Sponsorship ---
status:
Product: Koha
Version: Main
Hardware: All
OS: All
Status: NEW
Severity: normal
Priority: P5 - low
Component: Searching - Elasticsearch
Assignee: [email protected]
Reporter: [email protected]
QA Contact: [email protected]
Target Milestone: ---
When Elasticsearch cannot be reached, es_indexer_daemon.pl either marks the
index jobs
'failed' (see bug 44023) or, once that is fixed, resets them to 'new' (bug
42669). Neither
recovers with JobsNotificationMethod = STOMP, which is the default: the message
is
acknowledged before commit() runs, and the daemon only reads the
background_jobs table in
polling mode, so a job reset to 'new' is never picked up again and the update
is lost.
Tested with bug 44023 applied and STOMP: Elasticsearch stopped, a job enqueued,
job reset
to 'new', Elasticsearch started again: two minutes later the job is still
'new'. With
polling the same job finishes.
The patch keeps the batch in the forked worker that already owns it and retries
it every
ES_RETRY_DELAY seconds (default 10), for at most ES_RETRY_MAX_WAIT seconds
(default
3600). The jobs stay 'started' while waiting and finish as soon as
Elasticsearch answers.
After ES_RETRY_MAX_WAIT they are marked 'failed', like any indexing error.
SIGTERM, SIGINT,
or the parent daemon disappearing while waiting marks the jobs 'failed' and the
worker
exits. This works for STOMP and polling, and does not depend on the polling
change in
bug 43729.
Errors are classified by is_connection_error(): NoNodes, Timeout, Cxn and
Unavailable, plus
transport failures reported with the pseudo status 599 (HTTP::Tiny), which
Search::Elasticsearch reports as type Request when an open connection is reset.
Real
indexing errors still mark the jobs 'failed'.
The indexing of a batch moves to index_batch() so it can be called again. This
overlaps
with bug 43569, which changes the same lines of the daemon; the conflict is
mechanical.
Depends on bug 44022: without it a worker that has seen one failed request
rejects every
healthy response afterwards and never recovers.
Tested with the daemon from main and a real STOMP broker (ktd): outage -> the
job stays
'started' and finishes after Elasticsearch is back; ES_RETRY_MAX_WAIT reached
-> 'failed';
SIGTERM to the worker, or only to the parent -> 'failed' and the worker exits.
Not tested:
authority batches.
--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
