https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43020
Bug ID: 43020
Summary: One bad record can abort an entire OAI-PMH harvest
Initiative type: ---
Sponsorship ---
status:
Product: Koha
Version: Main
Hardware: All
OS: All
Status: NEW
Severity: normal
Priority: P5 - low
Component: Cataloging
Assignee: [email protected]
Reporter: [email protected]
QA Contact: [email protected]
CC: [email protected]
Depends on: 35104
Target Milestone: ---
Koha::OAI::Client::Harvester::processRecord has no error handling around
MARC::Record->new_from_xml when decoding a harvested record's metadata, and
no handling for Koha::Exceptions::Metadata::Invalid around the subsequent
AddBiblio/ModBiblio/AddAuthority/ModAuthority calls either. Since init()'s
harvest loop has no try/catch around processRecord, either failure dies
uncaught, aborting the entire harvest_oai.pl cron run partway through - no
completion log entry, no report email, and every remaining record in that
batch is left unprocessed.
This was found while auditing other places in the codebase that ingest
externally-sourced MARC data, prompted by bug 35104 (which fixed the same
class of problem - one bad record silently aborting a whole batch/job -
in the staged MARC import commit/revert code paths).
Test plan:
1. Configure an OAI-PMH server under Administration > OAI-PMH harvesting
servers, with a record type of biblio.
2. Arrange for one harvested record to have metadata that cannot be decoded
as MARCXML (e.g. an empty <datafield> element, or any malformed MARC).
3. Run: misc/cronjobs/harvest_oai.pl -r <server id>
4. Before this patch: the cron job dies partway through, with no harvest
report and no indication of which record caused it.
5. After this patch: the harvest completes. The bad record is counted as
in_error in the harvest report/email, its identifier and decoding error
are logged, and every other record in the batch is still processed
normally.
6. prove t/db_dependent/Koha/OAIHarvester.t
Referenced Bugs:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35104
[Bug 35104] We should warn when attempting to save MARC records that contain
characters invalid in XML
--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[email protected]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
