Re: [Koha] Zebra rebuild frequency
Paul A schreef op vr 14-11-2014 om 19:53 [-0500]: It's been running every minute for two years ;=} using Under older versions, there was a chance of two zebra reindexes crashing into each other, and I think that if this happened, it could be possible that a change would get lost. Running every minute would increase the chances of that happening. Newer versions (3.16 I think) lock the reindexing process so two runs can't collide. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
Paul, could you confirm that you use Zebra in DOM mode? It seems Zebra isn't configured properly, and so doesn't know where to pick up unique ID for biblio records (biblionumber field). Without an ID to identify biblio records, Zebra can't neither delete nor update a biblio record. Check your Zebra index definition file: biblio-koha-indexdefs.xml. At the beginning of the file, you should have a line like this one: idmarc:datafield[@tag='999']/marc:subfield[@code='c']/id My assumption is that you haven't this. This could happen if the upgrade isn't done properly, for a reason or another. I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. It works for the vast majority of the Koha crowd. Zebra is obviously designed to be able to add/edit/delete records. It's a minimum :-) You have to figure out what's wrong with your installation. Kind regards, -- ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
Paul's XSLT was using 001 for the id. El sáb, nov 15, 2014 05:39 AM, Frédéric Demians frede...@tamil.fr escribió: Paul, could you confirm that you use Zebra in DOM mode? It seems Zebra isn't configured properly, and so doesn't know where to pick up unique ID for biblio records (biblionumber field). Without an ID to identify biblio records, Zebra can't neither delete nor update a biblio record. Check your Zebra index definition file: biblio-koha-indexdefs.xml. At the beginning of the file, you should have a line like this one: idmarc:datafield[@tag='999']/marc:subfield[@code='c']/id My assumption is that you haven't this. This could happen if the upgrade isn't done properly, for a reason or another. I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. It works for the vast majority of the Koha crowd. Zebra is obviously designed to be able to add/edit/delete records. It's a minimum :-) You have to figure out what's wrong with your installation. Kind regards, -- ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
I see a couple situations when you could need a reindex: - An upgrade explicitly states the need for it - Changes on the index definitions - Changing from GRS-1 to DOM indexing - Weird/unexpected search results The latter means something is going wrong. In that situation, please report back to us, because: - It can affect another users that didn't already noticed it - We could be able to fix it On 3.18 you'll be able to even disable the cronjob as there will be an indexing daemon running. Regards Tomás PS: Report back means sending an email to the list, connecting to the IRC channel, or even filling a bug. I'd recommend IRC for a better and more effective report/feedback interaction, and we could end up filling a bug for it. El Thu Nov 13 2014 at 20:15:36, Robin Sheat (ro...@catalyst.net.nz) escribió: Steven Nickerson schreef op do 13-11-2014 om 07:42 [-0500]: Aside from this rebuild, is there some other complete Zebra rebuild recommended, and if so on what frequency? If you're seeing strange search results, in that they differ from what the detail page says, then do a reindex. Instructions are in the FAQ: http://koha-community.org/faqs/zebra-indexing-wont-work- fix-it-aka-search-stuff-up-help/ you can mostly jump to the using the packages bit. As for how often, we don't tend to do it unless things are looking strange. If things _are_ looking strange, then it's often the first thing to do. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
At 01:42 PM 11/14/2014 +, Tomas Cohen Arazi wrote: I see a couple situations when you could need a reindex: - An upgrade explicitly states the need for it - Changes on the index definitions - Changing from GRS-1 to DOM indexing - Weird/unexpected search results 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? Best -- Paul The latter means something is going wrong. In that situation, please report back to us, because: - It can affect another users that didn't already noticed it - We could be able to fix it On 3.18 you'll be able to even disable the cronjob as there will be an indexing daemon running. Regards Tomás PS: Report back means sending an email to the list, connecting to the IRC channel, or even filling a bug. I'd recommend IRC for a better and more effective report/feedback interaction, and we could end up filling a bug for it. El Thu Nov 13 2014 at 20:15:36, Robin Sheat (ro...@catalyst.net.nz) escribió: Steven Nickerson schreef op do 13-11-2014 om 07:42 [-0500]: Aside from this rebuild, is there some other complete Zebra rebuild recommended, and if so on what frequency? If you're seeing strange search results, in that they differ from what the detail page says, then do a reindex. Instructions are in the FAQ: http://koha-community.org/faqs/zebra-indexing-wont-work- fix-it-aka-search-stuff-up-help/ you can mostly jump to the using the packages bit. As for how often, we don't tend to do it unless things are looking strange. If things _are_ looking strange, then it's often the first thing to do. -- Robin Sheat Catalyst IT Ltd. â +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. http://NavalMarineArchive.com and http://UltraMarine.ca ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
In my instance(s), I have few enough records that I just run a full re-index daily. There have been a few times along the way where incremental indexing might manage to miss a record. Doing a full index regularly during the overnight hours just makes sure everything is in there. Probably not necessary, but I can't see where it hurts. Tom On Fri, Nov 14, 2014 at 10:36 AM, Owen Leonard oleon...@myacpl.org wrote: 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha -- *Tom Hanstra* *Sr. Systems Administrator* hans...@nd.edu http://library.nd.edu/ ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
Paul, I'm sorry you suffered that on your Koha, but I've never seen it. I'm only aware of a situation that could trigger that on NORMARC (recently fixed) using DOM indexing. Otherwise, we've used 3.0, 3.8 and 3.12 without any similar behaviour. El Fri Nov 14 2014 at 12:50:02, Tom Hanstra (hans...@nd.edu) escribió: In my instance(s), I have few enough records that I just run a full re-index daily. There have been a few times along the way where incremental indexing might manage to miss a record. Doing a full index regularly during the overnight hours just makes sure everything is in there. Probably not necessary, but I can't see where it hurts. Tom On Fri, Nov 14, 2014 at 10:36 AM, Owen Leonard oleon...@myacpl.org wrote: 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha -- *Tom Hanstra* *Sr. Systems Administrator* hans...@nd.edu http://library.nd.edu/ ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
At 04:20 PM 11/14/2014 +, Tomas Cohen Arazi wrote: Paul, I'm sorry you suffered that on your Koha, but I've never seen it. I'm only aware of a situation that could trigger that on NORMARC (recently fixed) using DOM indexing. [snip] 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. [snip] It is true on our production box (and its backup; sandbox in other use) Koha: 3.08.05.000 on Linux 3.2.0-31-generic (12.04LTS), Perl version: 5.014002, MySQL version: mysql 5.5.24, Apache version: 2.2.22, Zebra: 2.0.44 Pls see http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=kent+beyond+reef, two biblios for Kent, Beyond the reef: one is good, the other has been deleted, gives a 404 on the OPAC page, but will not go away from Zebra by cron job alone (it will go away next time I CLI re-index.) I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. Slightly different case, http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=steele+worcester -- this was produced by overwriting an existing biblio (old, incomplete) with a new Z39.50, and adding an item -- it's the same biblio, but the dead instance will not disappear until I re-index. This is a further example of the (for us) need to CLI re-index as the cronjob does not handle it. Best -- Paul --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. http://NavalMarineArchive.com and http://UltraMarine.ca ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
Greetings, And what cronjobs do you currently have? Perhaps you don't even have your partial reindex cronjob installed? GPML, Mark Tompsett ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
On 2014-11-15, at 9:49 AM, Paul A wrote: At 04:20 PM 11/14/2014 +, Tomas Cohen Arazi wrote: Paul, I'm sorry you suffered that on your Koha, but I've never seen it. I'm only aware of a situation that could trigger that on NORMARC (recently fixed) using DOM indexing. [snip] 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. [snip] It is true on our production box (and its backup; sandbox in other use) Koha: 3.08.05.000 on Linux 3.2.0-31-generic (12.04LTS), Perl version: 5.014002, MySQL version: mysql 5.5.24, Apache version: 2.2.22, Zebra: 2.0.44 Pls see http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=kent+beyond+reef, two biblios for Kent, Beyond the reef: one is good, the other has been deleted, gives a 404 on the OPAC page, but will not go away from Zebra by cron job alone (it will go away next time I CLI re-index.) I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. Slightly different case, http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=steele+worcester -- this was produced by overwriting an existing biblio (old, incomplete) with a new Z39.50, and adding an item -- it's the same biblio, but the dead instance will not disappear until I re-index. This is a further example of the (for us) need to CLI re-index as the cronjob does not handle it. you should show an example of your cron file - its probably just misconfigured? ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
At 12:53 PM 11/15/2014 +1300, Mason James wrote: On 2014-11-15, at 9:49 AM, Paul A wrote: At 04:20 PM 11/14/2014 +, Tomas Cohen Arazi wrote: Paul, I'm sorry you suffered that on your Koha, but I've never seen it. I'm only aware of a situation that could trigger that on NORMARC (recently fixed) using DOM indexing. [snip] 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. [snip] It is true on our production box (and its backup; sandbox in other use) Koha: 3.08.05.000 on Linux 3.2.0-31-generic (12.04LTS), Perl version: 5.014002, MySQL version: mysql 5.5.24, Apache version: 2.2.22, Zebra: 2.0.44 Pls see http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=kent+beyond+reef, two biblios for Kent, Beyond the reef: one is good, the other has been deleted, gives a 404 on the OPAC page, but will not go away from Zebra by cron job alone (it will go away next time I CLI re-index.) I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. Slightly different case, http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=steele+worcester -- this was produced by overwriting an existing biblio (old, incomplete) with a new Z39.50, and adding an item -- it's the same biblio, but the dead instance will not disappear until I re-index. This is a further example of the (for us) need to CLI re-index as the cronjob does not handle it. you should show an example of your cron file - its probably just misconfigured? It's been running every minute for two years ;=} using KOHA_CONF=/etc/koha/koha-conf.xml KOHAPATH=/usr/share/koha PERL5LIB=$KOHAPATH/lib */1 * * * *koha$KOHAPATH/bin/migration_tools/rebuild_zebra.pl -a -b -z /dev/null 21 paul@nelson:/$ grep CRON /var/log/syslog [snip]Nov 14 21:22:01 nelson CRON[23795]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:23:01 nelson CRON[23842]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:24:01 nelson CRON[23930]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:25:01 nelson CRON[24054]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:26:01 nelson CRON[24139]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:27:01 nelson CRON[24182]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:28:02 nelson CRON[24310]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Best -- Paul --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. http://NavalMarineArchive.com and http://UltraMarine.ca ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
On 2014-11-15, at 1:53 PM, Paul A wrote: At 12:53 PM 11/15/2014 +1300, Mason James wrote: On 2014-11-15, at 9:49 AM, Paul A wrote: At 04:20 PM 11/14/2014 +, Tomas Cohen Arazi wrote: Paul, I'm sorry you suffered that on your Koha, but I've never seen it. I'm only aware of a situation that could trigger that on NORMARC (recently fixed) using DOM indexing. [snip] 3.8.5 (production) and from my test notes 3.12.various and 3.14.2 required a complete biblio re-index after deleting a biblio record. Has this been fixed? I don't think that was ever true. [snip] It is true on our production box (and its backup; sandbox in other use) Koha: 3.08.05.000 on Linux 3.2.0-31-generic (12.04LTS), Perl version: 5.014002, MySQL version: mysql 5.5.24, Apache version: 2.2.22, Zebra: 2.0.44 Pls see http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=kent+beyond+reef, two biblios for Kent, Beyond the reef: one is good, the other has been deleted, gives a 404 on the OPAC page, but will not go away from Zebra by cron job alone (it will go away next time I CLI re-index.) I have a feeling that when zebra says incremental it means just that -- adding new data, but not deleting what has gone from MySQL. Slightly different case, http://opac.navalmarinearchive.com/cgi-bin/koha/opac-search.pl?q=steele+worcester -- this was produced by overwriting an existing biblio (old, incomplete) with a new Z39.50, and adding an item -- it's the same biblio, but the dead instance will not disappear until I re-index. This is a further example of the (for us) need to CLI re-index as the cronjob does not handle it. you should show an example of your cron file - its probably just misconfigured? It's been running every minute for two years ;=} using KOHA_CONF=/etc/koha/koha-conf.xml KOHAPATH=/usr/share/koha PERL5LIB=$KOHAPATH/lib */1 * * * *koha$KOHAPATH/bin/migration_tools/rebuild_zebra.pl -a -b -z /dev/null 21 paul@nelson:/$ grep CRON /var/log/syslog [snip]Nov 14 21:22:01 nelson CRON[23795]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:23:01 nelson CRON[23842]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:24:01 nelson CRON[23930]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:25:01 nelson CRON[24054]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:26:01 nelson CRON[24139]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:27:01 nelson CRON[24182]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Nov 14 21:28:02 nelson CRON[24310]: (koha) CMD (/usr/share/koha/bin/migration_tools/rebuild_zebra.pl -z -b -a /tmp/cron_koha.log 21) Best -- Paul try using the -v arg, it will show you what it is actually doing $ /usr/share/koha/bin/migration_tools/rebuild_zebra.pl -a -b -z -v Zebra configuration information Zebra biblio directory = /var/lib/koha/mastcaro/biblios Zebra authorities directory = /var/lib/koha/mastcaro/authorities Koha directory = /usr/share/koha Lockfile= /var/lock/zebra_koha_mastcaro/rebuild/rebuild..LCK BIBLIONUMBER in : 999$c BIBLIOITEMNUMBER in : 999$d exporting authority Records exported: 0 Records exported: 0 REINDEXING zebra exporting biblio 1. Records exported: 1 DELETED BIB, REMOVED FROM ZEBRA Records exported: 0 REINDEXING zebra CLEANING $ ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
[Koha] Zebra rebuild frequency
Hi everyone, I know the topic of Zebra index rebuilding comes up periodically and I've tried to find an answer to this but can't seem to locate one. I have the package version of 3.16.02 installed on Debian and have the following default cron entry in /etc/cron.d/koha-common: */5 * * * * root test -x /usr/sbin/koha-rebuild-zebra koha-rebuild-zebra -q $(koha-list --enabled) Aside from this rebuild, is there some other complete Zebra rebuild recommended, and if so on what frequency? Thanks! Steve ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Re: [Koha] Zebra rebuild frequency
Steven Nickerson schreef op do 13-11-2014 om 07:42 [-0500]: Aside from this rebuild, is there some other complete Zebra rebuild recommended, and if so on what frequency? If you're seeing strange search results, in that they differ from what the detail page says, then do a reindex. Instructions are in the FAQ: http://koha-community.org/faqs/zebra-indexing-wont-work-fix-it-aka-search-stuff-up-help/ you can mostly jump to the using the packages bit. As for how often, we don't tend to do it unless things are looking strange. If things _are_ looking strange, then it's often the first thing to do. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF ___ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
