[Koha-devel] Record indexer integration (Zebra)
An effort is being made to update the INSTALL.* files and to aid our users in running Koha almost auto-magically (read .deb install method, copy and paste instructions for src install in debian/ubuntu, etc). A missing piece (from my point of view) is the integration of the record indexer daemon into Koha (currently the default is to reindex -z every 5', and there are links to using Koha::Contrib::Tamil tools for having a *daemon* to do it). I vote for doing an effort to have Koha::Contrib::Tamil integrated into Koha. Once it is in the source tree this will: - Avoid Koha version-specific incompatibilities (read ease of maintenance). - Provide a simple way (SysV) to run the daemon. - Make it possible for the daemon to evolve (not sure, but I read that jcamins had some patches to adapt the Koha::Contrib::Tamil to the koha-common package multi-instance schema). Even if we move to a SolR/Zebra/Whatever: if we support Zebra it will be needed. As spoken with FrédericD, this implementation of the indexer lacks some features (?) as record sanitization, or ISO record indexing instead of MARCXML, but maybe we conclude it is not needed anymore there. Or it could be reintroduced once is in the source tree. As I see it, once it is integrated in the Koha source its easier to patch and make it evolve. Regards To+ ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Hi all, Just for information: We (at BibLibre) have developped an indexer daemon (named IndexRecordQueue). It's available on our git repository (branch dev/solr). Direct link: http://git.biblibre.com/?p=koha;a=blob;f=misc/solr/IndexRecordQueue.pl;h=9f1b03ba538eeab80f8261c1b9b78d14d075732f;hb=2175b9241de5c7a66b94de2abc84f2eb9db2e83a I can do some doc if someone is interested. Regards, Jonathan 2012/7/25 Tomas Cohen Arazi > An effort is being made to update the INSTALL.* files and to aid our > users in running Koha almost auto-magically (read .deb install method, > copy and paste instructions for src install in debian/ubuntu, etc). > > A missing piece (from my point of view) is the integration of the > record indexer daemon into Koha (currently the default is to reindex > -z every 5', and there are links to using Koha::Contrib::Tamil tools > for having a *daemon* to do it). > > I vote for doing an effort to have Koha::Contrib::Tamil integrated > into Koha. Once it is in the source tree this will: > > - Avoid Koha version-specific incompatibilities (read ease of maintenance). > - Provide a simple way (SysV) to run the daemon. > - Make it possible for the daemon to evolve (not sure, but I read that > jcamins had some patches to adapt the Koha::Contrib::Tamil to the > koha-common package multi-instance schema). Even if we move to a > SolR/Zebra/Whatever: if we support Zebra it will be needed. > > As spoken with FrédericD, this implementation of the indexer lacks > some features (?) as record sanitization, or ISO record indexing > instead of MARCXML, but maybe we conclude it is not needed anymore > there. Or it could be reintroduced once is in the source tree. > > As I see it, once it is integrated in the Koha source its easier to > patch and make it evolve. > > Regards > To+ > ___ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Hi Jonathan, Il 26/07/2012 11:32, Jonathan Druart ha scritto: > Hi all, > > Just for information: We (at BibLibre) have developped an indexer daemon > (named IndexRecordQueue). >> It's available on our git repository (branch dev/solr). > > I can do some doc if someone is interested. this script is only for solr or do you think can be used also for Zebra ? Bye Zeno -- Dott. Zeno Tajoli tajoliAT_SPAM_no_prendiATcilea.it fax +39 02 2135520 CILEA - Consorzio Interuniversitario http://www.cilea.it/disclaimer ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Hi Tomas, Il 25/07/2012 15:38, Tomas Cohen Arazi ha scritto: > An effort is being made to update the INSTALL.* files and to aid our > users in running Koha almost auto-magically (read .deb install method, > copy and paste instructions for src install in debian/ubuntu, etc). > > A missing piece (from my point of view) is the integration of the > record indexer daemon into Koha (currently the default is to reindex > -z every 5', and there are links to using Koha::Contrib::Tamil tools > for having a *daemon* to do it). I have done the doc and wiki modifications to insert the links to Koha::Contrib::Tamil tools. So for me is clarly of to insert them in main Koha git. Attention that it need many dependencies. Bye Zeno Tajoli -- Dott. Zeno Tajoli tajoliAT_SPAM_no_prendiATcilea.it fax +39 02 2135520 CILEA - Consorzio Interuniversitario http://www.cilea.it/disclaimer ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Zeno, To adapt for Zebra, I think you just have to modify the routine sub launch_indexation Currently, it calls C4::Search::Engine::Solr::IndexRecord( $recordtype, $recordids ); 2012/7/26 Zeno Tajoli > Hi Jonathan, > > Il 26/07/2012 11:32, Jonathan Druart ha scritto: > > Hi all, > > > > Just for information: We (at BibLibre) have developped an indexer daemon > > (named IndexRecordQueue). > >> It's available on our git repository (branch dev/solr). > > > > I can do some doc if someone is interested. > > this script is only for solr or do you think can be used also for Zebra ? > > Bye > Zeno > -- > Dott. Zeno Tajoli > tajoliAT_SPAM_no_prendiATcilea.it > fax +39 02 2135520 > CILEA - Consorzio Interuniversitario > http://www.cilea.it/disclaimer > ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Can we move Search::Engine::Solr into Koha:: namespace and create a Zebra version that implements the same API? (havent read the code yet, sorry). If there was a consensus on the way Koha::Contrib::Tamil does the indexing, the algorithms could be reused. Once we had this, we could introduce some more scripts/files or instructions so we have the indexing daemon ready for production, OOTB. I think this is the *only* obscure part in the Koha deployment, that we could have solved a while ago. No matter which approach/API is chosen. Regards To+ On Thu, Jul 26, 2012 at 7:24 AM, Jonathan Druart wrote: > Zeno, > To adapt for Zebra, I think you just have to modify the routine > sub launch_indexation > > Currently, it calls C4::Search::Engine::Solr::IndexRecord( $recordtype, > $recordids ); > > > 2012/7/26 Zeno Tajoli >> >> Hi Jonathan, >> >> Il 26/07/2012 11:32, Jonathan Druart ha scritto: >> > Hi all, >> > >> > Just for information: We (at BibLibre) have developped an indexer daemon >> > (named IndexRecordQueue). >> >> It's available on our git repository (branch dev/solr). >> > >> > I can do some doc if someone is interested. >> >> this script is only for solr or do you think can be used also for Zebra ? >> >> Bye >> Zeno >> -- >> Dott. Zeno Tajoli >> tajoliAT_SPAM_no_prendiATcilea.it >> fax +39 02 2135520 >> CILEA - Consorzio Interuniversitario >> http://www.cilea.it/disclaimer > > > > ___ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
>Can we move Search::Engine::Solr into Koha:: namespace and create a >Zebra version that implements the same API? (havent read the code yet, >sorry). Have a loot at Bug 8233 ( http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8233) and the wiki page : http://wiki.koha-community.org/wiki/SearchEngine_Layer_RFC Work in progress ;-) 2012/7/26 Tomas Cohen Arazi > Can we move Search::Engine::Solr into Koha:: namespace and create a > Zebra version that implements the same API? (havent read the code yet, > sorry). If there was a consensus on the way Koha::Contrib::Tamil does > the indexing, the algorithms could be reused. Once we had this, we > could introduce some more scripts/files or instructions so we have the > indexing daemon ready for production, OOTB. > > I think this is the *only* obscure part in the Koha deployment, that > we could have solved a while ago. No matter which approach/API is > chosen. > > Regards > To+ > > > On Thu, Jul 26, 2012 at 7:24 AM, Jonathan Druart > wrote: > > Zeno, > > To adapt for Zebra, I think you just have to modify the routine > > sub launch_indexation > > > > Currently, it calls C4::Search::Engine::Solr::IndexRecord( $recordtype, > > $recordids ); > > > > > > 2012/7/26 Zeno Tajoli > >> > >> Hi Jonathan, > >> > >> Il 26/07/2012 11:32, Jonathan Druart ha scritto: > >> > Hi all, > >> > > >> > Just for information: We (at BibLibre) have developped an indexer > daemon > >> > (named IndexRecordQueue). > >> >> It's available on our git repository (branch dev/solr). > >> > > >> > I can do some doc if someone is interested. > >> > >> this script is only for solr or do you think can be used also for Zebra > ? > >> > >> Bye > >> Zeno > >> -- > >> Dott. Zeno Tajoli > >> tajoliAT_SPAM_no_prendiATcilea.it > >> fax +39 02 2135520 > >> CILEA - Consorzio Interuniversitario > >> http://www.cilea.it/disclaimer > > > > > > > > ___ > > Koha-devel mailing list > > Koha-devel@lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Re: [Koha-devel] Record indexer integration (Zebra)
Is this going to be on master soon? On Thu, Jul 26, 2012 at 6:32 AM, Jonathan Druart wrote: > Hi all, > > Just for information: We (at BibLibre) have developped an indexer daemon > (named IndexRecordQueue). > It's available on our git repository (branch dev/solr). > > Direct link: > http://git.biblibre.com/?p=koha;a=blob;f=misc/solr/IndexRecordQueue.pl;h=9f1b03ba538eeab80f8261c1b9b78d14d075732f;hb=2175b9241de5c7a66b94de2abc84f2eb9db2e83a > > I can do some doc if someone is interested. > > Regards, > Jonathan > > 2012/7/25 Tomas Cohen Arazi >> >> An effort is being made to update the INSTALL.* files and to aid our >> users in running Koha almost auto-magically (read .deb install method, >> copy and paste instructions for src install in debian/ubuntu, etc). >> >> A missing piece (from my point of view) is the integration of the >> record indexer daemon into Koha (currently the default is to reindex >> -z every 5', and there are links to using Koha::Contrib::Tamil tools >> for having a *daemon* to do it). >> >> I vote for doing an effort to have Koha::Contrib::Tamil integrated >> into Koha. Once it is in the source tree this will: >> >> - Avoid Koha version-specific incompatibilities (read ease of >> maintenance). >> - Provide a simple way (SysV) to run the daemon. >> - Make it possible for the daemon to evolve (not sure, but I read that >> jcamins had some patches to adapt the Koha::Contrib::Tamil to the >> koha-common package multi-instance schema). Even if we move to a >> SolR/Zebra/Whatever: if we support Zebra it will be needed. >> >> As spoken with FrédericD, this implementation of the indexer lacks >> some features (?) as record sanitization, or ISO record indexing >> instead of MARCXML, but maybe we conclude it is not needed anymore >> there. Or it could be reintroduced once is in the source tree. >> >> As I see it, once it is integrated in the Koha source its easier to >> patch and make it evolve. >> >> Regards >> To+ >> ___ >> Koha-devel mailing list >> Koha-devel@lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > ___ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
