Re: [sword-devel] Location of prebuilt lucene indexes.
*And Bible* allows the user to create the search index for a module if it does not already exist. This is useful in these situations: (a) The user is currently not [reliably] connected to the Internet, (b) Martin has not yet made the downloadable index for new/updated modules. And Bible does not have a UI option to remove a search index, though this could be done manually by means of the file system, assuming you know where to look within the Android directory structure. David -- View this message in context: http://sword-dev.350566.n4.nabble.com/Location-of-prebuilt-lucene-indexes-tp4651756p4651788.html Sent from the SWORD Dev mailing list archive at Nabble.com. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Hi David, To delete an index in And Bible you long-press the desired module in the Documents list to reveal module and index delete actions in the context menu. Martin On 24 January 2013 12:36, David Haslam dfh...@googlemail.com wrote: *And Bible* allows the user to create the search index for a module if it does not already exist. This is useful in these situations: (a) The user is currently not [reliably] connected to the Internet, (b) Martin has not yet made the downloadable index for new/updated modules. And Bible does not have a UI option to remove a search index, though this could be done manually by means of the file system, assuming you know where to look within the Android directory structure. David -- View this message in context: http://sword-dev.350566.n4.nabble.com/Location-of-prebuilt-lucene-indexes-tp4651756p4651788.html Sent from the SWORD Dev mailing list archive at Nabble.com. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Thanks all for your replies. I am using Lucene.net which is a .net port of the java Lucene. I have been able to consume an index created by xiphos which I assume uses clucene. However based on the replies here I think I will generate my own indexes to avoid and compatibility issues. Daniel Hughes On Fri, Jan 25, 2013 at 2:10 AM, Martin Denham mjden...@gmail.com wrote: Hi David, To delete an index in And Bible you long-press the desired module in the Documents list to reveal module and index delete actions in the context menu. Martin On 24 January 2013 12:36, David Haslam dfh...@googlemail.com wrote: *And Bible* allows the user to create the search index for a module if it does not already exist. This is useful in these situations: (a) The user is currently not [reliably] connected to the Internet, (b) Martin has not yet made the downloadable index for new/updated modules. And Bible does not have a UI option to remove a search index, though this could be done manually by means of the file system, assuming you know where to look within the Android directory structure. David -- View this message in context: http://sword-dev.350566.n4.nabble.com/Location-of-prebuilt-lucene-indexes-tp4651756p4651788.html Sent from the SWORD Dev mailing list archive at Nabble.com. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Hi Daniel, I put the question to Martin Denham (the developer of And Bible). He replied: (lightly edited by me) And Bible's indexes are here: http://www.crosswire.org/and-bible/indices/v1/ but they are only guaranteed to work with And Bible and almost definitely won't work with SWORD C++ front ends. PocketSword's indices are located separately and are incompatible. DMSmith has all the complex details on why they won't work with other apps. I based the AB downloadable indices framework on PocketSword and here are the developer's comments during our initial conversation: - clucene is version dependent, so you need to use the same version of clucene as I am. - there is a different index per module version, so if a module is updated to v1.2 from v1.1, I need to rebuild the index. - if a module doesn't have a version number, I've given it 0.0 as the version number. - the ZIP files are named MODULE_NAME-VERSION.zip (eg KJV-1.3.zip) - the URL is http://www.crosswire.org/pocketsword/indices/v1/ - I am planning on updating SWORD to use the latest version of clucene at some point, which is why there is currently the v1 in the URL -- each time I update clucene in PocketSword, I will bump that number, so people using the new version of PS will have new indexes, but people still on the old version will still have the old ones available. - of course, if clucene is updated in PS, that new version of PS will have to go through and delete all the old lucene folders in each of the modules installed, as they will no longer be valid! :) David Haslam -- View this message in context: http://sword-dev.350566.n4.nabble.com/Location-of-prebuilt-lucene-indexes-tp4651756p4651763.html Sent from the SWORD Dev mailing list archive at Nabble.com. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Daniel, the indexes themselves are created using the utility mkfastmod. If built, the lucene indexes typically sit off of the modules directory directly .. ie: .../modules/texts/ztext/kjv/lucene Administrators of a repo, you can generate the lucene directories, and have them automatically associated with the modules you are sharing by generating the index and placing it directly off the module location in your repo, ie: /pub/sword/raw/modules/texts/ztext/kjv/lucene (you can use rsync -av to copy a lucene directory from one location to another). If you look at many of the modules in the Institute for Scripture Research repo, those repo administrators have done this already (look at modules tbn, tkci, tkli, ttr, uzv for example). The nice thing about when repo administrators do this, is when you get a module from that repo, you also automatically get the lucene index. I do the same for my own repo. If Crosswire were to generate lucene indexes for all of the modules they host, all module downloads would automatically come with them (at least this seems to be my experience using pocketsword - only the modules that have had these directories created seem to have search index's automatic). Even so, you can generate these indexes yourself. ~A On Wed, Jan 23, 2013 at 1:12 AM, Daniel Hughes tramps...@gmail.com wrote: I noticed that some of the front ends offer the option of downloading prebuilt lucene indexes. However I can't seem to find them on ftp://ftp.crosswire.org Where are they hosted? ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Von: Andrew Thule thules...@gmail.com The nice thing about when repo administrators do this, is when you get a module from that repo, you also automatically get the lucene index. The reason that this is not done are manifold - including that sword and jsword indices are different, that not all platforms have the same lucene/clucene incarnation etc. Most users are better off not to use downloaded indices, but to create indices suitable for their specific frontend. Peter ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
On Jan 23, 2013, at 11:02 AM, Peter von Kaehne ref...@gmx.net wrote: Von: Andrew Thule thules...@gmail.com The nice thing about when repo administrators do this, is when you get a module from that repo, you also automatically get the lucene index. The reason that this is not done are manifold - including that sword and jsword indices are different, that not all platforms have the same lucene/clucene incarnation etc. Most users are better off not to use downloaded indices, but to create indices suitable for their specific frontend. To add to what Peter said, this mailing list's archives have extensive discussion on why we don't do this and why we have done it for mobile devices (only PocketSword and AndBible, and specific to them only). Also, the transfer time for a lucene index exceeds the time of a recent computer to build them, e.g. my laptop. As a Bible Desktop user, I'd really rather not transfer an index that is unusable by Bible Desktop. In Him, DM ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Peter, DM, Thanks for your response. I wasn't being critical and I agree. There are Pros and Cons either way (resource usage and download time being considerations) as you point out. I also agree it's ultimately it's up to the repo administrator and the user, to decide for themselves what works best for them. I happen to like having the index included as part of the repo, because I'm not limited by resources, and I don't mind the wait. Regardless, it is still a matter of preference (based upon considerations, such as the ones you cite). ~A On Wednesday, January 23, 2013, DM Smith wrote: On Jan 23, 2013, at 11:02 AM, Peter von Kaehne ref...@gmx.netjavascript:; wrote: Von: Andrew Thule thules...@gmail.com javascript:; The nice thing about when repo administrators do this, is when you get a module from that repo, you also automatically get the lucene index. The reason that this is not done are manifold - including that sword and jsword indices are different, that not all platforms have the same lucene/clucene incarnation etc. Most users are better off not to use downloaded indices, but to create indices suitable for their specific frontend. To add to what Peter said, this mailing list's archives have extensive discussion on why we don't do this and why we have done it for mobile devices (only PocketSword and AndBible, and specific to them only). Also, the transfer time for a lucene index exceeds the time of a recent computer to build them, e.g. my laptop. As a Bible Desktop user, I'd really rather not transfer an index that is unusable by Bible Desktop. In Him, DM ___ sword-devel mailing list: sword-devel@crosswire.org javascript:; http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Von: Andrew Thule thules...@gmail.com There are Pros and Cons either way (resource usage and download time being considerations) as you point out. There are no pros and only contras. And I certainly have not pointed out any pros. Nor has DM. Please stop using my name as supposedly in agreement with you. I find this offensive. FWIW - If you run a repo for anyone other than yourself with indeces included, you will create a significant risk of malfunction. For anyone other than the user of the specific version of libsword and clucene. As such no one expects from you anything anymore, so, further explanations are probably pointless. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
To add to the discussion. Step ships data files and later indexes as part of the installation of software plus automatically after installation of any module. We considered shipping indexes but decided against it. This is because indexes can become corrupt quite easily and can be incompatible for no obvious reasons. We also want to be able to update data later with a lot of people having bad Internet connections. Since we're using Lucene as our data store as well as our indexing engine we need to ship most of the text data anyway. So functionality needs to be there anyway to manage indexing. As DM says it can often be done in background very fast. Chris On 23 Jan 2013 18:20, Peter von Kaehne ref...@gmx.net wrote: Von: Andrew Thule thules...@gmail.com There are Pros and Cons either way (resource usage and download time being considerations) as you point out. There are no pros and only contras. And I certainly have not pointed out any pros. Nor has DM. Please stop using my name as supposedly in agreement with you. I find this offensive. FWIW - If you run a repo for anyone other than yourself with indeces included, you will create a significant risk of malfunction. For anyone other than the user of the specific version of libsword and clucene. As such no one expects from you anything anymore, so, further explanations are probably pointless. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Peter, if I offended you, I hope you'll forgive me. ~A On Wednesday, January 23, 2013, Peter von Kaehne wrote: Von: Andrew Thule thules...@gmail.com javascript:; There are Pros and Cons either way (resource usage and download time being considerations) as you point out. There are no pros and only contras. And I certainly have not pointed out any pros. Nor has DM. Please stop using my name as supposedly in agreement with you. I find this offensive. FWIW - If you run a repo for anyone other than yourself with indeces included, you will create a significant risk of malfunction. For anyone other than the user of the specific version of libsword and clucene. As such no one expects from you anything anymore, so, further explanations are probably pointless. ___ sword-devel mailing list: sword-devel@crosswire.org javascript:; http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
My suggestion would be to avoid pre-built indices. However, given I'm the one who first went about making them, I think there are pros in certain situations. :P In my situation, I get to be gatekeeper over the entire solution (PocketSword), where users cannot change the version of clucene or libsword behind my back. And so I can be aware of when things might break ( increment my v1 bit of the download URL of the indices). BTW, I would strongly suggest that no one else use either the and-bible or PocketSword pre-built indices due to incompatibilities that have been explained in other posts. :) I'd also strongly suggest that no one else creates new ones for another platform, unless it's absolutely necessary. They are messy, as you can probably gather from all these posts! :) Thanks, ybic nic... :) On 24/01/2013, at 5:18 AM, Peter von Kaehne ref...@gmx.net wrote: Von: Andrew Thule thules...@gmail.com There are Pros and Cons either way (resource usage and download time being considerations) as you point out. There are no pros and only contras. And I certainly have not pointed out any pros. Nor has DM. Please stop using my name as supposedly in agreement with you. I find this offensive. FWIW - If you run a repo for anyone other than yourself with indeces included, you will create a significant risk of malfunction. For anyone other than the user of the specific version of libsword and clucene. As such no one expects from you anything anymore, so, further explanations are probably pointless. ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
Re: [sword-devel] Location of prebuilt lucene indexes.
Thanks for making them .. It is appreciated! ... I find them incredibly useful (and I was surprised they worked by way of a repo too!) I am sympathetic to many of the arguements presented against creating them in repos, and I do create them for myself on the desktop (so to speak) so I also see the value in that - that said, I almost entirely depend on the repo indices in PocketSword (a most marvellous tool) where I cannot create them, so cannot entirely see myself giving up that mode. I haven't yet spent much time with the And-bible though I down own an Asus ePad. ~A On Wednesday, January 23, 2013, Nic Carter wrote: My suggestion would be to avoid pre-built indices. However, given I'm the one who first went about making them, I think there are pros in certain situations. :P In my situation, I get to be gatekeeper over the entire solution (PocketSword), where users cannot change the version of clucene or libsword behind my back. And so I can be aware of when things might break ( increment my v1 bit of the download URL of the indices). BTW, I would strongly suggest that no one else use either the and-bible or PocketSword pre-built indices due to incompatibilities that have been explained in other posts. :) I'd also strongly suggest that no one else creates new ones for another platform, unless it's absolutely necessary. They are messy, as you can probably gather from all these posts! :) Thanks, ybic nic... :) On 24/01/2013, at 5:18 AM, Peter von Kaehne ref...@gmx.net javascript:; wrote: Von: Andrew Thule thules...@gmail.com javascript:; There are Pros and Cons either way (resource usage and download time being considerations) as you point out. There are no pros and only contras. And I certainly have not pointed out any pros. Nor has DM. Please stop using my name as supposedly in agreement with you. I find this offensive. FWIW - If you run a repo for anyone other than yourself with indeces included, you will create a significant risk of malfunction. For anyone other than the user of the specific version of libsword and clucene. As such no one expects from you anything anymore, so, further explanations are probably pointless. ___ sword-devel mailing list: sword-devel@crosswire.org javascript:; http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org javascript:; http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
[sword-devel] Location of prebuilt lucene indexes.
I noticed that some of the front ends offer the option of downloading prebuilt lucene indexes. However I can't seem to find them on ftp://ftp.crosswire.org Where are they hosted? ___ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
