Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Sorry, have completely missed this thread until someone pointed out to me just now. Some clarifications: commit=true is for the plugin only. I added it to make it consistent with Solr's behavior. Stand-alone Zoie does not require this, it is in more of an autocommit mode since it assumes a constant and busy stream of indexing events. So I think what we will do is add an autocommit parameter to the ZoieUpdateHandler and default it to true. With this, you don't need to have commit=true to see the new docs. -John On Wed, Apr 14, 2010 at 5:25 PM, qaz dfine.precis...@hotmail.com wrote: do you mean that the plugin for solr doesn't have all the functionalities of the standalone zoie or do you mean zoie just simply cannot handle large indexes? by really really small, what exactly are we talking about here? are there any better ways for NRT? maybe in solr 1.5? thanks -- View this message in context: http://n3.nabble.com/ANN-Zoie-Solr-Plugin-Zoie-Solr-Plugin-enables-real-time-update-functionality-for-Apache-Solr-1-4-tp506099p719893.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
do you mean that the plugin for solr doesn't have all the functionalities of the standalone zoie or do you mean zoie just simply cannot handle large indexes? by really really small, what exactly are we talking about here? are there any better ways for NRT? maybe in solr 1.5? thanks -- View this message in context: http://n3.nabble.com/ANN-Zoie-Solr-Plugin-Zoie-Solr-Plugin-enables-real-time-update-functionality-for-Apache-Solr-1-4-tp506099p719893.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
On Mar 23, 2010, at 7:29 PM, brad anderson wrote: I see, so when you do a commit it adds it to Zoie's ramdirectory. So, could you just commit after every document without having a performance impact and have real time search? Not likely, maybe on really, really small indexes. Zoie also does a writethrough, AIUI, to a file based index. Thanks, Brad On 20 March 2010 00:34, Janne Majaranta janne.majara...@gmail.com wrote: To my understanding it adds a in-memory index which holds the recent commits and which is flushed to the main index based on the config options. Not sure if it helps to get solr near real time. I am evaluating it currently, and I am really not sure if it adds anything because of the cache regeneration of solr on every commit ?? -Janne Lähetetty iPodista brad anderson solrinter...@gmail.com kirjoitti 19.3.2010 kello 20.53: Indeed, which is why I'm wondering what is Zoie adding if you still need to commit to search recent documents. Does anyone know? Thanks, Brad On 18 March 2010 19:41, Erik Hatcher erik.hatc...@gmail.com wrote: When I don't do the commit, I cannot search the documents I've indexed. - that's exactly how Solr without Zoie works, and it's how Lucene itself works. Gotta commit to see the documents indexed. Erik On Mar 18, 2010, at 5:41 PM, brad anderson wrote: Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs. -- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem using Solr/Lucene: http://www.lucidimagination.com/search
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
I see, so when you do a commit it adds it to Zoie's ramdirectory. So, could you just commit after every document without having a performance impact and have real time search? Thanks, Brad On 20 March 2010 00:34, Janne Majaranta janne.majara...@gmail.com wrote: To my understanding it adds a in-memory index which holds the recent commits and which is flushed to the main index based on the config options. Not sure if it helps to get solr near real time. I am evaluating it currently, and I am really not sure if it adds anything because of the cache regeneration of solr on every commit ?? -Janne Lähetetty iPodista brad anderson solrinter...@gmail.com kirjoitti 19.3.2010 kello 20.53: Indeed, which is why I'm wondering what is Zoie adding if you still need to commit to search recent documents. Does anyone know? Thanks, Brad On 18 March 2010 19:41, Erik Hatcher erik.hatc...@gmail.com wrote: When I don't do the commit, I cannot search the documents I've indexed. - that's exactly how Solr without Zoie works, and it's how Lucene itself works. Gotta commit to see the documents indexed. Erik On Mar 18, 2010, at 5:41 PM, brad anderson wrote: Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
To my understanding it adds a in-memory index which holds the recent commits and which is flushed to the main index based on the config options. Not sure if it helps to get solr near real time. I am evaluating it currently, and I am really not sure if it adds anything because of the cache regeneration of solr on every commit ?? -Janne Lähetetty iPodista brad anderson solrinter...@gmail.com kirjoitti 19.3.2010 kello 20.53: Indeed, which is why I'm wondering what is Zoie adding if you still need to commit to search recent documents. Does anyone know? Thanks, Brad On 18 March 2010 19:41, Erik Hatcher erik.hatc...@gmail.com wrote: When I don't do the commit, I cannot search the documents I've indexed. - that's exactly how Solr without Zoie works, and it's how Lucene itself works. Gotta commit to see the documents indexed. Erik On Mar 18, 2010, at 5:41 PM, brad anderson wrote: Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Indeed, which is why I'm wondering what is Zoie adding if you still need to commit to search recent documents. Does anyone know? Thanks, Brad On 18 March 2010 19:41, Erik Hatcher erik.hatc...@gmail.com wrote: When I don't do the commit, I cannot search the documents I've indexed. - that's exactly how Solr without Zoie works, and it's how Lucene itself works. Gotta commit to see the documents indexed. Erik On Mar 18, 2010, at 5:41 PM, brad anderson wrote: Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
When I don't do the commit, I cannot search the documents I've indexed. - that's exactly how Solr without Zoie works, and it's how Lucene itself works. Gotta commit to see the documents indexed. Erik On Mar 18, 2010, at 5:41 PM, brad anderson wrote: Tried following their tutorial for plugging zoie into solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Server It appears it only allows you to search on documents after you do a commit? Am I missing something here, or does plugin not doing anything. Their tutorial tells you to do a commit when you index the docs: curl http://localhost:8983/solr/update/csv?commit=true --data-binary @books.csv -H 'Content-type:text/plain; charset=utf-8' When I don't do the commit, I cannot search the documents I've indexed. Thanks, Brad On 9 March 2010 23:34, Don Werve d...@madwombat.com wrote: 2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
I think Don is talking about Zoie - it requires a long uniqueKey. On Tue, Mar 9, 2010 at 10:18 AM, Lance Norskog goks...@gmail.com wrote: Solr unique ids can be any type. The QueryElevateComponent complains if the unique id is not a string, but you can comment out the QEC. I have one benchmark test with 2 billion documents with an integer id. Works great. On Mon, Mar 8, 2010 at 5:06 PM, Don Werve d...@madwombat.com wrote: Too bad it requires integer (long) primary keys... :/ 2010/3/8 Ian Holsman li...@holsman.net I just saw this on twitter, and thought you guys would be interested.. I haven't tried it, but it looks interesting. http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin Thanks for the RT Shalin! -- Lance Norskog goks...@gmail.com -- Regards, Shalin Shekhar Mangar.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
2010/3/9 Shalin Shekhar Mangar shalinman...@gmail.com I think Don is talking about Zoie - it requires a long uniqueKey. Yep; we're using UUIDs.
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Too bad it requires integer (long) primary keys... :/ 2010/3/8 Ian Holsman li...@holsman.net I just saw this on twitter, and thought you guys would be interested.. I haven't tried it, but it looks interesting. http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin Thanks for the RT Shalin!
Re: [ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
Solr unique ids can be any type. The QueryElevateComponent complains if the unique id is not a string, but you can comment out the QEC. I have one benchmark test with 2 billion documents with an integer id. Works great. On Mon, Mar 8, 2010 at 5:06 PM, Don Werve d...@madwombat.com wrote: Too bad it requires integer (long) primary keys... :/ 2010/3/8 Ian Holsman li...@holsman.net I just saw this on twitter, and thought you guys would be interested.. I haven't tried it, but it looks interesting. http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin Thanks for the RT Shalin! -- Lance Norskog goks...@gmail.com
[ANN] Zoie Solr Plugin - Zoie Solr Plugin enables real-time update functionality for Apache Solr 1.4+
I just saw this on twitter, and thought you guys would be interested.. I haven't tried it, but it looks interesting. http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin Thanks for the RT Shalin!
