I tried that and the solution looked so clumsy . I need to commit the to read anything was making things difficult DB provides me 'immediate' reads . I am sure performance will be hit anyway. Is Lucene write much faster than DB (embedded) writes? http://www.h2database.com/html/performance.html
On Thu, Dec 4, 2008 at 10:07 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote: > A database, just to store uncommitted documents in case they might be > updated, seems like it will have a pretty major impact on indexing > performance. A lucene-only implementation would seem to be much > lighter on resources. > > -Yonik > > On Thu, Dec 4, 2008 at 11:32 AM, Noble Paul നോബിള് नोब्ळ् > <[EMAIL PROTECTED]> wrote: >> The solution will be an UpdateRequestProcessor (which itself is >> pluggable).I am implementing a JDBC based one. I'll test with H2 and >> MySql (and may be Derby) >> >> We will ship the H2 (embedded) jar >> >> >> >> >> >> >> On Thu, Dec 4, 2008 at 9:53 PM, Ryan McKinley <[EMAIL PROTECTED]> wrote: >>> Again, I would hope that solr builds a storage agnostic solution. >>> >>> As long as we have a simple interface to load/store documents, it should be >>> easy to write a JDBC/ehcache/disk/Cassandra/whatever implementation. >>> >>> ryan >>> >>> >>> On Dec 4, 2008, at 10:29 AM, Noble Paul നോബിള് नोब्ळ् wrote: >>> >>>> Cassandra does not meet our requirements. >>>> we do not need that kind of scalability >>>> >>>> Moreover its future is uncertain and they are trying to incubate it into >>>> Solr >>>> >>>> >>>> On Thu, Dec 4, 2008 at 8:52 PM, Sami Siren <[EMAIL PROTECTED]> wrote: >>>>> >>>>> Yet another possibility: http://wiki.apache.org/incubator/Cassandra >>>>> >>>>> It at least claims to be scalable, no personal experience. >>>>> >>>>> -- >>>>> Sami Siren >>>>> >>>>> Noble Paul ??????? ?????? wrote: >>>>>> >>>>>> Another persistence solution is ehcache with diskstore. It even has >>>>>> replication >>>>>> >>>>>> I have never used ehcache . So I cannot comment on it >>>>>> >>>>>> any comments? >>>>>> >>>>>> --Noble >>>>>> >>>>>> On Wed, Dec 3, 2008 at 8:50 PM, Noble Paul ??????? ?????? >>>>>> <[EMAIL PROTECTED]> wrote: >>>>>> >>>>>>> >>>>>>> On Wed, Dec 3, 2008 at 5:52 PM, Grant Ingersoll <[EMAIL PROTECTED]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> On Dec 3, 2008, at 1:28 AM, Noble Paul ??????? ?????? wrote: >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> The code can be written against JDBC. But we need to test the DDL and >>>>>>>>> data types on al the supported DBs >>>>>>>>> >>>>>>>>> But , which one would we like to ship with Solr as a default option? >>>>>>>>> >>>>>>>> >>>>>>>> Why do we need a default option? Is this something that is intended >>>>>>>> to >>>>>>>> be >>>>>>>> on by default? Or, do you mean just to have one for unit tests to >>>>>>>> work? >>>>>>>> >>>>>>> >>>>>>> Default does not mean that it is enabled bby default. But if it is >>>>>>> enabled I can have defaults for stuff like driver, url , DDL etc. And >>>>>>> the user may not need to provide an extra jar >>>>>>> >>>>>>>> >>>>>>>> I don't know if it is still the case, but I often find embedded dbs to >>>>>>>> be >>>>>>>> quite annoying since you often can't connect to them from other >>>>>>>> clients >>>>>>>> outside of the JVM which makes debugging harder. Of course, maybe I >>>>>>>> just >>>>>>>> don't know the tricks to do it. Derby is one DB that you can still >>>>>>>> connect >>>>>>>> to even when it is embedded. >>>>>>>> >>>>>>> >>>>>>> Embedded is the best bet for us because of performance reasons and >>>>>>> zero management. >>>>>>> The users can still read the data through Solr itself . >>>>>>> >>>>>>>> >>>>>>>> Also, whatever is chosen needs to scale to millions of documents, and >>>>>>>> I >>>>>>>> wonder about an embedded DB doing that. I also have a hard time >>>>>>>> believing >>>>>>>> that both a DB w/ millions of docs and Solr can live on the same >>>>>>>> machine, >>>>>>>> which is presumably what an embedded DB must do. Presumably, it also >>>>>>>> needs >>>>>>>> to be able to be replicated, right? >>>>>>>> >>>>>>> >>>>>>> millions of docs.? >>>>>>> then you must configure a remote DB for storage reasons >>>>>>> and must manage the replication separately >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> H2 looks impressive. the jar (small) is just 667KB and the memory >>>>>>>>> footprint is small too >>>>>>>>> --Noble >>>>>>>>> >>>>>>>>> On Wed, Dec 3, 2008 at 10:30 AM, Ryan McKinley <[EMAIL PROTECTED]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> check http://www.h2database.com/ in my view the best embedded DB >>>>>>>>>> out >>>>>>>>>> there. >>>>>>>>>> >>>>>>>>>> from the maker of HSQLDB... is second round. >>>>>>>>>> >>>>>>>>>> However, from anything solr, I would hope it would just rely on >>>>>>>>>> JDBC. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Dec 2, 2008, at 12:08 PM, Shalin Shekhar Mangar wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> HSQLDB has a limit of upto 8GB of data. In Solr, you might want to >>>>>>>>>>> go >>>>>>>>>>> beyond >>>>>>>>>>> that without a commit. >>>>>>>>>>> >>>>>>>>>>> On Tue, Dec 2, 2008 at 10:33 PM, Dawid Weiss >>>>>>>>>>> <[EMAIL PROTECTED]>wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Isn't HSQLDB an option? Its performance ranges a lot depending on >>>>>>>>>>>> the >>>>>>>>>>>> volume of data and queries, but otherwise the license looks >>>>>>>>>>>> BSDish. >>>>>>>>>>>> >>>>>>>>>>>> http://hsqldb.org/web/hsqlLicense.html >>>>>>>>>>>> >>>>>>>>>>>> Dawid >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Regards, >>>>>>>>>>> Shalin Shekhar Mangar. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> --Noble Paul >>>>>>>>> >>>>>>>> >>>>>>>> -------------------------- >>>>>>>> Grant Ingersoll >>>>>>>> >>>>>>>> Lucene Helpful Hints: >>>>>>>> http://wiki.apache.org/lucene-java/BasicsOfPerformance >>>>>>>> http://wiki.apache.org/lucene-java/LuceneFAQ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> --Noble Paul >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> --Noble Paul >>> >>> >> >> >> >> -- >> --Noble Paul >> > -- --Noble Paul
