Incremantally updating a VERY LARGE field - Is this possibe ?

e index later on. In my situation, (a) eventually causes a heap space error. Here's part of the handler code. Thanks much Thanks -- View this message in context: http://lucene.472066.n3.nabble.com/Incremantally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3881945.html Sent fro

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

Here's part of the handler code. > > > > Thanks much > > Thanks > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Incremantally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3881945.html > Sent from the Solr - User mailing list archive at Nabble.com. >

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

> > In my situation, (a) eventually causes a heap space error. > > > > > > > > > > Here's part of the handler code. > > > > > > > > Thanks much > > > > Thanks > > > > -- > > View this message in context: > > > http://lucene.472066.n3.nabble.com/Incremantally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3881945.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > -- Sincerely yours Mikhail Khludnev ge...@yandex.ru <http://www.griddynamics.com>

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

; > > a. adds to the in-memory field map and > > > b. attempts to write EVERYTHING to the index later on. > > > > > > In my situation, (a) eventually causes a heap space error. > > > > > > > > > > > > > > > Here's part of the handler code. > > > > > > > > > > > > Thanks much > > > > > > Thanks > > > > > > -- > > > View this message in context: > > > > > > http://lucene.472066.n3.nabble.com/Incremantally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3881945.html > > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > > > > > -- > Sincerely yours > Mikhail Khludnev > ge...@yandex.ru > > <http://www.griddynamics.com> > >

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

Thanks. Increasing max. heap space is not a scalable option as it reduces the ability of the system to scale with multiple concurrent index requests. The use case is indexing a set of text files which we have no control over i.e. could be small or large. -- View this message in context: http:/

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

> Updating a single field is not possible in solr. The whole record has to > be rewritten. Unfortunate. Lucene allows it. -- View this message in context: http://lucene.472066.n3.nabble.com/Incrementally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3885253.html Sent from the Solr -

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

On Wed, Apr 4, 2012 at 3:14 PM, vybe3142 wrote: > >> Updating a single field is not possible in solr.  The whole record has to >> be rewritten. > > Unfortunate. Lucene allows it. I think you're mistaken - the same limitations apply to Lucene. -Yonik lucenerevolution.com - Lucene/Solr Open Source

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

I believe we are talking about two different things. The original question was about incrementally building up a field during indexing, right? After a document is committed, a field cannot be separately updated, that is true in both Lucene and Solr. wunder On Apr 4, 2012, at 12:20 PM, Yonik S

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

depending on you jvm version, -XX:+UseCompressedStrings would help alleviate the problem. It did help me before. xab -- View this message in context: http://lucene.472066.n3.nabble.com/Incrementally-updating-a-VERY-LARGE-field-Is-this-possibe-tp3881945p3885493.html Sent from the Solr - User mail

Re: Incremantally updating a VERY LARGE field - Is this possibe ?

Yonik Seeley-2-2 wrote > > On Wed, Apr 4, 2012 at 3:14 PM, vybe3142 wrote: >> >>> Updating a single field is not possible in solr.  The whole record has >>> to >>> be rewritten. >> >> Unfortunate. Lucene allows it. > > I think you're mistaken - the same limitations apply to Lucene.