We had the same problem, we wanted to add new tokens to a field ( reagrdless of it being stored on unstored ) We also wanted to explore the possiblity of adding the adding new fields to a document.
Deleting and re-adding document didn't seem to scale because it involves re-tokenizing documents and big document use a lot of cpu of tokenize. One solution, which I have been trying to work on, involves reading ".tis" and ".tii" files and resurrecting the unstored tokens. Do any of you have any insights into into how to go about it or any other pointers. -Manish --- Dmitry Serebrennikov <[EMAIL PROTECTED]> wrote: > Victor Hadianto wrote: > > >On Thu, 22 Aug 2002 23:14, Otis Gospodnetic wrote: > > > > > >>That's the very top question/answer in Lucene FAQ > at jGuru: > >>http://www.jguru.com/faq/Lucene > >> > >> > > > >Hi Otis, > > > >Yep I realise that, but I think you haven't read my > question closely. My > >problem is not simply delete/add the new document, > but what happen with the > >fields thare are unstored. If all the fields in my > documents are stored then > >it should be fine, but unfortunately not in our > current situation. > > > >Has anyone else ever came across this problem? > > > Yes, generally, there are two answers -- either make > all fields stored > or use some other database for the storage of the > "master" documents. > The first approach expands the use of Lucene to the > point where it > becomes a database rather then an indexing engine. > This makes sense in > some applications, but generally degrades in > performance. The second > approach is preferred, but it increases complexity > of the overall solution. > > I suppose it might be possible to add a lower-level > support for > extracting the terms from the unstored fields of the > initial document > and re-indexing them for the new document... > > Dmitry. > > > > > > > > >>Otis > >> > >> > > > >Victor > > > > > > > > > >>--- Victor Hadianto <[EMAIL PROTECTED]> wrote: > >> > >> > >>>Hi, > >>> > >>>I asked this question yesterday on the user-list > and so far there is > >>>no > >>>reply. I post again this question on the dev-list > hoping that someone > >>>can > >>>answer it here. > >>> > >>>We have a situation where we have a large > collection of documents, > >>>which > >>>consist of both stored and unstored fields, and > we'd like to > >>>add/modify a > >>>stored field on an existing document. > >>> > >>>It seems the only way this can be achieved is to > delete the document, > >>>and > >>>then re-create it. However, this will only > perserve stored fields, > >>>the > >>>unstored field information will be lost. > >>> > >>>In our application, the unstored fields consist > of very large data, > >>>and it > >>>would not be desirable to store them. > >>> > >>>Are there any ways in getting around this > problem? Thanks. > >>> > >>>-- > >>>Victor Hadianto > >>> > >>>NUIX Pty Ltd > >>>Level 8, 143 York Street, Sydney 2000 > >>>Phone: (02) 9283 9010 > >>>Fax: (02) 9283 9020 > >>> > >>>This message is intended only for the named > recipient. If you are not > >>>the > >>>intended recipient you are notified that > disclosing, copying, > >>>distributing > >>>or taking any action in reliance on the contents > of this message or > >>>attachment is strictly prohibited. > >>> > >>>-- > >>>To unsubscribe, e-mail: > >>><mailto:[EMAIL PROTECTED]> > >>>For additional commands, e-mail: > >>><mailto:[EMAIL PROTECTED]> > >>> > >>> > >>__________________________________________________ > >>Do You Yahoo!? > >>HotJobs - Search Thousands of New Jobs > >>http://www.hotjobs.com > >> > >> > > > > > > > > > > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > __________________________________________________ Do You Yahoo!? Yahoo! Finance - Get real-time stock quotes http://finance.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
