Thanks Li. It makes sense
On Sat, Aug 27, 2011 at 7:28 PM, Li Li <[email protected]> wrote:
> It will affect the entire index because it 's a parameter of IndexWriter.
> but you can modify it anytime you like before IndexWriter.addDocument.
> If you want to truncate different fields with different maxLength. you
> should avoid multithreads' race condition.
> maybe you can add a TokenFilter to the end of analyzer train
>
> AtomicInteger field1Counter;
> public final boolean incrementToken() throws IOException {
> if(field1Counter.get()>=maxLength) return false;
> if (input.incrementToken()) {
> field1Counter.incrementAndGet();
> return true;
> }else{
> return false;
> }
> }
> }
> On Sun, Aug 28, 2011 at 9:45 AM, roz dev <[email protected]> wrote:
>
> > Hi All
> >
> > I have a question regarding MaxFieldLength. Is it a limit for number of
> > tokens in 1 field per document or entire index?
> >
> > Example:
> >
> > If MaxFieldLength is set to 100 and I add a document which has 105 tokens
> > for 1 field then I expect 5 tokens to be ignored.
> > but, if i add another document which has 95 tokens for same field then
> > these
> > 95 tokens should be added.
> >
> > Please advise.
> >
> > Thanks
> > Saroj
> >
>
