Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-17 Thread Michael McCandless
It is expected: those are the "prefix" terms, which come after all the full-precision numeric terms. But I'm not sure why you see 0s ... the bytes should be unique for every term you get back from the TermsEnum. Mike McCandless http://blog.mikemccandless.com On Mon, Nov 17, 2014 at 10:39 AM, B

RE: Iterating TermsEnum for Long field produces zero values at the end

2014-11-17 Thread Uwe Schindler
Hi, > It is expected: those are the "prefix" terms, which come after all the full- > precision numeric terms. > > But I'm not sure why you see 0s ... the bytes should be unique for every term > you get back from the TermsEnum. That's easy to explain: The lower precision terms at the end have mo

Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-17 Thread Barry Coughlan
Makes sense, thanks. I switched the implementation to a FieldCache with no noticeable performance difference: private Longs cacheDocIds() throws IOException { AtomicReader wrapped = SlowCompositeReaderWrapper.wrap(reader); Longs vals = FieldCache.DEFAULT.getLongs(wrapped, "id", false);

Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-17 Thread Michael McCandless
It's better to use doc values than field cache, if you can. Mike McCandless http://blog.mikemccandless.com On Mon, Nov 17, 2014 at 2:55 PM, Barry Coughlan wrote: > Makes sense, thanks. I switched the implementation to a FieldCache with no > noticeable performance difference: > > private Longs

Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-18 Thread Barry Coughlan
Hi Michael, Indexing: private NumericDocValuesField idField = new NumericDocValuesField("id", 0); Reading: private NumericDocValues cacheDocIds() throws IOException { AtomicReader wrapped = SlowCompositeReaderWrapper.wrap(reader); return DocValues.getNumeric(wrapped, "id

Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-18 Thread Barry Coughlan
Never mind, I got it: MultiDocValues.getNumericValues(final IndexReader r, final String field) Barry On Tue, Nov 18, 2014 at 12:05 PM, Barry Coughlan wrote: > Hi Michael, > > Indexing: > > private NumericDocValuesField idField = new > NumericDocValuesField("id", 0); > > Reading: > > pri

Re: Iterating TermsEnum for Long field produces zero values at the end

2014-11-18 Thread Michael McCandless
FieldCache is (will be?) already gone in 5.0: it's moved to the "misc" module. It is slow the first time you use it since it must walk all postings doing the inversion. It is also a heap hog compared to doc values which get more dev attention and try to be more careful in how they spend heap. If