In our case, we're trying to optimize document() retrieval and we found that
disabling the String interning in the Field constructor improved performance
dramatically. I agree that interning should be an option on the constructor.
For document retrieval, at least for a small of amount of fields, t
I don't think it is just the performance gain of equals() where intern
() matters.
It also reduces memory consumption dramatically when working with
large collections of documents in memory - although this could also
be done with constants, there is nothing in Java to enforce it (thus
the
True. However, in the case where you are processing Documents one at a time
and discarding them (e.g. We use hitCollector to process all documents from
a search), or memory is not an issue, it would be nice to have the ability
to disable the interning for performance sake.
Robert Engels wrote
On 2/23/07, James Kennedy <[EMAIL PROTECTED]> wrote:
In our case, we're trying to optimize document() retrieval and we found that
disabling the String interning in the Field constructor improved performance
dramatically. I agree that interning should be an option on the constructor.
Out of cur
James Kennedy wrote:
True. However, in the case where you are processing Documents one at a time
and discarding them (e.g. We use hitCollector to process all documents from
a search), or memory is not an issue, it would be nice to have the ability
to disable the interning for performance sake.
: Accessing documents from a hit-collector is not advised. It is
: generally best to compose queries and filters to reduce the number of
: matches. When that's not feasible, a hit collector that uses a
: FieldCache to filter by or collect field values is much faster than
: accessing documents.
Roughly search time of 10,000,000 documents (3 fields each) was cut in half.
However, keep in mind that we're using slightly modified lucene document
retrieval code. Using a HitCollector to aggregate search results.
Mike Klaas wrote:
>
> On 2/23/07, James Kennedy <[EMAIL PROTECTED]> wrote:
>>
On Feb 23, 2007, at 10:28 AM, James Kennedy wrote:
True. However, in the case where you are processing Documents one
at a time
and discarding them (e.g. We use hitCollector to process all
documents from
a search), or memory is not an issue, it would be nice to have the
ability
to disabl
Field constructor, avoiding String.intern()
In our case, we're trying to optimize document() retrieval and we found that
disabling the String interning in the Field constructor improved performance
dramatically. I agree that interning should be an option on the constructor.
For document retrieval, at
