: > Doesn't this cause a problem for highly interactive and large indexes? Since
: > every update to the index requires the rewriting of the norms, and
: > constructing a new array.
:
: The original complaint was primarily about search-time memory size, not
: update speed. I like the proposed pat
Robert Engels wrote:
Doesn't this cause a problem for highly interactive and large indexes? Since
every update to the index requires the rewriting of the norms, and
constructing a new array.
The original complaint was primarily about search-time memory size, not
update speed. I like the propo
From: Doug Cutting [mailto:[EMAIL PROTECTED]
Sent: Monday, October 10, 2005 2:15 PM
To: java-dev@lucene.apache.org
Subject: Re: Eliminating norms ... completley
Chris Hostetter wrote:
> 2) Can you think of a clean way for individual applications to eliminate
> norms (via subclassing
Chris Hostetter wrote:
2) Can you think of a clean way for individual applications to eliminate
norms (via subclassing the lucene code base - ie: no patching)
Can't you simply subclass FilterIndexReader and override norms() to
return a cached dummy array of Similarity.encodeNorm(1.0f) f
Paul, thanx for your suggestions. It seems like they mostly address the
issue of improving search time, by eliminting the need to read the norm
files from disk -- but the spead of the query isn't as big of a concern
for us as the memory footprint.
As I understand it, the point when we are reall
Here's my patch for the indexing and IndexReader side of things. Works like
a charm.
http://issues.apache.org/jira/browse/LUCENE-448
This doesn't yet include performance enhancements to scorers when norms
aren't included,
but I first wanted to make sure things were completely backward compatible
On Friday 07 October 2005 23:18, Chris Hostetter wrote:
>
...
> So I'm curious what you guys think...
>
> 1) Regarding the root problem: is there any other things you can think
> of besides norms[] that would contribute to the memory foot print
> needed by a large number of indexed f
I'm approaching it the same as term vectors... make the norms optional on a
per field basis.
My first cut is returning a dummy norm array filled in with the equiv of
1.0f so I didn't have to go modify all the queries/weights/scorers that
retrieve the norms. For best performance, those scorers shoul
I did exactly this in my custom lucene, since the array of a byte per
document is extremely wasteful in a lot of applications. I just changed the
code to return null from getNorms() and modified the callers to treat a null
array as always 1 for any document.
-Original Message-
From: Chris
: 2) Can you think of a clean way for individual applications to eliminate
: norms (via subclassing the lucene code base - ie: no patching)
For completeness, I should mention that one thing I briefly considered was
writing a new Directory implimentation that would proxy to FSDirectory,
but
Hi, Chris,
Turning off norm looks like a very interesting problem to me. I remember
that in Lucene Road Map for 2.0, there is a requirement to turn off indexing
for some information, such as proximity.
Maybe optionally turning off the norm could be an experiment to show case
how to turn off the p
11 matches
Mail list logo
