[
https://issues.apache.org/jira/browse/LUCENE-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854621#action_12854621
]
Yonik Seeley commented on LUCENE-2380:
--------------------------------------
bq. We could also do shared byte[] blocks (private), with a public method to
retrieve the BytesRef for a given doc?
Absolutely! Now that we are in control, it would be a crime not not share the
byte[]
Seems like one should pass in a BytesRef to be filled in... that would be most
efficient for people doing simple stuff like compare docid1 to docid2.
Returning a reused BytesRef could also work (as TermsEnum does) but it's less
efficient for anything needing a state of more than 1 BytesRef since it then
requires copying.
We can further save space by putting the length as a vInt in the byte[] - most
would be a single byte.
Then we just need an int[] as an index into the byte[]... or potentially packed
ints.
We'll also need an implementation that can span multiple byte[]s for larger
than 2GB support. The correct byte[] to look into is then simply a function of
the docid (as is done in Solr's UnInvertedField).
We could possibly play games with the offsets into the byte[] too - encode as a
delta against the average instead of an absolute offset. So offset =
average_size * ord + get_delta(ord)
> Add FieldCache.getTermBytes, to load term data as byte[]
> --------------------------------------------------------
>
> Key: LUCENE-2380
> URL: https://issues.apache.org/jira/browse/LUCENE-2380
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Michael McCandless
> Fix For: 3.1
>
>
> With flex, a term is now an opaque byte[] (typically, utf8 encoded unicode
> string, but not necessarily), so we need to push this up the search stack.
> FieldCache now has getStrings and getStringIndex; we need corresponding
> methods to load terms as native byte[], since in general they may not be
> representable as String. This should be quite a bit more RAM efficient too,
> for US ascii content since each character would then use 1 byte not 2.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
