[
https://issues.apache.org/jira/browse/LUCENE-510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580739#action_12580739
]
Hiroaki Kawai commented on LUCENE-510:
--------------------------------------
To the first reason of performance issue,
{quote}
CharsetEncoder encoder = Charset.forName("UTF-8").newEncoder();
CharBuffer cb = CharBuffer.allocate(5000);
ByteBuffer bb = ByteBuffer.allocate(5000);
byte[] bbArray = bb.array();
UnicodeUtil.UTF8Result utf8Result = new UnicodeUtil.UTF8Result();
t0 = System.currentTimeMillis();
for(int i=0;i<count;i++) { cb.clear(); cb.put(strings[i]); cb.flip();
bb.clear(); encoder.reset(); encoder.encode(cb, bb, true); }
{quote}
How about this code?
CharsetEncoder encoder = Charset.forName("UTF-8").newEncoder();
CharBuffer cb;
ByteBuffer bb = ByteBuffer.allocate(5000);
byte[] bbArray = bb.array();
UnicodeUtil.UTF8Result utf8Result = new UnicodeUtil.UTF8Result();
t0 = System.currentTimeMillis();
for(int i=0;i<count;i++) {
bb.clear();
encoder.reset();
encoder.encode(CharBuffer.wrap(strings[i]), bb, true);
}
> IndexOutput.writeString() should write length in bytes
> ------------------------------------------------------
>
> Key: LUCENE-510
> URL: https://issues.apache.org/jira/browse/LUCENE-510
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Affects Versions: 2.1
> Reporter: Doug Cutting
> Assignee: Michael McCandless
> Attachments: LUCENE-510.patch, LUCENE-510.take2.patch,
> SortExternal.java, strings.diff, TestSortExternal.java
>
>
> We should change the format of strings written to indexes so that the length
> of the string is in bytes, not Java characters. This issue has been
> discussed at:
> http://www.mail-archive.com/java-dev@lucene.apache.org/msg01970.html
> We must increment the file format number to indicate this change. At least
> the format number in the segments file should change.
> I'm targetting this for 2.1, i.e., we shouldn't commit it to trunk until
> after 2.0 is released, to minimize incompatible changes between 1.9 and 2.0
> (other than removal of deprecated features).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
