[ https://issues.apache.org/jira/browse/LUCENE-4678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13550488#comment-13550488 ]
Dawid Weiss commented on LUCENE-4678: ------------------------------------- This looks very cool! I looked at the patch briefly but I need to apply it to make sense of the whole picture. :) {code} + while(skip > 0) { + buffer.writeByte((byte) 0); + skip--; + } {code} this doesn't look particularly efficient but I didn't get the context where it's actually used from the patch so maybe it's all right. > FST should use paged byte[] instead of single contiguous byte[] > --------------------------------------------------------------- > > Key: LUCENE-4678 > URL: https://issues.apache.org/jira/browse/LUCENE-4678 > Project: Lucene - Core > Issue Type: Improvement > Components: core/FSTs > Reporter: Michael McCandless > Assignee: Michael McCandless > Fix For: 4.2, 5.0 > > Attachments: LUCENE-4678.patch, LUCENE-4678.patch > > > The single byte[] we use today has several limitations, eg it limits us to < > 2.1 GB FSTs (and suggesters in the wild are getting close to this limit), and > it causes big RAM spikes during building when a the array has to grow. > I took basically the same approach as LUCENE-3298, but I want to break out > this patch separately from changing all int -> long for > 2.1 GB support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org