[ https://issues.apache.org/jira/browse/LUCENE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand updated LUCENE-7351: --------------------------------- Attachment: LUCENE-7351.patch Updated patch. It now specializes both reading doc ids into an array and feeding a visitor, which seems to help get the performance back to what it is on master, or at least less than 1% slower (not easy to distinguish minor slowdowns to noise at this stage). It has 3 cases: - increasing doc ids, which is expected to happen for either sorted segments or when all docs in a block have the same value. In that case, we delta-encode using vints. - doc ids requiring less than 24 bits, which are encoded on 3 bytes. - doc ids requiring less than 32 bits, which are encoded on 4 bytes like on master today. I think it's ready to go? > BKDWriter should compress doc ids when all values in a block are the same > ------------------------------------------------------------------------- > > Key: LUCENE-7351 > URL: https://issues.apache.org/jira/browse/LUCENE-7351 > Project: Lucene - Core > Issue Type: Improvement > Reporter: Adrien Grand > Priority: Minor > Attachments: LUCENE-7351.patch, LUCENE-7351.patch > > > BKDWriter writes doc ids using 4 bytes per document. I think it should > compress similarly to postings when all docs in a block have the same packed > value. This can happen either when a field has a default value which is > common across documents or when quantization makes the number of unique > values so small that a large index will necessarily have blocks that all > contain the same value (eg. there are only 63490 unique half-float values). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org