siddharthteotia commented on a change in pull request #4993: Support Text
column type in Pinot (both offline and realtime)
URL: https://github.com/apache/incubator-pinot/pull/4993#discussion_r368136714
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/indexsegment/mutable/MutableSegmentImpl.java
##########
@@ -418,28 +474,35 @@ private void addForwardIndex(GenericRow row, int docId,
Map<String, Object> dict
}
}
} else {
+ // MV column: always dictionary encoded
int[] dictIds = (int[]) dictIdMap.get(column);
((FixedByteSingleColumnMultiValueReaderWriter)
_indexReaderWriterMap.get(column)).setIntArray(docId, dictIds);
}
}
}
- private void addInvertedIndex(int docId, Map<String, Object> dictIdMap) {
+ private void addInvertedIndex(GenericRow row, int docId, Map<String, Object>
dictIdMap) {
// Update inverted index at last
// NOTE: inverted index have to be updated at last because once it gets
updated, the latest record will become
// queryable
for (FieldSpec fieldSpec : _physicalFieldSpecs) {
String column = fieldSpec.getName();
- RealtimeInvertedIndexReader invertedIndex =
_invertedIndexMap.get(column);
+ InvertedIndexReader invertedIndex = _invertedIndexMap.get(column);
if (invertedIndex != null) {
- if (fieldSpec.isSingleValueField()) {
- invertedIndex.add(((Integer) dictIdMap.get(column)), docId);
+ if (invertedIndex instanceof RealtimeLuceneTextIndexReader) {
Review comment:
Discussed offline....The existing RealtimeInvertedIndexReader is actually
implemented as a reader and writer without a proper interface. Since that is
orthogonal piece of work and requires cleanup of existing interfaces, a
follow-up PR will come to cleanup this interface and then we can avoid
typecasting.
Also, right now typecasting is not that bad -- done only when realtime
segment is destroyed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
