mcvsubbu commented on a change in pull request #4954: Support schema evolution for consuming segments URL: https://github.com/apache/incubator-pinot/pull/4954#discussion_r368326346
########## File path: pinot-core/src/main/java/org/apache/pinot/core/indexsegment/mutable/MutableSegmentImpl.java ########## @@ -467,13 +483,22 @@ public SegmentMetadata getSegmentMetadata() { return physicalColumnNames; } + @Override + public Set<String> getColumnNamesForSelectStar() { + return Sets.union(getPhysicalColumnNames(), _newlyAddedColumnsFieldMap.keySet()); + } + @Override public ColumnDataSource getDataSource(String columnName) { FieldSpec fieldSpec = _schema.getFieldSpecFor(columnName); - if (fieldSpec.isVirtualColumn()) { + if ((fieldSpec == null && _newlyAddedColumnsFieldMap.containsKey(columnName)) || fieldSpec.isVirtualColumn()) { Review comment: We should have a different column provider here that should be optimized better than the virtual column provider. The new column provider should not be built every time, but should stay in memory once built. What may be updated is the number of rows. That will minimize additional allocation during query processing. Add a new column provider factory (say, ConstantValueColumnProviderFactory) that has a map from the col name to the provider. The map is built lazily as columns are added to the query. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org