mayankshriv commented on issue #9770: URL: https://github.com/apache/pinot/issues/9770#issuecomment-1309226060
In case of realtime ingestion, the following happens: - The column does get stored wtih no-dict (or raw values). - The column is also sorted on raw values, and that is reflected in metadata as well. - However, sorted raw values != sorted index, and this leads to full scan. While this is a valid expected behavior, I recommend we should change the precedence. If user specifies sorted as well as no-dict in real-time table, it would be better to default to dictionary based sorted index, so as to avoid full scan. I recommend this because the penalty of full scan is much higher than dict vs no dict storage overhead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
