codope commented on PR #11969: URL: https://github.com/apache/hudi/pull/11969#issuecomment-2372870237
> actually on second thought, why do we even need to store stats for these cols. Why can't we just ignore such cols. Then even on the query engine integration, query planning will proceed as though there are no stats. If not, we might make a call to col stats only to realize that stats are not of much use to prune anything. @nsivabalan The record and array type are comparable in Avro - https://avro.apache.org/docs/1.12.0/specification/#sort-order. It's only with Spark record type that error is thrown. Hence, the condition is dependent on the record type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
