kunal642 commented on a change in pull request #4241:
URL: https://github.com/apache/carbondata/pull/4241#discussion_r767429906
##########
File path:
integration/spark/src/main/spark3.1/org/apache/spark/sql/CarbonToSparkAdapter.scala
##########
@@ -180,9 +180,16 @@ object CarbonToSparkAdapter extends SparkVersionAdapter {
}
}
- def getDataFilter(partitionSet: AttributeSet, filter: Seq[Expression]):
Seq[Expression] = {
+ def getDataFilter(partitionSet: AttributeSet, filter: Seq[Expression],
+ partitionFilter: Seq[Expression]): Seq[Expression] = {
filter.filter {
- case _: DynamicPruningSubquery => false
+ case dp: DynamicPruningSubquery =>
+ // if filter does not exists in partition filter, then push down the
filter to spark
+ if (partitionFilter.exists(_.semanticEquals(dp))) {
Review comment:
no need of if else, directly return
!partitionFilter.exists(_.semanticEquals(dp))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
