yyanyy commented on code in PR #55252:
URL: https://github.com/apache/spark/pull/55252#discussion_r3054397422
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala:
##########
@@ -157,13 +157,22 @@ case class DataSourceV2Relation(
* @param keyGroupedPartitioning if set, the partitioning expressions that are
used to split the
* rows in the scan across different partitions
* @param ordering if set, the ordering provided by the scan
+ * @param pushedFilters Catalyst expressions for filters that were fully
pushed to the data
+ * source and do not appear as post-scan filters
*/
case class DataSourceV2ScanRelation(
relation: DataSourceV2Relation,
scan: Scan,
output: Seq[AttributeReference],
keyGroupedPartitioning: Option[Seq[Expression]] = None,
- ordering: Option[Seq[SortOrder]] = None) extends LeafNode with
NamedRelation {
+ ordering: Option[Seq[SortOrder]] = None,
+ pushedFilters: Seq[Expression] = Seq.empty) extends LeafNode with
NamedRelation {
+
+ // TODO: Override validConstraints to return ExpressionSet(pushedFilters) so
that pushed
Review Comment:
Thank you both for the review! I'd like to defer that in a later PR to avoid
blocking this one for too long for adjustments for golden files and others that
might take quite some investigation time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
