stanyao commented on code in PR #54972:
URL: https://github.com/apache/spark/pull/54972#discussion_r3125195784
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala:
##########
@@ -831,6 +837,30 @@ object V2ScanRelationPushDown extends Rule[LogicalPlan]
with PredicateHelper {
}
def pushDownSample(plan: LogicalPlan): LogicalPlan = plan.transform {
+ case sample: Sample if sample.sampleMethod == SampleMethod.System =>
+ sample.child match {
+ case PhysicalOperation(_, Nil, sHolder: ScanBuilderHolder) =>
+ val tableSample = TableSampleInfo(
+ sample.lowerBound,
+ sample.upperBound,
+ sample.withReplacement,
+ sample.seed.getOrElse((math.random() * 1000).toLong),
Review Comment:
The reason 1000 is used here is to be consistent with
SampleExec.resolvedSeed in basicPhysicalOperators.scala:362. Based on community
convention, I'd keep this as is to be consistent with existing patterns and
keep this PR focused and open a JIRA issue to fix both places at once
([SPARK-56573](https://issues.apache.org/jira/browse/SPARK-56573)). What do you
think, @szehon-ho ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
