[ https://issues.apache.org/jira/browse/SPARK-34844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307479#comment-17307479 ]
Takeshi Yamamuro commented on SPARK-34844: ------------------------------------------ The current logic can cause any issue? I feel there are no big difference between the current one and the proposed one though. > JDBCRelation columnPartition function includes the first stride in the lower > partition > -------------------------------------------------------------------------------------- > > Key: SPARK-34844 > URL: https://issues.apache.org/jira/browse/SPARK-34844 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Jason Yarbrough > Priority: Minor > > Currently, columnPartition in JDBCRelation contains logic that adds the first > stride into the lower partition. Because of this, the lower bound isn't used > as the ceiling for the lower partition. > For example, say we have data 0-10, 10 partitions, and the lowerBound is set > to 1. The lower/first partition should contain anything < 1. However, in the > current implementation, it would include anything < 2. > A possible easy fix would be changing the following code on line 132: > currentValue += stride > To: > if (i != 0) currentValue += stride > Or include currentValue += stride within the if statement on line 131... > although this creates a pretty bad looking side-effect. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org