[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed due to stride alignment.

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321997#comment-17321997 ] Apache Spark commented on SPARK-34843: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed due to stride alignment.

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321996#comment-17321996 ] Apache Spark commented on SPARK-34843: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed do to stride alignment.

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310273#comment-17310273 ] Jason Yarbrough commented on SPARK-34843: - [~dongjoon], that makes sense. Thank you for fixing

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed do to stride alignment.

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310234#comment-17310234 ] Dongjoon Hyun commented on SPARK-34843: --- Thank you for the suggestion and PR, [~hanover-fiste].

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size. Upper bound is skewed do to stride alignment.

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308978#comment-17308978 ] Jason Yarbrough commented on SPARK-34843: - PR created: [Spark 34843 - JDBCRelation

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308343#comment-17308343 ] Jason Yarbrough commented on SPARK-34843: - I've modified the formula slightly (reflected in the

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307502#comment-17307502 ] Jason Yarbrough commented on SPARK-34843: - I'm definitely interested. Thank you :) Fortunately,

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307498#comment-17307498 ] Takeshi Yamamuro commented on SPARK-34843: -- If the overflow is rare, it might be worth

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307494#comment-17307494 ] Jason Yarbrough commented on SPARK-34843: - Updated the formula. > JDBCRelation columnPartition

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307493#comment-17307493 ] Jason Yarbrough commented on SPARK-34843: - That's a good example! Thank you for sharing that. I

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307483#comment-17307483 ] Takeshi Yamamuro commented on SPARK-34843: -- How about `upperBound=Long.MaxValue,

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307477#comment-17307477 ] Jason Yarbrough commented on SPARK-34843: - >From what I can see, there is protection in the code

[jira] [Commented] (SPARK-34843) JDBCRelation columnPartition function improperly determines stride size

[ https://issues.apache.org/jira/browse/SPARK-34843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17307469#comment-17307469 ] Takeshi Yamamuro commented on SPARK-34843: -- To mitigate overflow issues, I think we use the