[jira] [Assigned] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33108: - Assignee: Dongjoon Hyun > Remove sbt-dependency-graph SBT plugin >

[jira] [Resolved] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33108. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29997 [https://

[jira] [Commented] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

2020-10-09 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211563#comment-17211563 ] Yang Jie commented on SPARK-33104: -- Is this an inevitable problem? `mvn test` can pass,

[jira] [Created] (SPARK-33109) Upgrade to SBT 1.4 and support `dependencyTree` back

2020-10-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33109: - Summary: Upgrade to SBT 1.4 and support `dependencyTree` back Key: SPARK-33109 URL: https://issues.apache.org/jira/browse/SPARK-33109 Project: Spark Issue

[jira] [Updated] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33094: - Fix Version/s: 2.4.8 > ORC format does not propagate Hadoop config from DS options to underlying

[jira] [Updated] (SPARK-33079) Replace the existing Maven job for Scala 2.13 in Github Actions with SBT job

2020-10-09 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-33079: --- Summary: Replace the existing Maven job for Scala 2.13 in Github Actions with SBT job (was:

[jira] [Commented] (SPARK-32082) Project Zen: Improving Python usability

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211560#comment-17211560 ] Hyukjin Kwon commented on SPARK-32082: -- Nope, these are all the JIRAs linked here.

[jira] [Comment Edited] (SPARK-32082) Project Zen: Improving Python usability

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211560#comment-17211560 ] Hyukjin Kwon edited comment on SPARK-32082 at 10/10/20, 4:55 AM: -

[jira] [Assigned] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33102: Assignee: Gabor Somogyi > Use stringToSeq on SQL list typed parameters >

[jira] [Resolved] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33102. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29989 [https://gi

[jira] [Resolved] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33105. -- Fix Version/s: 3.1.0 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/29

[jira] [Commented] (SPARK-32907) adaptively blockify instances

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211550#comment-17211550 ] Apache Spark commented on SPARK-32907: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-32907) adaptively blockify instances

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211549#comment-17211549 ] Apache Spark commented on SPARK-32907: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211547#comment-17211547 ] Apache Spark commented on SPARK-33108: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33108: Assignee: Apache Spark > Remove sbt-dependency-graph SBT plugin > ---

[jira] [Assigned] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33108: Assignee: (was: Apache Spark) > Remove sbt-dependency-graph SBT plugin >

[jira] [Updated] (SPARK-32907) adaptively blockify instances

2020-10-09 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-32907: - Attachment: blockify_svc_perf_20201010.xlsx > adaptively blockify instances > --

[jira] [Created] (SPARK-33108) Remove sbt-dependency-graph SBT plugin

2020-10-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33108: - Summary: Remove sbt-dependency-graph SBT plugin Key: SPARK-33108 URL: https://issues.apache.org/jira/browse/SPARK-33108 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33107) Remove hive-2.3 workaround code

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33107: Assignee: (was: Apache Spark) > Remove hive-2.3 workaround code > ---

[jira] [Assigned] (SPARK-33107) Remove hive-2.3 workaround code

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33107: Assignee: Apache Spark > Remove hive-2.3 workaround code > --

[jira] [Commented] (SPARK-33107) Remove hive-2.3 workaround code

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211526#comment-17211526 ] Apache Spark commented on SPARK-33107: -- User 'wangyum' has created a pull request f

[jira] [Created] (SPARK-33107) Remove hive-2.3 workaround code

2020-10-09 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33107: --- Summary: Remove hive-2.3 workaround code Key: SPARK-33107 URL: https://issues.apache.org/jira/browse/SPARK-33107 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33045) Implement built-in LIKE ANY and LIKE ALL UDF

2020-10-09 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211522#comment-17211522 ] jiaan.geng commented on SPARK-33045: I'm working on. > Implement built-in LIKE ANY

[jira] [Assigned] (SPARK-33080) Replace compiler reporter with more robust and maintainable solution

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33080: Assignee: Apache Spark > Replace compiler reporter with more robust and maintainable solu

[jira] [Assigned] (SPARK-33080) Replace compiler reporter with more robust and maintainable solution

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33080: Assignee: (was: Apache Spark) > Replace compiler reporter with more robust and mainta

[jira] [Commented] (SPARK-33080) Replace compiler reporter with more robust and maintainable solution

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211494#comment-17211494 ] Apache Spark commented on SPARK-33080: -- User 'gemelen' has created a pull request f

[jira] [Created] (SPARK-33106) Fix sbt resolvers clash

2020-10-09 Thread Denis Pyshev (Jira)
Denis Pyshev created SPARK-33106: Summary: Fix sbt resolvers clash Key: SPARK-33106 URL: https://issues.apache.org/jira/browse/SPARK-33106 Project: Spark Issue Type: Improvement Com

[jira] [Updated] (SPARK-33104) Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33104: -- Summary: Fix `YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil`

[jira] [Commented] (SPARK-32082) Project Zen: Improving Python usability

2020-10-09 Thread Andrew Malone Melo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211474#comment-17211474 ] Andrew Malone Melo commented on SPARK-32082: > Potentially better interface

[jira] [Assigned] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33105: Assignee: (was: Apache Spark) > Broken installation of source packages on AppVeyor >

[jira] [Assigned] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33105: Assignee: Apache Spark > Broken installation of source packages on AppVeyor > ---

[jira] [Commented] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211456#comment-17211456 ] Apache Spark commented on SPARK-33105: -- User 'zero323' has created a pull request f

[jira] [Commented] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211453#comment-17211453 ] Bruce Robbins commented on SPARK-33098: --- I  left out one case, which I added to th

[jira] [Updated] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-33098: -- Description: Comparing a partition column against a literal with the wrong type works if you

[jira] [Updated] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-33105: --- Description: It looks like AppVeyor configuration is broken, which leads to failure

[jira] [Created] (SPARK-33105) Broken installation of source packages on AppVeyor

2020-10-09 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-33105: -- Summary: Broken installation of source packages on AppVeyor Key: SPARK-33105 URL: https://issues.apache.org/jira/browse/SPARK-33105 Project: Spark

[jira] [Updated] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-33098: -- Description: Comparing a partition column against a literal with the wrong type works if you

[jira] [Updated] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-33098: -- Description: Comparing a partition column against a literal with the wrong type works if you

[jira] [Assigned] (SPARK-31972) Improve heurestic for selecting nodes for scale down to take into account graceful decommission cost

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31972: Assignee: Apache Spark > Improve heurestic for selecting nodes for scale down to take int

[jira] [Commented] (SPARK-31972) Improve heurestic for selecting nodes for scale down to take into account graceful decommission cost

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211408#comment-17211408 ] Apache Spark commented on SPARK-31972: -- User 'holdenk' has created a pull request f

[jira] [Assigned] (SPARK-31972) Improve heurestic for selecting nodes for scale down to take into account graceful decommission cost

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31972: Assignee: (was: Apache Spark) > Improve heurestic for selecting nodes for scale down

[jira] [Commented] (SPARK-32881) NoSuchElementException occurs during decommissioning

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211407#comment-17211407 ] Apache Spark commented on SPARK-32881: -- User 'holdenk' has created a pull request f

[jira] [Assigned] (SPARK-32881) NoSuchElementException occurs during decommissioning

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32881: Assignee: Apache Spark > NoSuchElementException occurs during decommissioning > -

[jira] [Assigned] (SPARK-32881) NoSuchElementException occurs during decommissioning

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32881: Assignee: (was: Apache Spark) > NoSuchElementException occurs during decommissioning

[jira] [Created] (SPARK-33104) Fix YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil

2020-10-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33104: - Summary: Fix YarnClusterSuite.yarn-cluster should respect conf overrides in SparkHadoopUtil Key: SPARK-33104 URL: https://issues.apache.org/jira/browse/SPARK-33104

[jira] [Resolved] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-9686. - Resolution: Duplicate > Spark Thrift server doesn't return correct JDBC metadata > -

[jira] [Created] (SPARK-33103) Custom Schema with Custom RDD reorders columns when more than 4 added

2020-10-09 Thread Justin Mays (Jira)
Justin Mays created SPARK-33103: --- Summary: Custom Schema with Custom RDD reorders columns when more than 4 added Key: SPARK-33103 URL: https://issues.apache.org/jira/browse/SPARK-33103 Project: Spark

[jira] [Commented] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211312#comment-17211312 ] Aoyuan Liao commented on SPARK-32069: - [~Gengliang.Wang] If no one is working on it,

[jira] [Commented] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211305#comment-17211305 ] Aoyuan Liao commented on SPARK-9686: [~srowen] I think SPARK-28426 fixed this. > Spa

[jira] [Commented] (SPARK-33039) Misleading watermark calculation in structure streaming

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211294#comment-17211294 ] Sean R. Owen commented on SPARK-33039: -- Yeah I think that's an OK resolution. Thank

[jira] [Resolved] (SPARK-33039) Misleading watermark calculation in structure streaming

2020-10-09 Thread Sandish Kumar HN (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandish Kumar HN resolved SPARK-33039. -- Resolution: Invalid > Misleading watermark calculation in structure streaming > --

[jira] [Commented] (SPARK-33039) Misleading watermark calculation in structure streaming

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211288#comment-17211288 ] Aoyuan Liao commented on SPARK-33039: - [~srowen] This is actually not a bug. The use

[jira] [Resolved] (SPARK-31430) Bug in the approximate quantile computation.

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31430. -- Resolution: Duplicate > Bug in the approximate quantile computation. > ---

[jira] [Resolved] (SPARK-16859) History Server storage information is missing

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-16859. -- Resolution: Not A Problem Sounds fine. It's very old in any event. > History Server storage i

[jira] [Commented] (SPARK-31430) Bug in the approximate quantile computation.

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211286#comment-17211286 ] Sean R. Owen commented on SPARK-31430: -- Sounds good, I usually mark as a Duplicate.

[jira] [Commented] (SPARK-16859) History Server storage information is missing

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211284#comment-17211284 ] Aoyuan Liao commented on SPARK-16859: - [~srowen] After configuring "spark.eventLog.l

[jira] [Commented] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

2020-10-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211283#comment-17211283 ] Sean R. Owen commented on SPARK-9686: - [~EveLiao] probably - do you know what other i

[jira] [Commented] (SPARK-31430) Bug in the approximate quantile computation.

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211282#comment-17211282 ] Aoyuan Liao commented on SPARK-31430: - [~srowen] This is already fixed. > Bug in th

[jira] [Commented] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

2020-10-09 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211280#comment-17211280 ] Aoyuan Liao commented on SPARK-9686: [~srowen] This is resolved on 3.0.1. Should we m

[jira] [Resolved] (SPARK-33062) Make DataFrameReader.jdbc work for DataSource V2

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-33062. Resolution: Not A Problem > Make DataFrameReader.jdbc work for DataSource V2 > --

[jira] [Resolved] (SPARK-33010) Make DataFrameWriter.jdbc work for DataSource V2

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-33010. Resolution: Not A Problem > Make DataFrameWriter.jdbc work for DataSource V2 > --

[jira] [Assigned] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33081: Assignee: Apache Spark > Support ALTER TABLE in JDBC v2 Table Catalog: update type and nu

[jira] [Assigned] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33081: Assignee: (was: Apache Spark) > Support ALTER TABLE in JDBC v2 Table Catalog: update

[jira] [Assigned] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33102: Assignee: (was: Apache Spark) > Use stringToSeq on SQL list typed parameters > --

[jira] [Assigned] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33102: Assignee: Apache Spark > Use stringToSeq on SQL list typed parameters > -

[jira] [Commented] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211166#comment-17211166 ] Apache Spark commented on SPARK-33102: -- User 'gaborgsomogyi' has created a pull req

[jira] [Reopened] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reopened SPARK-33081: > Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of > columns (DB2 dialect

[jira] [Issue Comment Deleted] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-33081: --- Comment: was deleted (was: This is done by smaller subtasks) > Support ALTER TABLE in JDBC v2 Table

[jira] [Commented] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211144#comment-17211144 ] Huaxin Gao commented on SPARK-33081: This is done by smaller subtasks > Support ALT

[jira] [Resolved] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-09 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-33081. Resolution: Not A Problem > Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullabil

[jira] [Commented] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1723#comment-1723 ] Gabor Somogyi commented on SPARK-33102: --- Filing a PR soon... > Use stringToSeq on

[jira] [Created] (SPARK-33102) Use stringToSeq on SQL list typed parameters

2020-10-09 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-33102: - Summary: Use stringToSeq on SQL list typed parameters Key: SPARK-33102 URL: https://issues.apache.org/jira/browse/SPARK-33102 Project: Spark Issue Type: Bu

[jira] [Comment Edited] (SPARK-25080) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2020-10-09 Thread Anika Kelhanka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211011#comment-17211011 ] Anika Kelhanka edited comment on SPARK-25080 at 10/9/20, 2:24 PM:

[jira] [Commented] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211040#comment-17211040 ] Apache Spark commented on SPARK-33098: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211043#comment-17211043 ] Apache Spark commented on SPARK-33098: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33098: Assignee: (was: Apache Spark) > Exception when using 'in' to compare a partition colu

[jira] [Assigned] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33098: Assignee: Apache Spark > Exception when using 'in' to compare a partition column to a lit

[jira] [Commented] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211025#comment-17211025 ] Apache Spark commented on SPARK-33094: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211022#comment-17211022 ] Apache Spark commented on SPARK-33094: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-25080) NPE in HiveShim$.toCatalystDecimal(HiveShim.scala:110)

2020-10-09 Thread Anika Kelhanka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211011#comment-17211011 ] Anika Kelhanka commented on SPARK-25080: I am able to produce this issue while q

[jira] [Comment Edited] (SPARK-32924) Web UI sort on duration is wrong

2020-10-09 Thread Rakesh Raushan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210977#comment-17210977 ] Rakesh Raushan edited comment on SPARK-32924 at 10/9/20, 1:49 PM:

[jira] [Commented] (SPARK-32924) Web UI sort on duration is wrong

2020-10-09 Thread Rakesh Raushan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210977#comment-17210977 ] Rakesh Raushan commented on SPARK-32924: I thinking its due to string sorting. O

[jira] [Updated] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33094: - Fix Version/s: 3.0.2 > ORC format does not propagate Hadoop config from DS options to underlying

[jira] [Updated] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33101: - Fix Version/s: 3.0.2 2.4.8 > LibSVM format does not propagate Hadoop config f

[jira] [Commented] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210808#comment-17210808 ] Apache Spark commented on SPARK-33101: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210807#comment-17210807 ] Apache Spark commented on SPARK-33101: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210799#comment-17210799 ] Apache Spark commented on SPARK-33094: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33094) ORC format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210798#comment-17210798 ] Apache Spark commented on SPARK-33094: -- User 'MaxGekk' has created a pull request f

[jira] [Resolved] (SPARK-32896) Add DataStreamWriter.table API

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32896. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29767 [https://

[jira] [Resolved] (SPARK-33099) Respect executor idle timeout conf in ExecutorPodsAllocator

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33099. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29981 [https://

[jira] [Assigned] (SPARK-33099) Respect executor idle timeout conf in ExecutorPodsAllocator

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33099: - Assignee: Dongjoon Hyun > Respect executor idle timeout conf in ExecutorPodsAllocator >

[jira] [Resolved] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33101. --- Resolution: Fixed Issue resolved by pull request 29984 [https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-33093) Why do my Spark 3 jobs fail to use external shuffle service on YARN?

2020-10-09 Thread Julien (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210715#comment-17210715 ] Julien commented on SPARK-33093: That worked, [~yumwang]; thanks! > Why do my Spark 3 j

[jira] [Commented] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210692#comment-17210692 ] Apache Spark commented on SPARK-33101: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210694#comment-17210694 ] Apache Spark commented on SPARK-33101: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33101: Assignee: Maxim Gekk (was: Apache Spark) > LibSVM format does not propagate Hadoop confi

[jira] [Assigned] (SPARK-33101) LibSVM format does not propagate Hadoop config from DS options to underlying HDFS file system

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33101: Assignee: Apache Spark (was: Maxim Gekk) > LibSVM format does not propagate Hadoop confi

[jira] [Commented] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210691#comment-17210691 ] Apache Spark commented on SPARK-13860: -- User 'leanken' has created a pull request f

[jira] [Commented] (SPARK-33098) Exception when using 'in' to compare a partition column to a literal with the wrong type

2020-10-09 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17210689#comment-17210689 ] Peter Toth commented on SPARK-33098: I've started to look into this issue. > Except

[jira] [Updated] (SPARK-33100) Support parse the sql statements with c-style comments

2020-10-09 Thread feiwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-33100: Description: Now the spark-sql does not support parse the sql statements with C-style comments. For the s

  1   2   >