[jira] [Commented] (SPARK-33085) "Master removed our application" error leads to FAILED driver status instead of KILLED driver status

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213647#comment-17213647 ] Hyukjin Kwon commented on SPARK-33085: -- Can you show the reproducible steps? > "Ma

[jira] [Commented] (SPARK-33113) [SparkR] gapply works with arrow disabled, fails with arrow enabled stringsAsFactors=TRUE

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213643#comment-17213643 ] Hyukjin Kwon commented on SPARK-33113: -- It works with Spark 3.0.0 too. Can you show

[jira] [Commented] (SPARK-33113) [SparkR] gapply works with arrow disabled, fails with arrow enabled stringsAsFactors=TRUE

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213640#comment-17213640 ] Hyukjin Kwon commented on SPARK-33113: -- It works in my local in Spark dev branch: {

[jira] [Updated] (SPARK-33113) [SparkR] gapply works with arrow disabled, fails with arrow enabled stringsAsFactors=TRUE

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33113: - Description: Running in databricks on Azure {code} library("arrow") library("SparkR") df <- as

[jira] [Updated] (SPARK-33113) [SparkR] gapply works with arrow disabled, fails with arrow enabled stringsAsFactors=TRUE

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33113: - Description: Running in databricks on Azure {code} library("arrow") library("SparkR") df <- as

[jira] [Updated] (SPARK-33113) [SparkR] gapply works with arrow disabled, fails with arrow enabled stringsAsFactors=TRUE

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33113: - Description: Running in databricks on Azure {code} library("arrow") library("SparkR") df <- as

[jira] [Commented] (SPARK-33120) Lazy Load of SparkContext.addFiles

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213635#comment-17213635 ] Hyukjin Kwon commented on SPARK-33120: -- Why don't you just upload your files into H

[jira] [Commented] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213629#comment-17213629 ] Hyukjin Kwon commented on SPARK-33133: -- cc [~kabhwan] FYI > History server fails w

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213612#comment-17213612 ] Jungtaek Lim commented on SPARK-33136: -- Note that AppendData in branch-2.4 is also

[jira] [Comment Edited] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213558#comment-17213558 ] Dongjoon Hyun edited comment on SPARK-29536 at 10/14/20, 3:18 AM:

[jira] [Updated] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29536: -- Affects Version/s: 2.4.7 > PySpark does not work with Python 3.8.0 > -

[jira] [Commented] (SPARK-29536) PySpark does not work with Python 3.8.0

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213558#comment-17213558 ] Dongjoon Hyun commented on SPARK-29536: --- Hi, [~hyukjin.kwon]. Apache Spark 2.4.7 a

[jira] [Resolved] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33134. -- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-33134: Assignee: Maxim Gekk > Incorrect nested complex JSON fields raise an exception >

[jira] [Commented] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Thejdeep (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213550#comment-17213550 ] Thejdeep commented on SPARK-33138: -- Can I look into this please ? > unify temp view an

[jira] [Created] (SPARK-33142) SQL temp view should store SQL text as well

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33142: --- Summary: SQL temp view should store SQL text as well Key: SPARK-33142 URL: https://issues.apache.org/jira/browse/SPARK-33142 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-33141) capture SQL configs when creating permanent views

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33141: --- Summary: capture SQL configs when creating permanent views Key: SPARK-33141 URL: https://issues.apache.org/jira/browse/SPARK-33141 Project: Spark Issue Type: S

[jira] [Created] (SPARK-33140) make Analyzer and Optimizer rules using SQLConf.get

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33140: --- Summary: make Analyzer and Optimizer rules using SQLConf.get Key: SPARK-33140 URL: https://issues.apache.org/jira/browse/SPARK-33140 Project: Spark Issue Type:

[jira] [Created] (SPARK-33139) protect setActiveSession and clearActiveSession

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33139: --- Summary: protect setActiveSession and clearActiveSession Key: SPARK-33139 URL: https://issues.apache.org/jira/browse/SPARK-33139 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and perman

[jira] [Created] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33138: --- Summary: unify temp view and permanent view behaviors Key: SPARK-33138 URL: https://issues.apache.org/jira/browse/SPARK-33138 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-33138) unify temp view and permanent view behaviors

2020-10-13 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33138: Description: Currently, temp view store mapping of temp view name and its logicalPlan, and permane

[jira] [Created] (SPARK-33137) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect)

2020-10-13 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-33137: -- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (PostgreSQL dialect) Key: SPARK-33137 URL: https://issues.apache.org/jira/browse/SPARK-33

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213461#comment-17213461 ] Apache Spark commented on SPARK-33136: -- User 'HeartSaVioR' has created a pull reque

[jira] [Assigned] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33136: Assignee: Apache Spark > Handling nullability for complex types is broken during resoluti

[jira] [Assigned] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33136: Assignee: (was: Apache Spark) > Handling nullability for complex types is broken duri

[jira] [Commented] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213448#comment-17213448 ] Jungtaek Lim commented on SPARK-33136: -- will submit a PR soon. > Handling nullabil

[jira] [Created] (SPARK-33136) Handling nullability for complex types is broken during resolution of V2 write command

2020-10-13 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-33136: Summary: Handling nullability for complex types is broken during resolution of V2 write command Key: SPARK-33136 URL: https://issues.apache.org/jira/browse/SPARK-33136

[jira] [Commented] (SPARK-29625) Spark Structure Streaming Kafka Wrong Reset Offset twice

2020-10-13 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213336#comment-17213336 ] L. C. Hsieh commented on SPARK-29625: - How did you specify the starting offset? > S

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [19], "playerId": 1

[jira] [Commented] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-13 Thread Aoyuan Liao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213312#comment-17213312 ] Aoyuan Liao commented on SPARK-33071: - The master branch has the same bug. > Join w

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213224#comment-17213224 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33132: - Assignee: akiyamaneko > The 'Shuffle Read Size / Records' field in Stage Summary metric

[jira] [Resolved] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33132. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30030 [https://

[jira] [Commented] (SPARK-33071) Join with ambiguous column succeeding but giving wrong output

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213205#comment-17213205 ] Wenchen Fan commented on SPARK-33071: - To confirm: is this a long-standing bug that

[jira] [Assigned] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33135: Assignee: Apache Spark > Use listLocatedStatus from FileSystem implementations >

[jira] [Resolved] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33129. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30028 [https://

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33129: - Assignee: Prashant Sharma > Since the sbt version is now upgraded, old `test-only` need

[jira] [Assigned] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33135: Assignee: (was: Apache Spark) > Use listLocatedStatus from FileSystem implementations

[jira] [Commented] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213203#comment-17213203 ] Apache Spark commented on SPARK-33135: -- User 'sunchao' has created a pull request f

[jira] [Created] (SPARK-33135) Use listLocatedStatus from FileSystem implementations

2020-10-13 Thread Chao Sun (Jira)
Chao Sun created SPARK-33135: Summary: Use listLocatedStatus from FileSystem implementations Key: SPARK-33135 URL: https://issues.apache.org/jira/browse/SPARK-33135 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-33134: --- Description: The code below: {code:scala} val pokerhand_raw = Seq("""[{"cards": [11], "playerId"

[jira] [Commented] (SPARK-25547) Pluggable jdbc connection factory

2020-10-13 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213152#comment-17213152 ] Gabor Somogyi commented on SPARK-25547: --- [~fsauer65] JDBC connection provider API

[jira] [Commented] (SPARK-33128) mismatched input since Spark 3.0

2020-10-13 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213137#comment-17213137 ] Yang Jie commented on SPARK-33128: -- set `spark.sql.ansi.enabled` to true can work with

[jira] [Assigned] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-13860: --- Assignee: Leanken.Lin > TPCDS query 39 returns wrong results compared to TPC official resul

[jira] [Reopened] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-13860: - > TPCDS query 39 returns wrong results compared to TPC official result set > --

[jira] [Resolved] (SPARK-13860) TPCDS query 39 returns wrong results compared to TPC official result set

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-13860. - Fix Version/s: 3.1.0 Resolution: Fixed > TPCDS query 39 returns wrong results compared to

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33134: Assignee: (was: Apache Spark) > Incorrect nested complex JSON fields raise an excepti

[jira] [Assigned] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33134: Assignee: Apache Spark > Incorrect nested complex JSON fields raise an exception > --

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213109#comment-17213109 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213108#comment-17213108 ] Apache Spark commented on SPARK-33134: -- User 'MaxGekk' has created a pull request f

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-33132: Fix Version/s: (was: 3.0.2) > The 'Shuffle Read Size / Records' field in Stage Summary metrics

[jira] [Assigned] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33125: --- Assignee: jiaan.geng > Improve the error when Lead and Lag are not allowed to specify windo

[jira] [Resolved] (SPARK-33125) Improve the error when Lead and Lag are not allowed to specify window frame

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33125. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30021 [https://gith

[jira] [Created] (SPARK-33134) Incorrect nested complex JSON fields raise an exception

2020-10-13 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-33134: -- Summary: Incorrect nested complex JSON fields raise an exception Key: SPARK-33134 URL: https://issues.apache.org/jira/browse/SPARK-33134 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33081. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29972 [https://gith

[jira] [Assigned] (SPARK-33081) Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (DB2 dialect)

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33081: --- Assignee: Huaxin Gao > Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullab

[jira] [Updated] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-33133: - Description: We have run into an issue where our history server fails to load new applications,

[jira] [Resolved] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32858. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29792 [https://gith

[jira] [Assigned] (SPARK-32858) UnwrapCastInBinaryComparison: support other numeric types

2020-10-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32858: --- Assignee: Chao Sun > UnwrapCastInBinaryComparison: support other numeric types > --

[jira] [Created] (SPARK-33133) History server fails when loading invalid rolling event logs

2020-10-13 Thread Adam Binford (Jira)
Adam Binford created SPARK-33133: Summary: History server fails when loading invalid rolling event logs Key: SPARK-33133 URL: https://issues.apache.org/jira/browse/SPARK-33133 Project: Spark

[jira] [Resolved] (SPARK-33115) `kvstore` and `unsafe` doc tasks fail

2020-10-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-33115. -- Fix Version/s: 3.1.0 3.0.2 Assignee: Denis Pyshev Resolution

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33132: Assignee: Apache Spark > The 'Shuffle Read Size / Records' field in Stage Summary metrics

[jira] [Commented] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213073#comment-17213073 ] Apache Spark commented on SPARK-33132: -- User 'akiyamaneko' has created a pull reque

[jira] [Assigned] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33132: Assignee: (was: Apache Spark) > The 'Shuffle Read Size / Records' field in Stage Summ

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Summary: The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefined'

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Description: Spark Version: 3.0.1 Description: The 'Shuffle Read Size / Records' field in Stage Summary

[jira] [Updated] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-33132: - Attachment: Stage Summary shows NaN undefined.png > The 'Shuffle Read Size / Records' field in Stage Sum

[jira] [Created] (SPARK-33132) The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind'

2020-10-13 Thread echohlne (Jira)
echohlne created SPARK-33132: Summary: The 'Shuffle Read Size / Records' field in Stage Summary metrics was shown as 'NaN Undefind' Key: SPARK-33132 URL: https://issues.apache.org/jira/browse/SPARK-33132

[jira] [Commented] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213068#comment-17213068 ] Apache Spark commented on SPARK-33131: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33131: Assignee: Apache Spark > Fix grouping sets with having clause can not resolve qualified c

[jira] [Assigned] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33131: Assignee: (was: Apache Spark) > Fix grouping sets with having clause can not resolve

[jira] [Created] (SPARK-33131) Fix grouping sets with having clause can not resolve qualified col name

2020-10-13 Thread ulysses you (Jira)
ulysses you created SPARK-33131: --- Summary: Fix grouping sets with having clause can not resolve qualified col name Key: SPARK-33131 URL: https://issues.apache.org/jira/browse/SPARK-33131 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-32681) PySpark type hints support

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] echohlne updated SPARK-32681: - Comment: was deleted (was: (y)) > PySpark type hints support > -- > >

[jira] [Commented] (SPARK-32681) PySpark type hints support

2020-10-13 Thread echohlne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213065#comment-17213065 ] echohlne commented on SPARK-32681: -- (y) > PySpark type hints support > ---

[jira] [Resolved] (SPARK-32295) Add not null and size > 0 filters before inner explode to benefit from predicate pushdown

2020-10-13 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32295. -- Fix Version/s: 3.1.0 Assignee: Tanel Kiis Resolution: Fixed Resolved b

[jira] [Created] (SPARK-33130) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect)

2020-10-13 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-33130: --- Summary: Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect) Key: SPARK-33130 URL: https://issues.apache.org/jira

[jira] [Commented] (SPARK-21708) Migrate build to sbt 1.3.13

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213033#comment-17213033 ] Apache Spark commented on SPARK-21708: -- User 'ScrapCodes' has created a pull reques

[jira] [Commented] (SPARK-21708) Migrate build to sbt 1.3.13

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213032#comment-17213032 ] Apache Spark commented on SPARK-21708: -- User 'ScrapCodes' has created a pull reques

[jira] [Commented] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213031#comment-17213031 ] Apache Spark commented on SPARK-33129: -- User 'ScrapCodes' has created a pull reques

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33129: Assignee: (was: Apache Spark) > Since the sbt version is now upgraded, old `test-only

[jira] [Assigned] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33129: Assignee: Apache Spark > Since the sbt version is now upgraded, old `test-only` needs to

[jira] [Updated] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-33129: Description: Follow up to SPARK-21708, updating the references to test-only with testOnly.

[jira] [Created] (SPARK-33129) Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly`

2020-10-13 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-33129: --- Summary: Since the sbt version is now upgraded, old `test-only` needs to be replaced with `testOnly` Key: SPARK-33129 URL: https://issues.apache.org/jira/browse/SPARK-33129

[jira] [Assigned] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32069: Assignee: (was: Apache Spark) > Improve error message on reading unexpected directory

[jira] [Assigned] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32069: Assignee: Apache Spark > Improve error message on reading unexpected directory which is n

[jira] [Commented] (SPARK-32069) Improve error message on reading unexpected directory which is not a table partition

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213012#comment-17213012 ] Apache Spark commented on SPARK-32069: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-10-13 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213010#comment-17213010 ] Ismaël Mejía commented on SPARK-27733: -- [~csun] This is excellent news! We will get

[jira] [Commented] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213002#comment-17213002 ] Apache Spark commented on SPARK-32978: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32978: Assignee: Apache Spark > Incorrect number of dynamic part metric > --

[jira] [Commented] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17213001#comment-17213001 ] Apache Spark commented on SPARK-32978: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-32978) Incorrect number of dynamic part metric

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32978: Assignee: (was: Apache Spark) > Incorrect number of dynamic part metric > ---

[jira] [Created] (SPARK-33128) mismatched input since Spark 3.0

2020-10-13 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33128: --- Summary: mismatched input since Spark 3.0 Key: SPARK-33128 URL: https://issues.apache.org/jira/browse/SPARK-33128 Project: Spark Issue Type: Bug Comp

[jira] [Assigned] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33095: Assignee: Apache Spark > Support ALTER TABLE in JDBC v2 Table Catalog: add, update type a

[jira] [Assigned] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33095: Assignee: (was: Apache Spark) > Support ALTER TABLE in JDBC v2 Table Catalog: add, up

[jira] [Commented] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212989#comment-17212989 ] Apache Spark commented on SPARK-33095: -- User 'ScrapCodes' has created a pull reques

[jira] [Updated] (SPARK-33095) Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MySQL dialect)

2020-10-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-33095: Description: Override the default SQL strings for: ALTER TABLE UPDATE COLUMN TYPE ALTER TA

[jira] [Assigned] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32229: Assignee: (was: Apache Spark) > Application entry parsing fails because DriverWrapper

[jira] [Commented] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17212968#comment-17212968 ] Apache Spark commented on SPARK-32229: -- User 'gaborgsomogyi' has created a pull req

[jira] [Assigned] (SPARK-32229) Application entry parsing fails because DriverWrapper registered instead of the normal driver

2020-10-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32229: Assignee: Apache Spark > Application entry parsing fails because DriverWrapper registered

  1   2   >