[jira] [Created] (SPARK-31975) Throw user facing error when use WindowFunction directly

2020-06-11 Thread ulysses you (Jira)
ulysses you created SPARK-31975: --- Summary: Throw user facing error when use WindowFunction directly Key: SPARK-31975 URL: https://issues.apache.org/jira/browse/SPARK-31975 Project: Spark Issue

[jira] [Commented] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133984#comment-17133984 ] Apache Spark commented on SPARK-26905: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133983#comment-17133983 ] Apache Spark commented on SPARK-26905: -- User 'maropu' has created a pull request fo

[jira] [Resolved] (SPARK-31912) Normalize all binary comparison expressions

2020-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-31912. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28734 [https://gith

[jira] [Assigned] (SPARK-31912) Normalize all binary comparison expressions

2020-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-31912: --- Assignee: Yuming Wang > Normalize all binary comparison expressions > -

[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133931#comment-17133931 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-31967) Loading jobs UI page takes 40 seconds

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133930#comment-17133930 ] Apache Spark commented on SPARK-31967: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-31967) Loading jobs UI page takes 40 seconds

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31967: Assignee: Apache Spark > Loading jobs UI page takes 40 seconds >

[jira] [Assigned] (SPARK-31967) Loading jobs UI page takes 40 seconds

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31967: Assignee: (was: Apache Spark) > Loading jobs UI page takes 40 seconds > -

[jira] [Commented] (SPARK-31887) Date casting to string is giving wrong value

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133925#comment-17133925 ] Hyukjin Kwon commented on SPARK-31887: -- {code} ➜ ~ cat /tmp/test1.csv/* a,b 2020-0

[jira] [Resolved] (SPARK-31887) Date casting to string is giving wrong value

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31887. -- Resolution: Cannot Reproduce I can't reproduce this in the current master. Probably fixed some

[jira] [Resolved] (SPARK-31928) Flaky test: StreamingDeduplicationSuite.test no-data flag

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31928. -- Resolution: Duplicate > Flaky test: StreamingDeduplicationSuite.test no-data flag > --

[jira] [Commented] (SPARK-31930) Pandas_udf does not properly return ArrayType

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133921#comment-17133921 ] Hyukjin Kwon commented on SPARK-31930: -- Seems like it depends on which version you

[jira] [Resolved] (SPARK-31930) Pandas_udf does not properly return ArrayType

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31930. -- Resolution: Cannot Reproduce > Pandas_udf does not properly return ArrayType > ---

[jira] [Commented] (SPARK-31947) Solve string value error about Date/Timestamp in ScriptTransform

2020-06-11 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133915#comment-17133915 ] angerszhu commented on SPARK-31947: --- [~hyukjin.kwon] Yea, later > Solve string value

[jira] [Updated] (SPARK-31930) Pandas_udf does not properly return ArrayType

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31930: - Priority: Major (was: Blocker) > Pandas_udf does not properly return ArrayType > --

[jira] [Resolved] (SPARK-31943) SPARK-31500 introduces breaking changes in 2.4.6

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31943. -- Resolution: Won't Fix {{Collect}} is under catalyst package which is private. It isn't an API.

[jira] [Commented] (SPARK-31947) Solve string value error about Date/Timestamp in ScriptTransform

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133905#comment-17133905 ] Hyukjin Kwon commented on SPARK-31947: -- Can you fill the JIRA description? > Solve

[jira] [Resolved] (SPARK-31961) Add a class in spark with all Kafka configuration key available as string

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31961. -- Resolution: Won't Fix > Add a class in spark with all Kafka configuration key available as str

[jira] [Updated] (SPARK-31961) Add a class in spark with all Kafka configuration key available as string

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31961: - Target Version/s: (was: 2.4.6) > Add a class in spark with all Kafka configuration key availab

[jira] [Commented] (SPARK-31962) Provide option to load files after a specified date when reading from a folder path

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133902#comment-17133902 ] Hyukjin Kwon commented on SPARK-31962: -- cc [~kabhwan] FYI > Provide option to load

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Lin Gang Deng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133898#comment-17133898 ] Lin Gang Deng commented on SPARK-31955: --- [~hyukjin.kwon] , the  difference between

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133894#comment-17133894 ] Yuming Wang commented on SPARK-31955: - It seems this is a Hive issue. Maybe we shoul

[jira] [Resolved] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31880. -- Resolution: Invalid > Adjacent value parsing not supported for Localized Patterns because of J

[jira] [Reopened] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-31880: -- > Adjacent value parsing not supported for Localized Patterns because of JDK bug > ---

[jira] [Resolved] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31880. -- Resolution: Not A Problem > Adjacent value parsing not supported for Localized Patterns becaus

[jira] [Updated] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31880: - Parent: SPARK-31408 Issue Type: Sub-task (was: Bug) > Adjacent value parsing not suppor

[jira] [Commented] (SPARK-28169) Spark can’t push down partition predicate for OR expression

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133886#comment-17133886 ] Apache Spark commented on SPARK-28169: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133887#comment-17133887 ] Hyukjin Kwon commented on SPARK-31880: -- Okay, thanks for clarification. Let's be di

[jira] [Commented] (SPARK-28169) Spark can’t push down partition predicate for OR expression

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133885#comment-17133885 ] Apache Spark commented on SPARK-28169: -- User 'AngersZh' has created a pull requ

[jira] [Resolved] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31955. -- Resolution: Incomplete > Beeline discard the last line of the sql file when submited to thrif

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133884#comment-17133884 ] Hyukjin Kwon commented on SPARK-31955: -- I am going to leave it resolved until enoug

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133883#comment-17133883 ] Hyukjin Kwon commented on SPARK-31955: -- [~denglg] Please show the __exact__ reprodu

[jira] [Updated] (SPARK-25186) Stabilize Data Source V2 API

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25186: - Target Version/s: 3.1.0 > Stabilize Data Source V2 API > - > >

[jira] [Updated] (SPARK-25186) Stabilize Data Source V2 API

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25186: - Target Version/s: (was: 3.0.0) > Stabilize Data Source V2 API > -

[jira] [Commented] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133880#comment-17133880 ] Kent Yao commented on SPARK-31880: -- This should be fixed as ’Not a Problem‘ because we

[jira] [Commented] (SPARK-19169) columns changed orc table encouter 'IndexOutOfBoundsException' when read the old schema files

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133879#comment-17133879 ] Hyukjin Kwon commented on SPARK-19169: -- [~angerszhuuu] can you file a new JIRA with

[jira] [Commented] (SPARK-30399) Bucketing does not compatible with partitioning in practice

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133878#comment-17133878 ] Hyukjin Kwon commented on SPARK-30399: -- Okay but Spark 2.3.0 is EOL. Mind checking

[jira] [Commented] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133874#comment-17133874 ] Hyukjin Kwon commented on SPARK-31880: -- I converted this into a separate issue beca

[jira] [Updated] (SPARK-31880) Adjacent value parsing not supported for Localized Patterns because of JDK bug

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31880: - Parent: (was: SPARK-31408) Issue Type: Bug (was: Sub-task) > Adjacent value parsing

[jira] [Resolved] (SPARK-31899) Forbid datetime pattern letter u

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31899. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/28

[jira] [Updated] (SPARK-31827) Fail datetime parsing/formatting if detect the Java 8 bug of stand-alone form

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31827: - Summary: Fail datetime parsing/formatting if detect the Java 8 bug of stand-alone form (was: fa

[jira] [Updated] (SPARK-31899) Forbid datetime pattern letter u

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31899: - Summary: Forbid datetime pattern letter u (was: forbid datetime pattern letter u) > Forbid dat

[jira] [Resolved] (SPARK-31827) fail datetime parsing/formatting if detect the Java 8 bug of stand-alone form

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31827. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in > fail datetime parsing/formatting if

[jira] [Comment Edited] (SPARK-31827) fail datetime parsing/formatting if detect the Java 8 bug of stand-alone form

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133868#comment-17133868 ] Hyukjin Kwon edited comment on SPARK-31827 at 6/12/20, 3:09 AM: --

[jira] [Commented] (SPARK-31799) Spark Datasource Tables Creating Incorrect Hive Metadata

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133866#comment-17133866 ] Hyukjin Kwon commented on SPARK-31799: -- JSON itself isn't a completely self-describ

[jira] [Updated] (SPARK-31705) Rewrite join condition to conjunctive normal form

2020-06-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31705: Fix Version/s: 3.1.0 > Rewrite join condition to conjunctive normal form > ---

[jira] [Created] (SPARK-31974) Stop tracking speculative shuffle files in ExecutorMonitor

2020-06-11 Thread Holden Karau (Jira)
Holden Karau created SPARK-31974: Summary: Stop tracking speculative shuffle files in ExecutorMonitor Key: SPARK-31974 URL: https://issues.apache.org/jira/browse/SPARK-31974 Project: Spark Is

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Lin Gang Deng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133822#comment-17133822 ] Lin Gang Deng commented on SPARK-31955: --- [~dongjoon] As  you said, EOL is the key

[jira] [Commented] (SPARK-31973) Add ability to disable Sort,Spill in Partial aggregation

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133810#comment-17133810 ] Apache Spark commented on SPARK-31973: -- User 'karuppayya' has created a pull reques

[jira] [Updated] (SPARK-31916) StringConcat can overflow `length`, leads to StringIndexOutOfBoundsException

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-31916: - Fix Version/s: 3.1.0 > StringConcat can overflow `length`, leads to StringIndexOutOfBoun

[jira] [Commented] (SPARK-31973) Add ability to disable Sort,Spill in Partial aggregation

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133809#comment-17133809 ] Apache Spark commented on SPARK-31973: -- User 'karuppayya' has created a pull reques

[jira] [Assigned] (SPARK-31973) Add ability to disable Sort,Spill in Partial aggregation

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31973: Assignee: Apache Spark > Add ability to disable Sort,Spill in Partial aggregation >

[jira] [Assigned] (SPARK-31973) Add ability to disable Sort,Spill in Partial aggregation

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31973: Assignee: (was: Apache Spark) > Add ability to disable Sort,Spill in Partial aggregat

[jira] [Resolved] (SPARK-31916) StringConcat can overflow `length`, leads to StringIndexOutOfBoundsException

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-31916. -- Fix Version/s: 3.0.1 Assignee: Dilip Biswal Resolution: Fixed Resolved

[jira] [Created] (SPARK-31973) Add ability to disable Sort,Spill in Partial aggregation

2020-06-11 Thread Karuppayya (Jira)
Karuppayya created SPARK-31973: -- Summary: Add ability to disable Sort,Spill in Partial aggregation Key: SPARK-31973 URL: https://issues.apache.org/jira/browse/SPARK-31973 Project: Spark Issue T

[jira] [Comment Edited] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133215#comment-17133215 ] Takeshi Yamamuro edited comment on SPARK-26905 at 6/11/20, 11:48 PM: -

[jira] [Comment Edited] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133215#comment-17133215 ] Takeshi Yamamuro edited comment on SPARK-26905 at 6/11/20, 11:47 PM: -

[jira] [Comment Edited] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133215#comment-17133215 ] Takeshi Yamamuro edited comment on SPARK-26905 at 6/11/20, 11:46 PM: -

[jira] [Comment Edited] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133215#comment-17133215 ] Takeshi Yamamuro edited comment on SPARK-26905 at 6/11/20, 11:44 PM: -

[jira] [Created] (SPARK-31972) Improve heurestic for selecting nodes for scale down to take into account graceful decommission cost

2020-06-11 Thread Holden Karau (Jira)
Holden Karau created SPARK-31972: Summary: Improve heurestic for selecting nodes for scale down to take into account graceful decommission cost Key: SPARK-31972 URL: https://issues.apache.org/jira/browse/SPARK-319

[jira] [Commented] (SPARK-31197) Exit the executor once all tasks & migrations are finished

2020-06-11 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133776#comment-17133776 ] Holden Karau commented on SPARK-31197: -- I'm working on this, will post PR soon. >

[jira] [Commented] (SPARK-31198) Use graceful decommissioning as part of dynamic scaling

2020-06-11 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133777#comment-17133777 ] Holden Karau commented on SPARK-31198: -- I'm going to start working on this (on top

[jira] [Resolved] (SPARK-31765) Upgrade HtmlUnit >= 2.37.0

2020-06-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31765. -- Fix Version/s: 3.1.0 Assignee: Kousuke Saruta Resolution: Fixed Resolved by ht

[jira] [Updated] (SPARK-31924) Create remote shuffle service reference implementation

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31924: -- Affects Version/s: (was: 3.0.0) 3.1.0 > Create remote shuffle servi

[jira] [Updated] (SPARK-31924) Create remote shuffle service reference implementation

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31924: -- Target Version/s: (was: 3.0.0) > Create remote shuffle service reference implementation > --

[jira] [Updated] (SPARK-31924) Create remote shuffle service reference implementation

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31924: -- Fix Version/s: (was: 3.0.0) > Create remote shuffle service reference implementation > ---

[jira] [Resolved] (SPARK-31935) Hadoop file system config should be effective in data source options

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31935. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-31935) Hadoop file system config should be effective in data source options

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31935: - Assignee: Gengliang Wang > Hadoop file system config should be effective in data source

[jira] [Resolved] (SPARK-21117) Built-in SQL Function Support - WIDTH_BUCKET

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-21117. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28764 [https://

[jira] [Assigned] (SPARK-21117) Built-in SQL Function Support - WIDTH_BUCKET

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-21117: - Assignee: Takeshi Yamamuro > Built-in SQL Function Support - WIDTH_BUCKET > ---

[jira] [Assigned] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31971: Assignee: Apache Spark (was: Kousuke Saruta) > Add pagination support for all jobs timel

[jira] [Assigned] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31971: Assignee: Kousuke Saruta (was: Apache Spark) > Add pagination support for all jobs timel

[jira] [Commented] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133679#comment-17133679 ] Apache Spark commented on SPARK-31971: -- User 'sarutak' has created a pull request f

[jira] [Updated] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-31971: --- Description: If there are lots of jobs, rendering performance of all jobs timeline can sign

[jira] [Updated] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-31971: --- Description: If there are lots of jobs, rendering performance of all jobs timeline can sign

[jira] [Created] (SPARK-31971) Add pagination support for all jobs timeline

2020-06-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-31971: -- Summary: Add pagination support for all jobs timeline Key: SPARK-31971 URL: https://issues.apache.org/jira/browse/SPARK-31971 Project: Spark Issue Type:

[jira] [Updated] (SPARK-30616) Introduce TTL config option for SQL Parquet Metadata Cache

2020-06-11 Thread Yaroslav Tkachenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaroslav Tkachenko updated SPARK-30616: --- Description: >From >[documentation|https://spark.apache.org/docs/2.4.4/sql-data-sou

[jira] [Updated] (SPARK-30616) Introduce TTL config option for SQL Parquet Metadata Cache

2020-06-11 Thread Yaroslav Tkachenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaroslav Tkachenko updated SPARK-30616: --- Description: >From >[documentation|https://spark.apache.org/docs/2.4.4/sql-data-sou

[jira] [Updated] (SPARK-30616) Introduce TTL config option for SQL Parquet Metadata Cache

2020-06-11 Thread Yaroslav Tkachenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaroslav Tkachenko updated SPARK-30616: --- Description: >From >[documentation|https://spark.apache.org/docs/2.4.4/sql-data-sou

[jira] [Commented] (SPARK-30616) Introduce TTL config option for SQL Parquet Metadata Cache

2020-06-11 Thread Yaroslav Tkachenko (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133596#comment-17133596 ] Yaroslav Tkachenko commented on SPARK-30616: The fix can be adding "expireAf

[jira] [Updated] (SPARK-31960) Only populate Hadoop classpath for no-hadoop build

2020-06-11 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-31960: Parent: SPARK-31582 Issue Type: Sub-task (was: Bug) > Only populate Hadoop classpath for no-hadoo

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133521#comment-17133521 ] Dongjoon Hyun commented on SPARK-31955: --- [~denglg]. Please add a new line characte

[jira] [Resolved] (SPARK-31705) Rewrite join condition to conjunctive normal form

2020-06-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31705. Resolution: Fixed > Rewrite join condition to conjunctive normal form > --

[jira] [Commented] (SPARK-31705) Rewrite join condition to conjunctive normal form

2020-06-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133435#comment-17133435 ] Gengliang Wang commented on SPARK-31705: The issue is resolved in https://github

[jira] [Assigned] (SPARK-31705) Rewrite join condition to conjunctive normal form

2020-06-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-31705: -- Assignee: Gengliang Wang (was: Yuming Wang) > Rewrite join condition to conjunctive

[jira] [Commented] (SPARK-31950) Extract SQL keywords from the generated parser class in TableIdentifierParserSuite

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133363#comment-17133363 ] Apache Spark commented on SPARK-31950: -- User 'maropu' has created a pull request fo

[jira] [Commented] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j.properties

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133351#comment-17133351 ] Apache Spark commented on SPARK-31970: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j.properties

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31970: Assignee: (was: Apache Spark) > Make MDC configuration step be consistent between set

[jira] [Assigned] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j.properties

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31970: Assignee: Apache Spark > Make MDC configuration step be consistent between setLocalProper

[jira] [Updated] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j.properties

2020-06-11 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi updated SPARK-31970: - Summary: Make MDC configuration step be consistent between setLocalProperty and log4j.properties (was: Make MDC

[jira] [Created] (SPARK-31970) Make MDC configuration step be consistent between setLocalProperty and log4j

2020-06-11 Thread wuyi (Jira)
wuyi created SPARK-31970: Summary: Make MDC configuration step be consistent between setLocalProperty and log4j Key: SPARK-31970 URL: https://issues.apache.org/jira/browse/SPARK-31970 Project: Spark

[jira] [Commented] (SPARK-31939) Fix Parsing day of year when year field pattern is missing

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133309#comment-17133309 ] Apache Spark commented on SPARK-31939: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-31939) Fix Parsing day of year when year field pattern is missing

2020-06-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133308#comment-17133308 ] Apache Spark commented on SPARK-31939: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-31955) Beeline discard the last line of the sql file when submited to thriftserver via beeline

2020-06-11 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133298#comment-17133298 ] YoungGyu Chun commented on SPARK-31955: --- [~denglg]    I cannot reproduce this lo

[jira] [Updated] (SPARK-31954) delete duplicate test cases in hivequerysuite

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31954: - Fix Version/s: (was: 3.0.0) 3.0.1 > delete duplicate test cases in hivequ

[jira] [Resolved] (SPARK-31954) delete duplicate test cases in hivequerysuite

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31954. -- Fix Version/s: 3.0.0 2.4.7 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-31954) delete duplicate test cases in hivequerysuite

2020-06-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31954: Assignee: philipse > delete duplicate test cases in hivequerysuite >

[jira] [Updated] (SPARK-31969) StreamingJobProgressListener threw an exception java.util.NoSuchElementException for Long Running Streaming Job

2020-06-11 Thread ThimmeGowda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ThimmeGowda updated SPARK-31969: Component/s: Web UI Spark Core Scheduler > StreamingJobProgressL

[jira] [Updated] (SPARK-31969) StreamingJobProgressListener threw an exception java.util.NoSuchElementException for Long Running Streaming Job

2020-06-11 Thread ThimmeGowda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ThimmeGowda updated SPARK-31969: Attachment: executor_log > StreamingJobProgressListener threw an exception > java.util.NoSuchElem

  1   2   >