[jira] [Created] (SPARK-32684) Scrip transform hive serde/default-serde mode null value keep same with hive

2020-08-21 Thread angerszhu (Jira)
angerszhu created SPARK-32684: - Summary: Scrip transform hive serde/default-serde mode null value keep same with hive Key: SPARK-32684 URL: https://issues.apache.org/jira/browse/SPARK-32684 Project:

[jira] [Commented] (SPARK-32287) Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181923#comment-17181923 ] Apache Spark commented on SPARK-32287: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Commented] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-21 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182040#comment-17182040 ] Nicholas Chammas commented on SPARK-32686: -- Not sure if I have the "Affects Version" field set

[jira] [Created] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-21 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-32686: Summary: Un-deprecate inferring DataFrame schema from list of dictionaries Key: SPARK-32686 URL: https://issues.apache.org/jira/browse/SPARK-32686 Project:

[jira] [Updated] (SPARK-32684) Scrip transform hive serde/default-serde mode null value keep same with hive

2020-08-21 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32684: -- Description: Hive serde default NULL value is '\N' {code:java} String nullString = tbl.getProperty(

[jira] [Updated] (SPARK-32684) Scrip transform hive serde/default-serde mode null value keep same with hive

2020-08-21 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32684: -- Parent: SPARK-31936 Issue Type: Sub-task (was: Bug) > Scrip transform hive

[jira] [Commented] (SPARK-31608) Add a hybrid KVStore to make UI loading faster

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181969#comment-17181969 ] Apache Spark commented on SPARK-31608: -- User 'baohe-zhang' has created a pull request for this

[jira] [Commented] (SPARK-31608) Add a hybrid KVStore to make UI loading faster

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181968#comment-17181968 ] Apache Spark commented on SPARK-31608: -- User 'baohe-zhang' has created a pull request for this

[jira] [Created] (SPARK-32685) Script transform hive serde default field.delimit is '\t'

2020-08-21 Thread angerszhu (Jira)
angerszhu created SPARK-32685: - Summary: Script transform hive serde default field.delimit is '\t' Key: SPARK-32685 URL: https://issues.apache.org/jira/browse/SPARK-32685 Project: Spark Issue

[jira] [Updated] (SPARK-32684) Scrip transform hive serde/default-serde mode null value keep same with hive as '\\N'

2020-08-21 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32684: -- Summary: Scrip transform hive serde/default-serde mode null value keep same with hive as '\\N' (was:

[jira] [Assigned] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32682: Assignee: (was: Apache Spark) > Use workflow_dispatch to enable manual test triggers

[jira] [Commented] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181765#comment-17181765 ] Apache Spark commented on SPARK-32682: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32682: Assignee: Apache Spark > Use workflow_dispatch to enable manual test triggers >

[jira] [Commented] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181767#comment-17181767 ] Apache Spark commented on SPARK-32682: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Description: >From the docs, the pattern F should give a week of the month.

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Summary: Datetime Pattern F not working expected (was: Datetime Pattern F works wrongly.) > Datetime

[jira] [Created] (SPARK-32683) Datetime Pattern F works wrongly.

2020-08-21 Thread Daeho Ro (Jira)
Daeho Ro created SPARK-32683: Summary: Datetime Pattern F works wrongly. Key: SPARK-32683 URL: https://issues.apache.org/jira/browse/SPARK-32683 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Summary: Datetime Pattern F not working as expected (was: Datetime Pattern F not working expected) >

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Environment: Windows 10 Pro * with Jupyter Lab Docker Image ** for the spark 3.0 ** python 3.8.5 **

[jira] [Updated] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-21 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-32635: - Description: When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181868#comment-17181868 ] Robert Joseph Evans commented on SPARK-32672: - So I am able to reduce the corruption down to

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181867#comment-17181867 ] Thomas Graves commented on SPARK-32672: --- [~cltlfcjin]  Please do not be changing priority just

[jira] [Commented] (SPARK-32635) When pyspark.sql.functions.lit() function is used with dataframe cache, it returns wrong result

2020-08-21 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181800#comment-17181800 ] Vinod KC commented on SPARK-32635: -- [~sascha.baumanns], updated code section. Thanks > When

[jira] [Created] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-32682: Summary: Use workflow_dispatch to enable manual test triggers Key: SPARK-32682 URL: https://issues.apache.org/jira/browse/SPARK-32682 Project: Spark

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Description: h3. Background >From the

[jira] [Resolved] (SPARK-32682) Use workflow_dispatch to enable manual test triggers

2020-08-21 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32682. -- Fix Version/s: 3.1.0 Assignee: Takeshi Yamamuro Resolution: Fixed

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Description: >From the docs, the pattern F should give a week of the month.

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Description: >From the docs, the pattern F should give a week of the month.

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Attachment: 주석 2020-08-21 213522.png > Datetime Pattern F not working as expected >

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Attachment: (was: 주석 2020-08-21 213522.png) > Datetime Pattern F not working as expected >

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Attachment: comment.png > Datetime Pattern F not working as expected >

[jira] [Commented] (SPARK-29654) Add configuration to allow disabling registration of static sources to the metrics system

2020-08-21 Thread Pavol Knapek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181827#comment-17181827 ] Pavol Knapek commented on SPARK-29654: -- Is there any chance to have this backported to a minor

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181873#comment-17181873 ] Robert Joseph Evans commented on SPARK-32672: - OK reading through the code I understand what

[jira] [Assigned] (SPARK-32680) CTAS with V2 catalog wrongly accessed unresolved query

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32680: Assignee: Apache Spark > CTAS with V2 catalog wrongly accessed unresolved query >

[jira] [Commented] (SPARK-32680) CTAS with V2 catalog wrongly accessed unresolved query

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181912#comment-17181912 ] Apache Spark commented on SPARK-32680: -- User 'linhongliu-db' has created a pull request for this

[jira] [Assigned] (SPARK-32680) CTAS with V2 catalog wrongly accessed unresolved query

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32680: Assignee: (was: Apache Spark) > CTAS with V2 catalog wrongly accessed unresolved

[jira] [Assigned] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32672: Assignee: Apache Spark > Data corruption in some cached compressed boolean columns >

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181910#comment-17181910 ] Apache Spark commented on SPARK-32672: -- User 'revans2' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32672: Assignee: (was: Apache Spark) > Data corruption in some cached compressed boolean

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181909#comment-17181909 ] Apache Spark commented on SPARK-32672: -- User 'revans2' has created a pull request for this issue:

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Robert Joseph Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-32672: Attachment: small_bad.snappy.parquet > Data corruption in some cached compressed

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Environment: Windows 10 Pro * with Jupyter Lab - Docker Image  **

[jira] [Updated] (SPARK-32683) Datetime Pattern F not working as expected

2020-08-21 Thread Daeho Ro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daeho Ro updated SPARK-32683: - Environment: Windows 10 Pro * with Jupyter Lab - Docker Image  **

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32672: -- Affects Version/s: 2.3.4 > Data corruption in some cached compressed boolean columns >

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32672: -- Affects Version/s: 2.2.3 > Data corruption in some cached compressed boolean columns >

[jira] [Resolved] (SPARK-32662) CountVectorizerModel: Remove requirement for minimum vocabulary size

2020-08-21 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-32662. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29482

[jira] [Assigned] (SPARK-32662) CountVectorizerModel: Remove requirement for minimum vocabulary size

2020-08-21 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao reassigned SPARK-32662: -- Assignee: Jatin Puri > CountVectorizerModel: Remove requirement for minimum vocabulary size

[jira] [Resolved] (SPARK-32669) expression unit tests should explore all cases that can lead to null result

2020-08-21 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32669. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-32642) Add support for ESS in Spark sidecar

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182160#comment-17182160 ] Apache Spark commented on SPARK-32642: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32642) Add support for ESS in Spark sidecar

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182158#comment-17182158 ] Apache Spark commented on SPARK-32642: -- User 'holdenk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32642) Add support for ESS in Spark sidecar

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32642: Assignee: Apache Spark (was: Holden Karau) > Add support for ESS in Spark sidecar >

[jira] [Assigned] (SPARK-32642) Add support for ESS in Spark sidecar

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32642: Assignee: Holden Karau (was: Apache Spark) > Add support for ESS in Spark sidecar >

[jira] [Updated] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32672: -- Affects Version/s: (was: 2.2.3) > Data corruption in some cached compressed boolean

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182156#comment-17182156 ] Dongjoon Hyun commented on SPARK-32672: --- Thank you, [~revans2]. > Data corruption in some cached

[jira] [Assigned] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32686: Assignee: Apache Spark > Un-deprecate inferring DataFrame schema from list of

[jira] [Commented] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182045#comment-17182045 ] Apache Spark commented on SPARK-32686: -- User 'nchammas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32686) Un-deprecate inferring DataFrame schema from list of dictionaries

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32686: Assignee: (was: Apache Spark) > Un-deprecate inferring DataFrame schema from list of

[jira] [Resolved] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32672. -- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution:

[jira] [Assigned] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32672: Assignee: Robert Joseph Evans > Data corruption in some cached compressed boolean

[jira] [Updated] (SPARK-32646) ORC predicate pushdown should work with case-insensitive analysis

2020-08-21 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32646: Affects Version/s: 3.0.0 > ORC predicate pushdown should work with case-insensitive analysis >

[jira] [Commented] (SPARK-32646) ORC predicate pushdown should work with case-insensitive analysis

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182218#comment-17182218 ] Apache Spark commented on SPARK-32646: -- User 'viirya' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182225#comment-17182225 ] Hyukjin Kwon edited comment on SPARK-32672 at 8/22/20, 4:57 AM:

[jira] [Comment Edited] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182225#comment-17182225 ] Hyukjin Kwon edited comment on SPARK-32672 at 8/22/20, 2:27 AM:

[jira] [Commented] (SPARK-32672) Data corruption in some cached compressed boolean columns

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182225#comment-17182225 ] Hyukjin Kwon commented on SPARK-32672: -- [~revans2] is a PMC, and it is a correctness issue. This

[jira] [Commented] (SPARK-26164) [SQL] Allow FileFormatWriter to write multiple partitions/buckets without sort

2020-08-21 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182239#comment-17182239 ] Cheng Su commented on SPARK-26164: -- [~cloud_fan], [~hyukjin.kwon] - wondering how do you think of this

sub

2020-08-21 Thread Bowen Li

[jira] [Created] (SPARK-32680) CTAS with V2 catalog wrongly accessed unresolved query

2020-08-21 Thread Linhong Liu (Jira)
Linhong Liu created SPARK-32680: --- Summary: CTAS with V2 catalog wrongly accessed unresolved query Key: SPARK-32680 URL: https://issues.apache.org/jira/browse/SPARK-32680 Project: Spark Issue

[jira] [Commented] (SPARK-32648) Remove unused DELETE_ACTION in FileStreamSinkLog

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181735#comment-17181735 ] Apache Spark commented on SPARK-32648: -- User 'michal-wieleba' has created a pull request for this

[jira] [Assigned] (SPARK-32648) Remove unused DELETE_ACTION in FileStreamSinkLog

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32648: Assignee: Apache Spark > Remove unused DELETE_ACTION in FileStreamSinkLog >

[jira] [Assigned] (SPARK-32648) Remove unused DELETE_ACTION in FileStreamSinkLog

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32648: Assignee: (was: Apache Spark) > Remove unused DELETE_ACTION in FileStreamSinkLog >

[jira] [Created] (SPARK-32681) PySpark type hints support

2020-08-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32681: Summary: PySpark type hints support Key: SPARK-32681 URL: https://issues.apache.org/jira/browse/SPARK-32681 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-32681) PySpark type hints support

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32681: - Parent: (was: SPARK-32082) Issue Type: Improvement (was: Sub-task) > PySpark type

[jira] [Commented] (SPARK-29471) "TaskResultLost (result lost from block manager)" error message is misleading in case result fetch is caused by client-side network connectivity issues

2020-08-21 Thread Chang chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181736#comment-17181736 ] Chang chen commented on SPARK-29471: I met the same issue, and eventually find the reason is caused

[jira] [Assigned] (SPARK-32663) TransportClient getting closed when there are outstanding requests to the server

2020-08-21 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-32663: --- Assignee: Attila Zsolt Piros > TransportClient getting closed when there

[jira] [Resolved] (SPARK-32663) TransportClient getting closed when there are outstanding requests to the server

2020-08-21 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-32663. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181637#comment-17181637 ] Dongjoon Hyun commented on SPARK-32517: --- So, the failure situation looks like this. For the

[jira] [Updated] (SPARK-32669) expression unit tests should explore all cases that can lead to null result

2020-08-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-32669: Description: Add document to {{ExpressionEvalHelper}}, and ask people to explore all the cases

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181641#comment-17181641 ] Dongjoon Hyun commented on SPARK-32517: --- BTW, this is not a bug in a new code because

[jira] [Updated] (SPARK-32677) Cache function after create

2020-08-21 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32677: Summary: Cache function after create (was: Cache function after create function) > Cache

[jira] [Updated] (SPARK-32677) Cache function directly after create

2020-08-21 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32677: Description: Change `CreateFunctionCommand` code that add class check before create function and

[jira] [Updated] (SPARK-32677) Cache function after create function

2020-08-21 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32677: Summary: Cache function after create function (was: Add class check before create function) >

[jira] [Commented] (SPARK-32677) Cache function directly after create

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181651#comment-17181651 ] Apache Spark commented on SPARK-32677: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-32677) Cache function directly after create

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32677: Assignee: Apache Spark > Cache function directly after create >

[jira] [Commented] (SPARK-32677) Cache function directly after create

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181650#comment-17181650 ] Apache Spark commented on SPARK-32677: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-32677) Cache function directly after create

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32677: Assignee: (was: Apache Spark) > Cache function directly after create >

[jira] [Created] (SPARK-32678) Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code

2020-08-21 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32678: --- Summary: Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code Key: SPARK-32678 URL: https://issues.apache.org/jira/browse/SPARK-32678 Project:

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181635#comment-17181635 ] Dongjoon Hyun commented on SPARK-32517: --- Thank you for informing that, [~huangtianhua]. I'll take

[jira] [Assigned] (SPARK-32678) Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32678: Assignee: Apache Spark > Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ

[jira] [Assigned] (SPARK-32678) Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32678: Assignee: (was: Apache Spark) > Rename EmptyHashedRelationWithAllNullKeys and

[jira] [Commented] (SPARK-32678) Rename EmptyHashedRelationWithAllNullKeys and simplify NAAJ generated code

2020-08-21 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181677#comment-17181677 ] Apache Spark commented on SPARK-32678: -- User 'leanken' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32667) Scrip transformation no-serde mode when column less then output length , Use null fill

2020-08-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32667: --- Assignee: angerszhu > Scrip transformation no-serde mode when column less then output

[jira] [Resolved] (SPARK-32667) Scrip transformation no-serde mode when column less then output length , Use null fill

2020-08-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32667. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29500

[jira] [Created] (SPARK-32679) update "no-serde" in the codebase in other TRANSFORM PRs.

2020-08-21 Thread angerszhu (Jira)
angerszhu created SPARK-32679: - Summary: update "no-serde" in the codebase in other TRANSFORM PRs. Key: SPARK-32679 URL: https://issues.apache.org/jira/browse/SPARK-32679 Project: Spark Issue

[jira] [Assigned] (SPARK-32674) Add suggestion for parallel directory listing in tuning doc

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32674: Assignee: Chao Sun > Add suggestion for parallel directory listing in tuning doc >

[jira] [Resolved] (SPARK-32674) Add suggestion for parallel directory listing in tuning doc

2020-08-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32674. -- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution:

[jira] [Resolved] (SPARK-32646) ORC predicate pushdown should work with case-insensitive analysis

2020-08-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32646. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29457

[jira] [Updated] (SPARK-32669) expression unit tests should explore all cases that can lead to null result

2020-08-21 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-32669: Summary: expression unit tests should explore all cases that can lead to null result (was: test

[jira] [Created] (SPARK-32677) Add class check before create function

2020-08-21 Thread ulysses you (Jira)
ulysses you created SPARK-32677: --- Summary: Add class check before create function Key: SPARK-32677 URL: https://issues.apache.org/jira/browse/SPARK-32677 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32517) Add StorageLevel.DISK_ONLY_3

2020-08-21 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181642#comment-17181642 ] Dongjoon Hyun commented on SPARK-32517: --- Lastly, thank you for your persistent interest on ARM

[jira] [Updated] (SPARK-32677) Cache function directly after create

2020-08-21 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-32677: Summary: Cache function directly after create (was: Cache function after create) > Cache

  1   2   >