[jira] [Created] (SPARK-28682) ANSI SQL: Collation Support

2019-08-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28682: --- Summary: ANSI SQL: Collation Support Key: SPARK-28682 URL: https://issues.apache.org/jira/browse/SPARK-28682 Project: Spark Issue Type: Sub-task Comp

[jira] [Updated] (SPARK-27986) Support Aggregate Expressions with filter

2019-08-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27986: Summary: Support Aggregate Expressions with filter (was: Support Aggregate Expressions) > Suppor

[jira] [Updated] (SPARK-28680) redundant code, or possible bug in Partitioner that could mess up check against spark.default.parallelism

2019-08-09 Thread Chris Bedford (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Bedford updated SPARK-28680: -- Environment: master = local[*] (was: This is a suggestion to reduce (what I think is) some co

[jira] [Updated] (SPARK-28680) redundant code, or possible bug in Partitioner that could mess up check against spark.default.parallelism

2019-08-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28680: Description: This is a suggestion to reduce (what I think is) some code redundancy. Looking at thi

[jira] [Assigned] (SPARK-28676) Avoid Excessive logging from ContextCleaner

2019-08-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28676: - Assignee: Ajith S > Avoid Excessive logging from ContextCleaner > -

[jira] [Resolved] (SPARK-28676) Avoid Excessive logging from ContextCleaner

2019-08-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28676. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25396 [https://

[jira] [Updated] (SPARK-28681) Spark SQL unable to read Hive ORC table with numeric column name

2019-08-09 Thread Brian Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Zhang updated SPARK-28681: Affects Version/s: 2.4.0 > Spark SQL unable to read Hive ORC table with numeric column name >

[jira] [Created] (SPARK-28681) Spark SQL unable to read Hive ORC table with numeric column name

2019-08-09 Thread Brian Zhang (JIRA)
Brian Zhang created SPARK-28681: --- Summary: Spark SQL unable to read Hive ORC table with numeric column name Key: SPARK-28681 URL: https://issues.apache.org/jira/browse/SPARK-28681 Project: Spark

[jira] [Created] (SPARK-28680) redundant code, or possible bug in Partitioner that could mess up check against spark.default.parallelism

2019-08-09 Thread Chris Bedford (JIRA)
Chris Bedford created SPARK-28680: - Summary: redundant code, or possible bug in Partitioner that could mess up check against spark.default.parallelism Key: SPARK-28680 URL: https://issues.apache.org/jira/browse/SP

[jira] [Resolved] (SPARK-28656) Support `millennium`, `century` and `decade` at `extract()`

2019-08-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28656. --- Resolution: Fixed Issue resolved by pull request 25388 [https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-28650) Fix the guarantee of ForeachWriter

2019-08-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904085#comment-16904085 ] Shixiong Zhu commented on SPARK-28650: -- Go ahead. I'm not working on this. For the

[jira] [Updated] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

2019-08-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-28651: - Docs Text: All fields of the Structured Streaming's file source schema will be forced to be null

[jira] [Commented] (SPARK-28679) Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified

2019-08-09 Thread Alessandro Bellina (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16904004#comment-16904004 ] Alessandro Bellina commented on SPARK-28679: Working on it. > Spark Yarn Re

[jira] [Created] (SPARK-28679) Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified

2019-08-09 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-28679: - Summary: Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified Key: SPARK-28679 URL: https://issues.apache.org/jira/browse/SPARK-286

[jira] [Updated] (SPARK-21481) Add indexOf method in ml.feature.HashingTF similar to mllib.feature.HashingTF

2019-08-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21481: -- Priority: Minor (was: Major) > Add indexOf method in ml.feature.HashingTF similar to mllib.feature.Ha

[jira] [Assigned] (SPARK-27371) Standalone master receives resource info from worker and allocate driver/executor properly

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-27371: - Assignee: wuyi > Standalone master receives resource info from worker and allocate > d

[jira] [Resolved] (SPARK-27368) Design: Standalone supports GPU scheduling

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27368. --- Resolution: Fixed resolving since code committed > Design: Standalone supports GPU scheduli

[jira] [Resolved] (SPARK-27371) Standalone master receives resource info from worker and allocate driver/executor properly

2019-08-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-27371. --- Resolution: Fixed Fix Version/s: 3.0.0 > Standalone master receives resource info fro

[jira] [Created] (SPARK-28678) Specify that start index is 1-based in docstring of pyspark.sql.functions.slice

2019-08-09 Thread Sivam Pasupathipillai (JIRA)
Sivam Pasupathipillai created SPARK-28678: - Summary: Specify that start index is 1-based in docstring of pyspark.sql.functions.slice Key: SPARK-28678 URL: https://issues.apache.org/jira/browse/SPARK-28678

[jira] [Commented] (SPARK-28653) Create table using DDL statement should not auto create the destination folder

2019-08-09 Thread pavithra ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903834#comment-16903834 ] pavithra ramachandran commented on SPARK-28653: --- i will check this > Crea

[jira] [Updated] (SPARK-28677) "Select All" checkbox in StagePage doesn't work properly

2019-08-09 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-28677: --- Attachment: Screenshot from 2019-08-09 18-46-05.png > "Select All" checkbox in StagePage doe

[jira] [Created] (SPARK-28677) "Select All" checkbox in StagePage doesn't work properly

2019-08-09 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-28677: -- Summary: "Select All" checkbox in StagePage doesn't work properly Key: SPARK-28677 URL: https://issues.apache.org/jira/browse/SPARK-28677 Project: Spark

[jira] [Resolved] (SPARK-27884) Deprecate Python 2 support in Spark 3.0

2019-08-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27884. -- Resolution: Done > Deprecate Python 2 support in Spark 3.0 > -

[jira] [Resolved] (SPARK-27889) Make development scripts under dev/ support Python 3

2019-08-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27889. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25289 [https://gi

[jira] [Resolved] (SPARK-28651) Streaming file source doesn't change the schema to nullable automatically

2019-08-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28651. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25382 [https://gi

[jira] [Resolved] (SPARK-28163) Kafka ignores user configuration on FETCH_OFFSET_NUM_RETRY and FETCH_OFFSET_RETRY_INTERVAL_MS

2019-08-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28163. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24967 [https://gith

[jira] [Assigned] (SPARK-28163) Kafka ignores user configuration on FETCH_OFFSET_NUM_RETRY and FETCH_OFFSET_RETRY_INTERVAL_MS

2019-08-09 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28163: --- Assignee: Gabor Somogyi > Kafka ignores user configuration on FETCH_OFFSET_NUM_RETRY and >

[jira] [Created] (SPARK-28676) Avoid Excessive logging from ContextCleaner

2019-08-09 Thread Ajith S (JIRA)
Ajith S created SPARK-28676: --- Summary: Avoid Excessive logging from ContextCleaner Key: SPARK-28676 URL: https://issues.apache.org/jira/browse/SPARK-28676 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-28674) Spark should support select into from where as PostgreSQL supports

2019-08-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903663#comment-16903663 ] Yuming Wang commented on SPARK-28674: - Duplicate SPARK-28329? > Spark should suppor

[jira] [Updated] (SPARK-28675) Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions

2019-08-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28675: Description: Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions to consistent wit

[jira] [Updated] (SPARK-28675) Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions

2019-08-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28675: Summary: Replace CatalogUtils.maskCredentials with SQLConf.get.redactOptions (was: Remove Catalog

[jira] [Created] (SPARK-28675) Remove CatalogUtils.maskCredentials

2019-08-09 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28675: --- Summary: Remove CatalogUtils.maskCredentials Key: SPARK-28675 URL: https://issues.apache.org/jira/browse/SPARK-28675 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-28674) Spark should support select into from where as PostgreSQL supports

2019-08-09 Thread ABHISHEK KUMAR GUPTA (JIRA)
ABHISHEK KUMAR GUPTA created SPARK-28674: Summary: Spark should support select into from where as PostgreSQL supports Key: SPARK-28674 URL: https://issues.apache.org/jira/browse/SPARK-28674