[jira] [Resolved] (SPARK-29983) Add an independent config for an optional INTERVAL clause

2019-11-22 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29983. -- Resolution: Won't Fix > Add an independent config for an optional INTERVAL clause >

[jira] [Resolved] (SPARK-29971) Multiple possible buffer leaks in TransportFrameDecoder and TransportCipher

2019-11-22 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-29971. Fix Version/s: 3.0.0 Assignee: Norman Maurer Resolution:

[jira] [Updated] (SPARK-29427) Add API to convert RelationalGroupedDataset to KeyValueGroupedDataset

2019-11-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29427: -- Summary: Add API to convert RelationalGroupedDataset to KeyValueGroupedDataset (was: Create

[jira] [Resolved] (SPARK-29427) Create KeyValueGroupedDataset in a relational way

2019-11-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29427. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26509

[jira] [Assigned] (SPARK-29427) Create KeyValueGroupedDataset in a relational way

2019-11-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29427: - Assignee: L. C. Hsieh > Create KeyValueGroupedDataset in a relational way >

[jira] [Updated] (SPARK-30003) Prevent stack over flow in unknown hint resolution

2019-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30003: - Summary: Prevent stack over flow in unknown hint resolution (was: Prevent stack over flow in

[jira] [Updated] (SPARK-30003) Prevent stack over flow in hint resolution

2019-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-30003: - Summary: Prevent stack over flow in hint resolution (was: Prevent stack over flow in unknown

[jira] [Updated] (SPARK-30002) Reuse SparkSession in pyspark via Gateway

2019-11-22 Thread Kaushal Prajapati (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaushal Prajapati updated SPARK-30002: -- Description: In PySpark, we create SparkContext via user spark configurations or the

[jira] [Created] (SPARK-30003) Prevent stack over flow in unknown hint resolution

2019-11-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-30003: Summary: Prevent stack over flow in unknown hint resolution Key: SPARK-30003 URL: https://issues.apache.org/jira/browse/SPARK-30003 Project: Spark Issue

[jira] [Commented] (SPARK-11664) Add methods to get bisecting k-means cluster structure

2019-11-22 Thread Kavitha Rajendran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-11664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980293#comment-16980293 ] Kavitha Rajendran commented on SPARK-11664: --- Hi, I need to retrieve the tree structure

[jira] [Resolved] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-11-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29707. -- Resolution: Won't Fix > Make PartitionFilters and PushedFilters abbreviate configurable in

[jira] [Updated] (SPARK-29707) Make PartitionFilters and PushedFilters abbreviate configurable in metadata

2019-11-22 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29707: - Priority: Minor (was: Major) > Make PartitionFilters and PushedFilters abbreviate configurable

[jira] [Created] (SPARK-30002) Reuse SparkSession in pyspark via Gateway

2019-11-22 Thread Kaushal Prajapati (Jira)
Kaushal Prajapati created SPARK-30002: - Summary: Reuse SparkSession in pyspark via Gateway Key: SPARK-30002 URL: https://issues.apache.org/jira/browse/SPARK-30002 Project: Spark Issue

[jira] [Commented] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980128#comment-16980128 ] angerszhu commented on SPARK-29998: --- [~cloud_fan] No, not same problem but all caused by disk broken.

[jira] [Commented] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980126#comment-16980126 ] Wenchen Fan commented on SPARK-29998: - Does

[jira] [Commented] (SPARK-30001) can't lookup v1 tables whose names specify the session catalog

2019-11-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980101#comment-16980101 ] Wenchen Fan commented on SPARK-30001: - cc [~rdblue] [~imback82] > can't lookup v1 tables whose

[jira] [Created] (SPARK-30001) can't lookup v1 tables whose names specify the session catalog

2019-11-22 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-30001: --- Summary: can't lookup v1 tables whose names specify the session catalog Key: SPARK-30001 URL: https://issues.apache.org/jira/browse/SPARK-30001 Project: Spark

[jira] [Commented] (SPARK-29966) Add version method in TableCatalog to avoid load table twice

2019-11-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980090#comment-16980090 ] Wenchen Fan commented on SPARK-29966: - yup > Add version method in TableCatalog to avoid load table

[jira] [Updated] (SPARK-30000) Trim the string when cast string type to decimals

2019-11-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-3: - Description: https://bugs.openjdk.java.net/browse/JDK-8170259

[jira] [Commented] (SPARK-30000) Trim the string when cast string type to decimals

2019-11-22 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980087#comment-16980087 ] Kent Yao commented on SPARK-3: -- working on this. > Trim the string when cast string type to

[jira] [Created] (SPARK-30000) Trim the string when cast string type to decimals

2019-11-22 Thread Kent Yao (Jira)
Kent Yao created SPARK-3: Summary: Trim the string when cast string type to decimals Key: SPARK-3 URL: https://issues.apache.org/jira/browse/SPARK-3 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-28023) Trim the string when cast string type to other types

2019-11-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28023. - Fix Version/s: 3.0.0 Assignee: Kent Yao Resolution: Fixed > Trim the string

[jira] [Reopened] (SPARK-28023) Trim the string when cast string type to other types

2019-11-22 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-28023: - > Trim the string when cast string type to other types >

[jira] [Commented] (SPARK-29999) Writing empty partition via DSv2 implementation of FileStreamSink throws FileNotFoundException

2019-11-22 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980068#comment-16980068 ] Jungtaek Lim commented on SPARK-2: -- Will submit a patch soon. > Writing empty partition via

[jira] [Created] (SPARK-29999) Writing empty partition via DSv2 implementation of FileStreamSink throws FileNotFoundException

2019-11-22 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-2: Summary: Writing empty partition via DSv2 implementation of FileStreamSink throws FileNotFoundException Key: SPARK-2 URL: https://issues.apache.org/jira/browse/SPARK-2

[jira] [Commented] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16980024#comment-16980024 ] angerszhu commented on SPARK-29998: --- [~yumwang][~dongjoon] [~cloud_fan] For this problem, I think we

[jira] [Updated] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29998: -- Description: Recently, I meet one situation: One NodeManager's disk is broken. when task begin to

[jira] [Updated] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-29998: -- Description: Recently, I meet one situation: One NodeManager's disk is broken. when task begin to

[jira] [Created] (SPARK-29998) A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails

2019-11-22 Thread angerszhu (Jira)
angerszhu created SPARK-29998: - Summary: A corrupted hard disk causes the task to execute repeatedly on a machine until the job fails Key: SPARK-29998 URL: https://issues.apache.org/jira/browse/SPARK-29998

[jira] [Comment Edited] (SPARK-29995) Structured Streaming file-sink log grow indefinitely

2019-11-22 Thread zhang liming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979997#comment-16979997 ] zhang liming edited comment on SPARK-29995 at 11/22/19 9:19 AM: May I

[jira] [Commented] (SPARK-29995) Structured Streaming file-sink log grow indefinitely

2019-11-22 Thread zhang liming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979997#comment-16979997 ] zhang liming commented on SPARK-29995: -- May I delete the previous contents of the latest .compact

[jira] [Updated] (SPARK-29997) Show job name for empty jobs in WebUI

2019-11-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-29997: --- Summary: Show job name for empty jobs in WebUI (was: Show the job name for empty jobs in

[jira] [Created] (SPARK-29997) Show the job name for empty jobs in WebUI

2019-11-22 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-29997: -- Summary: Show the job name for empty jobs in WebUI Key: SPARK-29997 URL: https://issues.apache.org/jira/browse/SPARK-29997 Project: Spark Issue Type: