[jira] [Updated] (SPARK-27943) Implement default constraint with Column for Hive table

[ https://issues.apache.org/jira/browse/SPARK-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-27943: --- Summary: Implement default constraint with Column for Hive table (was: Add default constraint when

[jira] [Updated] (SPARK-27943) Add default constraint when create hive table

[ https://issues.apache.org/jira/browse/SPARK-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-27943: --- Description: Default constraint with column is ANSI standard. Hive 3.0+ has supported default const

[jira] [Commented] (SPARK-27798) from_avro can modify variables in other rows in local mode

[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856370#comment-16856370 ] Gengliang Wang commented on SPARK-27798: [~viirya] I am currently working on oth

[jira] [Commented] (SPARK-27798) from_avro can modify variables in other rows in local mode

[ https://issues.apache.org/jira/browse/SPARK-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856362#comment-16856362 ] Liang-Chi Hsieh commented on SPARK-27798: - Is anyone working one this? If none,

[jira] [Created] (SPARK-27952) Built-in function: lag

Zhu, Lipeng created SPARK-27952: --- Summary: Built-in function: lag Key: SPARK-27952 URL: https://issues.apache.org/jira/browse/SPARK-27952 Project: Spark Issue Type: Sub-task Component

[jira] [Updated] (SPARK-27951) Built-in function: NTH_VALUE

[ https://issues.apache.org/jira/browse/SPARK-27951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-27951: Description: |{{nth_value({{value}}{{any}}, {{nth}}{{integer}})}}|{{same type as  }}{{value}}|retu

[jira] [Created] (SPARK-27951) Built-in function: NTH_VALUE

Zhu, Lipeng created SPARK-27951: --- Summary: Built-in function: NTH_VALUE Key: SPARK-27951 URL: https://issues.apache.org/jira/browse/SPARK-27951 Project: Spark Issue Type: Sub-task Com

[jira] [Updated] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27949: Issue Type: Sub-task (was: Improvement) Parent: SPARK-27764 > Support ANSI SQL grammar `s

[jira] [Updated] (SPARK-27578) Support INTERVAL ... HOUR TO SECOND syntax

[ https://issues.apache.org/jira/browse/SPARK-27578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-27578: Issue Type: Sub-task (was: Improvement) Parent: SPARK-27764 > Support INTERVAL ... HOUR T

[jira] [Resolved] (SPARK-27948) GPU Scheduling - Use ResouceName to represent resource names

[ https://issues.apache.org/jira/browse/SPARK-27948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27948. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Updated] (SPARK-27950) Additional DynamoDB and CloudWatch config

[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric S Meisel updated SPARK-27950: -- Priority: Minor (was: Major) > Additional DynamoDB and CloudWatch config > --

[jira] [Updated] (SPARK-27909) Fix CTE substitution dependence on ResolveRelations throwing AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-27909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27909: -- Fix Version/s: 3.0.0 > Fix CTE substitution dependence on ResolveRelations throwing AnalysisEx

[jira] [Commented] (SPARK-27950) Additional DynamoDB and CloudWatch config

[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856275#comment-16856275 ] Apache Spark commented on SPARK-27950: -- User 'etspaceman' has created a pull reques

[jira] [Assigned] (SPARK-27950) Additional DynamoDB and CloudWatch config

[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27950: Assignee: (was: Apache Spark) > Additional DynamoDB and CloudWatch config > -

[jira] [Assigned] (SPARK-27950) Additional DynamoDB and CloudWatch config

[ https://issues.apache.org/jira/browse/SPARK-27950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27950: Assignee: Apache Spark > Additional DynamoDB and CloudWatch config >

[jira] [Created] (SPARK-27950) Additional DynamoDB and CloudWatch config

Eric S Meisel created SPARK-27950: - Summary: Additional DynamoDB and CloudWatch config Key: SPARK-27950 URL: https://issues.apache.org/jira/browse/SPARK-27950 Project: Spark Issue Type: Impro

[jira] [Updated] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27949: -- Priority: Minor (was: Major) > Support ANSI SQL grammar `substring(string_expression from n1

[jira] [Updated] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27949: -- Issue Type: Improvement (was: Bug) > Support ANSI SQL grammar `substring(string_expression fr

[jira] [Assigned] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27949: Assignee: Apache Spark > Support ANSI SQL grammar `substring(string_expression from n1 [f

[jira] [Assigned] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

[ https://issues.apache.org/jira/browse/SPARK-27949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27949: Assignee: (was: Apache Spark) > Support ANSI SQL grammar `substring(string_expression

[jira] [Created] (SPARK-27949) Support ANSI SQL grammar `substring(string_expression from n1 [for n2])`

Zhu, Lipeng created SPARK-27949: --- Summary: Support ANSI SQL grammar `substring(string_expression from n1 [for n2])` Key: SPARK-27949 URL: https://issues.apache.org/jira/browse/SPARK-27949 Project: Spark

[jira] [Assigned] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27947: Assignee: (was: Apache Spark) > ParsedStatement subclass toString may throw ClassCast

[jira] [Assigned] (SPARK-27948) GPU Scheduling - Use ResouceName to represent resource names

[ https://issues.apache.org/jira/browse/SPARK-27948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27948: Assignee: Xingbo Jiang (was: Apache Spark) > GPU Scheduling - Use ResouceName to represe

[jira] [Assigned] (SPARK-27948) GPU Scheduling - Use ResouceName to represent resource names

[ https://issues.apache.org/jira/browse/SPARK-27948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27948: Assignee: Apache Spark (was: Xingbo Jiang) > GPU Scheduling - Use ResouceName to represe

[jira] [Assigned] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27947: Assignee: Apache Spark > ParsedStatement subclass toString may throw ClassCastException >

[jira] [Created] (SPARK-27948) GPU Scheduling - Use ResouceName to represent resource names

Xingbo Jiang created SPARK-27948: Summary: GPU Scheduling - Use ResouceName to represent resource names Key: SPARK-27948 URL: https://issues.apache.org/jira/browse/SPARK-27948 Project: Spark

[jira] [Updated] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27947: --- Description: In ParsedStatement.productIterator, `case mapArg: Map[_, _]` may match any Map type, t

[jira] [Updated] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27947: --- Description: In ParsedStatement.productIterator, `case mapArg: Map[_, _]` may match any Map type, t

[jira] [Updated] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27947: --- Description: In ParsedStatement.productIterator, `case mapArg: Map[_, _]` may match any Map type, t

[jira] [Updated] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27947: --- Description: In ParsedStatement.productIterator, "case mapArg: Map[_, _]" may match any Map type, t

[jira] [Created] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

John Zhuge created SPARK-27947: -- Summary: ParsedStatement subclass toString may throw ClassCastException Key: SPARK-27947 URL: https://issues.apache.org/jira/browse/SPARK-27947 Project: Spark I

[jira] [Updated] (SPARK-27947) ParsedStatement subclass toString may throw ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-27947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-27947: --- Description: In ParsedStatement.productIterator, "case mapArg: Map[_, _]" may match any Map type, t

[jira] [Resolved] (SPARK-27366) Spark scheduler internal changes to support GPU scheduling

[ https://issues.apache.org/jira/browse/SPARK-27366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-27366. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24374 [https://

[jira] [Comment Edited] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855948#comment-16855948 ] John Bauer edited comment on SPARK-17025 at 6/4/19 11:12 PM: -

[jira] [Comment Edited] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855948#comment-16855948 ] John Bauer edited comment on SPARK-17025 at 6/4/19 11:10 PM: -

[jira] [Assigned] (SPARK-27724) DataSourceV2: Add RTAS logical operation

[ https://issues.apache.org/jira/browse/SPARK-27724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27724: Assignee: (was: Apache Spark) > DataSourceV2: Add RTAS logical operation > --

[jira] [Assigned] (SPARK-27724) DataSourceV2: Add RTAS logical operation

[ https://issues.apache.org/jira/browse/SPARK-27724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27724: Assignee: Apache Spark > DataSourceV2: Add RTAS logical operation > -

[jira] [Resolved] (SPARK-27909) Fix CTE substitution dependence on ResolveRelations throwing AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-27909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27909. - Resolution: Fixed Assignee: Ryan Blue > Fix CTE substitution dependence on ResolveRelations throwi

[jira] [Updated] (SPARK-27909) Fix CTE substitution dependence on ResolveRelations throwing AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-27909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27909: Issue Type: Improvement (was: Bug) > Fix CTE substitution dependence on ResolveRelations throwing Analysi

[jira] [Commented] (SPARK-27909) Fix CTE substitution dependence on ResolveRelations throwing AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-27909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856138#comment-16856138 ] Xiao Li commented on SPARK-27909: - We do not support recursive CTE. I change it to an im

[jira] [Created] (SPARK-27946) Hive DDL to Spark DDL conversion feature for "show create table"

Xiao Li created SPARK-27946: --- Summary: Hive DDL to Spark DDL conversion feature for "show create table" Key: SPARK-27946 URL: https://issues.apache.org/jira/browse/SPARK-27946 Project: Spark Issue

[jira] [Updated] (SPARK-27946) Hive DDL to Spark DDL conversion feature for "show create table"

[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27946: Issue Type: Improvement (was: Bug) > Hive DDL to Spark DDL conversion feature for "show create table" > -

[jira] [Updated] (SPARK-27946) Hive DDL to Spark DDL conversion USING "show create table"

[ https://issues.apache.org/jira/browse/SPARK-27946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27946: Summary: Hive DDL to Spark DDL conversion USING "show create table" (was: Hive DDL to Spark DDL conversio

[jira] [Updated] (SPARK-27900) Spark driver will not exit due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Summary: Spark driver will not exit due to an oom error (was: Spark will not exit

[jira] [Updated] (SPARK-27900) Spark will not exit due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Description: This affects Spark on K8s at least as pods will run forever. A spark

[jira] [Updated] (SPARK-27900) Spark will not exit due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Description: This affects Spark on K8s at least as pods will run forever and makes

[jira] [Updated] (SPARK-27900) Spark will not exit due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Summary: Spark will not exit due to an oom error (was: Spark on K8s will not repo

[jira] [Updated] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Component/s: (was: Kubernetes) > Spark on K8s will not report container failur

[jira] [Updated] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-27900: Component/s: Spark Core > Spark on K8s will not report container failure due to an

[jira] [Assigned] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27900: Assignee: (was: Apache Spark) > Spark on K8s will not report container failure due to

[jira] [Assigned] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27900: Assignee: Apache Spark > Spark on K8s will not report container failure due to an oom err

[jira] [Issue Comment Deleted] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-20202: - Comment: was deleted (was: gentle ping) > Remove references to org.spark-project.hive > ---

[jira] [Issue Comment Deleted] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-20202: - Comment: was deleted (was: bump) > Remove references to org.spark-project.hive > --

[jira] [Resolved] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg resolved SPARK-27567. --- Resolution: Not A Bug In our case, issue was caused by someone changing the IP addre

[jira] [Commented] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855996#comment-16855996 ] Dmitry Goldenberg commented on SPARK-27567: --- OK, it appears that the IP addres

[jira] [Resolved] (SPARK-27939) Defining a schema with VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27939. -- Resolution: Not A Problem > Defining a schema with VectorUDT > ---

[jira] [Comment Edited] (SPARK-27939) Defining a schema with VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855969#comment-16855969 ] Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:13 PM:

[jira] [Commented] (SPARK-27939) Defining a schema with VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855969#comment-16855969 ] Bryan Cutler commented on SPARK-27939: -- Another problem with Python {{Row}} class

[jira] [Comment Edited] (SPARK-27939) Defining a schema with VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855966#comment-16855966 ] Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:11 PM:

[jira] [Commented] (SPARK-27939) Defining a schema with VectorUDT

[ https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855966#comment-16855966 ] Bryan Cutler commented on SPARK-27939: -- The problem is the {{Row}} class sorts the

[jira] [Assigned] (SPARK-27945) Make minimal changes to support columnar processing

[ https://issues.apache.org/jira/browse/SPARK-27945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27945: Assignee: Apache Spark > Make minimal changes to support columnar processing > --

[jira] [Assigned] (SPARK-27945) Make minimal changes to support columnar processing

[ https://issues.apache.org/jira/browse/SPARK-27945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27945: Assignee: (was: Apache Spark) > Make minimal changes to support columnar processing >

[jira] [Commented] (SPARK-17025) Cannot persist PySpark ML Pipeline model that includes custom Transformer

[ https://issues.apache.org/jira/browse/SPARK-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855948#comment-16855948 ] John Bauer commented on SPARK-17025: [~Hadar] [~yug95] I wrote a minimal example of

[jira] [Updated] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

[ https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27805: - Affects Version/s: (was: 3.1.0) 2.4.3 > toPandas does not propagate S

[jira] [Resolved] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

[ https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-27805. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24677 [https://gi

[jira] [Assigned] (SPARK-27805) toPandas does not propagate SparkExceptions with arrow enabled

[ https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler reassigned SPARK-27805: Assignee: David Vogelbacher > toPandas does not propagate SparkExceptions with arrow enab

[jira] [Commented] (SPARK-27888) Python 2->3 migration guide for PySpark users

[ https://issues.apache.org/jira/browse/SPARK-27888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855896#comment-16855896 ] Hyukjin Kwon commented on SPARK-27888: -- Hm, Pyspark as far as I can tell should wor

[jira] [Updated] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-27396: Epic Name: Public APIs for extended Columnar Processing Support > SPIP: Public API

[jira] [Created] (SPARK-27945) Make minimal changes to support columnar processing

Robert Joseph Evans created SPARK-27945: --- Summary: Make minimal changes to support columnar processing Key: SPARK-27945 URL: https://issues.apache.org/jira/browse/SPARK-27945 Project: Spark

[jira] [Updated] (SPARK-27396) SPIP: Public APIs for extended Columnar Processing Support

[ https://issues.apache.org/jira/browse/SPARK-27396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated SPARK-27396: Issue Type: Epic (was: Improvement) > SPIP: Public APIs for extended Columnar Pro

[jira] [Updated] (SPARK-27933) Extracting common purge "behaviour" to the parent StreamExecution

[ https://issues.apache.org/jira/browse/SPARK-27933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-27933: Summary: Extracting common purge "behaviour" to the parent StreamExecution (was: Introduc

[jira] [Commented] (SPARK-27888) Python 2->3 migration guide for PySpark users

[ https://issues.apache.org/jira/browse/SPARK-27888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855838#comment-16855838 ] Xiangrui Meng commented on SPARK-27888: --- It would be nice if we can find some PySp

[jira] [Assigned] (SPARK-27884) Deprecate Python 2 support in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-27884: - Assignee: Xiangrui Meng > Deprecate Python 2 support in Spark 3.0 > ---

[jira] [Commented] (SPARK-27888) Python 2->3 migration guide for PySpark users

[ https://issues.apache.org/jira/browse/SPARK-27888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855832#comment-16855832 ] Xiangrui Meng commented on SPARK-27888: --- This JIRA is not to inform users that Pyt

[jira] [Resolved] (SPARK-27886) Add Apache Spark project to https://python3statement.org/

[ https://issues.apache.org/jira/browse/SPARK-27886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-27886. --- Resolution: Done > Add Apache Spark project to https://python3statement.org/ > -

[jira] [Updated] (SPARK-27942) Note that Python 2.7 is deprecated in Spark documentation

[ https://issues.apache.org/jira/browse/SPARK-27942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27942: -- Component/s: Documentation > Note that Python 2.7 is deprecated in Spark documentation > -

[jira] [Resolved] (SPARK-27942) Note that Python 2.7 is deprecated in Spark documentation

[ https://issues.apache.org/jira/browse/SPARK-27942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27942. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via https://github.com/apache

[jira] [Resolved] (SPARK-27772) SQLTestUtils Refactoring

[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27772. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24747 [https://github.c

[jira] [Assigned] (SPARK-27772) SQLTestUtils Refactoring

[ https://issues.apache.org/jira/browse/SPARK-27772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27772: - Assignee: William Wong > SQLTestUtils Refactoring > > >

[jira] [Reopened] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg reopened SPARK-27567: --- Issue not resolved, it appears intermittently. We have a QA box with Kafka installed an

[jira] [Updated] (SPARK-18569) Support R formula arithmetic

[ https://issues.apache.org/jira/browse/SPARK-18569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18569: -- Affects Version/s: 2.4.3 Issue Type: Improvement (was: Sub-task) Parent:

[jira] [Resolved] (SPARK-18570) Consider supporting other R formula operators

[ https://issues.apache.org/jira/browse/SPARK-18570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18570. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24764 [https://github.c

[jira] [Assigned] (SPARK-18570) Consider supporting other R formula operators

[ https://issues.apache.org/jira/browse/SPARK-18570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-18570: - Assignee: Ozan Cicekci > Consider supporting other R formula operators > --

[jira] [Assigned] (SPARK-27944) Unify the behavior of checking empty output column names

[ https://issues.apache.org/jira/browse/SPARK-27944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27944: Assignee: Apache Spark > Unify the behavior of checking empty output column names > -

[jira] [Assigned] (SPARK-27944) Unify the behavior of checking empty output column names

[ https://issues.apache.org/jira/browse/SPARK-27944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27944: Assignee: (was: Apache Spark) > Unify the behavior of checking empty output column na

[jira] [Commented] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855598#comment-16855598 ] Stavros Kontopoulos commented on SPARK-27900: - Setting Thread.setDefaultUnca

[jira] [Created] (SPARK-27944) Unify the behavior of checking empty output column names

zhengruifeng created SPARK-27944: Summary: Unify the behavior of checking empty output column names Key: SPARK-27944 URL: https://issues.apache.org/jira/browse/SPARK-27944 Project: Spark Issu

[jira] [Assigned] (SPARK-27943) Add default constraint when create hive table

[ https://issues.apache.org/jira/browse/SPARK-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27943: Assignee: Apache Spark > Add default constraint when create hive table >

[jira] [Assigned] (SPARK-27943) Add default constraint when create hive table

[ https://issues.apache.org/jira/browse/SPARK-27943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27943: Assignee: (was: Apache Spark) > Add default constraint when create hive table > -

[jira] [Commented] (SPARK-27895) Spark streaming - RDD filter is always refreshing providing updated filtered items

[ https://issues.apache.org/jira/browse/SPARK-27895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855547#comment-16855547 ] Ilias Karalis commented on SPARK-27895: --- The results of RDD filter change all the

[jira] [Updated] (SPARK-27895) Spark streaming - RDD filter is always refreshing providing updated filtered items

[ https://issues.apache.org/jira/browse/SPARK-27895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilias Karalis updated SPARK-27895: -- Description: Spark streaming: 2.4.x Scala: 2.11.11     foreachRDD of DStream, in case fil

[jira] [Created] (SPARK-27943) Add default constraint when create hive table

jiaan.geng created SPARK-27943: -- Summary: Add default constraint when create hive table Key: SPARK-27943 URL: https://issues.apache.org/jira/browse/SPARK-27943 Project: Spark Issue Type: New Fea

[jira] [Updated] (SPARK-27941) Serverless Spark in the Cloud

[ https://issues.apache.org/jira/browse/SPARK-27941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuheng Dai updated SPARK-27941: Description: Public cloud providers have started offering serverless container services. For exam

[jira] [Comment Edited] (SPARK-27900) Spark on K8s will not report container failure due to an oom error

[ https://issues.apache.org/jira/browse/SPARK-27900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854449#comment-16854449 ] Stavros Kontopoulos edited comment on SPARK-27900 at 6/4/19 9:13 AM: -

[jira] [Updated] (SPARK-27941) Serverless Spark in the Cloud

[ https://issues.apache.org/jira/browse/SPARK-27941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuheng Dai updated SPARK-27941: Description: Public cloud providers have started offering serverless container services. For exam

[jira] [Updated] (SPARK-27941) Serverless Spark in the Cloud

[ https://issues.apache.org/jira/browse/SPARK-27941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuheng Dai updated SPARK-27941: Description: Public cloud providers have started offering serverless container services. For exam

[jira] [Updated] (SPARK-27941) Serverless Spark in the Cloud

[ https://issues.apache.org/jira/browse/SPARK-27941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuheng Dai updated SPARK-27941: Description: Public cloud providers have started offering serverless container services. For exam

[jira] [Updated] (SPARK-27873) Csv reader, adding a corrupt record column causes error if enforceSchema=false

[ https://issues.apache.org/jira/browse/SPARK-27873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27873: - Fix Version/s: 2.4.4 > Csv reader, adding a corrupt record column causes error if enforceSchema=

[jira] [Updated] (SPARK-27884) Deprecate Python 2 support in Spark 3.0

[ https://issues.apache.org/jira/browse/SPARK-27884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27884: - Labels: release-notes (was: ) > Deprecate Python 2 support in Spark 3.0 > -

[jira] [Commented] (SPARK-27888) Python 2->3 migration guide for PySpark users

[ https://issues.apache.org/jira/browse/SPARK-27888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855420#comment-16855420 ] Hyukjin Kwon commented on SPARK-27888: -- [~mengxr], I know it's good to inform that

  1   2   >