[jira] [Updated] (SPARK-33391) element_at with CreateArray not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33391: Description: var df = spark.sql("select element_at(array(3, 2, 1), 0)") df.printSchema() df =

[jira] [Updated] (SPARK-33391) element_at with CreateArray not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33391: Summary: element_at with CreateArray not respect one based index (was: element_at not respect

[jira] [Created] (SPARK-33391) element_at not respect one based index

2020-11-08 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33391: --- Summary: element_at not respect one based index Key: SPARK-33391 URL: https://issues.apache.org/jira/browse/SPARK-33391 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-33390) Make Literal support char array

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33390: Assignee: Apache Spark > Make Literal support char array >

[jira] [Commented] (SPARK-33390) Make Literal support char array

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228385#comment-17228385 ] Apache Spark commented on SPARK-33390: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-33390) Make Literal support char array

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33390: Assignee: (was: Apache Spark) > Make Literal support char array >

[jira] [Created] (SPARK-33390) Make Literal support char array

2020-11-08 Thread ulysses you (Jira)
ulysses you created SPARK-33390: --- Summary: Make Literal support char array Key: SPARK-33390 URL: https://issues.apache.org/jira/browse/SPARK-33390 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-11-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32405: --- Assignee: Huaxin Gao > Apply table options while creating tables in JDBC Table Catalog >

[jira] [Resolved] (SPARK-32405) Apply table options while creating tables in JDBC Table Catalog

2020-11-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32405. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30154

[jira] [Created] (SPARK-33389) make internal classes of SparkSession always using active SQLConf

2020-11-08 Thread Lu Lu (Jira)
Lu Lu created SPARK-33389: - Summary: make internal classes of SparkSession always using active SQLConf Key: SPARK-33389 URL: https://issues.apache.org/jira/browse/SPARK-33389 Project: Spark Issue

[jira] [Resolved] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33387. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30293

[jira] [Assigned] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33387: - Assignee: Dongjoon Hyun > Support ordered shuffle block migration >

[jira] [Updated] (SPARK-33140) make all sub-class of Rule[QueryPlan] using SQLConf.get

2020-11-08 Thread Lu Lu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu Lu updated SPARK-33140: -- Summary: make all sub-class of Rule[QueryPlan] using SQLConf.get (was: make Analyzer rules using

[jira] [Commented] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228356#comment-17228356 ] Apache Spark commented on SPARK-33140: -- User 'linhongliu-db' has created a pull request for this

[jira] [Commented] (SPARK-33140) make Analyzer rules using SQLConf.get

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228357#comment-17228357 ] Apache Spark commented on SPARK-33140: -- User 'linhongliu-db' has created a pull request for this

[jira] [Updated] (SPARK-33371) Support Python 3.9+ in PySpark

2020-11-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33371: -- Fix Version/s: 3.0.2 > Support Python 3.9+ in PySpark > -- > >

[jira] [Updated] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33387: -- Description: Since the current shuffle block migration works in a random order, the failure

[jira] [Created] (SPARK-33388) Merge In and InSet predicate

2020-11-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33388: --- Summary: Merge In and InSet predicate Key: SPARK-33388 URL: https://issues.apache.org/jira/browse/SPARK-33388 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33387: Assignee: Apache Spark > Support ordered shuffle block migration >

[jira] [Assigned] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33387: Assignee: (was: Apache Spark) > Support ordered shuffle block migration >

[jira] [Commented] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228341#comment-17228341 ] Apache Spark commented on SPARK-33387: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33387: Assignee: (was: Apache Spark) > Support ordered shuffle block migration >

[jira] [Created] (SPARK-33387) Support ordered shuffle block migration

2020-11-08 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-33387: - Summary: Support ordered shuffle block migration Key: SPARK-33387 URL: https://issues.apache.org/jira/browse/SPARK-33387 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-11-08 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-33244: Description: * The code paths of `spark.table` and `spark.read.table` should be the same. This

[jira] [Updated] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-11-08 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-33244: Description: * The code paths of `spark.table` and `spark.read.table` should be the same. This

[jira] [Updated] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-11-08 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-33244: Description: The code paths of `spark.table` and `spark.read.table` should be the same. This

[jira] [Updated] (SPARK-33244) Unify the code paths for spark.table and spark.read.table

2020-11-08 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-33244: Summary: Unify the code paths for spark.table and spark.read.table (was: Block reading streaming

[jira] [Commented] (SPARK-33166) Provide Search Function in Spark docs site

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228335#comment-17228335 ] Apache Spark commented on SPARK-33166: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33166) Provide Search Function in Spark docs site

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33166: Assignee: Apache Spark > Provide Search Function in Spark docs site >

[jira] [Assigned] (SPARK-33166) Provide Search Function in Spark docs site

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33166: Assignee: (was: Apache Spark) > Provide Search Function in Spark docs site >

[jira] [Updated] (SPARK-33386) Accessing array elements should failed if index is out of bound.

2020-11-08 Thread Leanken.Lin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leanken.Lin updated SPARK-33386: Description: When ansi mode enabled, accessing array element should failed with exception, but

[jira] [Created] (SPARK-33386) Accessing array elements should failed if index is out of bound.

2020-11-08 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33386: --- Summary: Accessing array elements should failed if index is out of bound. Key: SPARK-33386 URL: https://issues.apache.org/jira/browse/SPARK-33386 Project: Spark

[jira] [Resolved] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33384. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30290

[jira] [Assigned] (SPARK-33385) Bucket pruning support IsNaN

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33385: Assignee: (was: Apache Spark) > Bucket pruning support IsNaN >

[jira] [Assigned] (SPARK-33385) Bucket pruning support IsNaN

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33385: Assignee: Apache Spark > Bucket pruning support IsNaN > > >

[jira] [Commented] (SPARK-33385) Bucket pruning support IsNaN

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228324#comment-17228324 ] Apache Spark commented on SPARK-33385: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-33385) Bucket pruning support IsNaN

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228323#comment-17228323 ] Apache Spark commented on SPARK-33385: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-33385) Bucket pruning support IsNaN

2020-11-08 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-33385: --- Summary: Bucket pruning support IsNaN Key: SPARK-33385 URL: https://issues.apache.org/jira/browse/SPARK-33385 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-33352) Fix procedure-like declaration compilation warning in Scala 2.13

2020-11-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-33352: Assignee: Yang Jie > Fix procedure-like declaration compilation warning in Scala 2.13 >

[jira] [Resolved] (SPARK-33352) Fix procedure-like declaration compilation warning in Scala 2.13

2020-11-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-33352. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30255

[jira] [Commented] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228270#comment-17228270 ] Apache Spark commented on SPARK-33384: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33384: Assignee: Apache Spark (was: L. C. Hsieh) > Delete temporary file when cancelling

[jira] [Commented] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228269#comment-17228269 ] Apache Spark commented on SPARK-33384: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33384: Assignee: L. C. Hsieh (was: Apache Spark) > Delete temporary file when cancelling

[jira] [Created] (SPARK-33384) Delete temporary file when cancelling writing to final path even underlying stream throwing error

2020-11-08 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-33384: --- Summary: Delete temporary file when cancelling writing to final path even underlying stream throwing error Key: SPARK-33384 URL: https://issues.apache.org/jira/browse/SPARK-33384

[jira] [Assigned] (SPARK-33141) capture SQL configs when creating permanent views

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33141: Assignee: (was: Apache Spark) > capture SQL configs when creating permanent views >

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228069#comment-17228069 ] Apache Spark commented on SPARK-33141: -- User 'luluorta' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33141) capture SQL configs when creating permanent views

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33141: Assignee: Apache Spark > capture SQL configs when creating permanent views >

[jira] [Commented] (SPARK-33141) capture SQL configs when creating permanent views

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228030#comment-17228030 ] Apache Spark commented on SPARK-33141: -- User 'luluorta' has created a pull request for this issue:

[jira] [Updated] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-08 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Wollschläger updated SPARK-33383: --- Description: When I asked [a question on

[jira] [Updated] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-08 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Wollschläger updated SPARK-33383: --- Description: When I asked [a question on

[jira] [Updated] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-08 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Wollschläger updated SPARK-33383: --- Description: When I asked [a question on

[jira] [Updated] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-08 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-33383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Wollschläger updated SPARK-33383: --- Description: When I asked [a question on

[jira] [Created] (SPARK-33383) Improve performance of Column.isin Expression

2020-11-08 Thread Jira
Felix Wollschläger created SPARK-33383: -- Summary: Improve performance of Column.isin Expression Key: SPARK-33383 URL: https://issues.apache.org/jira/browse/SPARK-33383 Project: Spark

[jira] [Resolved] (SPARK-30527) Add IsNotNull filter when use In, InSet and InSubQuery

2020-11-08 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-30527. - Resolution: Invalid > Add IsNotNull filter when use In, InSet and InSubQuery >

[jira] [Resolved] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-11-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32860. -- Fix Version/s: 3.1.0 3.0.2 Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-32860) Encoders::bean doc incorrectly states maps are not supported

2020-11-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32860: Assignee: Dan Ziemba > Encoders::bean doc incorrectly states maps are not supported >

[jira] [Commented] (SPARK-33371) Support Python 3.9+ in PySpark

2020-11-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227971#comment-17227971 ] Apache Spark commented on SPARK-33371: -- User 'HyukjinKwon' has created a pull request for this