[jira] [Commented] (SPARK-35347) Use MethodUtils for method looking up in Invoke and StaticInvoke

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343036#comment-17343036 ] Apache Spark commented on SPARK-35347: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-35347) Use MethodUtils for method looking up in Invoke and StaticInvoke

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343035#comment-17343035 ] Apache Spark commented on SPARK-35347: -- User 'viirya' has created a pull request fo

[jira] [Updated] (SPARK-35384) Improve performance for InvokeLike.invoke

2021-05-11 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-35384: - Issue Type: Improvement (was: Bug) > Improve performance for InvokeLike.invoke > --

[jira] [Created] (SPARK-35384) Improve performance for InvokeLike.invoke

2021-05-11 Thread Chao Sun (Jira)
Chao Sun created SPARK-35384: Summary: Improve performance for InvokeLike.invoke Key: SPARK-35384 URL: https://issues.apache.org/jira/browse/SPARK-35384 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343017#comment-17343017 ] Apache Spark commented on SPARK-35383: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35383: Assignee: Apache Spark > Improve s3a magic committer support by inferring missing configs

[jira] [Assigned] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35383: Assignee: (was: Apache Spark) > Improve s3a magic committer support by inferring miss

[jira] [Updated] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35383: -- Description: {code} Exception in thread "main" org.apache.hadoop.fs.s3a.commit.PathCommitExcep

[jira] [Created] (SPARK-35383) Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-35383: - Summary: Improve s3a magic committer support by inferring missing configs Key: SPARK-35383 URL: https://issues.apache.org/jira/browse/SPARK-35383 Project: Spark

[jira] [Commented] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343007#comment-17343007 ] Apache Spark commented on SPARK-35381: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35381: Assignee: (was: Apache Spark) > Fix lambda variable name issues in nested DataFrame f

[jira] [Commented] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343006#comment-17343006 ] Apache Spark commented on SPARK-35381: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35381: Assignee: Apache Spark > Fix lambda variable name issues in nested DataFrame functions in

[jira] [Commented] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343003#comment-17343003 ] Apache Spark commented on SPARK-35364: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35364: Assignee: Apache Spark > Renaming the existing Koalas related codes. > --

[jira] [Assigned] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35364: Assignee: (was: Apache Spark) > Renaming the existing Koalas related codes. > ---

[jira] [Commented] (SPARK-35364) Renaming the existing Koalas related codes.

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343002#comment-17343002 ] Apache Spark commented on SPARK-35364: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342998#comment-17342998 ] Apache Spark commented on SPARK-35380: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342996#comment-17342996 ] Apache Spark commented on SPARK-35380: -- User 'yaooqinn' has created a pull request

[jira] [Assigned] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35380: Assignee: (was: Apache Spark) > Support loading SparkSessionExtensions from ServiceLo

[jira] [Assigned] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35380: Assignee: Apache Spark > Support loading SparkSessionExtensions from ServiceLoader >

[jira] [Created] (SPARK-35382) Fix lambda variable name issues in nested DataFrame functions in Python APIs

2021-05-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35382: Summary: Fix lambda variable name issues in nested DataFrame functions in Python APIs Key: SPARK-35382 URL: https://issues.apache.org/jira/browse/SPARK-35382 Project:

[jira] [Created] (SPARK-35381) Fix lambda variable name issues in nested DataFrame functions in R APIs

2021-05-11 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-35381: Summary: Fix lambda variable name issues in nested DataFrame functions in R APIs Key: SPARK-35381 URL: https://issues.apache.org/jira/browse/SPARK-35381 Project: Spar

[jira] [Created] (SPARK-35380) Support loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-35380: Summary: Support loading SparkSessionExtensions from ServiceLoader Key: SPARK-35380 URL: https://issues.apache.org/jira/browse/SPARK-35380 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35379: Assignee: (was: Apache Spark) > Improve InferFiltersFromConstraints rule performance

[jira] [Assigned] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35379: Assignee: Apache Spark > Improve InferFiltersFromConstraints rule performance when parsin

[jira] [Commented] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342992#comment-17342992 ] Apache Spark commented on SPARK-35379: -- User 'wankunde' has created a pull request

[jira] [Created] (SPARK-35379) Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread Wan Kun (Jira)
Wan Kun created SPARK-35379: --- Summary: Improve InferFiltersFromConstraints rule performance when parsing spark sql Key: SPARK-35379 URL: https://issues.apache.org/jira/browse/SPARK-35379 Project: Spark

[jira] [Commented] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342984#comment-17342984 ] Apache Spark commented on SPARK-35378: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35378: Assignee: (was: Apache Spark) > Convert LeafRunnableCommand to LocalRelation when que

[jira] [Assigned] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35378: Assignee: Apache Spark > Convert LeafRunnableCommand to LocalRelation when query with CTE

[jira] [Commented] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342983#comment-17342983 ] Apache Spark commented on SPARK-35378: -- User 'beliefer' has created a pull request

[jira] [Updated] (SPARK-35378) Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Convert LeafRunnableCommand to LocalRelation when query with CTE (was: Support LeafRunnabl

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342981#comment-17342981 ] Yuming Wang commented on SPARK-35365: - {noformat} -- 2.4 org.apache.spark.sql.cataly

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark3.1.1_report_originalsql > spark3.1.1 use too long time to analyze table fields > ---

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Description: Currently, Spark doesn't support LeafRunnableCommand as sub query. Because the LeafRunn

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342978#comment-17342978 ] yao commented on SPARK-35365: - [~yumwang] I have upload two files,  I removed some columns a

[jira] [Comment Edited] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342978#comment-17342978 ] yao edited comment on SPARK-35365 at 5/12/21, 3:04 AM: --- [~yumwang]

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Support LeafRunnableCommand as sub query (was: Support LeafRunnableCommand as sub query so

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark3.11report > spark3.1.1 use too long time to analyze table fields > -

[jira] [Updated] (SPARK-35378) Support LeafRunnableCommand as sub query so as supports query command of CTE

2021-05-11 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-35378: --- Summary: Support LeafRunnableCommand as sub query so as supports query command of CTE (was: Support

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Attachment: spark2.4report > spark3.1.1 use too long time to analyze table fields > --

[jira] [Created] (SPARK-35378) Support LeafRunnableCommand as subQuery so as support query command with CTE

2021-05-11 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-35378: -- Summary: Support LeafRunnableCommand as subQuery so as support query command with CTE Key: SPARK-35378 URL: https://issues.apache.org/jira/browse/SPARK-35378 Project: Spa

[jira] [Commented] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342975#comment-17342975 ] Apache Spark commented on SPARK-35377: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35377: Assignee: Apache Spark (was: Kousuke Saruta) > Add JS linter to GA > ---

[jira] [Assigned] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35377: Assignee: Kousuke Saruta (was: Apache Spark) > Add JS linter to GA > ---

[jira] [Commented] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342974#comment-17342974 ] Apache Spark commented on SPARK-35377: -- User 'sarutak' has created a pull request f

[jira] [Updated] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35377: --- Issue Type: Improvement (was: Bug) > Add JS linter to GA > --- > >

[jira] [Created] (SPARK-35377) Add JS linter to GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35377: -- Summary: Add JS linter to GA Key: SPARK-35377 URL: https://issues.apache.org/jira/browse/SPARK-35377 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-35365) spark3.1.1 use too long time to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yao updated SPARK-35365: Summary: spark3.1.1 use too long time to analyze table fields (was: spark3.1.1 use too long to analyze table fiel

[jira] [Commented] (SPARK-35365) spark3.1.1 use too long to analyze table fields

2021-05-11 Thread yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342962#comment-17342962 ] yao commented on SPARK-35365: - [~yumwang] thanks, I will have a try, then give the results

[jira] [Assigned] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35376: Assignee: Apache Spark > Fallback config should override defaultValue > -

[jira] [Assigned] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35376: Assignee: (was: Apache Spark) > Fallback config should override defaultValue > --

[jira] [Commented] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342959#comment-17342959 ] Apache Spark commented on SPARK-35376: -- User 'ulysses-you' has created a pull reque

[jira] [Resolved] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35361. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32507 [https://gi

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35361: Assignee: Chao Sun > Improve performance for ApplyFunctionExpression > --

[jira] [Created] (SPARK-35376) Fallback config should override defaultValue

2021-05-11 Thread XiDuo You (Jira)
XiDuo You created SPARK-35376: - Summary: Fallback config should override defaultValue Key: SPARK-35376 URL: https://issues.apache.org/jira/browse/SPARK-35376 Project: Spark Issue Type: Improvemen

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yikf updated SPARK-35356: - Description: In the case of externalCatalog is InMemoryCatalog, Using a different SparkSession will report the

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread yikf (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342949#comment-17342949 ] yikf commented on SPARK-35356: -- Sorry, my version of Spark is 3.0.0, Please visit https://

[jira] [Commented] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342947#comment-17342947 ] Hyukjin Kwon commented on SPARK-35375: -- Fixed in https://github.com/apache/spark/pu

[jira] [Resolved] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35375. -- Fix Version/s: 3.2.0 3.1.2 Resolution: Fixed > Use Jinja2 < 3.0.0 fo

[jira] [Commented] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342938#comment-17342938 ] L. C. Hsieh commented on SPARK-35356: - Is the affect version wrong? Do you mean 3.0.

[jira] [Updated] (SPARK-35356) Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-35356: Fix Version/s: (was: 3.0.0) > Fix issue of the createTable when externalCatalog is InMemoryCat

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342937#comment-17342937 ] L. C. Hsieh commented on SPARK-35371: - Oh, I think it was fixed by SPARK-34829. > S

[jira] [Commented] (SPARK-35371) Scala UDF returning string or complex type applied to array members returns wrong data

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342936#comment-17342936 ] L. C. Hsieh commented on SPARK-35371: - I just ran the example in both current master

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Summary: Use Jinja2 < 3.0.0 for Python linter dependency in GA (was: Use Jinja2 < 3.0.0 for

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Assigned] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35375: Assignee: Kousuke Saruta (was: Apache Spark) > Use Jinja2 < 3.0.0 for linter dependency

[jira] [Assigned] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35375: Assignee: Apache Spark (was: Kousuke Saruta) > Use Jinja2 < 3.0.0 for linter dependency

[jira] [Commented] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342926#comment-17342926 ] Apache Spark commented on SPARK-35375: -- User 'sarutak' has created a pull request f

[jira] [Updated] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35375: --- Description: >From a few hours ago, Python linter fails in GA. The latest Jinja 3.0.0 seems

[jira] [Created] (SPARK-35375) Use Jinja2 < 3.0.0 for linter dependency in GA

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35375: -- Summary: Use Jinja2 < 3.0.0 for linter dependency in GA Key: SPARK-35375 URL: https://issues.apache.org/jira/browse/SPARK-35375 Project: Spark Issue Type

[jira] [Commented] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342793#comment-17342793 ] Apache Spark commented on SPARK-35361: -- User 'sunchao' has created a pull request f

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35361: Assignee: (was: Apache Spark) > Improve performance for ApplyFunctionExpression > ---

[jira] [Assigned] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35361: Assignee: Apache Spark > Improve performance for ApplyFunctionExpression > --

[jira] [Commented] (SPARK-35361) Improve performance for ApplyFunctionExpression

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342792#comment-17342792 ] Apache Spark commented on SPARK-35361: -- User 'sunchao' has created a pull request f

[jira] [Commented] (SPARK-34205) Add pipe API to Dataset

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342788#comment-17342788 ] L. C. Hsieh commented on SPARK-34205: - This is basically driven by our internal cust

[jira] [Resolved] (SPARK-34205) Add pipe API to Dataset

2021-05-11 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-34205. - Resolution: Won't Fix > Add pipe API to Dataset > --- > > Ke

[jira] [Resolved] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35372. Fix Version/s: 3.2.0 Assignee: Hyukjin Kwon Resolution: Fixed This issue w

[jira] [Commented] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342733#comment-17342733 ] Apache Spark commented on SPARK-35374: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35374: Assignee: Apache Spark (was: Kousuke Saruta) > Add string-to-number conversion support t

[jira] [Assigned] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35374: Assignee: Kousuke Saruta (was: Apache Spark) > Add string-to-number conversion support t

[jira] [Commented] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342732#comment-17342732 ] Apache Spark commented on SPARK-35374: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-35374) Add string-to-number conversion support to JacksonParser

2021-05-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-35374: -- Summary: Add string-to-number conversion support to JacksonParser Key: SPARK-35374 URL: https://issues.apache.org/jira/browse/SPARK-35374 Project: Spark

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Apache Spark (was: Sean R. Owen) > Verify checksums of downloaded artifacts in

[jira] [Commented] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342709#comment-17342709 ] Apache Spark commented on SPARK-35373: -- User 'srowen' has created a pull request fo

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Apache Spark (was: Sean R. Owen) > Verify checksums of downloaded artifacts in

[jira] [Assigned] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35373: Assignee: Sean R. Owen (was: Apache Spark) > Verify checksums of downloaded artifacts in

[jira] [Created] (SPARK-35373) Verify checksums of downloaded artifacts in build/mvn

2021-05-11 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-35373: Summary: Verify checksums of downloaded artifacts in build/mvn Key: SPARK-35373 URL: https://issues.apache.org/jira/browse/SPARK-35373 Project: Spark Issue T

[jira] [Commented] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342645#comment-17342645 ] Apache Spark commented on SPARK-35013: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35013: Assignee: (was: Apache Spark) > Spark allows to set spark.driver.cores=0 > --

[jira] [Assigned] (SPARK-35013) Spark allows to set spark.driver.cores=0

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35013: Assignee: Apache Spark > Spark allows to set spark.driver.cores=0 > -

[jira] [Commented] (SPARK-35370) IllegalArgumentException when loading a PipelineModel with Spark 3

2021-05-11 Thread Avenash Kabeera (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342559#comment-17342559 ] Avenash Kabeera commented on SPARK-35370: - Some additional details.  My model wa

[jira] [Resolved] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

2021-05-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35229. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32381 [https:

[jira] [Assigned] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

2021-05-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35229: -- Assignee: Kousuke Saruta > Spark Job web page is extremely slow while there are more

[jira] [Assigned] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35372: Assignee: (was: Apache Spark) > JDK 11 compilation failure due to StackOverflowError

[jira] [Commented] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342503#comment-17342503 ] Apache Spark commented on SPARK-35372: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342504#comment-17342504 ] Apache Spark commented on SPARK-35372: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-35372) JDK 11 compilation failure due to StackOverflowError

2021-05-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35372: Assignee: Apache Spark > JDK 11 compilation failure due to StackOverflowError > -

  1   2   >