[jira] [Resolved] (SPARK-32539) Disallow FileSystem.get(Configuration conf) by default

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32539. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29357

[jira] [Assigned] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32518: --- Assignee: wuyi > CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all

[jira] [Resolved] (SPARK-32518) CoarseGrainedSchedulerBackend.maxNumConcurrentTasks should consider all kinds of resources

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32518. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29332

[jira] [Updated] (SPARK-32556) Fix release script to uri encode the user provided passwords.

2020-08-05 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-32556: Description: As I was trying to do the release using the docker {code:java}

[jira] [Created] (SPARK-32556) Fix release script to uri encode the user provided passwords.

2020-08-05 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-32556: --- Summary: Fix release script to uri encode the user provided passwords. Key: SPARK-32556 URL: https://issues.apache.org/jira/browse/SPARK-32556 Project: Spark

[jira] [Commented] (SPARK-32526) Let sql/catalyst module tests pass for Scala 2.13

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172040#comment-17172040 ] Apache Spark commented on SPARK-32526: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-32526) Let sql/catalyst module tests pass for Scala 2.13

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172039#comment-17172039 ] Apache Spark commented on SPARK-32526: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-32555) Add unique ID on QueryExecution to enable listeners to deduplicate

2020-08-05 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-32555: Summary: Add unique ID on QueryExecution to enable listeners to deduplicate Key: SPARK-32555 URL: https://issues.apache.org/jira/browse/SPARK-32555 Project: Spark

[jira] [Commented] (SPARK-32540) Eliminate filter clause in aggregate

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171893#comment-17171893 ] Apache Spark commented on SPARK-32540: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32540) Eliminate filter clause in aggregate

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32540: Assignee: (was: Apache Spark) > Eliminate filter clause in aggregate >

[jira] [Commented] (SPARK-32540) Eliminate filter clause in aggregate

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171892#comment-17171892 ] Apache Spark commented on SPARK-32540: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32540) Eliminate filter clause in aggregate

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32540: Assignee: Apache Spark > Eliminate filter clause in aggregate >

[jira] [Updated] (SPARK-32540) Eliminate filter clause in aggregate

2020-08-05 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-32540: --- Summary: Eliminate filter clause in aggregate (was: Eliminate distinct that with filter clause) >

[jira] [Commented] (SPARK-32544) Bucketing and Partitioning information are not passed on to non FileFormat datasource writes

2020-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171880#comment-17171880 ] Hyukjin Kwon commented on SPARK-32544: -- This will be deal with together with DataSource V2. >

[jira] [Resolved] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32547. -- Resolution: Invalid > Cant able to process Timestamp 0001-01-01T00:00:00.000+ with

[jira] [Commented] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171879#comment-17171879 ] Hyukjin Kwon commented on SPARK-32547: -- I think this is more because of a limitation of Python

[jira] [Commented] (SPARK-32551) Ambiguous self join error in non self join with window

2020-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171874#comment-17171874 ] Hyukjin Kwon commented on SPARK-32551: -- {{spark.sql.analyzer.failAmbiguousSelfJoin}} isn't a

[jira] [Commented] (SPARK-32554) Update the k8s document according to the current development status

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171869#comment-17171869 ] Apache Spark commented on SPARK-32554: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32554) Update the k8s document according to the current development status

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32554: Assignee: Apache Spark > Update the k8s document according to the current development

[jira] [Commented] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171870#comment-17171870 ] Takeshi Yamamuro commented on SPARK-32550: -- Looks nice. Thanks for the info. > Make

[jira] [Assigned] (SPARK-32554) Update the k8s document according to the current development status

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32554: Assignee: (was: Apache Spark) > Update the k8s document according to the current

[jira] [Created] (SPARK-32554) Update the k8s document according to the current development status

2020-08-05 Thread Takeshi Yamamuro (Jira)
Takeshi Yamamuro created SPARK-32554: Summary: Update the k8s document according to the current development status Key: SPARK-32554 URL: https://issues.apache.org/jira/browse/SPARK-32554 Project:

[jira] [Commented] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Muhammad Samir Khan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171868#comment-17171868 ] Muhammad Samir Khan commented on SPARK-32550: - [~maropu] added some benchmark results in

[jira] [Resolved] (SPARK-32519) test of org.apache.spark.sql.kafka010.KafkaSourceStressSuite failed for aarch64

2020-08-05 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua resolved SPARK-32519. -- Resolution: Fixed > test of org.apache.spark.sql.kafka010.KafkaSourceStressSuite failed for

[jira] [Commented] (SPARK-32519) test of org.apache.spark.sql.kafka010.KafkaSourceStressSuite failed for aarch64

2020-08-05 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171860#comment-17171860 ] huangtianhua commented on SPARK-32519: -- It works on aarch64 platform, thanks all. > test of

[jira] [Created] (SPARK-32553) Spark application failed due to stage fatch failed without retry

2020-08-05 Thread wangshengjie (Jira)
wangshengjie created SPARK-32553: Summary: Spark application failed due to stage fatch failed without retry Key: SPARK-32553 URL: https://issues.apache.org/jira/browse/SPARK-32553 Project: Spark

[jira] [Assigned] (SPARK-32552) complete documenting table_valued function

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32552: Assignee: Apache Spark > complete documenting table_valued function >

[jira] [Commented] (SPARK-32552) complete documenting table_valued function

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171855#comment-17171855 ] Apache Spark commented on SPARK-32552: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32552) complete documenting table_valued function

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32552: Assignee: (was: Apache Spark) > complete documenting table_valued function >

[jira] [Created] (SPARK-32552) complete documenting table_valued function

2020-08-05 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-32552: -- Summary: complete documenting table_valued function Key: SPARK-32552 URL: https://issues.apache.org/jira/browse/SPARK-32552 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31198) Use graceful decommissioning as part of dynamic scaling

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171850#comment-17171850 ] Apache Spark commented on SPARK-31198: -- User 'holdenk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171843#comment-17171843 ] Takeshi Yamamuro commented on SPARK-32547: -- Thanks for the report. I checked that

[jira] [Commented] (SPARK-29767) Core dump happening on executors while doing simple union of Data Frames

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171835#comment-17171835 ] Takeshi Yamamuro commented on SPARK-29767: -- Thanks for the bug report, but I think we couldn't

[jira] [Created] (SPARK-32551) Ambiguous self join error in non self join with window

2020-08-05 Thread kanika dhuria (Jira)
kanika dhuria created SPARK-32551: - Summary: Ambiguous self join error in non self join with window Key: SPARK-32551 URL: https://issues.apache.org/jira/browse/SPARK-32551 Project: Spark

[jira] [Updated] (SPARK-32551) Ambiguous self join error in non self join with window

2020-08-05 Thread kanika dhuria (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kanika dhuria updated SPARK-32551: -- Description: Following code fails ambiguous self join analysis, even when it doesn't have

[jira] [Commented] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171832#comment-17171832 ] Apache Spark commented on SPARK-32550: -- User 'msamirkhan' has created a pull request for this

[jira] [Assigned] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32550: Assignee: (was: Apache Spark) > Make SpecificInternalRow constructors faster by

[jira] [Assigned] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32550: Assignee: Apache Spark > Make SpecificInternalRow constructors faster by using while

[jira] [Commented] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171831#comment-17171831 ] Apache Spark commented on SPARK-32550: -- User 'msamirkhan' has created a pull request for this

[jira] [Comment Edited] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171828#comment-17171828 ] Takeshi Yamamuro edited comment on SPARK-32550 at 8/5/20, 11:32 PM:

[jira] [Comment Edited] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171828#comment-17171828 ] Takeshi Yamamuro edited comment on SPARK-32550 at 8/5/20, 11:30 PM:

[jira] [Resolved] (SPARK-31197) Exit the executor once all tasks & migrations are finished

2020-08-05 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31197. -- Fix Version/s: 3.1.0 Target Version/s: 3.1.0 Resolution: Fixed > Exit the

[jira] [Assigned] (SPARK-31197) Exit the executor once all tasks & migrations are finished

2020-08-05 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau reassigned SPARK-31197: Assignee: Holden Karau > Exit the executor once all tasks & migrations are finished >

[jira] [Commented] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171828#comment-17171828 ] Takeshi Yamamuro commented on SPARK-32550: -- The proposed improvement can affect end-2-end

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171827#comment-17171827 ] Takeshi Yamamuro commented on SPARK-32515: -- Could you narrow down the issue, first? At least,

[jira] [Created] (SPARK-32550) Make SpecificInternalRow constructors faster by using while loops instead of maps

2020-08-05 Thread Muhammad Samir Khan (Jira)
Muhammad Samir Khan created SPARK-32550: --- Summary: Make SpecificInternalRow constructors faster by using while loops instead of maps Key: SPARK-32550 URL: https://issues.apache.org/jira/browse/SPARK-32550

[jira] [Commented] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171810#comment-17171810 ] Apache Spark commented on SPARK-32549: -- User 'liangz1' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32549: Assignee: (was: Apache Spark) > Add column name in _infer_schema error message >

[jira] [Commented] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171811#comment-17171811 ] Apache Spark commented on SPARK-32549: -- User 'liangz1' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32549: Assignee: Apache Spark > Add column name in _infer_schema error message >

[jira] [Updated] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Liang Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Zhang updated SPARK-32549: Description: The current error message from _infer_type in _infer_schema only includes the

[jira] [Created] (SPARK-32549) Add column name in _infer_schema error message

2020-08-05 Thread Liang Zhang (Jira)
Liang Zhang created SPARK-32549: --- Summary: Add column name in _infer_schema error message Key: SPARK-32549 URL: https://issues.apache.org/jira/browse/SPARK-32549 Project: Spark Issue Type:

[jira] [Reopened] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang reopened SPARK-32515: - > Distinct Function Weird Bug > --- > > Key: SPARK-32515 >

[jira] [Assigned] (SPARK-32548) Add Application attemptId support to SQL Rest API

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32548: Assignee: (was: Apache Spark) > Add Application attemptId support to SQL Rest API >

[jira] [Assigned] (SPARK-32548) Add Application attemptId support to SQL Rest API

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32548: Assignee: Apache Spark > Add Application attemptId support to SQL Rest API >

[jira] [Commented] (SPARK-32548) Add Application attemptId support to SQL Rest API

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171778#comment-17171778 ] Apache Spark commented on SPARK-32548: -- User 'erenavsarogullari' has created a pull request for

[jira] [Updated] (SPARK-32522) Using pyspark with a MultiLayerPerceptron model given inconsistent outputs if a large amount of data is fed into it and at least one of the model outputs is fed to a Pyt

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32522: -- Affects Version/s: (was: 3.1.0) 3.0.0 > Using pyspark with a

[jira] [Commented] (SPARK-32522) Using pyspark with a MultiLayerPerceptron model given inconsistent outputs if a large amount of data is fed into it and at least one of the model outputs is fed to a P

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171773#comment-17171773 ] Dongjoon Hyun commented on SPARK-32522: --- Is this correctness issue? > Using pyspark with a

[jira] [Commented] (SPARK-29767) Core dump happening on executors while doing simple union of Data Frames

2020-08-05 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171724#comment-17171724 ] Udit Mehrotra commented on SPARK-29767: --- The issue has been open for quite sometime. Can someone

[jira] [Updated] (SPARK-32548) Add Application attemptId support to SQL Rest API

2020-08-05 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-32548: --- Description: Currently, Spark Public Rest APIs support Application attemptId except

[jira] [Created] (SPARK-32548) Add Application attemptId support to SQL Rest API

2020-08-05 Thread Eren Avsarogullari (Jira)
Eren Avsarogullari created SPARK-32548: -- Summary: Add Application attemptId support to SQL Rest API Key: SPARK-32548 URL: https://issues.apache.org/jira/browse/SPARK-32548 Project: Spark

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171682#comment-17171682 ] Jayce Jiang edited comment on SPARK-32515 at 8/5/20, 7:10 PM: -- Okay, I am

[jira] [Commented] (SPARK-28210) Shuffle Storage API: Reads

2020-08-05 Thread Tianchen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171696#comment-17171696 ] Tianchen Zhang commented on SPARK-28210: Thank you [~mcheah] for the input. Sure I will start

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171686#comment-17171686 ] Jayce Jiang commented on SPARK-32515: - I just did it in google collab. The same results happen. The

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayce Jiang updated SPARK-32515: Attachment: Screen_Shot_2020-08-05_at_2.46.42_PM.png > Distinct Function Weird Bug >

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171682#comment-17171682 ] Jayce Jiang edited comment on SPARK-32515 at 8/5/20, 6:45 PM: -- Okay, I am

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171682#comment-17171682 ] Jayce Jiang edited comment on SPARK-32515 at 8/5/20, 6:42 PM: -- Okay, The

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

2020-08-05 Thread Jayce Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171682#comment-17171682 ] Jayce Jiang commented on SPARK-32515: - Okay, The expect results is what is I show in the

[jira] [Assigned] (SPARK-32529) Spark 3.0 History Server May Never Finish One Round Log Dir Scan

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32529: - Assignee: Yan Xiaole > Spark 3.0 History Server May Never Finish One Round Log Dir

[jira] [Resolved] (SPARK-32529) Spark 3.0 History Server May Never Finish One Round Log Dir Scan

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32529. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-32543) Remove arrow::as_tibble usage in SparkR

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32543: - Assignee: Hyukjin Kwon > Remove arrow::as_tibble usage in SparkR >

[jira] [Resolved] (SPARK-32543) Remove arrow::as_tibble usage in SparkR

2020-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32543. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29361

[jira] [Commented] (SPARK-32530) SPIP: Kotlin support for Apache Spark

2020-08-05 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171633#comment-17171633 ] Sean R. Owen commented on SPARK-32530: -- I know there was an email thread about this. For reference,

[jira] [Updated] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Manjunath H (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath H updated SPARK-32547: Description: Spark Version : 3.0.0 Below is the sample code to reproduce the problem with

[jira] [Updated] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Manjunath H (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath H updated SPARK-32547: Description: Spark Version : 3.0.0 Below is the sample code to reproduce the problem with

[jira] [Updated] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Manjunath H (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath H updated SPARK-32547: Description: Spark Version : 3.0.0 Below is the sample code to reproduce the problem with

[jira] [Updated] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Manjunath H (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath H updated SPARK-32547: Description: Spark Version : 3.0.0 Below is the sample code to reproduce the problem with

[jira] [Created] (SPARK-32547) Cant able to process Timestamp 0001-01-01T00:00:00.000+0000 with TimestampType

2020-08-05 Thread Manjunath Hatti (Jira)
Manjunath Hatti created SPARK-32547: --- Summary: Cant able to process Timestamp 0001-01-01T00:00:00.000+ with TimestampType Key: SPARK-32547 URL: https://issues.apache.org/jira/browse/SPARK-32547

[jira] [Commented] (SPARK-32546) SHOW VIEWS fails with MetaException ... ClassNotFoundException

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171604#comment-17171604 ] Apache Spark commented on SPARK-32546: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32546) SHOW VIEWS fails with MetaException ... ClassNotFoundException

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32546: Assignee: (was: Apache Spark) > SHOW VIEWS fails with MetaException ...

[jira] [Assigned] (SPARK-32546) SHOW VIEWS fails with MetaException ... ClassNotFoundException

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32546: Assignee: Apache Spark > SHOW VIEWS fails with MetaException ... ClassNotFoundException

[jira] [Commented] (SPARK-32546) SHOW VIEWS fails with MetaException ... ClassNotFoundException

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171605#comment-17171605 ] Apache Spark commented on SPARK-32546: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-32546) SHOW VIEWS fails with MetaException ... ClassNotFoundException

2020-08-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32546: -- Summary: SHOW VIEWS fails with MetaException ... ClassNotFoundException Key: SPARK-32546 URL: https://issues.apache.org/jira/browse/SPARK-32546 Project: Spark

[jira] [Assigned] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32485: --- Assignee: Michael Munday > RecordBinaryComparatorSuite test failures on big-endian systems

[jira] [Resolved] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32485. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29259

[jira] [Updated] (SPARK-32542) add a batch for optimizing logicalPlan

2020-08-05 Thread karl wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] karl wang updated SPARK-32542: -- Description: Split an expand into several smallExpand,which contains the Specified number of

[jira] [Created] (SPARK-32545) Applications with Maven dependencies fail when submitted in cluster mode on K8s

2020-08-05 Thread Samuel Suter (Jira)
Samuel Suter created SPARK-32545: Summary: Applications with Maven dependencies fail when submitted in cluster mode on K8s Key: SPARK-32545 URL: https://issues.apache.org/jira/browse/SPARK-32545

[jira] [Commented] (SPARK-32542) add a batch for optimizing logicalPlan

2020-08-05 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171492#comment-17171492 ] Rohit Mishra commented on SPARK-32542: -- [~wangshikai], Can you please add a description to this

[jira] [Created] (SPARK-32544) Bucketing and Partitioning information are not passed on to non FileFormat datasource writes

2020-08-05 Thread Rahij Ramsharan (Jira)
Rahij Ramsharan created SPARK-32544: --- Summary: Bucketing and Partitioning information are not passed on to non FileFormat datasource writes Key: SPARK-32544 URL:

[jira] [Commented] (SPARK-31460) spark-sql-kafka source in spark 2.4.4 causes reading stream failure frequently

2020-08-05 Thread Akanksha goel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171464#comment-17171464 ] Akanksha goel commented on SPARK-31460: --- I am using Spark 2.4 and Kafka 2.0, and getting this

[jira] [Resolved] (SPARK-32501) Inconsistent NULL conversions to strings

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32501. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29311

[jira] [Assigned] (SPARK-32501) Inconsistent NULL conversions to strings

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32501: --- Assignee: Maxim Gekk > Inconsistent NULL conversions to strings >

[jira] [Resolved] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32402. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29324

[jira] [Assigned] (SPARK-32402) Implement ALTER TABLE in JDBC Table Catalog

2020-08-05 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32402: --- Assignee: Huaxin Gao > Implement ALTER TABLE in JDBC Table Catalog >

[jira] [Commented] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171431#comment-17171431 ] Apache Spark commented on SPARK-32485: -- User 'mundaym' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32485: Assignee: (was: Apache Spark) > RecordBinaryComparatorSuite test failures on

[jira] [Commented] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17171430#comment-17171430 ] Apache Spark commented on SPARK-32485: -- User 'mundaym' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32485) RecordBinaryComparatorSuite test failures on big-endian systems

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32485: Assignee: Apache Spark > RecordBinaryComparatorSuite test failures on big-endian systems

[jira] [Updated] (SPARK-32543) Remove arrow::as_tibble usage in SparkR

2020-08-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32543: - Priority: Trivial (was: Major) > Remove arrow::as_tibble usage in SparkR >

[jira] [Assigned] (SPARK-32543) Remove arrow::as_tibble usage in SparkR

2020-08-05 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32543: Assignee: (was: Apache Spark) > Remove arrow::as_tibble usage in SparkR >

  1   2   >