[jira] [Updated] (SPARK-32104) Avoid full outer join OOM on skewed dataset

2020-06-26 Thread Zuo Dao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuo Dao updated SPARK-32104: Description: SPARK-24985 changed {{SortMergeJoinExec}} to use  {{ExternalAppendOnlyUnsafeRowArray}} on 

[jira] [Resolved] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32058. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28897

[jira] [Assigned] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32058: - Assignee: Dongjoon Hyun > Use Apache Hadoop 3.2.0 dependency by default >

[jira] [Assigned] (SPARK-31845) DAGSchedulerSuite: Reuse completeNextStageWithFetchFailure and support checkAnswer

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31845: - Assignee: jiaan.geng > DAGSchedulerSuite: Reuse completeNextStageWithFetchFailure and

[jira] [Resolved] (SPARK-31845) DAGSchedulerSuite: Reuse completeNextStageWithFetchFailure and support checkAnswer

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31845. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28866

[jira] [Commented] (SPARK-31843) DAGSchedulerSuite: For the pattern of complete + assert, extract the general method

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146741#comment-17146741 ] Dongjoon Hyun commented on SPARK-31843: --- I consolidate this into SPARK-31845, [~beliefer] ~ >

[jira] [Commented] (SPARK-32051) Dataset.foreachPartition returns object

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146740#comment-17146740 ] Sean R. Owen commented on SPARK-32051: -- Yes, it's because of Scala 2.12 SAM support. Some overloads

[jira] [Closed] (SPARK-31843) DAGSchedulerSuite: For the pattern of complete + assert, extract the general method

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-31843. - > DAGSchedulerSuite: For the pattern of complete + assert, extract the general > method >

[jira] [Updated] (SPARK-31845) DAGSchedulerSuite: Reuse completeNextStageWithFetchFailure and support checkAnswer

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31845: -- Component/s: Tests > DAGSchedulerSuite: Reuse completeNextStageWithFetchFailure and support

[jira] [Resolved] (SPARK-31843) DAGSchedulerSuite: For the pattern of complete + assert, extract the general method

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31843. --- Resolution: Duplicate > DAGSchedulerSuite: For the pattern of complete + assert, extract

[jira] [Assigned] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32099: - Assignee: Guy Khazma > Remove broken link in cloud integration documentation >

[jira] [Resolved] (SPARK-32099) Remove broken link in cloud integration documentation

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32099. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32088: - Assignee: philipse (was: huangtianhua) > test of

[jira] [Assigned] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32088: - Assignee: huangtianhua > test of pyspark.sql.functions.timestamp_seconds failed if

[jira] [Resolved] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32088. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28932

[jira] [Resolved] (SPARK-32023) Generate spark streaming test jar with maven plugin maven-jar-plugin

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32023. --- Resolution: Cannot Reproduce Please see

[jira] [Closed] (SPARK-32023) Generate spark streaming test jar with maven plugin maven-jar-plugin

2020-06-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-32023. - > Generate spark streaming test jar with maven plugin maven-jar-plugin >

[jira] [Comment Edited] (SPARK-32051) Dataset.foreachPartition returns object

2020-06-26 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146722#comment-17146722 ] Jungtaek Lim edited comment on SPARK-32051 at 6/27/20, 1:42 AM: OK so it

[jira] [Commented] (SPARK-32051) Dataset.foreachPartition returns object

2020-06-26 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146722#comment-17146722 ] Jungtaek Lim commented on SPARK-32051: -- OK so it needs to explicitly return Unit. Thanks for the

[jira] [Created] (SPARK-32112) Add a method to calculate the number of parallel tasks that Spark can process at the same time

2020-06-26 Thread Noritaka Sekiyama (Jira)
Noritaka Sekiyama created SPARK-32112: - Summary: Add a method to calculate the number of parallel tasks that Spark can process at the same time Key: SPARK-32112 URL:

[jira] [Created] (SPARK-32111) Cleanup locks and docs in CoarseGrainedSchedulerBackend

2020-06-26 Thread Holden Karau (Jira)
Holden Karau created SPARK-32111: Summary: Cleanup locks and docs in CoarseGrainedSchedulerBackend Key: SPARK-32111 URL: https://issues.apache.org/jira/browse/SPARK-32111 Project: Spark

[jira] [Commented] (SPARK-32111) Cleanup locks and docs in CoarseGrainedSchedulerBackend

2020-06-26 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146613#comment-17146613 ] Holden Karau commented on SPARK-32111: -- I'm working on this since I'm in the code anyways for

[jira] [Created] (SPARK-32110) -0.0 vs 0.0 is inconsistent

2020-06-26 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-32110: --- Summary: -0.0 vs 0.0 is inconsistent Key: SPARK-32110 URL: https://issues.apache.org/jira/browse/SPARK-32110 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32109) SQL hash function handling of nulls makes collision too likely

2020-06-26 Thread koert kuipers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-32109: -- Description: this ticket is about org.apache.spark.sql.functions.hash and sparks handling of

[jira] [Created] (SPARK-32109) SQL hash function handling of nulls makes collision too likely

2020-06-26 Thread koert kuipers (Jira)
koert kuipers created SPARK-32109: - Summary: SQL hash function handling of nulls makes collision too likely Key: SPARK-32109 URL: https://issues.apache.org/jira/browse/SPARK-32109 Project: Spark

[jira] [Updated] (SPARK-32108) Silent mode of spark-sql is broken

2020-06-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-32108: --- Description: 1. I download the recent release Spark 3.0 from

[jira] [Created] (SPARK-32108) Silent mode of spark-sql is broken

2020-06-26 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32108: -- Summary: Silent mode of spark-sql is broken Key: SPARK-32108 URL: https://issues.apache.org/jira/browse/SPARK-32108 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-20249) Add summary for LinearSVCModel

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-20249: - Labels: (was: bulk-closed) > Add summary for LinearSVCModel > --

[jira] [Resolved] (SPARK-20249) Add summary for LinearSVCModel

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-20249. -- Fix Version/s: 3.1.0 Assignee: Huaxin Gao (was: Apache Spark) Resolution:

[jira] [Assigned] (SPARK-20249) Add summary for LinearSVCModel

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20249: Assignee: Apache Spark (was: Huaxin Gao) > Add summary for LinearSVCModel >

[jira] [Assigned] (SPARK-20249) Add summary for LinearSVCModel

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-20249: Assignee: Huaxin Gao > Add summary for LinearSVCModel > -- >

[jira] [Reopened] (SPARK-20249) Add summary for LinearSVCModel

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reopened SPARK-20249: -- > Add summary for LinearSVCModel > -- > > Key:

[jira] [Updated] (SPARK-19939) Add support for association rules in ML

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-19939: - Priority: Minor (was: Major) > Add support for association rules in ML >

[jira] [Resolved] (SPARK-19939) Add support for association rules in ML

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-19939. -- Fix Version/s: 3.1.0 Resolution: Fixed Resolved by

[jira] [Updated] (SPARK-19939) Add support for association rules in ML

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-19939: - Labels: (was: bulk-closed) > Add support for association rules in ML >

[jira] [Assigned] (SPARK-19939) Add support for association rules in ML

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-19939: Assignee: Huaxin Gao > Add support for association rules in ML >

[jira] [Reopened] (SPARK-19939) Add support for association rules in ML

2020-06-26 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reopened SPARK-19939: -- > Add support for association rules in ML > --- > >

[jira] [Updated] (SPARK-32107) Dask faster than Spark with a lot less iterations and better accuracy

2020-06-26 Thread Julian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian updated SPARK-32107: --- Description: Hello, I'm benchmarking k-means clustering Dask versus Spark. Right now these are only

[jira] [Updated] (SPARK-32107) Dask faster than Spark with a lot less iterations and better accuracy

2020-06-26 Thread Julian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian updated SPARK-32107: --- Description: Hello, I'm benchmarking k-means clustering Dask versus Spark. Right now these are only

[jira] [Created] (SPARK-32107) Dask faster than Spark with a lot less iterations and better accuracy

2020-06-26 Thread Julian (Jira)
Julian created SPARK-32107: -- Summary: Dask faster than Spark with a lot less iterations and better accuracy Key: SPARK-32107 URL: https://issues.apache.org/jira/browse/SPARK-32107 Project: Spark

[jira] [Assigned] (SPARK-32105) Refactor current script transform code

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32105: Assignee: Apache Spark > Refactor current script transform code >

[jira] [Assigned] (SPARK-32105) Refactor current script transform code

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32105: Assignee: (was: Apache Spark) > Refactor current script transform code >

[jira] [Commented] (SPARK-32105) Refactor current script transform code

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146351#comment-17146351 ] Apache Spark commented on SPARK-32105: -- User 'AngersZh' has created a pull request for this

[jira] [Created] (SPARK-32106) Implement script transform in sql/core

2020-06-26 Thread angerszhu (Jira)
angerszhu created SPARK-32106: - Summary: Implement script transform in sql/core Key: SPARK-32106 URL: https://issues.apache.org/jira/browse/SPARK-32106 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-32105) Refactor current script transform code

2020-06-26 Thread angerszhu (Jira)
angerszhu created SPARK-32105: - Summary: Refactor current script transform code Key: SPARK-32105 URL: https://issues.apache.org/jira/browse/SPARK-32105 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-31851) Redesign PySpark documentation

2020-06-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146278#comment-17146278 ] Hyukjin Kwon commented on SPARK-31851: -- Thanks, [~jijosg]. I'll add some sub tasks soon. I'm happy

[jira] [Comment Edited] (SPARK-31851) Redesign PySpark documentation

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146261#comment-17146261 ] Jijo Sunny edited comment on SPARK-31851 at 6/26/20, 12:14 PM: --- I can help

[jira] [Commented] (SPARK-31851) Redesign PySpark documentation

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146261#comment-17146261 ] Jijo Sunny commented on SPARK-31851: I can help with this, have some free time. > Redesign PySpark

[jira] [Assigned] (SPARK-32104) Avoid full outer join OOM on skewed dataset

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32104: Assignee: Apache Spark > Avoid full outer join OOM on skewed dataset >

[jira] [Commented] (SPARK-32104) Avoid full outer join OOM on skewed dataset

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146209#comment-17146209 ] Apache Spark commented on SPARK-32104: -- User 'sidedoorleftroad' has created a pull request for this

[jira] [Assigned] (SPARK-32104) Avoid full outer join OOM on skewed dataset

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32104: Assignee: (was: Apache Spark) > Avoid full outer join OOM on skewed dataset >

[jira] [Created] (SPARK-32104) Avoid full outer join OOM on skewed dataset

2020-06-26 Thread Zuo Dao (Jira)
Zuo Dao created SPARK-32104: --- Summary: Avoid full outer join OOM on skewed dataset Key: SPARK-32104 URL: https://issues.apache.org/jira/browse/SPARK-32104 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146158#comment-17146158 ] Apache Spark commented on SPARK-32088: -- User 'GuoPhilipse' has created a pull request for this

[jira] [Assigned] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32088: Assignee: Apache Spark > test of pyspark.sql.functions.timestamp_seconds failed if

[jira] [Assigned] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32088: Assignee: (was: Apache Spark) > test of pyspark.sql.functions.timestamp_seconds

[jira] [Commented] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146091#comment-17146091 ] Apache Spark commented on SPARK-32103: -- User 'PavithraRamachandran' has created a pull request for

[jira] [Assigned] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32103: Assignee: Apache Spark > Spark support IPV6 in yarn mode >

[jira] [Assigned] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32103: Assignee: (was: Apache Spark) > Spark support IPV6 in yarn mode >

[jira] [Commented] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146088#comment-17146088 ] Apache Spark commented on SPARK-32103: -- User 'PavithraRamachandran' has created a pull request for

[jira] [Commented] (SPARK-29999) Writing empty partition via DSv2 implementation of FileStreamSink throws FileNotFoundException

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146086#comment-17146086 ] Apache Spark commented on SPARK-2: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-29999) Writing empty partition via DSv2 implementation of FileStreamSink throws FileNotFoundException

2020-06-26 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146087#comment-17146087 ] Apache Spark commented on SPARK-2: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Issue Comment Deleted] (SPARK-31887) Date casting to string is giving wrong value

2020-06-26 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pavithra ramachandran updated SPARK-31887: -- Comment: was deleted (was: i would like to check this.) > Date casting to

[jira] [Commented] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146073#comment-17146073 ] pavithra ramachandran commented on SPARK-32103: --- i shall raise a pr soon > Spark support

[jira] [Created] (SPARK-32103) Spark support IPV6 in yarn mode

2020-06-26 Thread jobit mathew (Jira)
jobit mathew created SPARK-32103: Summary: Spark support IPV6 in yarn mode Key: SPARK-32103 URL: https://issues.apache.org/jira/browse/SPARK-32103 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Description: Java source code link in Spark api scaladoc is broken. This issue still persists for

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Description: Java source code link in Spark api scaladoc is broken. This issue still persists for

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Description: Java source code link in Spark api scaladoc is broken. The relative path expression

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Description: Java source code link in Spark api scaladoc is broken. The relative path expression

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Description: Java source code link in Spark api scaladoc is broken. The relative path expression

[jira] [Updated] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jijo Sunny updated SPARK-32102: --- Affects Version/s: (was: 2.1.2) (was: 2.0.2)

[jira] [Created] (SPARK-32102) Broken link to java source code in Spark Scala api doc

2020-06-26 Thread Jijo Sunny (Jira)
Jijo Sunny created SPARK-32102: -- Summary: Broken link to java source code in Spark Scala api doc Key: SPARK-32102 URL: https://issues.apache.org/jira/browse/SPARK-32102 Project: Spark Issue

[jira] [Created] (SPARK-32101) The name in the with clause when it is same as table name. And when that table name is used in the other places, it is not taking the table, it is considering the with c

2020-06-26 Thread Pavan Kothamasu (Jira)
Pavan Kothamasu created SPARK-32101: --- Summary: The name in the with clause when it is same as table name. And when that table name is used in the other places, it is not taking the table, it is considering the with clause.

[jira] [Commented] (SPARK-31525) Inconsistent result of df.head(1) and df.head()

2020-06-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146030#comment-17146030 ] Hyukjin Kwon commented on SPARK-31525: -- [~joshuahendinata] are you interested in submitting a PR?

[jira] [Commented] (SPARK-31525) Inconsistent result of df.head(1) and df.head()

2020-06-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146029#comment-17146029 ] Hyukjin Kwon commented on SPARK-31525: -- Yeah, I think it should return an empty list instead of

[jira] [Commented] (SPARK-31525) Inconsistent result of df.head(1) and df.head()

2020-06-26 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146026#comment-17146026 ] Xiao Li commented on SPARK-31525: - cc [~hyukjin.kwon] and [~ueshin] > Inconsistent result of df.head(1)