[jira] [Updated] (SPARK-26452) Suppressing exception in finally: Java heap space java.lang.OutOfMemoryError: Java heap space

2019-01-01 Thread tommy duan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tommy duan updated SPARK-26452: --- Description: In spark2.2.0 structured streaming program,the shell code of submit as follow: {code:ja

[jira] [Updated] (SPARK-26452) Suppressing exception in finally: Java heap space java.lang.OutOfMemoryError: Java heap space

2019-01-01 Thread tommy duan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tommy duan updated SPARK-26452: --- Description: In spark2.2.0 structured streaming program,the shell code of submit as follow: {code:ja

[jira] [Updated] (SPARK-26452) Suppressing exception in finally: Java heap space java.lang.OutOfMemoryError: Java heap space

2019-01-01 Thread tommy duan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tommy duan updated SPARK-26452: --- Description: In spark2.2.0 structured streaming program,the shell code of submit as follow: {code:ja

[jira] [Updated] (SPARK-26452) Suppressing exception in finally: Java heap space java.lang.OutOfMemoryError: Java heap space

2019-01-01 Thread tommy duan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tommy duan updated SPARK-26452: --- Description: In spark2.2.0 structured streaming program,the shell code of submit as follow:   {code

[jira] [Commented] (SPARK-26454) IllegegalArgument Exception is Thrown while creating new UDF with JAR

2019-01-01 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731769#comment-16731769 ] Udbhav Agrawal commented on SPARK-26454: CC [~sandeep-katta] [~sujith] [~ajithsh

[jira] [Updated] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26306: -- Priority: Minor (was: Major) > Flaky test: org.apache.spark.util.collection.SorterSuite > ---

[jira] [Resolved] (SPARK-26508) Address warning messages in Java by lgtm.com

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26508. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23420 [https://github.c

[jira] [Assigned] (SPARK-26508) Address warning messages in Java by lgtm.com

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26508: - Assignee: Kazuaki Ishizaki > Address warning messages in Java by lgtm.com > ---

[jira] [Assigned] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26306: Assignee: Apache Spark > Flaky test: org.apache.spark.util.collection.SorterSuite > -

[jira] [Assigned] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26306: Assignee: (was: Apache Spark) > Flaky test: org.apache.spark.util.collection.SorterSu

[jira] [Updated] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26306: -- Description: In PR builder the following issue appeared: {code:java} [info] org.apache.spark.

[jira] [Updated] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26306: -- Description: TestTimSort causes OOM frequently. {code:java} [info] org.apache.spark.util.coll

[jira] [Updated] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26306: -- Description: In PR builder the following issue appeared: {code:java} [info] org.apache.spark.

[jira] [Commented] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731734#comment-16731734 ] Dongjoon Hyun commented on SPARK-26306: --- I'm reopening this because this is a real

[jira] [Reopened] (SPARK-26306) Flaky test: org.apache.spark.util.collection.SorterSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-26306: --- > Flaky test: org.apache.spark.util.collection.SorterSuite > ---

[jira] [Issue Comment Deleted] (SPARK-25299) Use remote storage for persisting shuffle data

2019-01-01 Thread Carson Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carson Wang updated SPARK-25299: Comment: was deleted (was: I am on a vacation and will be back on January 2, 2019. Please expect

[jira] [Commented] (SPARK-26152) Flaky test: BroadcastSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731715#comment-16731715 ] Dongjoon Hyun commented on SPARK-26152: --- This bug is not reported correctly as we

[jira] [Resolved] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26227. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23184 [https://gi

[jira] [Assigned] (SPARK-26227) from_[csv|json] should accept schema_of_[csv|json] in R API

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26227: Assignee: Hyukjin Kwon > from_[csv|json] should accept schema_of_[csv|json] in R API > --

[jira] [Resolved] (SPARK-26374) Support new date/timestamp parser in HadoopFsRelationTest

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26374. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23417 [https://gi

[jira] [Updated] (SPARK-26152) Flaky test: BroadcastSuite

2019-01-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26152: -- Description: - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/jo

[jira] [Assigned] (SPARK-26374) Support new date/timestamp parser in HadoopFsRelationTest

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26374: Assignee: Maxim Gekk > Support new date/timestamp parser in HadoopFsRelationTest > --

[jira] [Resolved] (SPARK-26515) SQL date_format function for 2018-12-30 returns 2019 date

2019-01-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-26515. - Resolution: Not A Bug The problem is in your logic, namely in the format. You need {{}}, not

[jira] [Updated] (SPARK-26515) SQL date_format function for 2018-12-30 returns 2019 date

2019-01-01 Thread David Thomasson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Thomasson updated SPARK-26515: Description: When running a query that formats the date '*2018*-12-30 03:16:04' to '-

[jira] [Updated] (SPARK-26515) SQL date_format function for 2018-12-30 returns 2019 date

2019-01-01 Thread David Thomasson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Thomasson updated SPARK-26515: Description: When running a query that formats the date '*2018*-12-30 03:16:04' to '-

[jira] [Created] (SPARK-26515) SQL date_format function for 2018-12-30 returns 2019 date

2019-01-01 Thread David Thomasson (JIRA)
David Thomasson created SPARK-26515: --- Summary: SQL date_format function for 2018-12-30 returns 2019 date Key: SPARK-26515 URL: https://issues.apache.org/jira/browse/SPARK-26515 Project: Spark

[jira] [Resolved] (SPARK-25765) Add trainingCost to BisectingKMeans summary

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25765. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22764 [https://github.c

[jira] [Assigned] (SPARK-25765) Add trainingCost to BisectingKMeans summary

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25765: - Assignee: Marco Gaido > Add trainingCost to BisectingKMeans summary > -

[jira] [Commented] (SPARK-26454) IllegegalArgument Exception is Thrown while creating new UDF with JAR

2019-01-01 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731618#comment-16731618 ] Udbhav Agrawal commented on SPARK-26454: [~hyukjin.kwon] jar is registered when

[jira] [Resolved] (SPARK-24544) Print actual failure cause when look up function failed

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24544. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21790 [https://github.c

[jira] [Assigned] (SPARK-24544) Print actual failure cause when look up function failed

2019-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24544: - Assignee: zhoukang > Print actual failure cause when look up function failed >

[jira] [Assigned] (SPARK-26339) Behavior of reading files that start with underscore is confusing

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26339: Assignee: Apache Spark (was: Keiichi Hirobe) > Behavior of reading files that start with

[jira] [Assigned] (SPARK-26339) Behavior of reading files that start with underscore is confusing

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26339: Assignee: Keiichi Hirobe (was: Apache Spark) > Behavior of reading files that start with

[jira] [Assigned] (SPARK-26482) Use ConfigEntry for hardcoded configs for ui categories.

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26482: Assignee: (was: Apache Spark) > Use ConfigEntry for hardcoded configs for ui categori

[jira] [Assigned] (SPARK-26482) Use ConfigEntry for hardcoded configs for ui categories.

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26482: Assignee: Apache Spark > Use ConfigEntry for hardcoded configs for ui categories. > -

[jira] [Resolved] (SPARK-26511) java.lang.ClassCastException error when loading Spark MLlib model from parquet file

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26511. -- Resolution: Invalid I'm leaving this resolved for the current status. > java.lang.ClassCastEx

[jira] [Commented] (SPARK-26511) java.lang.ClassCastException error when loading Spark MLlib model from parquet file

2019-01-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731592#comment-16731592 ] Liang-Chi Hsieh commented on SPARK-26511: - The model you provided has a differen

[jira] [Updated] (SPARK-26510) Spark 2.3 regression (vs 1.6) upon caching a dataframe and using 'createOrReplaceTempView'

2019-01-01 Thread Hagai Attias (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hagai Attias updated SPARK-26510: - Summary: Spark 2.3 regression (vs 1.6) upon caching a dataframe and using 'createOrReplaceTempVi

[jira] [Updated] (SPARK-26510) Spark 2.3 regression (vs 1.6) upon caching a dataframe as a temporary view ('createOrReplaceTempView')

2019-01-01 Thread Hagai Attias (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hagai Attias updated SPARK-26510: - Summary: Spark 2.3 regression (vs 1.6) upon caching a dataframe as a temporary view ('createOrRe

[jira] [Updated] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26512: - Priority: Major (was: Critical) > Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10? >

[jira] [Updated] (SPARK-26510) Spark 2.3 regression (vs 1.6) when caching a dataframe and using 'createOrReplaceTempView'

2019-01-01 Thread Hagai Attias (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hagai Attias updated SPARK-26510: - Summary: Spark 2.3 regression (vs 1.6) when caching a dataframe and using 'createOrReplaceTempVi

[jira] [Updated] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-01 Thread Anubhav Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Jain updated SPARK-26512: - Priority: Minor (was: Major) > Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10? > --

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-01 Thread Anubhav Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731555#comment-16731555 ] Anubhav Jain commented on SPARK-26512: -- Thanks for the prompt reply and i will do t

[jira] [Commented] (SPARK-26510) Spark 2.3 change of behavior (vs 1.6) when caching a dataframe and using 'createOrReplaceTempView'

2019-01-01 Thread Hagai Attias (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731548#comment-16731548 ] Hagai Attias commented on SPARK-26510: -- is there a release version containing this

[jira] [Commented] (SPARK-26510) Spark 2.3 change of behavior (vs 1.6) when caching a dataframe and using 'createOrReplaceTempView'

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731552#comment-16731552 ] Hyukjin Kwon commented on SPARK-26510: -- I can confirm upcoming release (3.0) has a

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731550#comment-16731550 ] Hyukjin Kwon commented on SPARK-26512: -- Critical+ is reserved for committers. Pleas

[jira] [Updated] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

2019-01-01 Thread Anubhav Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Jain updated SPARK-26512: - Flags: Important Priority: Critical (was: Major) > Spark 2.4.0 is not working with Hadoo

[jira] [Commented] (SPARK-26449) Missing Dataframe.transform API in Python API

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731543#comment-16731543 ] Apache Spark commented on SPARK-26449: -- User 'chanansh' has created a pull request

[jira] [Assigned] (SPARK-26449) Missing Dataframe.transform API in Python API

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26449: Assignee: (was: Apache Spark) > Missing Dataframe.transform API in Python API > -

[jira] [Assigned] (SPARK-26449) Missing Dataframe.transform API in Python API

2019-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26449: Assignee: Apache Spark > Missing Dataframe.transform API in Python API >