[jira] [Updated] (SPARK-25942) Aggregate expressions shouldn't be resolved on AppendColumns

2018-11-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-25942: Description: Dataset.groupByKey will bring in new attributes from serializer. If key type

[jira] [Updated] (SPARK-25942) Aggregate expressions shouldn't be resolved on AppendColumns

2018-11-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-25942: Description: Dataset.groupByKey will bring in new attributes from serializer. If key type

[jira] [Updated] (SPARK-25942) Aggregate expressions shouldn't be resolved on AppendColumns

2018-11-12 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-25942: Summary: Aggregate expressions shouldn't be resolved on AppendColumns (was:

***UNCHECKED*** [jira] [Updated] (SPARK-25986) Banning throw new OutOfMemoryErrors

2018-11-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25986: -- Target Version/s: 3.0.0 Labels: release-notes (was: starter) Per the pull request, I

[jira] [Assigned] (SPARK-25986) Banning throw new Errors

2018-11-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25986: - Assignee: Yuanjian Li Description: Adding a linter rule to ban the construction of new

[jira] [Resolved] (SPARK-26030) Bump previousSparkVersion in MimaBuild.scala to be 2.4.0

2018-11-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26030. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22977

[jira] [Resolved] (SPARK-26003) Improve performance in SQLAppStatusListener

2018-11-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26003. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23002

[jira] [Assigned] (SPARK-26003) Improve performance in SQLAppStatusListener

2018-11-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26003: --- Assignee: Marco Gaido > Improve performance in SQLAppStatusListener >

[jira] [Created] (SPARK-26031) dataframe can't load correct after saving to local disk in cluster mode

2018-11-12 Thread Bihui Jin (JIRA)
Bihui Jin created SPARK-26031: - Summary: dataframe can't load correct after saving to local disk in cluster mode Key: SPARK-26031 URL: https://issues.apache.org/jira/browse/SPARK-26031 Project: Spark

[jira] [Assigned] (SPARK-26007) DataFrameReader.csv() should respect to spark.sql.columnNameOfCorruptRecord

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26007: Assignee: Maxim Gekk > DataFrameReader.csv() should respect to

[jira] [Updated] (SPARK-26012) Dynamic partition will fail when both '' and null values are taken as dynamic partition values simultaneously.

2018-11-12 Thread eaton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] eaton updated SPARK-26012: -- Issue Type: Bug (was: Improvement) > Dynamic partition will fail when both '' and null values are taken as

[jira] [Resolved] (SPARK-26007) DataFrameReader.csv() should respect to spark.sql.columnNameOfCorruptRecord

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26007. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23006

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684688#comment-16684688 ] Hyukjin Kwon commented on SPARK-26009: -- Please ask it to mailing list - people respond regardless

[jira] [Commented] (SPARK-24255) Require Java 8 in SparkR description

2018-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684668#comment-16684668 ] Felix Cheung commented on SPARK-24255: -- [~shivaram] I'm thinking if this is handling all version

[jira] [Commented] (SPARK-26030) Bump previousSparkVersion in MimaBuild.scala to be 2.4.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684669#comment-16684669 ] Apache Spark commented on SPARK-26030: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26030) Bump previousSparkVersion in MimaBuild.scala to be 2.4.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26030: Assignee: Wenchen Fan (was: Apache Spark) > Bump previousSparkVersion in

[jira] [Assigned] (SPARK-26030) Bump previousSparkVersion in MimaBuild.scala to be 2.4.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26030: Assignee: Apache Spark (was: Wenchen Fan) > Bump previousSparkVersion in

[jira] [Created] (SPARK-26030) Bump previousSparkVersion in MimaBuild.scala to be 2.4.0

2018-11-12 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26030: --- Summary: Bump previousSparkVersion in MimaBuild.scala to be 2.4.0 Key: SPARK-26030 URL: https://issues.apache.org/jira/browse/SPARK-26030 Project: Spark Issue

[jira] [Resolved] (SPARK-26010) SparkR vignette fails on CRAN on Java 11

2018-11-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-26010. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 3.0.0

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Bala Bharath Reddy Resapu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684665#comment-16684665 ] Bala Bharath Reddy Resapu commented on SPARK-26009: --- If this is in resolved state. Do

[jira] [Resolved] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26029. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.4.1 > Bump

[jira] [Commented] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684631#comment-16684631 ] Apache Spark commented on SPARK-26029: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26029: Assignee: (was: Apache Spark) > Bump previousSparkVersion in MimaBuild.scala to be

[jira] [Commented] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684632#comment-16684632 ] Apache Spark commented on SPARK-26029: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26029: Assignee: Apache Spark > Bump previousSparkVersion in MimaBuild.scala to be 2.3.0 >

[jira] [Created] (SPARK-26029) Bump previousSparkVersion in MimaBuild.scala to be 2.3.0

2018-11-12 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26029: --- Summary: Bump previousSparkVersion in MimaBuild.scala to be 2.3.0 Key: SPARK-26029 URL: https://issues.apache.org/jira/browse/SPARK-26029 Project: Spark Issue

[jira] [Comment Edited] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684567#comment-16684567 ] Jason Moore edited comment on SPARK-26027 at 11/13/18 12:49 AM:

[jira] [Commented] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684567#comment-16684567 ] Jason Moore commented on SPARK-26027: - Actually, the fact that scala-library 2.11.12 is downloaded

[jira] [Commented] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684560#comment-16684560 ] Hyukjin Kwon commented on SPARK-26008: -- Discussing a rough idea or wish should ideally start from a

[jira] [Resolved] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26008. -- Resolution: Invalid > Structured Streaming Manual clock for simulation >

[jira] [Resolved] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26009. -- Resolution: Invalid > Unable to fetch jar from remote repo while running spark-submit on

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684525#comment-16684525 ] Hyukjin Kwon commented on SPARK-26009: -- Let's file a bug when we're clear after talking in the

[jira] [Updated] (SPARK-25994) SPIP: DataFrame-based graph queries and algorithms

2018-11-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25994: -- Description: Copied from the SPIP doc: {quote} GraphX was one of the foundational pillars of

[jira] [Commented] (SPARK-25994) SPIP: DataFrame-based graph queries and algorithms

2018-11-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684524#comment-16684524 ] Xiangrui Meng commented on SPARK-25994: --- [~mju] Could you link the SPIP doc here and link the

[jira] [Created] (SPARK-26028) Design sketch for SPIP: DataFrame-based graph queries and algorithms

2018-11-12 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-26028: - Summary: Design sketch for SPIP: DataFrame-based graph queries and algorithms Key: SPARK-26028 URL: https://issues.apache.org/jira/browse/SPARK-26028 Project:

[jira] [Assigned] (SPARK-25994) SPIP: DataFrame-based graph queries and algorithms

2018-11-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25994: - Assignee: Martin Junghanns > SPIP: DataFrame-based graph queries and algorithms >

[jira] [Updated] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Moore updated SPARK-26027: Priority: Minor (was: Major) > Unable to build Spark for Scala 2.12 with Maven script provided >

[jira] [Commented] (SPARK-26022) PySpark Comparison with Pandas

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684502#comment-16684502 ] Hyukjin Kwon commented on SPARK-26022: -- Yup, will do. Thanks for cc'ing me. > PySpark Comparison

[jira] [Updated] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Moore updated SPARK-26027: Description: In ./build/mvn, from pom.xml is used to determine which Scala library to fetch but

[jira] [Updated] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Moore updated SPARK-26027: Description: In ./build/mvn, from pom.xml is used to determine which Scala library to fetch but

[jira] [Created] (SPARK-26027) Unable to build Spark for Scala 2.12 with Maven script provided

2018-11-12 Thread Jason Moore (JIRA)
Jason Moore created SPARK-26027: --- Summary: Unable to build Spark for Scala 2.12 with Maven script provided Key: SPARK-26027 URL: https://issues.apache.org/jira/browse/SPARK-26027 Project: Spark

[jira] [Commented] (SPARK-26025) Docker image build on dev builds is slow

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1668#comment-1668 ] Apache Spark commented on SPARK-26025: -- User 'vanzin' has created a pull request for this issue:

[jira] [Updated] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-12 Thread Long Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Long Cao updated SPARK-26026: - Description: For 2.3.x and beyond, it appears that published *-javadoc.jars are missing. For concrete

[jira] [Assigned] (SPARK-26025) Docker image build on dev builds is slow

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26025: Assignee: (was: Apache Spark) > Docker image build on dev builds is slow >

[jira] [Commented] (SPARK-26025) Docker image build on dev builds is slow

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684442#comment-16684442 ] Apache Spark commented on SPARK-26025: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26025) Docker image build on dev builds is slow

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26025: Assignee: Apache Spark > Docker image build on dev builds is slow >

[jira] [Created] (SPARK-26026) Published Scaladoc jars missing from Maven Central

2018-11-12 Thread Long Cao (JIRA)
Long Cao created SPARK-26026: Summary: Published Scaladoc jars missing from Maven Central Key: SPARK-26026 URL: https://issues.apache.org/jira/browse/SPARK-26026 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-12 Thread Julien Peloton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-26024: --- Affects Version/s: 2.3.0 2.3.1 > Dataset API:

[jira] [Created] (SPARK-26025) Docker image build on dev builds is slow

2018-11-12 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-26025: -- Summary: Docker image build on dev builds is slow Key: SPARK-26025 URL: https://issues.apache.org/jira/browse/SPARK-26025 Project: Spark Issue Type:

[jira] [Created] (SPARK-26024) Dataset API: repartitionByRange(...) has inconsistent behaviour

2018-11-12 Thread Julien Peloton (JIRA)
Julien Peloton created SPARK-26024: -- Summary: Dataset API: repartitionByRange(...) has inconsistent behaviour Key: SPARK-26024 URL: https://issues.apache.org/jira/browse/SPARK-26024 Project: Spark

[jira] [Commented] (SPARK-26023) Dumping truncated plans to a file

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684385#comment-16684385 ] Apache Spark commented on SPARK-26023: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26023) Dumping truncated plans to a file

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26023: Assignee: Apache Spark > Dumping truncated plans to a file >

[jira] [Created] (SPARK-26023) Dumping truncated plans to a file

2018-11-12 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26023: -- Summary: Dumping truncated plans to a file Key: SPARK-26023 URL: https://issues.apache.org/jira/browse/SPARK-26023 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-26023) Dumping truncated plans to a file

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26023: Assignee: (was: Apache Spark) > Dumping truncated plans to a file >

[jira] [Commented] (SPARK-26023) Dumping truncated plans to a file

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684384#comment-16684384 ] Apache Spark commented on SPARK-26023: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Reopened] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Tom Bar Yacov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bar Yacov reopened SPARK-26008: --- I believe this is more a wish not a question. I reopened to allow review of the wish and

[jira] [Comment Edited] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Tom Bar Yacov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684341#comment-16684341 ] Tom Bar Yacov edited comment on SPARK-26008 at 11/12/18 8:40 PM: - I

[jira] [Updated] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Tom Bar Yacov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bar Yacov updated SPARK-26008: -- Shepherd: Tathagata Das Description: Structured streaming Internal

[jira] [Commented] (SPARK-26022) PySpark Comparison with Pandas

2018-11-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684283#comment-16684283 ] Xiao Li commented on SPARK-26022: - [~hyukjin.kwon] Could you lead this effort to help the community

[jira] [Created] (SPARK-26022) PySpark Comparison with Pandas

2018-11-12 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26022: --- Summary: PySpark Comparison with Pandas Key: SPARK-26022 URL: https://issues.apache.org/jira/browse/SPARK-26022 Project: Spark Issue Type: Documentation

[jira] [Commented] (SPARK-26020) shuffle data from spark streaming not cleaned up when External Shuffle Service is enabled

2018-11-12 Thread Hidayat Teonadi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684200#comment-16684200 ] Hidayat Teonadi commented on SPARK-26020: - linking related ticket SPARK-17233 > shuffle data

[jira] [Commented] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684193#comment-16684193 ] Sean Owen commented on SPARK-26021: --- [~smilegator] and/or [~cloud_fan] might be interested in this?

[jira] [Created] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-12 Thread Sean Owen (JIRA)
Sean Owen created SPARK-26021: - Summary: -0.0 and 0.0 not treated consistently, doesn't match Hive Key: SPARK-26021 URL: https://issues.apache.org/jira/browse/SPARK-26021 Project: Spark Issue

[jira] [Commented] (SPARK-24834) Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

2018-11-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684191#comment-16684191 ] Sean Owen commented on SPARK-24834: --- I agree that doesn't look right at all. To be tidy, I'm going to

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Bala Bharath Reddy Resapu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684190#comment-16684190 ] Bala Bharath Reddy Resapu commented on SPARK-26009: --- Sure, But i think that this might

[jira] [Created] (SPARK-26020) shuffle data from spark streaming not cleaned up when External Shuffle Service is enabled

2018-11-12 Thread Hidayat Teonadi (JIRA)
Hidayat Teonadi created SPARK-26020: --- Summary: shuffle data from spark streaming not cleaned up when External Shuffle Service is enabled Key: SPARK-26020 URL: https://issues.apache.org/jira/browse/SPARK-26020

[jira] [Commented] (SPARK-24834) Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

2018-11-12 Thread Alon Doron (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684146#comment-16684146 ] Alon Doron commented on SPARK-24834: [~srowen] [~mcheah] - In hive, 0.0 and -0.0 are equal since

[jira] [Resolved] (SPARK-26013) Upgrade R tools version to 3.5.1 in AppVeyor build

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26013. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23011

[jira] [Assigned] (SPARK-25994) SPIP: DataFrame-based graph queries and algorithms

2018-11-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-25994: - Assignee: (was: Xiangrui Meng) > SPIP: DataFrame-based graph queries and

[jira] [Assigned] (SPARK-26013) Upgrade R tools version to 3.5.1 in AppVeyor build

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26013: Assignee: Hyukjin Kwon > Upgrade R tools version to 3.5.1 in AppVeyor build >

[jira] [Commented] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684098#comment-16684098 ] Apache Spark commented on SPARK-26015: -- User 'rvesse' has created a pull request for this issue:

[jira] [Commented] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684101#comment-16684101 ] Apache Spark commented on SPARK-26015: -- User 'rvesse' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26015: Assignee: (was: Apache Spark) > Include a USER directive in project provided Spark

[jira] [Assigned] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26015: Assignee: Apache Spark > Include a USER directive in project provided Spark Dockerfiles

[jira] [Created] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

2018-11-12 Thread Ruslan Dautkhanov (JIRA)
Ruslan Dautkhanov created SPARK-26019: - Summary: pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates() Key: SPARK-26019 URL:

[jira] [Commented] (SPARK-26018) Support Scalar subqueries in predicate push down to datasources

2018-11-12 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684073#comment-16684073 ] Marco Gaido commented on SPARK-26018: - I'll submit a PR for this once

[jira] [Created] (SPARK-26018) Support Scalar subqueries in predicate push down to datasources

2018-11-12 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-26018: --- Summary: Support Scalar subqueries in predicate push down to datasources Key: SPARK-26018 URL: https://issues.apache.org/jira/browse/SPARK-26018 Project: Spark

[jira] [Assigned] (SPARK-26014) Deprecate R < 3.4 support

2018-11-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26014: - Docs Text: Versions of R before 3.4 are deprecated in Spark 3.0.0. Assignee:

[jira] [Updated] (SPARK-26017) SVD++ error rate is high in the test suite.

2018-11-12 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26017: --- Description: In the test suite, "{color:#008000}Test SVD++ with mean square error on training set",

[jira] [Updated] (SPARK-26017) SVD++ error rate is high in the test suite.

2018-11-12 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26017: --- Attachment: image-2018-11-12-20-41-49-370.png > SVD++ error rate is high in the test suite. >

[jira] [Created] (SPARK-26017) SVD++ error rate is high in the test suite.

2018-11-12 Thread shahid (JIRA)
shahid created SPARK-26017: -- Summary: SVD++ error rate is high in the test suite. Key: SPARK-26017 URL: https://issues.apache.org/jira/browse/SPARK-26017 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683896#comment-16683896 ] Hyukjin Kwon commented on SPARK-26009: -- Let's move this to mailing list. Questions should go there.

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Bala Bharath Reddy Resapu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683888#comment-16683888 ] Bala Bharath Reddy Resapu commented on SPARK-26009: --- Its a question and i dont think

[jira] [Commented] (SPARK-25995) sparkR should ensure user args are after the argument used for the port

2018-11-12 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683876#comment-16683876 ] Thomas Graves commented on SPARK-25995: --- I haven't looked at the details but I would say whatever

[jira] [Created] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2018-11-12 Thread Chris Caspanello (JIRA)
Chris Caspanello created SPARK-26016: Summary: Encoding not working when using a map / mapPartitions call Key: SPARK-26016 URL: https://issues.apache.org/jira/browse/SPARK-26016 Project: Spark

[jira] [Updated] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2018-11-12 Thread Chris Caspanello (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Caspanello updated SPARK-26016: - Attachment: spark-sandbox.zip > Encoding not working when using a map / mapPartitions

[jira] [Updated] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2018-11-12 Thread Chris Caspanello (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Caspanello updated SPARK-26016: - Attachment: (was: spark-sandbox.zip) > Encoding not working when using a map /

[jira] [Updated] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2018-11-12 Thread Chris Caspanello (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Caspanello updated SPARK-26016: - Attachment: spark-sandbox.zip > Encoding not working when using a map / mapPartitions

[jira] [Assigned] (SPARK-26006) mllib Prefixspan

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26006: Assignee: Apache Spark > mllib Prefixspan > > > Key:

[jira] [Assigned] (SPARK-26006) mllib Prefixspan

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26006: Assignee: (was: Apache Spark) > mllib Prefixspan > > >

[jira] [Commented] (SPARK-26006) mllib Prefixspan

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683838#comment-16683838 ] Apache Spark commented on SPARK-26006: -- User 'shahidki31' has created a pull request for this

[jira] [Resolved] (SPARK-25949) Add test for optimizer rule PullOutPythonUDFInJoinCondition

2018-11-12 Thread Yuanjian Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li resolved SPARK-25949. - Resolution: Fixed Issue resolved by pull request 22955

[jira] [Comment Edited] (SPARK-2243) Support multiple SparkContexts in the same JVM

2018-11-12 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683600#comment-16683600 ] sam edited comment on SPARK-2243 at 11/12/18 11:24 AM: --- Big bonus of being able to

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

2018-11-12 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683600#comment-16683600 ] sam commented on SPARK-2243: Big bonus of being able to create and shutdown SparkContexts is to be able to

[jira] [Created] (SPARK-26015) Include a USER directive in project provided Spark Dockerfiles

2018-11-12 Thread Rob Vesse (JIRA)
Rob Vesse created SPARK-26015: - Summary: Include a USER directive in project provided Spark Dockerfiles Key: SPARK-26015 URL: https://issues.apache.org/jira/browse/SPARK-26015 Project: Spark

[jira] [Commented] (SPARK-25023) Clarify Spark security documentation

2018-11-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683550#comment-16683550 ] Apache Spark commented on SPARK-25023: -- User 'rvesse' has created a pull request for this issue:

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683448#comment-16683448 ] Hyukjin Kwon commented on SPARK-26009: -- Please avoid to set a blocker which is usually reserved for

[jira] [Commented] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683451#comment-16683451 ] Hyukjin Kwon commented on SPARK-26009: -- Is it a question or an issue? > Unable to fetch jar from

[jira] [Updated] (SPARK-26009) Unable to fetch jar from remote repo while running spark-submit on kubernetes

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26009: - Priority: Major (was: Blocker) > Unable to fetch jar from remote repo while running

[jira] [Resolved] (SPARK-26008) Structured Streaming Manual clock for simulation

2018-11-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26008. -- Resolution: Invalid Questions should be asked to mailing list (see

  1   2   >