[jira] [Assigned] (SPARK-25592) Bump master branch version to 3.0.0-SNAPSHOT

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25592: Assignee: Apache Spark (was: Xiao Li) > Bump master branch version to 3.0.0-SNAPSHOT > -

[jira] [Commented] (SPARK-25592) Bump master branch version to 3.0.0-SNAPSHOT

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16635058#comment-16635058 ] Apache Spark commented on SPARK-25592: -- User 'gatorsmile' has created a pull reques

[jira] [Assigned] (SPARK-25592) Bump master branch version to 3.0.0-SNAPSHOT

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25592: Assignee: Xiao Li (was: Apache Spark) > Bump master branch version to 3.0.0-SNAPSHOT > -

[jira] [Created] (SPARK-25592) Bump master branch version to 3.0.0-SNAPSHOT

2018-10-01 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25592: --- Summary: Bump master branch version to 3.0.0-SNAPSHOT Key: SPARK-25592 URL: https://issues.apache.org/jira/browse/SPARK-25592 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-25461) PySpark Pandas UDF outputs incorrect results when input columns contain None

2018-10-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16635048#comment-16635048 ] Liang-Chi Hsieh edited comment on SPARK-25461 at 10/2/18 5:27 AM:

[jira] [Commented] (SPARK-25461) PySpark Pandas UDF outputs incorrect results when input columns contain None

2018-10-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16635048#comment-16635048 ] Liang-Chi Hsieh commented on SPARK-25461: - I've looked more at this. We don't re

[jira] [Created] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-01 Thread Abdeali Kothari (JIRA)
Abdeali Kothari created SPARK-25591: --- Summary: PySpark Accumulators with multiple PythonUDFs Key: SPARK-25591 URL: https://issues.apache.org/jira/browse/SPARK-25591 Project: Spark Issue Typ

[jira] [Commented] (SPARK-15689) Data source API v2

2018-10-01 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634871#comment-16634871 ] Wenchen Fan commented on SPARK-15689: - So {{SupportsReportPartitioning}} is not powe

[jira] [Updated] (SPARK-25543) Confusing log messages at DEBUG level, in K8s mode.

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25543: -- Fix Version/s: (was: 2.4.1) (was: 2.5.0) 2.4.0 > Confusi

[jira] [Updated] (SPARK-23401) Improve test cases for all supported types and unsupported types

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23401: -- Fix Version/s: (was: 2.4.1) (was: 2.5.0) 2.4.0 > Improve

[jira] [Updated] (SPARK-25542) Flaky test: OpenHashMapSuite

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25542: -- Fix Version/s: (was: 2.4.1) 2.4.0 > Flaky test: OpenHashMapSuite >

[jira] [Updated] (SPARK-25572) SparkR tests failed on CRAN on Java 10

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25572: -- Target Version/s: (was: 2.4.1, 2.5.0) Fix Version/s: (was: 2.4.1)

[jira] [Updated] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25578: -- Target Version/s: (was: 2.4.1) > Update to Scala 2.12.7 > -- > >

[jira] [Updated] (SPARK-25570) Replace 2.3.1 with 2.3.2 in HiveExternalCatalogVersionsSuite

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25570: -- Fix Version/s: (was: 2.4.1) (was: 2.5.0) 2.4.0 > Replace

[jira] [Resolved] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25578. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22600 [https://github.c

[jira] [Assigned] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25578: - Assignee: Sean Owen > Update to Scala 2.12.7 > -- > > Key:

[jira] [Updated] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-01 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Heuer updated SPARK-25587: -- Description: In an attempt to replicate the following issue in ADAM, a library downstream of

[jira] [Updated] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2018-10-01 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-25590: --- Description: That jar file, which is packaged when the k8s profile is enabled, has a log4j

[jira] [Created] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2018-10-01 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-25590: -- Summary: kubernetes-model-2.0.0.jar masks default Spark logging config Key: SPARK-25590 URL: https://issues.apache.org/jira/browse/SPARK-25590 Project: Spark

[jira] [Commented] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634741#comment-16634741 ] Apache Spark commented on SPARK-25589: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25589: Assignee: (was: Apache Spark) > Add BloomFilterBenchmark > >

[jira] [Commented] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634740#comment-16634740 ] Apache Spark commented on SPARK-25589: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25589: Assignee: Apache Spark > Add BloomFilterBenchmark > > >

[jira] [Updated] (SPARK-25586) toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25586: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) This is not a bug; SPARK-25118

[jira] [Updated] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25589: -- Component/s: Tests > Add BloomFilterBenchmark > > > K

[jira] [Created] (SPARK-25589) Add BloomFilterBenchmark

2018-10-01 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-25589: - Summary: Add BloomFilterBenchmark Key: SPARK-25589 URL: https://issues.apache.org/jira/browse/SPARK-25589 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-25575) SQL tab in the spark UI doesn't have option of hiding tables, eventhough other UI tabs has.

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25575: - Assignee: shahid > SQL tab in the spark UI doesn't have option of hiding tables, eventhough >

[jira] [Resolved] (SPARK-25575) SQL tab in the spark UI doesn't have option of hiding tables, eventhough other UI tabs has.

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25575. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22592 [https://github.c

[jira] [Updated] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-01 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Heuer updated SPARK-25587: -- Description: In an attempt to replicate the following issue in ADAM, a library downstream of

[jira] [Updated] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-01 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Heuer updated SPARK-25588: -- Description: In ADAM, a library downstream of Spark, we use Avro to define a schema, generate

[jira] [Created] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-01 Thread Michael Heuer (JIRA)
Michael Heuer created SPARK-25588: - Summary: SchemaParseException: Can't redefine: list when reading from Parquet Key: SPARK-25588 URL: https://issues.apache.org/jira/browse/SPARK-25588 Project: Spark

[jira] [Updated] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-01 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Heuer updated SPARK-25587: -- Description: In an attempt to replicate the following issue in ADAM, a library downstream of

[jira] [Created] (SPARK-25587) NPE in Dataset when reading from Parquet as Product

2018-10-01 Thread Michael Heuer (JIRA)
Michael Heuer created SPARK-25587: - Summary: NPE in Dataset when reading from Parquet as Product Key: SPARK-25587 URL: https://issues.apache.org/jira/browse/SPARK-25587 Project: Spark Issue T

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-10-01 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634679#comment-16634679 ] John Bauer commented on SPARK-21542: The above is not as minimal as I would have lik

[jira] [Commented] (SPARK-21542) Helper functions for custom Python Persistence

2018-10-01 Thread John Bauer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634677#comment-16634677 ] John Bauer commented on SPARK-21542: {code:python} #!/usr/bin/env python3 # -*- cod

[jira] [Commented] (SPARK-15689) Data source API v2

2018-10-01 Thread Geoff Freeman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634650#comment-16634650 ] Geoff Freeman commented on SPARK-15689: --- I'm having trouble figuring out how to ex

[jira] [Comment Edited] (SPARK-15689) Data source API v2

2018-10-01 Thread Geoff Freeman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634650#comment-16634650 ] Geoff Freeman edited comment on SPARK-15689 at 10/1/18 9:24 PM: --

[jira] [Assigned] (SPARK-25586) toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25586: Assignee: (was: Apache Spark) > toString method of GeneralizedLinearRegressionTrainin

[jira] [Assigned] (SPARK-25586) toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25586: Assignee: Apache Spark > toString method of GeneralizedLinearRegressionTrainingSummary ru

[jira] [Commented] (SPARK-25586) toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634643#comment-16634643 ] Apache Spark commented on SPARK-25586: -- User 'ankuriitg' has created a pull request

[jira] [Created] (SPARK-25586) toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError

2018-10-01 Thread Ankur Gupta (JIRA)
Ankur Gupta created SPARK-25586: --- Summary: toString method of GeneralizedLinearRegressionTrainingSummary runs in infinite loop throwing StackOverflowError Key: SPARK-25586 URL: https://issues.apache.org/jira/browse

[jira] [Created] (SPARK-25585) Allow users to specify scale of result in Decimal arithmetic

2018-10-01 Thread Benito Kestelman (JIRA)
Benito Kestelman created SPARK-25585: Summary: Allow users to specify scale of result in Decimal arithmetic Key: SPARK-25585 URL: https://issues.apache.org/jira/browse/SPARK-25585 Project: Spark

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634534#comment-16634534 ] Apache Spark commented on SPARK-25538: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25538: Assignee: (was: Apache Spark) > incorrect row counts after distinct() > -

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634532#comment-16634532 ] Apache Spark commented on SPARK-25538: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25538: Assignee: Apache Spark > incorrect row counts after distinct() >

[jira] [Updated] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25578: -- Issue Type: Bug (was: Improvement) OK. I'm proposing it for 2.4.0 mostly because it _might_ be a bug

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634527#comment-16634527 ] Apache Spark commented on SPARK-25062: -- User 'peter-toth' has created a pull reques

[jira] [Commented] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634524#comment-16634524 ] Apache Spark commented on SPARK-25062: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25062: Assignee: (was: Apache Spark) > Clean up BlockLocations in FileStatus objects > -

[jira] [Assigned] (SPARK-25062) Clean up BlockLocations in FileStatus objects

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25062: Assignee: Apache Spark > Clean up BlockLocations in FileStatus objects >

[jira] [Commented] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634492#comment-16634492 ] Dongjoon Hyun commented on SPARK-25578: --- [~srowen]. Could you update the `Type` an

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634430#comment-16634430 ] Dongjoon Hyun commented on SPARK-25538: --- [~mgaido]'s PR, https://github.com/apache

[jira] [Resolved] (SPARK-25315) setting "auto.offset.reset" to "earliest" has no effect in Structured Streaming with Spark 2.3.1 and Kafka 1.0

2018-10-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-25315. -- Resolution: Not A Bug > setting "auto.offset.reset" to "earliest" has no effect in Structured

[jira] [Commented] (SPARK-25315) setting "auto.offset.reset" to "earliest" has no effect in Structured Streaming with Spark 2.3.1 and Kafka 1.0

2018-10-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634423#comment-16634423 ] Shixiong Zhu commented on SPARK-25315: -- Kafka’s own configurations should be set wi

[jira] [Commented] (SPARK-25582) Error in Spark logs when using the org.apache.spark:spark-sql_2.11:2.2.0 Java library

2018-10-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634389#comment-16634389 ] Marco Gaido commented on SPARK-25582: - Sorry, I linked the wrong JIRA in the PR. Ple

[jira] [Updated] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-25583: --- Priority: Minor (was: Trivial) > Add newly added History server related configurations in the documentation

[jira] [Assigned] (SPARK-25576) Fix lint failure in 2.2

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25576: Assignee: Apache Spark > Fix lint failure in 2.2 > --- > >

[jira] [Comment Edited] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634344#comment-16634344 ] Kazuaki Ishizaki edited comment on SPARK-25538 at 10/1/18 5:21 PM: ---

[jira] [Assigned] (SPARK-25576) Fix lint failure in 2.2

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25576: Assignee: (was: Apache Spark) > Fix lint failure in 2.2 > --- > >

[jira] [Resolved] (SPARK-25322) ML, Graph 2.4 QA: API: Experimental, DeveloperApi, final, sealed audit

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25322. --- Resolution: Done > ML, Graph 2.4 QA: API: Experimental, DeveloperApi, final, sealed audit >

[jira] [Resolved] (SPARK-25319) Spark MLlib, GraphX 2.4 QA umbrella

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25319. --- Resolution: Done Assignee: Weichen Xu (was: Joseph K. Bradley) > Spark MLlib, GraphX

[jira] [Resolved] (SPARK-25325) ML, Graph 2.4 QA: Update user guide for new features & APIs

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25325. --- Resolution: Won't Do > ML, Graph 2.4 QA: Update user guide for new features & APIs > ---

[jira] [Updated] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25323: -- Priority: Major (was: Critical) > ML 2.4 QA: API: Python API coverage > -

[jira] [Resolved] (SPARK-25323) ML 2.4 QA: API: Python API coverage

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25323. --- Resolution: Won't Do > ML 2.4 QA: API: Python API coverage > ---

[jira] [Updated] (SPARK-25325) ML, Graph 2.4 QA: Update user guide for new features & APIs

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25325: -- Priority: Major (was: Critical) > ML, Graph 2.4 QA: Update user guide for new features & APIs

[jira] [Resolved] (SPARK-25326) ML, Graph 2.4 QA: Programming guide update and migration guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25326. --- Resolution: Won't Do > ML, Graph 2.4 QA: Programming guide update and migration guide >

[jira] [Updated] (SPARK-25326) ML, Graph 2.4 QA: Programming guide update and migration guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25326: -- Priority: Major (was: Critical) > ML, Graph 2.4 QA: Programming guide update and migration gu

[jira] [Updated] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25584: -- Component/s: ML > Document libsvm data source in doc site > --

[jira] [Commented] (SPARK-25524) Spark datasource for image/libsvm user guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634347#comment-16634347 ] Xiangrui Meng commented on SPARK-25524: --- Marked as duplicate and create SPARK-2558

[jira] [Updated] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25584: -- Description: Currently, we only have Scala/Java API docs for libsvm data source. It would be n

[jira] [Updated] (SPARK-25347) Document image data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-25347: -- Component/s: ML > Document image data source in doc site > ---

[jira] [Created] (SPARK-25584) Document libsvm data source in doc site

2018-10-01 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-25584: - Summary: Document libsvm data source in doc site Key: SPARK-25584 URL: https://issues.apache.org/jira/browse/SPARK-25584 Project: Spark Issue Type: Story

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634344#comment-16634344 ] Kazuaki Ishizaki commented on SPARK-25538: -- This test case does not print {{63}

[jira] [Resolved] (SPARK-25524) Spark datasource for image/libsvm user guide

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-25524. --- Resolution: Duplicate > Spark datasource for image/libsvm user guide > -

[jira] [Commented] (SPARK-25378) ArrayData.toArray(StringType) assume UTF8String in 2.4

2018-10-01 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634335#comment-16634335 ] Xiangrui Meng commented on SPARK-25378: --- I don't think I'm the right person to dec

[jira] [Commented] (SPARK-25561) HiveClient.getPartitionsByFilter throws an exception if Hive retries directSql

2018-10-01 Thread Karthik Manamcheri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634332#comment-16634332 ] Karthik Manamcheri commented on SPARK-25561: I am working on a patch for thi

[jira] [Commented] (SPARK-25582) Error in Spark logs when using the org.apache.spark:spark-sql_2.11:2.2.0 Java library

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634290#comment-16634290 ] Apache Spark commented on SPARK-25582: -- User 'mgaido91' has created a pull request

[jira] [Assigned] (SPARK-25582) Error in Spark logs when using the org.apache.spark:spark-sql_2.11:2.2.0 Java library

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25582: Assignee: (was: Apache Spark) > Error in Spark logs when using the org.apache.spark:s

[jira] [Assigned] (SPARK-25582) Error in Spark logs when using the org.apache.spark:spark-sql_2.11:2.2.0 Java library

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25582: Assignee: Apache Spark > Error in Spark logs when using the org.apache.spark:spark-sql_2.

[jira] [Commented] (SPARK-25582) Error in Spark logs when using the org.apache.spark:spark-sql_2.11:2.2.0 Java library

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634284#comment-16634284 ] Apache Spark commented on SPARK-25582: -- User 'mgaido91' has created a pull request

[jira] [Commented] (SPARK-25544) Slow/failed convergence in Spark ML models due to internal predictor scaling

2018-10-01 Thread Andrew Crosby (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634268#comment-16634268 ] Andrew Crosby commented on SPARK-25544: --- SPARK-23537 contains what might be anothe

[jira] [Commented] (SPARK-23537) Logistic Regression without standardization

2018-10-01 Thread Andrew Crosby (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634263#comment-16634263 ] Andrew Crosby commented on SPARK-23537: --- The different results for standardization

[jira] [Assigned] (SPARK-18364) Expose metrics for YarnShuffleService

2018-10-01 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-18364: - Assignee: Marek Simunek > Expose metrics for YarnShuffleService > -

[jira] [Resolved] (SPARK-18364) Expose metrics for YarnShuffleService

2018-10-01 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-18364. --- Resolution: Fixed Fix Version/s: 2.5.0 > Expose metrics for YarnShuffleService >

[jira] [Commented] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634204#comment-16634204 ] Apache Spark commented on SPARK-25583: -- User 'shahidki31' has created a pull reques

[jira] [Commented] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634202#comment-16634202 ] Apache Spark commented on SPARK-25583: -- User 'shahidki31' has created a pull reques

[jira] [Assigned] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25583: Assignee: (was: Apache Spark) > Add newly added History server related configurations

[jira] [Assigned] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25583: Assignee: Apache Spark > Add newly added History server related configurations in the doc

[jira] [Created] (SPARK-25583) Add newly added History server related configurations in the documentation

2018-10-01 Thread shahid (JIRA)
shahid created SPARK-25583: -- Summary: Add newly added History server related configurations in the documentation Key: SPARK-25583 URL: https://issues.apache.org/jira/browse/SPARK-25583 Project: Spark

[jira] [Updated] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-25538: -- Priority: Blocker (was: Major) > incorrect row counts after distinct() >

[jira] [Assigned] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25578: Assignee: (was: Apache Spark) > Update to Scala 2.12.7 > -- > >

[jira] [Assigned] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25578: Assignee: Apache Spark > Update to Scala 2.12.7 > -- > >

[jira] [Commented] (SPARK-25578) Update to Scala 2.12.7

2018-10-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634111#comment-16634111 ] Apache Spark commented on SPARK-25578: -- User 'srowen' has created a pull request fo

[jira] [Commented] (SPARK-25538) incorrect row counts after distinct()

2018-10-01 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634106#comment-16634106 ] Marco Gaido commented on SPARK-25538: - I was able to reproduce also using limit inst

[jira] [Assigned] (SPARK-25510) Create new trait replace BenchmarkWithCodegen

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25510: - Assignee: Yuming Wang > Create new trait replace BenchmarkWithCodegen > --

[jira] [Updated] (SPARK-25510) Create a new trait SqlBasedBenchmark

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25510: -- Summary: Create a new trait SqlBasedBenchmark (was: Create new trait replace BenchmarkWithC

[jira] [Resolved] (SPARK-25510) Create new trait replace BenchmarkWithCodegen

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25510. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22484 [https://

[jira] [Resolved] (SPARK-25476) Refactor AggregateBenchmark to use main method

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25476. --- Resolution: Fixed Fix Version/s: 2.5.0 Issue resolved by pull request 22484 [https://

[jira] [Assigned] (SPARK-25476) Refactor AggregateBenchmark to use main method

2018-10-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-25476: - Assignee: Yuming Wang > Refactor AggregateBenchmark to use main method > --

  1   2   >