[jira] [Updated] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

[ https://issues.apache.org/jira/browse/SPARK-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-17517: - Target Version/s: (was: 2.1.0) External issue ID: (was: 12795) Fix Version/s: (was:

[jira] [Assigned] (SPARK-17123) Performing set operations that combine string and date / timestamp columns may result in generated projection code which doesn't compile

[ https://issues.apache.org/jira/browse/SPARK-17123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17123: Assignee: (was: Apache Spark) > Performing set operations that combine string and

[jira] [Commented] (SPARK-17123) Performing set operations that combine string and date / timestamp columns may result in generated projection code which doesn't compile

[ https://issues.apache.org/jira/browse/SPARK-17123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486354#comment-15486354 ] Apache Spark commented on SPARK-17123: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-17123) Performing set operations that combine string and date / timestamp columns may result in generated projection code which doesn't compile

[ https://issues.apache.org/jira/browse/SPARK-17123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17123: Assignee: Apache Spark > Performing set operations that combine string and date /

[jira] [Updated] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

[ https://issues.apache.org/jira/browse/SPARK-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-17517: - Description: For current `BroadcastHashJoinExec`, we generate join code for key is not unique like

[jira] [Commented] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

[ https://issues.apache.org/jira/browse/SPARK-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486335#comment-15486335 ] Apache Spark commented on SPARK-17517: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

[ https://issues.apache.org/jira/browse/SPARK-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17517: Assignee: (was: Apache Spark) > Improve generated Code for BroadcastHashJoinExec >

[jira] [Assigned] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

[ https://issues.apache.org/jira/browse/SPARK-17517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17517: Assignee: Apache Spark > Improve generated Code for BroadcastHashJoinExec >

[jira] [Created] (SPARK-17517) Improve generated Code for BroadcastHashJoinExec

Kent Yao created SPARK-17517: Summary: Improve generated Code for BroadcastHashJoinExec Key: SPARK-17517 URL: https://issues.apache.org/jira/browse/SPARK-17517 Project: Spark Issue Type:

[jira] [Updated] (SPARK-17502) Multiple Bugs in DDL Statements on Temporary Views

[ https://issues.apache.org/jira/browse/SPARK-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-17502: Description: - When the permanent tables/views do not exist but the temporary view exists, the expected

[jira] [Commented] (SPARK-17516) Current user info is not checked on STS in DML queries

[ https://issues.apache.org/jira/browse/SPARK-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486192#comment-15486192 ] Tao Li commented on SPARK-17516: cc [~bikassaha], [~thejas] > Current user info is not checked on STS in

[jira] [Commented] (SPARK-17516) Current user info is not checked on STS in DML queries

[ https://issues.apache.org/jira/browse/SPARK-17516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486177#comment-15486177 ] Tao Li commented on SPARK-17516: I have verified that switching to Hive.get() to get the Hive instance

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486131#comment-15486131 ] Cody Koeninger commented on SPARK-15406: I've got a minimal working Source and SourceProvider, at

[jira] [Created] (SPARK-17516) Current user info is not checked on STS in DML queries

Tao Li created SPARK-17516: -- Summary: Current user info is not checked on STS in DML queries Key: SPARK-17516 URL: https://issues.apache.org/jira/browse/SPARK-17516 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-17515) CollectLimit.execute() should perform per-partition limits

[ https://issues.apache.org/jira/browse/SPARK-17515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485986#comment-15485986 ] Apache Spark commented on SPARK-17515: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17515) CollectLimit.execute() should perform per-partition limits

[ https://issues.apache.org/jira/browse/SPARK-17515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17515: Assignee: Apache Spark (was: Josh Rosen) > CollectLimit.execute() should perform

[jira] [Assigned] (SPARK-17515) CollectLimit.execute() should perform per-partition limits

[ https://issues.apache.org/jira/browse/SPARK-17515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17515: Assignee: Josh Rosen (was: Apache Spark) > CollectLimit.execute() should perform

[jira] [Created] (SPARK-17515) CollectLimit.execute() should perform per-partition limits

Josh Rosen created SPARK-17515: -- Summary: CollectLimit.execute() should perform per-partition limits Key: SPARK-17515 URL: https://issues.apache.org/jira/browse/SPARK-17515 Project: Spark Issue

[jira] [Updated] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

[ https://issues.apache.org/jira/browse/SPARK-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kishor Patil updated SPARK-17511: - Description: While trying to reach launch multiple containers in pool, if running executors

[jira] [Assigned] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

[ https://issues.apache.org/jira/browse/SPARK-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17511: Assignee: (was: Apache Spark) > Dynamic allocation race condition: Containers getting

[jira] [Assigned] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

[ https://issues.apache.org/jira/browse/SPARK-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17511: Assignee: Apache Spark > Dynamic allocation race condition: Containers getting marked

[jira] [Commented] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

[ https://issues.apache.org/jira/browse/SPARK-17511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485941#comment-15485941 ] Apache Spark commented on SPARK-17511: -- User 'kishorvpatil' has created a pull request for this

[jira] [Updated] (SPARK-16728) migrate internal API for MLlib trees from spark.mllib to spark.ml

[ https://issues.apache.org/jira/browse/SPARK-16728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Feinberg updated SPARK-16728: -- Description: Currently, spark.ml trees rely on spark.mllib implementations. There are

[jira] [Commented] (SPARK-17514) df.take(1) and df.limit(1).collect() perform differently in Python

[ https://issues.apache.org/jira/browse/SPARK-17514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485821#comment-15485821 ] Apache Spark commented on SPARK-17514: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17514) df.take(1) and df.limit(1).collect() perform differently in Python

[ https://issues.apache.org/jira/browse/SPARK-17514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17514: Assignee: Josh Rosen (was: Apache Spark) > df.take(1) and df.limit(1).collect() perform

[jira] [Assigned] (SPARK-17514) df.take(1) and df.limit(1).collect() perform differently in Python

[ https://issues.apache.org/jira/browse/SPARK-17514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17514: Assignee: Apache Spark (was: Josh Rosen) > df.take(1) and df.limit(1).collect() perform

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485782#comment-15485782 ] Evan Zamir commented on SPARK-17508: [~bryanc] Oh, that helps a lot! I've been writing very light

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485780#comment-15485780 ] Tathagata Das commented on SPARK-15406: --- Hey all, I am working on the design doc right now. I will

[jira] [Created] (SPARK-17514) df.take(1) and df.limit(1).collect() perform differently in Python

Josh Rosen created SPARK-17514: -- Summary: df.take(1) and df.limit(1).collect() perform differently in Python Key: SPARK-17514 URL: https://issues.apache.org/jira/browse/SPARK-17514 Project: Spark

[jira] [Assigned] (SPARK-17513) StreamExecution should discard unneeded metadata

[ https://issues.apache.org/jira/browse/SPARK-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17513: Assignee: Apache Spark > StreamExecution should discard unneeded metadata >

[jira] [Assigned] (SPARK-17513) StreamExecution should discard unneeded metadata

[ https://issues.apache.org/jira/browse/SPARK-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17513: Assignee: (was: Apache Spark) > StreamExecution should discard unneeded metadata >

[jira] [Commented] (SPARK-17513) StreamExecution should discard unneeded metadata

[ https://issues.apache.org/jira/browse/SPARK-17513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485735#comment-15485735 ] Apache Spark commented on SPARK-17513: -- User 'frreiss' has created a pull request for this issue:

[jira] [Created] (SPARK-17513) StreamExecution should discard unneeded metadata

Frederick Reiss created SPARK-17513: --- Summary: StreamExecution should discard unneeded metadata Key: SPARK-17513 URL: https://issues.apache.org/jira/browse/SPARK-17513 Project: Spark Issue

[jira] [Created] (SPARK-17512) Specifying remote files for Python based Spark jobs in Yarn cluster mode not working

Udit Mehrotra created SPARK-17512: - Summary: Specifying remote files for Python based Spark jobs in Yarn cluster mode not working Key: SPARK-17512 URL: https://issues.apache.org/jira/browse/SPARK-17512

[jira] [Commented] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485718#comment-15485718 ] Pramit Choudhary commented on SPARK-16750: -- It seems the release branch was cut on 19th July and

[jira] [Commented] (SPARK-16441) Spark application hang when dynamic allocation is enabled

[ https://issues.apache.org/jira/browse/SPARK-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485711#comment-15485711 ] Ashwin Shankar commented on SPARK-16441: Hey [~Dhruve Ashar], we hit the same issue at Netflix

[jira] [Resolved] (SPARK-17474) Python UDF does not work between Sort and Limit

[ https://issues.apache.org/jira/browse/SPARK-17474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17474. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-16750) ML GaussianMixture training failed due to feature column type mistake

[ https://issues.apache.org/jira/browse/SPARK-16750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485531#comment-15485531 ] Pramit Choudhary commented on SPARK-16750: -- Did this fix make it to the release version 2.0.0.

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485530#comment-15485530 ] Bryan Cutler commented on SPARK-17508: -- [~zamir.e...@gmail.com] This is a gripe I have with ML

[jira] [Created] (SPARK-17511) Dynamic allocation race condition: Containers getting marked failed while releasing

Kishor Patil created SPARK-17511: Summary: Dynamic allocation race condition: Containers getting marked failed while releasing Key: SPARK-17511 URL: https://issues.apache.org/jira/browse/SPARK-17511

[jira] [Resolved] (SPARK-17485) Failed remote cached block reads can lead to whole job failure

[ https://issues.apache.org/jira/browse/SPARK-17485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17485. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485505#comment-15485505 ] Hyukjin Kwon commented on SPARK-17477: -- I left a related commnet

[jira] [Resolved] (SPARK-13406) NPE in LazilyGeneratedOrdering

[ https://issues.apache.org/jira/browse/SPARK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13406. Resolution: Duplicate > NPE in LazilyGeneratedOrdering > -- > >

[jira] [Updated] (SPARK-13406) NPE in LazilyGeneratedOrdering

[ https://issues.apache.org/jira/browse/SPARK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-13406: --- Assignee: (was: Josh Rosen) > NPE in LazilyGeneratedOrdering > -- >

[jira] [Resolved] (SPARK-2424) ApplicationState.MAX_NUM_RETRY should be configurable

[ https://issues.apache.org/jira/browse/SPARK-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2424. --- Resolution: Duplicate Fix Version/s: 2.1.0 2.0.1 1.6.3

[jira] [Resolved] (SPARK-14818) Move sketch and mllibLocal out from mima exclusion

[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-14818. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485420#comment-15485420 ] Apache Spark commented on SPARK-17463: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-5575) Artificial neural networks for MLlib deep learning

[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5575: Description: *Goal:* Implement various types of artificial neural networks *Motivation:*

[jira] [Assigned] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

[ https://issues.apache.org/jira/browse/SPARK-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17509: Assignee: (was: Apache Spark) > When wrapping catalyst datatype to Hive data type

[jira] [Commented] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

[ https://issues.apache.org/jira/browse/SPARK-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485327#comment-15485327 ] Apache Spark commented on SPARK-17509: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

[ https://issues.apache.org/jira/browse/SPARK-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17509: Assignee: Apache Spark > When wrapping catalyst datatype to Hive data type avoid pattern

[jira] [Assigned] (SPARK-15621) BatchEvalPythonExec fails with OOM

[ https://issues.apache.org/jira/browse/SPARK-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15621: -- Assignee: Davies Liu > BatchEvalPythonExec fails with OOM >

[jira] [Resolved] (SPARK-17483) Minor refactoring and cleanup in BlockManager block status reporting and block removal

[ https://issues.apache.org/jira/browse/SPARK-17483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17483. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15036

[jira] [Commented] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485130#comment-15485130 ] Alessio commented on SPARK-2352: Pretty strange that this post with such hype is still "In progress" after

[jira] [Commented] (SPARK-5575) Artificial neural networks for MLlib deep learning

[ https://issues.apache.org/jira/browse/SPARK-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485131#comment-15485131 ] Alessio commented on SPARK-5575: Pretty strange that this post with such hype is still "In progress" after

[jira] [Created] (SPARK-17510) Set Streaming MaxRate Independently For Multiple Streams

Jeff Nadler created SPARK-17510: --- Summary: Set Streaming MaxRate Independently For Multiple Streams Key: SPARK-17510 URL: https://issues.apache.org/jira/browse/SPARK-17510 Project: Spark Issue

[jira] [Updated] (SPARK-17409) Query in CTAS is Optimized Twice

[ https://issues.apache.org/jira/browse/SPARK-17409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17409: --- Labels: correctness (was: ) > Query in CTAS is Optimized Twice > >

[jira] [Commented] (SPARK-17494) Floor function rounds up during join

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484952#comment-15484952 ] Josh Rosen commented on SPARK-17494: This also seems to affect Spark 2.0, except there it always

[jira] [Updated] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

[ https://issues.apache.org/jira/browse/SPARK-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia updated SPARK-17509: Affects Version/s: 2.0.0 Description: Profiling a job, we saw that patten matching in

[jira] [Updated] (SPARK-17494) Floor function rounds up during join

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17494: --- Affects Version/s: 2.0.0 > Floor function rounds up during join >

[jira] [Updated] (SPARK-17494) Floor function rounds up during join

[ https://issues.apache.org/jira/browse/SPARK-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17494: --- Labels: correctness (was: ) > Floor function rounds up during join >

[jira] [Created] (SPARK-17509) When wrapping catalyst datatype to Hive data type avoid pattern matching

Sital Kedia created SPARK-17509: --- Summary: When wrapping catalyst datatype to Hive data type avoid pattern matching Key: SPARK-17509 URL: https://issues.apache.org/jira/browse/SPARK-17509 Project:

[jira] [Resolved] (SPARK-17503) Memory leak in Memory store when unable to cache the whole RDD in memory

[ https://issues.apache.org/jira/browse/SPARK-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-17503. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Fixed for 2.0.1 / 2.1.0 by

[jira] [Updated] (SPARK-17503) Memory leak in Memory store when unable to cache the whole RDD in memory

[ https://issues.apache.org/jira/browse/SPARK-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17503: --- Assignee: Sean Zhong > Memory leak in Memory store when unable to cache the whole RDD in memory >

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484861#comment-15484861 ] Evan Zamir commented on SPARK-17508: Just ran the same snippet of code setting weightCol="" and that

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484850#comment-15484850 ] Evan Zamir commented on SPARK-17508: Yep, I'm running 2.0.0. You can see in the error messages above

[jira] [Updated] (SPARK-17503) Memory leak in Memory store when unable to cache the whole RDD in memory

[ https://issues.apache.org/jira/browse/SPARK-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-17503: --- Target Version/s: 2.0.1, 2.1.0 (was: 2.1.0) > Memory leak in Memory store when unable to cache the

[jira] [Commented] (SPARK-17471) Add compressed method for Matrix class

[ https://issues.apache.org/jira/browse/SPARK-17471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484775#comment-15484775 ] Seth Hendrickson commented on SPARK-17471: -- [~yanboliang] Do you have any updates on this? We

[jira] [Assigned] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17463: Assignee: Shixiong Zhu (was: Apache Spark) > Serialization of accumulators in heartbeats

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484769#comment-15484769 ] Apache Spark commented on SPARK-17463: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17463: Assignee: Apache Spark (was: Shixiong Zhu) > Serialization of accumulators in heartbeats

[jira] [Commented] (SPARK-17321) YARN shuffle service should use good disk from yarn.nodemanager.local-dirs

[ https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484748#comment-15484748 ] Thomas Graves commented on SPARK-17321: --- Not sure I follow this comment. So you are using NM

[jira] [Assigned] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-17463: Assignee: Shixiong Zhu > Serialization of accumulators in heartbeats is not thread-safe >

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484732#comment-15484732 ] Matei Zaharia commented on SPARK-17445: --- Sounds good to me. > Reference an ASF page as the main

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484720#comment-15484720 ] Shixiong Zhu commented on SPARK-17463: -- [~joshrosen] I think we can just leave LongAccum as it is.

[jira] [Commented] (SPARK-17424) Dataset job fails from unsound substitution in ScalaReflect

[ https://issues.apache.org/jira/browse/SPARK-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484718#comment-15484718 ] Ryan Blue commented on SPARK-17424: --- I'm adding the above fix in a PR. This fix works for us (the job

[jira] [Commented] (SPARK-17424) Dataset job fails from unsound substitution in ScalaReflect

[ https://issues.apache.org/jira/browse/SPARK-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484715#comment-15484715 ] Apache Spark commented on SPARK-17424: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17424) Dataset job fails from unsound substitution in ScalaReflect

[ https://issues.apache.org/jira/browse/SPARK-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17424: Assignee: (was: Apache Spark) > Dataset job fails from unsound substitution in

[jira] [Assigned] (SPARK-17424) Dataset job fails from unsound substitution in ScalaReflect

[ https://issues.apache.org/jira/browse/SPARK-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17424: Assignee: Apache Spark > Dataset job fails from unsound substitution in ScalaReflect >

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Commented] (SPARK-17508) Setting weightCol to None in ML library causes an error

[ https://issues.apache.org/jira/browse/SPARK-17508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484704#comment-15484704 ] Sean Owen commented on SPARK-17508: --- This looks a lot like the problem solved in SPARK-14931 /

[jira] [Assigned] (SPARK-14818) Move sketch and mllibLocal out from mima exclusion

[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen reassigned SPARK-14818: -- Assignee: Josh Rosen > Move sketch and mllibLocal out from mima exclusion >

[jira] [Assigned] (SPARK-14818) Move sketch and mllibLocal out from mima exclusion

[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14818: Assignee: Apache Spark (was: Josh Rosen) > Move sketch and mllibLocal out from mima

[jira] [Assigned] (SPARK-14818) Move sketch and mllibLocal out from mima exclusion

[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14818: Assignee: Josh Rosen (was: Apache Spark) > Move sketch and mllibLocal out from mima

[jira] [Commented] (SPARK-14818) Move sketch and mllibLocal out from mima exclusion

[ https://issues.apache.org/jira/browse/SPARK-14818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484691#comment-15484691 ] Apache Spark commented on SPARK-14818: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Commented] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484629#comment-15484629 ] Gang Wu commented on SPARK-17477: - [~hyukjin.kwon] I agree with you. But both issues are targeting at

[jira] [Created] (SPARK-17508) Setting weightCol to None in ML library causes an error

Evan Zamir created SPARK-17508: -- Summary: Setting weightCol to None in ML library causes an error Key: SPARK-17508 URL: https://issues.apache.org/jira/browse/SPARK-17508 Project: Spark Issue

[jira] [Updated] (SPARK-17477) SparkSQL cannot handle schema evolution from Int -> Long when parquet files have Int as its type while hive metastore has Long as its type

[ https://issues.apache.org/jira/browse/SPARK-17477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17477: Target Version/s: (was: 2.1.0) > SparkSQL cannot handle schema evolution from Int -> Long when parquet

[jira] [Commented] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484602#comment-15484602 ] Chris Parmer commented on SPARK-15406: -- For my team, we are just primarily interested in the SQL /

[jira] [Assigned] (SPARK-17507) check weight vector size in ANN

[ https://issues.apache.org/jira/browse/SPARK-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17507: Assignee: (was: Apache Spark) > check weight vector size in ANN >

[jira] [Assigned] (SPARK-17507) check weight vector size in ANN

[ https://issues.apache.org/jira/browse/SPARK-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17507: Assignee: Apache Spark > check weight vector size in ANN >

[jira] [Commented] (SPARK-17507) check weight vector size in ANN

[ https://issues.apache.org/jira/browse/SPARK-17507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484586#comment-15484586 ] Apache Spark commented on SPARK-17507: -- User 'WeichenXu123' has created a pull request for this

[jira] [Created] (SPARK-17507) check weight vector size in ANN

Weichen Xu created SPARK-17507: -- Summary: check weight vector size in ANN Key: SPARK-17507 URL: https://issues.apache.org/jira/browse/SPARK-17507 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4633) Support gzip in spark.compression.io.codec

[ https://issues.apache.org/jira/browse/SPARK-4633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484488#comment-15484488 ] Adam Roberts commented on SPARK-4633: - Very interested in this and I know Nasser Ebrahim is also (full

[jira] [Updated] (SPARK-17506) Improve the check double values equality rule

[ https://issues.apache.org/jira/browse/SPARK-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17506: -- Priority: Minor (was: Critical) > Improve the check double values equality rule >

[jira] [Assigned] (SPARK-17506) Improve the check double values equality rule

[ https://issues.apache.org/jira/browse/SPARK-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17506: Assignee: Apache Spark > Improve the check double values equality rule >

[jira] [Commented] (SPARK-17506) Improve the check double values equality rule

[ https://issues.apache.org/jira/browse/SPARK-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484443#comment-15484443 ] Apache Spark commented on SPARK-17506: -- User 'jiangxb1987' has created a pull request for this

[jira] [Assigned] (SPARK-17506) Improve the check double values equality rule

[ https://issues.apache.org/jira/browse/SPARK-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17506: Assignee: (was: Apache Spark) > Improve the check double values equality rule >

  1   2   >