[jira] [Commented] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560180#comment-14560180 ] Konstantin Shaposhnikov commented on SPARK-7042: It looks like

[jira] [Updated] (SPARK-7104) Support model save/load in Python's Word2Vec

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7104: - Target Version/s: 1.5.0 (was: 1.4.0) Support model save/load in Python's Word2Vec

[jira] [Updated] (SPARK-7605) Python API for ElementwiseProduct

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7605: - Target Version/s: 1.5.0 Assignee: Manoj Kumar Python API for

[jira] [Updated] (SPARK-6263) Python MLlib API missing items: Utils

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6263: - Target Version/s: 1.5.0 (was: 1.4.0) Python MLlib API missing items: Utils

[jira] [Commented] (SPARK-7605) Python API for ElementwiseProduct

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560212#comment-14560212 ] Joseph K. Bradley commented on SPARK-7605: -- Updated for 1.5. I don't think we'll

[jira] [Commented] (SPARK-6263) Python MLlib API missing items: Utils

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560214#comment-14560214 ] Joseph K. Bradley commented on SPARK-6263: -- Updated for 1.5. I don't think we'll

[jira] [Updated] (SPARK-6099) Stabilize mllib ClassificationModel, RegressionModel APIs

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6099: - Target Version/s: 1.5.0 (was: 1.4.0) Stabilize mllib ClassificationModel,

[jira] [Updated] (SPARK-6099) Stabilize mllib ClassificationModel, RegressionModel APIs

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6099: - Target Version/s: 1.4.0 (was: 1.5.0) Stabilize mllib ClassificationModel,

[jira] [Resolved] (SPARK-7883) Fixing broken trainImplicit example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7883. -- Resolution: Fixed Fix Version/s: 1.4.0 1.0.3 1.6.0

[jira] [Updated] (SPARK-7883) Fixing broken trainImplicit example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7883: - Fix Version/s: (was: 1.6.0) Fixing broken trainImplicit example in MLlib Collaborative

[jira] [Commented] (SPARK-7865) Hadoop Filesystem for eventlog closed before sparkContext stopped

2015-05-26 Thread Zhang, Liye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560266#comment-14560266 ] Zhang, Liye commented on SPARK-7865: Thank [~vanzin]'s reply, I'm just thinking about

[jira] [Commented] (SPARK-7555) User guide update for ElasticNet

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560171#comment-14560171 ] Joseph K. Bradley commented on SPARK-7555: -- Thanks! User guide update for

[jira] [Commented] (SPARK-7576) User guide update for spark.ml ElementwiseProduct

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560170#comment-14560170 ] Joseph K. Bradley commented on SPARK-7576: -- Thanks! User guide update for

[jira] [Updated] (SPARK-7856) Scalable PCA implementation for tall and fat matrices

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7856: - Issue Type: Improvement (was: Bug) Scalable PCA implementation for tall and fat

[jira] [Commented] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560181#comment-14560181 ] Konstantin Shaposhnikov commented on SPARK-7042: It looks like

[jira] [Issue Comment Deleted] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-7042: --- Comment: was deleted (was: It looks like akka-zeromq_2.11 is only available

[jira] [Commented] (SPARK-7604) Python API for PCA and PCAModel

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560209#comment-14560209 ] Joseph K. Bradley commented on SPARK-7604: -- Updated for 1.5. I don't think we'll

[jira] [Assigned] (SPARK-7884) Allow Spark shuffle APIs to be more customizable

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7884: --- Assignee: Apache Spark Allow Spark shuffle APIs to be more customizable

[jira] [Assigned] (SPARK-7884) Allow Spark shuffle APIs to be more customizable

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7884: --- Assignee: (was: Apache Spark) Allow Spark shuffle APIs to be more customizable

[jira] [Created] (SPARK-7884) Allow Spark shuffle APIs to be more customizable

2015-05-26 Thread Matt Massie (JIRA)
Matt Massie created SPARK-7884: -- Summary: Allow Spark shuffle APIs to be more customizable Key: SPARK-7884 URL: https://issues.apache.org/jira/browse/SPARK-7884 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7884) Allow Spark shuffle APIs to be more customizable

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560241#comment-14560241 ] Apache Spark commented on SPARK-7884: - User 'massie' has created a pull request for

[jira] [Commented] (SPARK-3674) Add support for launching YARN clusters in spark-ec2

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560282#comment-14560282 ] Apache Spark commented on SPARK-3674: - User 'shivaram' has created a pull request for

[jira] [Updated] (SPARK-7824) Collapsing operator reordering and constant folding into a single batch to push down the single side.

2015-05-26 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei updated SPARK-7824: -- Summary: Collapsing operator reordering and constant folding into a single batch to push down

[jira] [Updated] (SPARK-7852) Use continuation when GLMs are run with multiple regParams

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7852: - Description: Per the discussion from

[jira] [Commented] (SPARK-7880) Silent failure if assembly jar is corrupted

2015-05-26 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560167#comment-14560167 ] Andrew Or commented on SPARK-7880: -- I was testing out RC2 for 1.4 and somehow ended up

[jira] [Updated] (SPARK-7852) Use continuation when GLMs are run with multiple regParams

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7852: - Summary: Use continuation when GLMs are run with multiple regParams (was: Add support

[jira] [Updated] (SPARK-6915) VectorIndexer improvements

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6915: - Target Version/s: (was: 1.4.0) VectorIndexer improvements --

[jira] [Updated] (SPARK-6634) Allow replacing columns in Transformers

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6634: - Target Version/s: 1.5.0 (was: 1.4.0) Allow replacing columns in Transformers

[jira] [Resolved] (SPARK-6295) spark.ml.Evaluator should have evaluate method not taking ParamMap

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6295. -- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Xiangrui Meng (was:

[jira] [Updated] (SPARK-7604) Python API for PCA and PCAModel

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7604: - Target Version/s: 1.5.0 Assignee: Yanbo Liang Python API for PCA and

[jira] [Updated] (SPARK-7461) Remove spark.ml Model, and have all Transformers have parent

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7461: - Target Version/s: 1.5.0 (was: 1.4.0) Remove spark.ml Model, and have all Transformers

[jira] [Updated] (SPARK-6261) Python MLlib API missing items: Feature

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6261: - Target Version/s: (was: 1.4.0) Python MLlib API missing items: Feature

[jira] [Issue Comment Deleted] (SPARK-7577) User guide update for Bucketizer

2015-05-26 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-7577: - Comment: was deleted (was: Oh, yes. Sorry, I forget it. Thanks for the reminder.) User guide update for

[jira] [Commented] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560166#comment-14560166 ] Konstantin Shaposhnikov commented on SPARK-7042: That is not true:

[jira] [Commented] (SPARK-7461) Remove spark.ml Model, and have all Transformers have parent

2015-05-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560205#comment-14560205 ] Joseph K. Bradley commented on SPARK-7461: -- Speaking with [~mengxr], we're going

[jira] [Commented] (SPARK-7699) Number of executors can be reduced from initial before work is scheduled

2015-05-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560227#comment-14560227 ] Sandy Ryza commented on SPARK-7699: --- I think tying this the AM-RM heartbeat would just

[jira] [Assigned] (SPARK-7867) Support revoke role ...

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7867: --- Assignee: (was: Apache Spark) Support revoke role ... -

[jira] [Assigned] (SPARK-7867) Support revoke role ...

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7867: --- Assignee: Apache Spark Support revoke role ... -

[jira] [Updated] (SPARK-7883) Fixing broken trainImplicit example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7883: - Assignee: Mike Dusenberry Fixing broken trainImplicit example in MLlib Collaborative Filtering

[jira] [Updated] (SPARK-7883) Fixing broken trainImplicit example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7883: - Target Version/s: 1.0.3, 1.1.2, 1.2.3, 1.3.2, 1.4.0 Affects Version/s: 1.0.2

[jira] [Resolved] (SPARK-7637) StructType.merge slow with large nenormalised tables O(N2)

2015-05-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7637. - Resolution: Fixed Fix Version/s: 1.5.0 StructType.merge slow with large

[jira] [Commented] (SPARK-7577) User guide update for Bucketizer

2015-05-26 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560278#comment-14560278 ] Xusen Yin commented on SPARK-7577: -- Oh, yes. Sorry, I forget it. Thanks for the reminder.

[jira] [Commented] (SPARK-7577) User guide update for Bucketizer

2015-05-26 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560279#comment-14560279 ] Xusen Yin commented on SPARK-7577: -- Oh, yes. Sorry, I forget it. Thanks for the reminder.

[jira] [Commented] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560283#comment-14560283 ] Konstantin Shaposhnikov commented on SPARK-7042: It looks like the Spark

[jira] [Created] (SPARK-7885) add config to control map aggregation in spark sql

2015-05-26 Thread jeanlyn (JIRA)
jeanlyn created SPARK-7885: -- Summary: add config to control map aggregation in spark sql Key: SPARK-7885 URL: https://issues.apache.org/jira/browse/SPARK-7885 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-7866) print the format string in dataframe explain

2015-05-26 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fei Wang closed SPARK-7866. --- Resolution: Won't Fix Wrong print in intelli idea, this is really ok, not a problem. print the format

[jira] [Assigned] (SPARK-7885) add config to control map aggregation in spark sql

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7885: --- Assignee: (was: Apache Spark) add config to control map aggregation in spark sql

[jira] [Assigned] (SPARK-7885) add config to control map aggregation in spark sql

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7885: --- Assignee: Apache Spark add config to control map aggregation in spark sql

[jira] [Commented] (SPARK-7885) add config to control map aggregation in spark sql

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560344#comment-14560344 ] Apache Spark commented on SPARK-7885: - User 'jeanlyn' has created a pull request for

[jira] [Closed] (SPARK-7734) make explode support struct type

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7734. -- Resolution: Not A Problem make explode support struct type

[jira] [Closed] (SPARK-7742) Figure out what to do with insertInto w.r.t. DataFrameWriter API

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7742. -- Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Yin Huai We decided to add insertInto

[jira] [Updated] (SPARK-7853) ClassNotFoundException for SparkSQL

2015-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7853: -- Description: Reproduce steps: {code} bin/spark-sql --jars

[jira] [Commented] (SPARK-4867) UDF clean up

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560434#comment-14560434 ] Reynold Xin commented on SPARK-4867: That's a good idea. I created SPARK-4867 for

[jira] [Created] (SPARK-7886) Add built-in expressions to FunctionRegistry

2015-05-26 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7886: -- Summary: Add built-in expressions to FunctionRegistry Key: SPARK-7886 URL: https://issues.apache.org/jira/browse/SPARK-7886 Project: Spark Issue Type:

[jira] [Commented] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560444#comment-14560444 ] Reynold Xin commented on SPARK-7550: [~chenghao] can you work on this one? Support

[jira] [Resolved] (SPARK-7858) DataSourceStrategy.createPhysicalRDD should use output schema when performing row conversions, not relation schema

2015-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-7858. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6400

[jira] [Resolved] (SPARK-7868) Ignores _temporary directories while listing files in HadoopFsRelation

2015-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-7868. - Issue resolved by pull request 6411 [https://github.com/apache/spark/pull/6411] Ignores _temporary

[jira] [Resolved] (SPARK-6012) Deadlock when asking for partitions from CoalescedRDD on top of a TakeOrdered operator

2015-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6012. - Resolution: Not A Problem I think we do not have this issue after 1.3. I am going to resolve it as Not A

[jira] [Updated] (SPARK-6012) Deadlock when asking for partitions from CoalescedRDD on top of a TakeOrdered operator

2015-05-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6012: Target Version/s: (was: 1.4.0) Deadlock when asking for partitions from CoalescedRDD on top of a

[jira] [Commented] (SPARK-7853) ClassNotFoundException for SparkSQL

2015-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560429#comment-14560429 ] Cheng Lian commented on SPARK-7853: --- OT: [~chenghao] Just edited the JIRA description.

[jira] [Updated] (SPARK-7886) Add built-in expressions to FunctionRegistry

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7886: --- Target Version/s: 1.5.0 Add built-in expressions to FunctionRegistry

[jira] [Commented] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560455#comment-14560455 ] Reynold Xin commented on SPARK-7550: cc [~yhuai]. I think the proposed design is to

[jira] [Updated] (SPARK-6923) Spark SQL CLI does not read Data Source schema correctly

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6923: --- Target Version/s: 1.5.0 (was: 1.4.0) Spark SQL CLI does not read Data Source schema correctly

[jira] [Assigned] (SPARK-7887) Remove EvaluatedType from SQL Expression

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7887: --- Assignee: Apache Spark (was: Reynold Xin) Remove EvaluatedType from SQL Expression

[jira] [Commented] (SPARK-7887) Remove EvaluatedType from SQL Expression

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560462#comment-14560462 ] Apache Spark commented on SPARK-7887: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-7887) Remove EvaluatedType from SQL Expression

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7887: --- Assignee: Reynold Xin (was: Apache Spark) Remove EvaluatedType from SQL Expression

[jira] [Commented] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

2015-05-26 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560450#comment-14560450 ] Cheng Hao commented on SPARK-7550: -- Similar issue with SPARK-6923 Support setting the

[jira] [Comment Edited] (SPARK-7550) Support setting the right schema serde when writing to Hive metastore

2015-05-26 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560450#comment-14560450 ] Cheng Hao edited comment on SPARK-7550 at 5/27/15 5:48 AM: ---

[jira] [Created] (SPARK-7887) Remove EvaluatedType from SQL Expression

2015-05-26 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-7887: -- Summary: Remove EvaluatedType from SQL Expression Key: SPARK-7887 URL: https://issues.apache.org/jira/browse/SPARK-7887 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-7699) Number of executors can be reduced from initial before work is scheduled

2015-05-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558748#comment-14558748 ] Sandy Ryza commented on SPARK-7699: --- We can't wait only on the initial allocation being

[jira] [Resolved] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7042. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6341

[jira] [Updated] (SPARK-7042) Spark version of akka-actor_2.11 is not compatible with the official akka-actor_2.11 2.3.x

2015-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7042: - Assignee: Konstantin Shaposhnikov Spark version of akka-actor_2.11 is not compatible with the official

[jira] [Commented] (SPARK-7699) Number of executors can be reduced from initial before work is scheduled

2015-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558752#comment-14558752 ] Sean Owen commented on SPARK-7699: -- Hm, waiting for x seconds also seems suboptimal

[jira] [Resolved] (SPARK-7110) when use saveAsNewAPIHadoopFile, sometimes it throws Delegation Token can be issued only with kerberos or web authentication

2015-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-7110. -- Resolution: Duplicate when use saveAsNewAPIHadoopFile, sometimes it throws Delegation Token can be

[jira] [Commented] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558758#comment-14558758 ] Santiago M. Mola commented on SPARK-7727: - [~chenghao] I think that is a good

[jira] [Commented] (SPARK-7699) Number of executors can be reduced from initial before work is scheduled

2015-05-26 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558757#comment-14558757 ] Sandy Ryza commented on SPARK-7699: --- I think delaying releasing them is exactly the

[jira] [Commented] (SPARK-7699) Number of executors can be reduced from initial before work is scheduled

2015-05-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558764#comment-14558764 ] Sean Owen commented on SPARK-7699: -- initialExecutors? I think it's there for the ramp-up

[jira] [Issue Comment Deleted] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-7727: Comment: was deleted (was: [~evacchi] I'm sorry I opened this duplicate for:

[jira] [Resolved] (SPARK-7823) [SQL] Batch, FixedPoint, Strategy should not be inner classes of class RuleExecutor

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola resolved SPARK-7823. - Resolution: Duplicate This is a duplicate of

[jira] [Comment Edited] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Edoardo Vacchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558796#comment-14558796 ] Edoardo Vacchi edited comment on SPARK-7727 at 5/26/15 7:44 AM:

[jira] [Commented] (SPARK-3846) KryoException when doing joins in SparkSQL

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558808#comment-14558808 ] Santiago M. Mola commented on SPARK-3846: - [~huangjs] Would you mind adding a

[jira] [Comment Edited] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Edoardo Vacchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558796#comment-14558796 ] Edoardo Vacchi edited comment on SPARK-7727 at 5/26/15 7:41 AM:

[jira] [Updated] (SPARK-7862) Query would hang when the using script has error output in SparkSQL

2015-05-26 Thread zhichao-li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhichao-li updated SPARK-7862: -- Description: Steps to reproduce: val data = (1 to 10).map { i = (i, i, i) } data.toDF(d1, d2,

[jira] [Comment Edited] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Edoardo Vacchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558796#comment-14558796 ] Edoardo Vacchi edited comment on SPARK-7727 at 5/26/15 7:44 AM:

[jira] [Updated] (SPARK-3846) KryoException when doing joins in SparkSQL

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-3846: Priority: Blocker (was: Major) KryoException when doing joins in SparkSQL

[jira] [Updated] (SPARK-3846) [SQL] Serialization exception (Kryo) on joins when enabling codegen

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-3846: Summary: [SQL] Serialization exception (Kryo) on joins when enabling codegen (was: [SQL]

[jira] [Closed] (SPARK-3846) [SQL] Serialization exception (Kryo) on joins when enabling codegen

2015-05-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-3846. -- Resolution: Duplicate [SQL] Serialization exception (Kryo) on joins when enabling codegen

[jira] [Created] (SPARK-7864) Clicking a job's DAG graph on Web UI kills the job as the link is broken

2015-05-26 Thread Carson Wang (JIRA)
Carson Wang created SPARK-7864: -- Summary: Clicking a job's DAG graph on Web UI kills the job as the link is broken Key: SPARK-7864 URL: https://issues.apache.org/jira/browse/SPARK-7864 Project: Spark

[jira] [Commented] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558777#comment-14558777 ] Santiago M. Mola commented on SPARK-7727: - [~evacchi] I'm sorry I opened this

[jira] [Assigned] (SPARK-7862) Query would hang when the using script has error output in SparkSQL

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7862: --- Assignee: Apache Spark Query would hang when the using script has error output in SparkSQL

[jira] [Commented] (SPARK-7562) Improve error reporting for expression data type mismatch

2015-05-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558836#comment-14558836 ] Wenchen Fan commented on SPARK-7562: After thinking about it, I think we can't just

[jira] [Assigned] (SPARK-7562) Improve error reporting for expression data type mismatch

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7562: --- Assignee: (was: Apache Spark) Improve error reporting for expression data type mismatch

[jira] [Assigned] (SPARK-7863) SimpleDateParam should not use SimpleDateFormat in multiple threads because SimpleDateFormat is not thread-safe

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7863: --- Assignee: (was: Apache Spark) SimpleDateParam should not use SimpleDateFormat in

[jira] [Commented] (SPARK-7863) SimpleDateParam should not use SimpleDateFormat in multiple threads because SimpleDateFormat is not thread-safe

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558841#comment-14558841 ] Apache Spark commented on SPARK-7863: - User 'zsxwing' has created a pull request for

[jira] [Assigned] (SPARK-7862) Query would hang when the using script has error output in SparkSQL

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7862: --- Assignee: (was: Apache Spark) Query would hang when the using script has error output

[jira] [Commented] (SPARK-7862) Query would hang when the using script has error output in SparkSQL

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558797#comment-14558797 ] Apache Spark commented on SPARK-7862: - User 'zhichao-li' has created a pull request

[jira] [Commented] (SPARK-7727) Avoid inner classes in RuleExecutor

2015-05-26 Thread Edoardo Vacchi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558796#comment-14558796 ] Edoardo Vacchi commented on SPARK-7727: --- [~smolav] that's fine, since I opened my PR

[jira] [Commented] (SPARK-5707) Enabling spark.sql.codegen throws ClassNotFound exception

2015-05-26 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14558823#comment-14558823 ] Santiago M. Mola commented on SPARK-5707: - This is probably a duplicate of

[jira] [Created] (SPARK-7863) SimpleDateParam should not use SimpleDateFormat in multiple threads because SimpleDateFormat is not thread-safe

2015-05-26 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-7863: --- Summary: SimpleDateParam should not use SimpleDateFormat in multiple threads because SimpleDateFormat is not thread-safe Key: SPARK-7863 URL:

[jira] [Assigned] (SPARK-7864) Clicking a job's DAG graph on Web UI kills the job as the link is broken

2015-05-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7864: --- Assignee: Apache Spark Clicking a job's DAG graph on Web UI kills the job as the link is

  1   2   3   >