[jira] [Created] (SPARK-4842) Use WeakTypeTags in ScalaReflection

2014-12-14 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-4842: --- Summary: Use WeakTypeTags in ScalaReflection Key: SPARK-4842 URL: https://issues.apache.org/jira/browse/SPARK-4842 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1406) PMML model evaluation support via MLib

2014-12-14 Thread Vincenzo Selvaggio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincenzo Selvaggio updated SPARK-1406: -- Attachment: SPARK-1406_v2.pdf Updated document with model supported so far:

[jira] [Commented] (SPARK-1406) PMML model evaluation support via MLib

2014-12-14 Thread Vincenzo Selvaggio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245894#comment-14245894 ] Vincenzo Selvaggio commented on SPARK-1406: --- Scala examples on usage of

[jira] [Closed] (SPARK-3640) KinesisUtils should accept a credentials object instead of forcing DefaultCredentialsProvider

2014-12-14 Thread Aniket Bhatnagar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Bhatnagar closed SPARK-3640. --- Resolution: Not a Problem Tested and Chris's suggestion of using EC2 IAM instance profile

[jira] [Commented] (SPARK-4841) Batch serializer bug in PySpark's RDD.zip

2014-12-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245996#comment-14245996 ] Xiangrui Meng commented on SPARK-4841: -- This is the commit that caused the bug:

[jira] [Commented] (SPARK-4838) StackOverflowError when serialization task

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246023#comment-14246023 ] Michael Armbrust commented on SPARK-4838: - Yeah, any more detail you can provide

[jira] [Updated] (SPARK-4782) Add inferSchema support for RDD[Map[String, Any]]

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4782: Target Version/s: 1.3.0 Add inferSchema support for RDD[Map[String, Any]]

[jira] [Updated] (SPARK-4782) Add inferSchema support for RDD[Map[String, Any]]

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4782: Affects Version/s: (was: 1.3.0) Add inferSchema support for RDD[Map[String, Any]]

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246024#comment-14246024 ] Michael Armbrust commented on SPARK-4814: - Either way, I don't think we should

[jira] [Updated] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4814: Target Version/s: 1.3.0 Enable assertions in SBT, Maven tests / AssertionError from Hive's

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246041#comment-14246041 ] Hari Shreedharan commented on SPARK-4826: - It looks like there is some issue with

[jira] [Updated] (SPARK-4684) Add a script to run JDBC server on Windows

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4684: Target Version/s: 1.3.0 Add a script to run JDBC server on Windows

[jira] [Commented] (SPARK-4775) Possible problem in a simple join? Getting duplicate rows and missing rows

2014-12-14 Thread Stephen Boesch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246084#comment-14246084 ] Stephen Boesch commented on SPARK-4775: --- Thanks v much Michael. You hit the nail on

[jira] [Closed] (SPARK-4775) Possible problem in a simple join? Getting duplicate rows and missing rows

2014-12-14 Thread Stephen Boesch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Boesch closed SPARK-4775. - Resolution: Not a Problem Possible problem in a simple join? Getting duplicate rows and missing

[jira] [Updated] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

2014-12-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4812: Assignee: Shixiong Zhu SparkPlan.codegenEnabled may be initialized to a wrong value

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246091#comment-14246091 ] Nicholas Chammas commented on SPARK-4826: - This raises an interesting test

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246122#comment-14246122 ] Nicholas Chammas commented on SPARK-4826: - I just cooked up a quick way of

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2014-12-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246135#comment-14246135 ] Apache Spark commented on SPARK-4814: - User 'srowen' has created a pull request for

[jira] [Created] (SPARK-4843) Squash ExecutorRunnable and ExecutorRunnableUtil hierarchy in yarn module

2014-12-14 Thread Kostas Sakellis (JIRA)
Kostas Sakellis created SPARK-4843: -- Summary: Squash ExecutorRunnable and ExecutorRunnableUtil hierarchy in yarn module Key: SPARK-4843 URL: https://issues.apache.org/jira/browse/SPARK-4843 Project:

[jira] [Created] (SPARK-4844) SGD should support custom sampling.

2014-12-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-4844: -- Summary: SGD should support custom sampling. Key: SPARK-4844 URL: https://issues.apache.org/jira/browse/SPARK-4844 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4826) Possible flaky tests in WriteAheadLogBackedBlockRDDSuite: java.lang.IllegalStateException: File exists and there is no append support!

2014-12-14 Thread Hari Shreedharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246270#comment-14246270 ] Hari Shreedharan commented on SPARK-4826: - I suspect that the nextString is

[jira] [Created] (SPARK-4845) Adding a parallelismRatio to control the partitions num of shuffledRDD

2014-12-14 Thread wangfei (JIRA)
wangfei created SPARK-4845: -- Summary: Adding a parallelismRatio to control the partitions num of shuffledRDD Key: SPARK-4845 URL: https://issues.apache.org/jira/browse/SPARK-4845 Project: Spark

[jira] [Commented] (SPARK-4845) Adding a parallelismRatio to control the partitions num of shuffledRDD

2014-12-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246286#comment-14246286 ] Apache Spark commented on SPARK-4845: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-4843) Squash ExecutorRunnable and ExecutorRunnableUtil hierarchy in yarn module

2014-12-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246318#comment-14246318 ] Apache Spark commented on SPARK-4843: - User 'ksakellis' has created a pull request for

[jira] [Created] (SPARK-4846) When the vocabulary size is large, Word2Vec may yield OutOfMemoryError: Requested array size exceeds VM limit

2014-12-14 Thread Joseph Tang (JIRA)
Joseph Tang created SPARK-4846: -- Summary: When the vocabulary size is large, Word2Vec may yield OutOfMemoryError: Requested array size exceeds VM limit Key: SPARK-4846 URL:

[jira] [Closed] (SPARK-2604) Spark Application hangs on yarn in edge case scenario of executor memory requirement

2014-12-14 Thread Twinkle Sachdeva (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Twinkle Sachdeva closed SPARK-2604. --- Resolution: Fixed Spark Application hangs on yarn in edge case scenario of executor memory

[jira] [Commented] (SPARK-4846) When the vocabulary size is large, Word2Vec may yield OutOfMemoryError: Requested array size exceeds VM limit

2014-12-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246337#comment-14246337 ] Apache Spark commented on SPARK-4846: - User 'jinntrance' has created a pull request

[jira] [Updated] (SPARK-4843) Squash ExecutorRunnable and ExecutorRunnableUtil hierarchy in yarn module

2014-12-14 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-4843: -- Assignee: Kostas Sakellis Squash ExecutorRunnable and ExecutorRunnableUtil hierarchy in yarn module

[jira] [Created] (SPARK-4847) extraStrategies cannot take effect in SQLContext

2014-12-14 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-4847: -- Summary: extraStrategies cannot take effect in SQLContext Key: SPARK-4847 URL: https://issues.apache.org/jira/browse/SPARK-4847 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4812) SparkPlan.codegenEnabled may be initialized to a wrong value

2014-12-14 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-4812: Description: The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`, which can

[jira] [Commented] (SPARK-4847) extraStrategies cannot take effect in SQLContext

2014-12-14 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246380#comment-14246380 ] Apache Spark commented on SPARK-4847: - User 'jerryshao' has created a pull request for

[jira] [Created] (SPARK-4848) On a stand-alone cluster, several worker-specific variables are read only on the master

2014-12-14 Thread Nathan Kronenfeld (JIRA)
Nathan Kronenfeld created SPARK-4848: Summary: On a stand-alone cluster, several worker-specific variables are read only on the master Key: SPARK-4848 URL: https://issues.apache.org/jira/browse/SPARK-4848