[jira] [Created] (SPARK-2231) dev/run-tests should include YARN and use a recent Hadoop version

2014-06-21 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2231: -- Summary: dev/run-tests should include YARN and use a recent Hadoop version Key: SPARK-2231 URL: https://issues.apache.org/jira/browse/SPARK-2231 Project: Spark

[jira] [Created] (SPARK-2230) Improvements to Jenkins QA Harness

2014-06-21 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2230: -- Summary: Improvements to Jenkins QA Harness Key: SPARK-2230 URL: https://issues.apache.org/jira/browse/SPARK-2230 Project: Spark Issue Type: Umbrella

[jira] [Resolved] (SPARK-1996) Remove use of special Maven repo for Akka

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1996. Resolution: Fixed Fix Version/s: (was: 1.0.1) 1.1.0 Fixed via

[jira] [Updated] (SPARK-1996) Remove use of special Maven repo for Akka

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1996: --- Assignee: Sean Owen > Remove use of special Maven repo for Akka > ---

[jira] [Commented] (SPARK-2222) Add multiclass evaluation metrics

2014-06-21 Thread Jun Xie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040034#comment-14040034 ] Jun Xie commented on SPARK-: Nice work. I am reading the implementation of Multiclass

[jira] [Updated] (SPARK-2229) SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException

2014-06-21 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2229: --- Summary: SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException (was: SizeBasedRollingPo

[jira] [Created] (SPARK-2229) SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException

2014-06-21 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2229: -- Summary: SizeBasedRollingPolicy throw an java.lang.IllegalArgumentException Key: SPARK-2229 URL: https://issues.apache.org/jira/browse/SPARK-2229 Project: Spark

[jira] [Commented] (SPARK-2222) Add multiclass evaluation metrics

2014-06-21 Thread Alexander Ulanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040029#comment-14040029 ] Alexander Ulanov commented on SPARK-: - Hi Jun, I've already implemented this

[jira] [Closed] (SPARK-1698) Improve spark integration

2014-06-21 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li closed SPARK-1698. -- Resolution: Won't Fix > Improve spark integration > - > > Key:

[jira] [Commented] (SPARK-1316) Remove use of Commons IO

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039991#comment-14039991 ] Sean Owen commented on SPARK-1316: -- PR: https://github.com/apache/spark/pull/1173 Actual

[jira] [Commented] (SPARK-2222) Add multiclass evaluation metrics

2014-06-21 Thread Jun Xie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039988#comment-14039988 ] Jun Xie commented on SPARK-: Hi, Alexander. I would like to take up this task. Would y

[jira] [Updated] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will be thrown in onStageCompleted

2014-06-21 Thread Baoxu Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoxu Shi updated SPARK-2228: - Summary: onStageSubmitted does not properly called so NoSuchElement will be thrown in onStageCompleted (

[jira] [Updated] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will throw in onStageCompleted

2014-06-21 Thread Baoxu Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoxu Shi updated SPARK-2228: - Affects Version/s: 1.0.0 > onStageSubmitted does not properly called so NoSuchElement will throw in > on

[jira] [Updated] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will throw in onStageCompleted

2014-06-21 Thread Baoxu Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoxu Shi updated SPARK-2228: - Description: We are using `SaveAsObjectFile` and `objectFile` to cut off lineage during iterative comput

[jira] [Created] (SPARK-2228) onStageSubmitted does not properly called so NoSuchElement will throw in onStageCompleted

2014-06-21 Thread Baoxu Shi (JIRA)
Baoxu Shi created SPARK-2228: Summary: onStageSubmitted does not properly called so NoSuchElement will throw in onStageCompleted Key: SPARK-2228 URL: https://issues.apache.org/jira/browse/SPARK-2228 Proje

[jira] [Commented] (SPARK-1996) Remove use of special Maven repo for Akka

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039965#comment-14039965 ] Sean Owen commented on SPARK-1996: -- PR: https://github.com/apache/spark/pull/1170 > Remo

[jira] [Comment Edited] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039953#comment-14039953 ] Patrick Wendell edited comment on SPARK-1392 at 6/21/14 9:15 PM: ---

[jira] [Resolved] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-1392. Resolution: Duplicate > Local spark-shell Runs Out of Memory With Default Settings > --

[jira] [Commented] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039953#comment-14039953 ] Patrick Wendell commented on SPARK-1392: Okay great, I confirmed this is fixed by

[jira] [Commented] (SPARK-1675) Make clear whether computePrincipalComponents requires centered data

2014-06-21 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039952#comment-14039952 ] Sandy Ryza commented on SPARK-1675: --- I think it still wouldn't hurt to add a remark that

[jira] [Updated] (SPARK-1675) Make clear whether computePrincipalComponents requires centered data

2014-06-21 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1675: -- Priority: Trivial (was: Major) > Make clear whether computePrincipalComponents requires centered data

[jira] [Updated] (SPARK-1675) Make clear whether computePrincipalComponents requires centered data

2014-06-21 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-1675: -- Summary: Make clear whether computePrincipalComponents requires centered data (was: Make clear whether

[jira] [Commented] (SPARK-721) Fix remaining deprecation warnings

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039951#comment-14039951 ] Sean Owen commented on SPARK-721: - This appears to be resolved as I don't think these warni

[jira] [Commented] (SPARK-1803) Rename test resources to be compatible with Windows FS

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039948#comment-14039948 ] Sean Owen commented on SPARK-1803: -- PR was committed so this is another that seems to be

[jira] [Commented] (SPARK-1046) Enable to build behind a proxy.

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039950#comment-14039950 ] Sean Owen commented on SPARK-1046: -- Is this stale / resolved? I don't see this in the cod

[jira] [Commented] (SPARK-1698) Improve spark integration

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039949#comment-14039949 ] Sean Owen commented on SPARK-1698: -- Can this be closed? the child tasks are closed and I

[jira] [Commented] (SPARK-1804) Mark 0.9.1 as released in JIRA

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039947#comment-14039947 ] Sean Owen commented on SPARK-1804: -- Looks like this can be closed as resolved. https://is

[jira] [Commented] (SPARK-1846) RAT checks should exclude logs/ directory

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039945#comment-14039945 ] Sean Owen commented on SPARK-1846: -- Just looking over some old JIRAs. This appears to be

[jira] [Commented] (SPARK-1392) Local spark-shell Runs Out of Memory With Default Settings

2014-06-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039946#comment-14039946 ] Patrick Wendell commented on SPARK-1392: I mentioned this on the pull request, but

[jira] [Commented] (SPARK-1675) Make clear whether computePrincipalComponents centers data

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039944#comment-14039944 ] Sean Owen commented on SPARK-1675: -- Is this still valid? Looking at the code, PCA is comp

[jira] [Commented] (SPARK-1478) Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-1915

2014-06-21 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039937#comment-14039937 ] Ted Malaska commented on SPARK-1478: OK I have made the changes requested. But I had

[jira] [Updated] (SPARK-2220) Fix remaining Hive Commands

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2220: Description: None of the following have an execution plan: {code} private[hive] case class

[jira] [Reopened] (SPARK-2227) Support "dfs" command

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reopened SPARK-2227: - Sorry, i'll reopen this since you already have a PR with just this change. > Support "dfs" c

[jira] [Resolved] (SPARK-2195) Parquet extraMetadata can contain key information

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2195. - Resolution: Fixed Fix Version/s: 1.1.0 1.0.1 > Parquet extraMet

[jira] [Commented] (SPARK-2195) Parquet extraMetadata can contain key information

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039915#comment-14039915 ] Michael Armbrust commented on SPARK-2195: - Yeah, thanks for taking care of this so

[jira] [Resolved] (SPARK-2227) Support "dfs" command

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2227. - Resolution: Duplicate > Support "dfs" command > - > >

[jira] [Updated] (SPARK-1800) Add broadcast hash join operator

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1800: Issue Type: Sub-task (was: Improvement) Parent: SPARK-2211 > Add broadcast hash jo

[jira] [Closed] (SPARK-2214) Broadcast Join (aka map join)

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-2214. --- Resolution: Duplicate > Broadcast Join (aka map join) > - > >

[jira] [Updated] (SPARK-2212) Hash Outer Joins

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2212: Priority: Major (was: Critical) > Hash Outer Joins > > >

[jira] [Updated] (SPARK-2212) Hash Outer Joins

2014-06-21 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2212: Summary: Hash Outer Joins (was: HashJoin) > Hash Outer Joins > > >

[jira] [Commented] (SPARK-1568) Spark 0.9.0 hangs reading s3

2014-06-21 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039854#comment-14039854 ] sam commented on SPARK-1568: When we upgrade to 1.0.0 I'll test this. This particular problem

[jira] [Commented] (SPARK-1138) Spark 0.9.0 does not work with Hadoop / HDFS

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039805#comment-14039805 ] Sean Owen commented on SPARK-1138: -- This is no longer observed in the unit tests. The com

[jira] [Commented] (SPARK-1339) Build error: org.eclipse.paho:mqtt-client

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039804#comment-14039804 ] Sean Owen commented on SPARK-1339: -- (Just cruising some old issues) I can't reproduce thi

[jira] [Commented] (SPARK-2223) Building and running tests with maven is extremely slow

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039802#comment-14039802 ] Sean Owen commented on SPARK-2223: -- On a latest-generation Macbook Pro here, a full 'mvn

[jira] [Commented] (SPARK-1568) Spark 0.9.0 hangs reading s3

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039797#comment-14039797 ] Sean Owen commented on SPARK-1568: -- Sam, did the other recent changes to S3 deps resolve

[jira] [Commented] (SPARK-1759) sbt/sbt package fail cause by directory

2014-06-21 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039792#comment-14039792 ] Sean Owen commented on SPARK-1759: -- As discussed at https://github.com/apache/spark/pull/

[jira] [Comment Edited] (SPARK-704) ConnectionManager sometimes cannot detect loss of sending connections

2014-06-21 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039742#comment-14039742 ] Mridul Muralidharan edited comment on SPARK-704 at 6/21/14 9:10 AM: -

[jira] [Commented] (SPARK-704) ConnectionManager sometimes cannot detect loss of sending connections

2014-06-21 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039742#comment-14039742 ] Mridul Muralidharan commented on SPARK-704: --- If remote node goes down, SendingCon