[jira] [Commented] (SPARK-1464) Update MLLib Examples to Use Breeze

[ https://issues.apache.org/jira/browse/SPARK-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972342#comment-13972342 ] Sean Owen commented on SPARK-1464: -- This is a duplicate of

[jira] [Created] (SPARK-1520) Spark assembly fails with Java 6

Patrick Wendell created SPARK-1520: -- Summary: Spark assembly fails with Java 6 Key: SPARK-1520 URL: https://issues.apache.org/jira/browse/SPARK-1520 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1520) Spark assembly fails with Java 6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Spark assembly fails with JRE6 for unknown reason

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Summary: Spark assembly fails with JRE6 for unknown reason (was: Spark assembly fails with

[jira] [Updated] (SPARK-1520) Spark assembly fails with Java 6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Spark assembly fails with JRE6 for unknown reason

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Component/s: MLlib Spark assembly fails with JRE6 for unknown reason

[jira] [Updated] (SPARK-1520) Spark assembly fails with JRE6 for unknown reason

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Component/s: Spark Core Spark assembly fails with JRE6 for unknown reason

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Summary: Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Created] (SPARK-1521) Take character set size into account when compressing in-memory string columns

Cheng Lian created SPARK-1521: - Summary: Take character set size into account when compressing in-memory string columns Key: SPARK-1521 URL: https://issues.apache.org/jira/browse/SPARK-1521 Project:

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Commented] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972397#comment-13972397 ] Sean Owen commented on SPARK-1520: -- Madness. One wild guess is that the breeze .jar files

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Commented] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972523#comment-13972523 ] Sean Owen commented on SPARK-1520: -- Regarding large numbers of files: are there INDEX.LST

[jira] [Created] (SPARK-1522) YARN ClientBase will throw a NPE if there is no YARN application specific classpath.

Bernardo Gomez Palacio created SPARK-1522: - Summary: YARN ClientBase will throw a NPE if there is no YARN application specific classpath. Key: SPARK-1522 URL:

[jira] [Commented] (SPARK-1522) YARN ClientBase will throw a NPE if there is no YARN application specific classpath.

[ https://issues.apache.org/jira/browse/SPARK-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972803#comment-13972803 ] Bernardo Gomez Palacio commented on SPARK-1522: ---

[jira] [Created] (SPARK-1524) TaskSetManager'd better not schedule tasks which has no preferred executorId using PROCESS_LOCAL in the first search process

YanTang Zhai created SPARK-1524: --- Summary: TaskSetManager'd better not schedule tasks which has no preferred executorId using PROCESS_LOCAL in the first search process Key: SPARK-1524 URL:

[jira] [Commented] (SPARK-1524) TaskSetManager'd better not schedule tasks which has no preferred executorId using PROCESS_LOCAL in the first search process

[ https://issues.apache.org/jira/browse/SPARK-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972864#comment-13972864 ] Mridul Muralidharan commented on SPARK-1524: The expectation is to fallback to

[jira] [Commented] (SPARK-1525) TaskSchedulerImpl should decrease availableCpus by spark.task.cpus not 1

[ https://issues.apache.org/jira/browse/SPARK-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972957#comment-13972957 ] witgo commented on SPARK-1525: -- The latest code already fix the bug. TaskSchedulerImpl

[jira] [Closed] (SPARK-1511) Update TestUtils.createCompiledClass() API to work with creating class file on different filesystem

[ https://issues.apache.org/jira/browse/SPARK-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Xianjin closed SPARK-1511. - Resolution: Fixed Fix Version/s: 1.0.0 Update TestUtils.createCompiledClass() API to work with

[jira] [Commented] (SPARK-1476) 2GB limit in spark for blocks

[ https://issues.apache.org/jira/browse/SPARK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972978#comment-13972978 ] Mridul Muralidharan commented on SPARK-1476: [~matei] We are having some

[jira] [Created] (SPARK-1526) Running spark driver program from my local machine

Idan Zalzberg created SPARK-1526: Summary: Running spark driver program from my local machine Key: SPARK-1526 URL: https://issues.apache.org/jira/browse/SPARK-1526 Project: Spark Issue Type:

[jira] [Created] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

Ye Xianjin created SPARK-1527: - Summary: rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1 Key: SPARK-1527 URL: https://issues.apache.org/jira/browse/SPARK-1527 Project:

[jira] [Commented] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

[ https://issues.apache.org/jira/browse/SPARK-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973067#comment-13973067 ] Sean Owen commented on SPARK-1527: -- {{toString()}} returns {{getPath()}} which may still

[jira] [Commented] (SPARK-1518) Spark master doesn't compile against hadoop-common trunk

[ https://issues.apache.org/jira/browse/SPARK-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973084#comment-13973084 ] witgo commented on SPARK-1518: -- As the hadoop API changes, some methods have been removed.

[jira] [Commented] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

[ https://issues.apache.org/jira/browse/SPARK-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973087#comment-13973087 ] Ye Xianjin commented on SPARK-1527: --- Yes. You are right. toString() may give relative

[jira] [Commented] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

[ https://issues.apache.org/jira/browse/SPARK-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973091#comment-13973091 ] Sean Owen commented on SPARK-1527: -- If the paths are only used locally, then an absolute

[jira] [Created] (SPARK-1528) Spark on Yarn: Add option for user to specify additional namenodes to get tokens from

Thomas Graves created SPARK-1528: Summary: Spark on Yarn: Add option for user to specify additional namenodes to get tokens from Key: SPARK-1528 URL: https://issues.apache.org/jira/browse/SPARK-1528

[jira] [Commented] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

[ https://issues.apache.org/jira/browse/SPARK-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973096#comment-13973096 ] Ye Xianjin commented on SPARK-1527: --- Yes, of course, sometimes we want absolute path,

[jira] [Commented] (SPARK-1527) rootDirs in DiskBlockManagerSuite doesn't get full path from rootDir0, rootDir1

[ https://issues.apache.org/jira/browse/SPARK-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973148#comment-13973148 ] Sean Owen commented on SPARK-1527: -- There are a number of other uses of File.getName(),

[jira] [Commented] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973291#comment-13973291 ] Xiangrui Meng commented on SPARK-1520: -- I'm using Java 6 JDK located at

[jira] [Commented] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973306#comment-13973306 ] Xiangrui Meng commented on SPARK-1520: -- When I try to use jar-1.6 to untar the

[jira] [Comment Edited] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973326#comment-13973326 ] Xiangrui Meng edited comment on SPARK-1520 at 4/17/14 7:59 PM:

[jira] [Commented] (SPARK-1520) Inclusion of breeze corrupts assembly when compiled with JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973326#comment-13973326 ] Xiangrui Meng commented on SPARK-1520: -- The quick fix may be removing fastutil. In

[jira] [Updated] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Summary: Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run

[jira] [Updated] (SPARK-1520) Assembly Jar with more than JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Summary: Assembly Jar with more than JDK7 and run on JDK6 (was: Inclusion of breeze

[jira] [Created] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

Patrick Wendell created SPARK-1529: -- Summary: Support setting spark.local.dirs to a hadoop FileSystem Key: SPARK-1529 URL: https://issues.apache.org/jira/browse/SPARK-1529 Project: Spark

[jira] [Updated] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Updated] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1520: --- Description: This is a real doozie - when compiling a Spark assembly with JDK7, the produced

[jira] [Commented] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973463#comment-13973463 ] Xiangrui Meng commented on SPARK-1520: -- It seems HyperLogLog doesn't need fastutil,

[jira] [Resolved] (SPARK-1464) Update MLLib Examples to Use Breeze

[ https://issues.apache.org/jira/browse/SPARK-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1464. -- Resolution: Duplicate Update MLLib Examples to Use Breeze

[jira] [Updated] (SPARK-1496) SparkContext.jarOfClass should return Option instead of a sequence

[ https://issues.apache.org/jira/browse/SPARK-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1496: --- Labels: release-notes (was: releasenotes) SparkContext.jarOfClass should return Option

[jira] [Updated] (SPARK-1496) SparkContext.jarOfClass should return Option instead of a sequence

[ https://issues.apache.org/jira/browse/SPARK-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1496: --- Labels: releasenotes (was: ) SparkContext.jarOfClass should return Option instead of a

[jira] [Updated] (SPARK-1496) SparkContext.jarOfClass should return Option instead of a sequence

[ https://issues.apache.org/jira/browse/SPARK-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1496: --- Labels: api-change (was: release-notes) SparkContext.jarOfClass should return Option

[jira] [Updated] (SPARK-964) Investigate the potential for using JDK 8 lambda expressions for the Java/Scala APIs

[ https://issues.apache.org/jira/browse/SPARK-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-964: -- Labels: api-change (was: ) Investigate the potential for using JDK 8 lambda expressions for

[jira] [Created] (SPARK-1530) Streaming UI test can hang indefinitely

Patrick Wendell created SPARK-1530: -- Summary: Streaming UI test can hang indefinitely Key: SPARK-1530 URL: https://issues.apache.org/jira/browse/SPARK-1530 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-1473) Feature selection for high dimensional datasets

[ https://issues.apache.org/jira/browse/SPARK-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973631#comment-13973631 ] Ignacio Zendejas commented on SPARK-1473: - Thanks for pointing this out Martin.

[jira] [Commented] (SPARK-1530) Streaming UI test can hang indefinitely

[ https://issues.apache.org/jira/browse/SPARK-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13973737#comment-13973737 ] Tathagata Das commented on SPARK-1530: -- I wonder what about Jenkins environment is

[jira] [Assigned] (SPARK-1520) Assembly Jar with more than 65536 files won't work when compiled on JDK7 and run on JDK6

[ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-1520: Assignee: Xiangrui Meng Assembly Jar with more than 65536 files won't work when compiled

[jira] [Created] (SPARK-1531) GraphX should have messageRDD to enable OutOfCore messages

Jianfeng created SPARK-1531: --- Summary: GraphX should have messageRDD to enable OutOfCore messages Key: SPARK-1531 URL: https://issues.apache.org/jira/browse/SPARK-1531 Project: Spark Issue Type: