[jira] [Updated] (SPARK-2469) Lower shuffle compression buffer memory usage

2014-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2469: --- Summary: Lower shuffle compression buffer memory usage (was: Lower shuffle compression memory

[jira] [Commented] (SPARK-2410) Thrift/JDBC Server

2014-07-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060388#comment-14060388 ] Cheng Lian commented on SPARK-2410: --- PR: https://github.com/apache/spark/pull/1399

[jira] [Updated] (SPARK-2469) Lower shuffle compression buffer memory usage

2014-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2469: --- Description: I was looking into the memory usage of shuffle and one annoying thing is the default

[jira] [Created] (SPARK-2470) Fix PEP 8 violations

2014-07-14 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2470: -- Summary: Fix PEP 8 violations Key: SPARK-2470 URL: https://issues.apache.org/jira/browse/SPARK-2470 Project: Spark Issue Type: Improvement Reporter:

[jira] [Updated] (SPARK-2471) Dependency issues after having sbt read from pom

2014-07-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2471: - Attachment: diff.list diff of the assembly jar content before () and after () PR #772

[jira] [Commented] (SPARK-1556) jets3t dep doesn't update properly with newer Hadoop versions

2014-07-14 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060443#comment-14060443 ] Xiangrui Meng commented on SPARK-1556: -- [~srowen] I saw you set jets3t's scope to

[jira] [Created] (SPARK-2472) Spark SQL Thrift server sometimes assigns wrong job group name

2014-07-14 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-2472: - Summary: Spark SQL Thrift server sometimes assigns wrong job group name Key: SPARK-2472 URL: https://issues.apache.org/jira/browse/SPARK-2472 Project: Spark

[jira] [Commented] (SPARK-1556) jets3t dep doesn't update properly with newer Hadoop versions

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060462#comment-14060462 ] Sean Owen commented on SPARK-1556: -- Yeah see PR comments -- runtime is correct because

[jira] [Commented] (SPARK-2136) Spark SQL does not disply the job description on web ui/ event log

2014-07-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060466#comment-14060466 ] Cheng Lian commented on SPARK-2136: --- The Thrift server has already supported this.

[jira] [Commented] (SPARK-2465) Use long as user / item ID for ALS

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060474#comment-14060474 ] Sean Owen commented on SPARK-2465: -- Yeah, more data is a cost for sure. I had though it

[jira] [Commented] (SPARK-2382) build error:

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060476#comment-14060476 ] Sean Owen commented on SPARK-2382: -- I don't think the build can work around a lack of

[jira] [Commented] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060498#comment-14060498 ] Sean Owen commented on SPARK-2278: -- (Oh right, sortBy is not in 1.0.0, it came after:

[jira] [Commented] (SPARK-2354) BitSet Range Expanded when creating new one

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060521#comment-14060521 ] Sean Owen commented on SPARK-2354: -- Yes, you could not just naively flip all the bits to

[jira] [Commented] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060543#comment-14060543 ] Mridul Muralidharan commented on SPARK-2468: We map the file content and

[jira] [Commented] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060545#comment-14060545 ] Mridul Muralidharan commented on SPARK-2468: Writing mmap'ed buffers are

[jira] [Comment Edited] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Hans Uhlig (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060668#comment-14060668 ] Hans Uhlig edited comment on SPARK-2278 at 7/14/14 2:19 PM: So

[jira] [Comment Edited] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Hans Uhlig (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060668#comment-14060668 ] Hans Uhlig edited comment on SPARK-2278 at 7/14/14 2:14 PM: So

[jira] [Comment Edited] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Hans Uhlig (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060668#comment-14060668 ] Hans Uhlig edited comment on SPARK-2278 at 7/14/14 2:22 PM: So

[jira] [Commented] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060677#comment-14060677 ] Sean Owen commented on SPARK-2278: -- Where are keys replicated? I don't get that point.

[jira] [Commented] (SPARK-2473) Direct graph-database interaction/connection

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060679#comment-14060679 ] Sean Owen commented on SPARK-2473: -- (If you're asking a question, I don't think a JIRA is

[jira] [Commented] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Hans Uhlig (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060691#comment-14060691 ] Hans Uhlig commented on SPARK-2278: --- I think I am not communicating this very well.

[jira] [Closed] (SPARK-2473) Direct graph-database interaction/connection

2014-07-14 Thread David Deisadze (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Deisadze closed SPARK-2473. - Resolution: Later I asked the spark mailing list community:

[jira] [Commented] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060718#comment-14060718 ] Sean Owen commented on SPARK-2278: -- groupBy / groupByKey vs collectBy / collectByKey?

[jira] [Commented] (SPARK-2154) Worker goes down.

2014-07-14 Thread siva venkat gogineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060728#comment-14060728 ] siva venkat gogineni commented on SPARK-2154: - It looks it has been fixed in

[jira] [Updated] (SPARK-2154) Worker goes down.

2014-07-14 Thread siva venkat gogineni (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] siva venkat gogineni updated SPARK-2154: Attachment: Sccreenhot at various states of driver ..jpg Worker goes down.

[jira] [Commented] (SPARK-546) Support full outer join and multiple join in a single shuffle

2014-07-14 Thread Aaron (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060763#comment-14060763 ] Aaron commented on SPARK-546: - I created a PR for a full outer join implementation here:

[jira] [Commented] (SPARK-2278) groupBy groupByKey should support custom comparator

2014-07-14 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060774#comment-14060774 ] Sean Owen commented on SPARK-2278: -- The more direct parallel certainly also exists, if

[jira] [Commented] (SPARK-2443) Reading from Partitioned Tables is Slow

2014-07-14 Thread Jerry Lam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060790#comment-14060790 ] Jerry Lam commented on SPARK-2443: -- I wonder if this fix can be easily merged into the

[jira] [Commented] (SPARK-2154) Worker goes down.

2014-07-14 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060813#comment-14060813 ] Aaron Davidson commented on SPARK-2154: --- Created this PR to hopefully fix that:

[jira] [Created] (SPARK-2474) In some cases, the Analyzer failed to resolve a table registered in OverrideCatalog

2014-07-14 Thread Yin Huai (JIRA)
Yin Huai created SPARK-2474: --- Summary: In some cases, the Analyzer failed to resolve a table registered in OverrideCatalog Key: SPARK-2474 URL: https://issues.apache.org/jira/browse/SPARK-2474 Project:

[jira] [Commented] (SPARK-2474) In some cases, the Analyzer failed to resolve a table registered in OverrideCatalog

2014-07-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060828#comment-14060828 ] Yin Huai commented on SPARK-2474: - I think the problem is the lookupRelation in

[jira] [Updated] (SPARK-2474) For a registered table in OverrideCatalog, the Analyzer failed to resolve references in the format of tableName.fieldName

2014-07-14 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2474: Summary: For a registered table in OverrideCatalog, the Analyzer failed to resolve references in the

[jira] [Comment Edited] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060937#comment-14060937 ] Reynold Xin edited comment on SPARK-2468 at 7/14/14 5:55 PM: -

[jira] [Commented] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060937#comment-14060937 ] Reynold Xin commented on SPARK-2468: We do use mmap for large blocks. However, most of

[jira] [Comment Edited] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060937#comment-14060937 ] Reynold Xin edited comment on SPARK-2468 at 7/14/14 5:57 PM: -

[jira] [Commented] (SPARK-1981) Add AWS Kinesis streaming support

2014-07-14 Thread Chris Fregly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061007#comment-14061007 ] Chris Fregly commented on SPARK-1981: - quick update: i completed all code, examples,

[jira] [Commented] (SPARK-2468) zero-copy shuffle network communication

2014-07-14 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061094#comment-14061094 ] Mridul Muralidharan commented on SPARK-2468: Ah, small files - those are

[jira] [Commented] (SPARK-1215) Clustering: Index out of bounds error

2014-07-14 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061095#comment-14061095 ] Joseph K. Bradley commented on SPARK-1215: -- Submitted fix as PR 1407:

[jira] [Updated] (SPARK-2449) Spark sql reflection code requires a constructor taking all the fields for the table

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2449: Target Version/s: 1.1.0 Spark sql reflection code requires a constructor taking all the

[jira] [Created] (SPARK-2475) Check whether #cores #receivers in local mode

2014-07-14 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-2475: Summary: Check whether #cores #receivers in local mode Key: SPARK-2475 URL: https://issues.apache.org/jira/browse/SPARK-2475 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2446) Add BinaryType support to Parquet I/O.

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2446: Target Version/s: 1.1.0 Add BinaryType support to Parquet I/O.

[jira] [Updated] (SPARK-2419) Misc updates to streaming programming guide

2014-07-14 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2419: - Description: This JIRA collects together a number of small issues that should be added to the

[jira] [Resolved] (SPARK-2443) Reading from Partitioned Tables is Slow

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-2443. - Resolution: Fixed Fix Version/s: 1.1.0 Reading from Partitioned Tables is Slow

[jira] [Updated] (SPARK-2443) Reading from Partitioned Tables is Slow

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2443: Fix Version/s: 1.0.2 Reading from Partitioned Tables is Slow

[jira] [Updated] (SPARK-2476) Have sbt-assembly include runtime dependencies in jar

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2476: --- Environment: (was: If possible, we should try to contribute the ability to include

[jira] [Created] (SPARK-2476) Have sbt-assembly include runtime dependencies in jar

2014-07-14 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2476: -- Summary: Have sbt-assembly include runtime dependencies in jar Key: SPARK-2476 URL: https://issues.apache.org/jira/browse/SPARK-2476 Project: Spark

[jira] [Updated] (SPARK-2476) Have sbt-assembly include runtime dependencies in jar

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2476: --- Description: If possible, we should try to contribute the ability to include runtime-scoped

[jira] [Resolved] (SPARK-1946) Submit stage after executors have been registered

2014-07-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1946. -- Resolution: Fixed Fix Version/s: 1.1.0 Submit stage after executors have been

[jira] [Resolved] (SPARK-1453) Improve the way Spark on Yarn waits for executors before starting

2014-07-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1453. -- Resolution: Duplicate duplicate of https://issues.apache.org/jira/browse/SPARK-1946 Improve

[jira] [Commented] (SPARK-2475) Check whether #cores #receivers in local mode

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061194#comment-14061194 ] Patrick Wendell commented on SPARK-2475: Good call! I've hit this one a few times

[jira] [Commented] (SPARK-2406) Partitioned Parquet Support

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061232#comment-14061232 ] Michael Armbrust commented on SPARK-2406: - I think there are two ways we can

[jira] [Created] (SPARK-2477) Using appendBias for adding intercept in GeneralizedLinearAlgorithm

2014-07-14 Thread DB Tsai (JIRA)
DB Tsai created SPARK-2477: -- Summary: Using appendBias for adding intercept in GeneralizedLinearAlgorithm Key: SPARK-2477 URL: https://issues.apache.org/jira/browse/SPARK-2477 Project: Spark Issue

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

2014-07-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Fix Version/s: (was: 1.0.0) Passing of JAVA_OPTS to YARN on command line

[jira] [Updated] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line

2014-07-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-1576: - Affects Version/s: (was: 1.0.0) Passing of JAVA_OPTS to YARN on command line

[jira] [Created] (SPARK-2478) Add Python APIs for decision tree

2014-07-14 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-2478: Summary: Add Python APIs for decision tree Key: SPARK-2478 URL: https://issues.apache.org/jira/browse/SPARK-2478 Project: Spark Issue Type: New Feature

Subscribe

2014-07-14 Thread Mubarak Seyed

[jira] [Updated] (SPARK-2479) Comparing floating-point numbers using relative error in UnitTests

2014-07-14 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-2479: --- Description: Due to rounding errors, most floating-point numbers end up being slightly imprecise. As long

[jira] [Commented] (SPARK-2446) Add BinaryType support to Parquet I/O.

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061382#comment-14061382 ] Michael Armbrust commented on SPARK-2446: - Note that this commit changes the

[jira] [Commented] (SPARK-1866) Closure cleaner does not null shadowed fields when outer scope is referenced

2014-07-14 Thread Kan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061501#comment-14061501 ] Kan Zhang commented on SPARK-1866: -- My previous comment may be less readable, let me try

[jira] [Updated] (SPARK-2419) Misc updates to streaming programming guide

2014-07-14 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-2419: - Description: This JIRA collects together a number of small issues that should be added to the

[jira] [Created] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2480: -- Summary: Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag Key: SPARK-2480 URL: https://issues.apache.org/jira/browse/SPARK-2480 Project: Spark

[jira] [Created] (SPARK-2481) The environment variables SPARK_HISTORY_OPTS is covered in spark-env.sh

2014-07-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2481: -- Summary: The environment variables SPARK_HISTORY_OPTS is covered in spark-env.sh Key: SPARK-2481 URL: https://issues.apache.org/jira/browse/SPARK-2481 Project: Spark

[jira] [Created] (SPARK-2482) Resolve sbt warnings during build

2014-07-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2482: -- Summary: Resolve sbt warnings during build Key: SPARK-2482 URL: https://issues.apache.org/jira/browse/SPARK-2482 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2458) Make failed application log visible on History Server

2014-07-14 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061536#comment-14061536 ] Masayoshi TSUZUKI commented on SPARK-2458: -- History server uses FsHistoryProvider

[jira] [Created] (SPARK-2483) HiveQL parses accessing struct fields in an array incorrectly.

2014-07-14 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2483: --- Summary: HiveQL parses accessing struct fields in an array incorrectly. Key: SPARK-2483 URL: https://issues.apache.org/jira/browse/SPARK-2483 Project: Spark

[jira] [Created] (SPARK-2484) By default does not run hive compatibility tests

2014-07-14 Thread Guoqiang Li (JIRA)
Guoqiang Li created SPARK-2484: -- Summary: By default does not run hive compatibility tests Key: SPARK-2484 URL: https://issues.apache.org/jira/browse/SPARK-2484 Project: Spark Issue Type:

[jira] [Created] (SPARK-2485) Usage of HiveClient not threadsafe.

2014-07-14 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-2485: --- Summary: Usage of HiveClient not threadsafe. Key: SPARK-2485 URL: https://issues.apache.org/jira/browse/SPARK-2485 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

2014-07-14 Thread wangfei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061620#comment-14061620 ] wangfei commented on SPARK-2243: Spark Job Server may new multiple SparkContext in one jvm

[jira] [Created] (SPARK-2486) Utils.getCallSite can crash under JVMTI profilers

2014-07-14 Thread William Benton (JIRA)
William Benton created SPARK-2486: - Summary: Utils.getCallSite can crash under JVMTI profilers Key: SPARK-2486 URL: https://issues.apache.org/jira/browse/SPARK-2486 Project: Spark Issue

[jira] [Commented] (SPARK-2486) Utils.getCallSite can crash under JVMTI profilers

2014-07-14 Thread William Benton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061625#comment-14061625 ] William Benton commented on SPARK-2486: --- A (trivial but functional) workaround is

[jira] [Updated] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2459: Target Version/s: 1.1.0 Affects Version/s: (was: 1.1.0)

[jira] [Commented] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-14 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061688#comment-14061688 ] Michael Armbrust commented on SPARK-2459: - Is the correct thing to do here to just

[jira] [Commented] (SPARK-2243) Support multiple SparkContexts in the same JVM

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061698#comment-14061698 ] Patrick Wendell commented on SPARK-2243: I think the Spark JobServer shares a

[jira] [Updated] (SPARK-2477) Using appendBias for adding intercept in GeneralizedLinearAlgorithm

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2477: --- Component/s: MLlib Using appendBias for adding intercept in GeneralizedLinearAlgorithm

[jira] [Resolved] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2480. Resolution: Not a Problem This is left there intentionally to guide users... why do you

[jira] [Updated] (SPARK-2467) Revert SparkBuild to publish-local to both .m2 and .ivy2.

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2467: --- Component/s: Build Revert SparkBuild to publish-local to both .m2 and .ivy2.

[jira] [Updated] (SPARK-2470) Fix PEP 8 violations

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2470: --- Component/s: PySpark Fix PEP 8 violations Key:

[jira] [Commented] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061707#comment-14061707 ] Guoqiang Li commented on SPARK-2480: I'm sorry, my description is not correct . I mean

[jira] [Comment Edited] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061707#comment-14061707 ] Guoqiang Li edited comment on SPARK-2480 at 7/15/14 5:50 AM: -

[jira] [Reopened] (SPARK-2480) Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li reopened SPARK-2480: Remove NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

[jira] [Updated] (SPARK-2480) Resolve sbt warnings NOTE: SPARK_YARN is deprecated, please use -Pyarn flag

2014-07-14 Thread Guoqiang Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guoqiang Li updated SPARK-2480: --- Summary: Resolve sbt warnings NOTE: SPARK_YARN is deprecated, please use -Pyarn flag (was: Remove

[jira] [Updated] (SPARK-2471) Dependency issues after having sbt read from pom

2014-07-14 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2471: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-2487 Dependency issues after having

[jira] [Created] (SPARK-2487) Follow up from SBT build refactor

2014-07-14 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-2487: -- Summary: Follow up from SBT build refactor Key: SPARK-2487 URL: https://issues.apache.org/jira/browse/SPARK-2487 Project: Spark Issue Type: Bug