[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

2014-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058552#comment-14058552 ] Sean Owen commented on SPARK-2398: -- [~gq] This does not really have to do with allocating

[jira] [Created] (SPARK-2445) MesosExecutorBackend crashes in fine grained mode

2014-07-11 Thread Dario Rexin (JIRA)
Dario Rexin created SPARK-2445: -- Summary: MesosExecutorBackend crashes in fine grained mode Key: SPARK-2445 URL: https://issues.apache.org/jira/browse/SPARK-2445 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2446) Add BinaryType support to Parquet I/O.

2014-07-11 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2446: Summary: Add BinaryType support to Parquet I/O. Key: SPARK-2446 URL: https://issues.apache.org/jira/browse/SPARK-2446 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-2446) Add BinaryType support to Parquet I/O.

2014-07-11 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058584#comment-14058584 ] Takuya Ueshin commented on SPARK-2446: -- PRed: https://github.com/apache/spark/pull/13

[jira] [Created] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2014-07-11 Thread Ted Malaska (JIRA)
Ted Malaska created SPARK-2447: -- Summary: Add common solution for sending upsert actions to HBase (put, deletes, and increment) Key: SPARK-2447 URL: https://issues.apache.org/jira/browse/SPARK-2447 Proje

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

2014-07-11 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058645#comment-14058645 ] Ted Malaska commented on SPARK-2447: tdas when you have time can you assign to me. Th

[jira] [Commented] (SPARK-1642) Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-2083

2014-07-11 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058648#comment-14058648 ] Ted Malaska commented on SPARK-1642: Hey Tdas. Yes now that SPARK-1478 is done I will

[jira] [Commented] (SPARK-2420) Change Spark build to minimize library conflicts

2014-07-11 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058757#comment-14058757 ] Xuefu Zhang commented on SPARK-2420: [~rxin] Is it possible to downgrade or shade guav

[jira] [Created] (SPARK-2448) Table name is not getting applied to their attributes after "registerAsTable"

2014-07-11 Thread Jerry Lam (JIRA)
Jerry Lam created SPARK-2448: Summary: Table name is not getting applied to their attributes after "registerAsTable" Key: SPARK-2448 URL: https://issues.apache.org/jira/browse/SPARK-2448 Project: Spark

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058881#comment-14058881 ] Thomas Graves commented on SPARK-2444: -- Can you be more specific, what are you lookin

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058909#comment-14058909 ] Sean Owen commented on SPARK-2444: -- In SPARK-2398, YARN was allocating 32.5GB for a 32GB

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058930#comment-14058930 ] Thomas Graves commented on SPARK-2444: -- Maybe I'm missing something in what you are a

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058943#comment-14058943 ] Sean Owen commented on SPARK-2444: -- That's all right. I'm mostly saying I expected a smal

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058987#comment-14058987 ] Marcelo Vanzin commented on SPARK-2444: --- As Tom mentions, the setting is documented.

[jira] [Created] (SPARK-2449) Spark sql reflection code requires a constructor taking all the fields for the table

2014-07-11 Thread Ian O Connell (JIRA)
Ian O Connell created SPARK-2449: Summary: Spark sql reflection code requires a constructor taking all the fields for the table Key: SPARK-2449 URL: https://issues.apache.org/jira/browse/SPARK-2449 Pr

[jira] [Created] (SPARK-2451) Enable to load config file for Akka

2014-07-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2451: - Summary: Enable to load config file for Akka Key: SPARK-2451 URL: https://issues.apache.org/jira/browse/SPARK-2451 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

2014-07-11 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059147#comment-14059147 ] Nishkam Ravi commented on SPARK-2444: - [~tgraves] Thanks for the pointer. I guess I wa

[jira] [Created] (SPARK-2450) Provide link to YARN executor logs on UI

2014-07-11 Thread Bill Havanki (JIRA)
Bill Havanki created SPARK-2450: --- Summary: Provide link to YARN executor logs on UI Key: SPARK-2450 URL: https://issues.apache.org/jira/browse/SPARK-2450 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-2437) Rename MAVEN_PROFILES to SBT_MAVEN_PROFILES and add SBT_MAVEN_PROPERTIES

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2437. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 1374 [https://

[jira] [Created] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Timothy Hunter (JIRA)
Timothy Hunter created SPARK-2452: - Summary: Multi-statement input to spark repl does not work Key: SPARK-2452 URL: https://issues.apache.org/jira/browse/SPARK-2452 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059225#comment-14059225 ] Patrick Wendell commented on SPARK-2452: This might be related to SPARK-1199 (/cc

[jira] [Created] (SPARK-2453) Compound lines in spark-shell cause compilation errors

2014-07-11 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-2453: - Summary: Compound lines in spark-shell cause compilation errors Key: SPARK-2453 URL: https://issues.apache.org/jira/browse/SPARK-2453 Project: Spark Issue

[jira] [Commented] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059267#comment-14059267 ] Patrick Wendell commented on SPARK-2452: I think we should narrow the description

[jira] [Updated] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2452: --- Assignee: Prashant Sharma > Multi-statement input to spark repl does not work > -

[jira] [Updated] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2452: --- Priority: Blocker (was: Major) > Multi-statement input to spark repl does not work > ---

[jira] [Updated] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2452: --- Target Version/s: 1.0.2 > Multi-statement input to spark repl does not work > ---

[jira] [Closed] (SPARK-2453) Compound lines in spark-shell cause compilation errors

2014-07-11 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson closed SPARK-2453. - Resolution: Duplicate https://issues.apache.org/jira/browse/SPARK-2452 > Compound lines in spark

[jira] [Updated] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2452: --- Affects Version/s: (was: 1.0.0) 1.0.1 > Multi-statement input to s

[jira] [Created] (SPARK-2454) Separate driver spark home from executor spark home

2014-07-11 Thread Andrew Or (JIRA)
Andrew Or created SPARK-2454: Summary: Separate driver spark home from executor spark home Key: SPARK-2454 URL: https://issues.apache.org/jira/browse/SPARK-2454 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2440) Enable HistoryServer to display lots of Application History

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2440: -- Fix Version/s: (was: 1.0.0) > Enable HistoryServer to display lots of Application History >

[jira] [Updated] (SPARK-2440) Enable HistoryServer to display lots of Application History

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2440: -- Affects Version/s: 1.0.0 > Enable HistoryServer to display lots of Application History > --

[jira] [Commented] (SPARK-2440) Enable HistoryServer to display lots of Application History

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059393#comment-14059393 ] Kousuke Saruta commented on SPARK-2440: --- I hit this issue using 1.0.0 but the code o

[jira] [Updated] (SPARK-2452) Multi-statement input to spark repl does not work

2014-07-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2452: --- Description: Here is an example: {code} scala> val x = 4 ; def f() = x x: Int = 4 f: ()Int

[jira] [Created] (SPARK-2455) VertexPartition is not serializable

2014-07-11 Thread Ankur Dave (JIRA)
Ankur Dave created SPARK-2455: - Summary: VertexPartition is not serializable Key: SPARK-2455 URL: https://issues.apache.org/jira/browse/SPARK-2455 Project: Spark Issue Type: Bug Compone

[jira] [Commented] (SPARK-2347) Graph object can not be set to StorageLevel.MEMORY_ONLY_SER

2014-07-11 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059419#comment-14059419 ] Ankur Dave commented on SPARK-2347: --- VertexPartition is actually supposed to be Serializ

[jira] [Commented] (SPARK-2455) VertexPartition is not serializable

2014-07-11 Thread Ankur Dave (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059436#comment-14059436 ] Ankur Dave commented on SPARK-2455: --- Proposed fix: https://github.com/apache/spark/pull/

[jira] [Created] (SPARK-2456) Scheduler refactoring

2014-07-11 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-2456: -- Summary: Scheduler refactoring Key: SPARK-2456 URL: https://issues.apache.org/jira/browse/SPARK-2456 Project: Spark Issue Type: Improvement Reporter:

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

2014-07-11 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059489#comment-14059489 ] Nishkam Ravi commented on SPARK-2398: - [~gq] [~sowen] I don't specify XmX, I'm only r

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

2014-07-11 Thread Nishkam Ravi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059494#comment-14059494 ] Nishkam Ravi commented on SPARK-2398: - In this case, it should be Spark taking care of

[jira] [Closed] (SPARK-2413) Upgrade junit_xml_listener to 0.5.1

2014-07-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai closed SPARK-2413. -- > Upgrade junit_xml_listener to 0.5.1 > --- > > Key: SPARK-2413 >

[jira] [Updated] (SPARK-2456) Scheduler refactoring

2014-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2456: --- Target Version/s: 1.2.0 > Scheduler refactoring > - > > Key: SPAR

[jira] [Commented] (SPARK-911) Support map pruning on sorted (K, V) RDD's

2014-07-11 Thread Aaron (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059548#comment-14059548 ] Aaron commented on SPARK-911: - pull request https://github.com/apache/spark/pull/1381/files >

[jira] [Created] (SPARK-2457) Inconsistent description in README about build option

2014-07-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2457: - Summary: Inconsistent description in README about build option Key: SPARK-2457 URL: https://issues.apache.org/jira/browse/SPARK-2457 Project: Spark Issue T

[jira] [Commented] (SPARK-2457) Inconsistent description in README about build option

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059571#comment-14059571 ] Kousuke Saruta commented on SPARK-2457: --- PR: https://github.com/apache/spark/pull/13

[jira] [Commented] (SPARK-1649) Figure out Nullability semantics for Array elements and Map values

2014-07-11 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059570#comment-14059570 ] Yin Huai commented on SPARK-1649: - My PR for SPARK-2179 (https://github.com/apache/spark/p

[jira] [Commented] (SPARK-1642) Upgrade FlumeInputDStream's FlumeReceiver to support FLUME-2083

2014-07-11 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059594#comment-14059594 ] Ted Malaska commented on SPARK-1642: OK I got it working and all the test passed. I'l

[jira] [Created] (SPARK-2458) Make failed application log visible on History Server

2014-07-11 Thread Masayoshi TSUZUKI (JIRA)
Masayoshi TSUZUKI created SPARK-2458: Summary: Make failed application log visible on History Server Key: SPARK-2458 URL: https://issues.apache.org/jira/browse/SPARK-2458 Project: Spark I

[jira] [Commented] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059606#comment-14059606 ] Kousuke Saruta commented on SPARK-1667: --- PR: https://github.com/apache/spark/pull/13

[jira] [Resolved] (SPARK-2457) Inconsistent description in README about build option

2014-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-2457. Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Kousuke Saruta > Inconsistent desc

[jira] [Created] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-11 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-2459: -- Summary: the user should be able to configure the resources used by JDBC server Key: SPARK-2459 URL: https://issues.apache.org/jira/browse/SPARK-2459 Project: Spark Iss

[jira] [Updated] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-11 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2459: --- Description: I'm trying the jdbc server I found that the jdbc server always occupies all cores in the cluste

[jira] [Commented] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059635#comment-14059635 ] Reynold Xin commented on SPARK-2459: Can you submit a pull request to fix this? > the

[jira] [Commented] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-11 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059636#comment-14059636 ] Nan Zhu commented on SPARK-2459: I discussed with [~liancheng], he is working on merging t

[jira] [Comment Edited] (SPARK-2459) the user should be able to configure the resources used by JDBC server

2014-07-11 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059636#comment-14059636 ] Nan Zhu edited comment on SPARK-2459 at 7/12/14 4:35 AM: - I discus

[jira] [Commented] (SPARK-2298) Show stage attempt in UI

2014-07-11 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059644#comment-14059644 ] Masayoshi TSUZUKI commented on SPARK-2298: -- PRed: https://github.com/apache/spark

[jira] [Resolved] (SPARK-1969) Public available online summarizer for mean, variance, min, and max

2014-07-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-1969. -- Resolution: Done Fix Version/s: 1.1.0 PR: https://github.com/apache/spark/pull/955 > Pu

[jira] [Updated] (SPARK-1969) Public available online summarizer for mean, variance, min, and max

2014-07-11 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-1969: - Assignee: DB Tsai > Public available online summarizer for mean, variance, min, and max > ---