[jira] [Commented] (SPARK-975) Spark Replay Debugger

2014-04-28 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983603#comment-13983603 ] Kousuke Saruta commented on SPARK-975: -- Hi [~lian cheng]. Are there any updates on

[jira] [Created] (SPARK-1660) Centralize the definition of property names and default values

2014-04-28 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-1660: - Summary: Centralize the definition of property names and default values Key: SPARK-1660 URL: https://issues.apache.org/jira/browse/SPARK-1660 Project: Spark

[jira] [Commented] (SPARK-1667) Should re-fetch when intermediate data for shuffle is lost

2014-04-29 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984900#comment-13984900 ] Kousuke Saruta commented on SPARK-1667: --- Now I'm trying to address this issue.

[jira] [Commented] (SPARK-1361) DAGScheduler throws NullPointerException occasionally

2014-04-30 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13986174#comment-13986174 ] Kousuke Saruta commented on SPARK-1361: --- Hi [~scwf], are there any update for this

[jira] [Updated] (SPARK-1667) re-fetch fails occasionally

2014-04-30 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-1667: -- Summary: re-fetch fails occasionally (was: Should re-fetch when intermediate data for shuffle

[jira] [Commented] (SPARK-1048) Create spark-site.xml or spark-site.yaml for configuration

2014-05-06 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991356#comment-13991356 ] Kousuke Saruta commented on SPARK-1048: --- Any update on this issue? Is someone trying

[jira] [Created] (SPARK-1742) Profiler for Spark

2014-05-06 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-1742: - Summary: Profiler for Spark Key: SPARK-1742 URL: https://issues.apache.org/jira/browse/SPARK-1742 Project: Spark Issue Type: Wish Reporter:

[jira] [Commented] (SPARK-1765) Modify a typo in monitoring.md

2014-05-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13992992#comment-13992992 ] Kousuke Saruta commented on SPARK-1765: --- I've pull-requested for PR-698. Modify a

[jira] [Created] (SPARK-1765) Modify a typo in monitoring.md

2014-05-14 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-1765: - Summary: Modify a typo in monitoring.md Key: SPARK-1765 URL: https://issues.apache.org/jira/browse/SPARK-1765 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2390: - Summary: Files in staging directory cannot be deleted and wastes the space of HDFS Key: SPARK-2390 URL: https://issues.apache.org/jira/browse/SPARK-2390 Project:

[jira] [Updated] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2390: -- Affects Version/s: 1.0.1 Files in staging directory cannot be deleted and wastes the space of

[jira] [Commented] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054041#comment-14054041 ] Kousuke Saruta commented on SPARK-2390: --- I modified the way to create a instance of

[jira] [Commented] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054298#comment-14054298 ] Kousuke Saruta commented on SPARK-2390: --- Thank you for your comment [~mridulm80]. I

[jira] [Commented] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054333#comment-14054333 ] Kousuke Saruta commented on SPARK-2390: --- Pull Requested at

[jira] [Comment Edited] (SPARK-2390) Files in staging directory cannot be deleted and wastes the space of HDFS

2014-07-07 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054333#comment-14054333 ] Kousuke Saruta edited comment on SPARK-2390 at 7/8/14 12:41 AM:

[jira] [Updated] (SPARK-2390) Files in .sparkStaging on HDFS cannot be deleted and wastes the space of HDFS

2014-07-10 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2390: -- Summary: Files in .sparkStaging on HDFS cannot be deleted and wastes the space of HDFS (was:

[jira] [Created] (SPARK-2440) Enable HistoryServer to display lots of Application History

2014-07-10 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2440: - Summary: Enable HistoryServer to display lots of Application History Key: SPARK-2440 URL: https://issues.apache.org/jira/browse/SPARK-2440 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

2014-07-10 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-1667: -- Comment: was deleted (was: Now I'm trying to address this issue.) Jobs never finish

[jira] [Updated] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

2014-07-10 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-1667: -- Description: If jobs execute shuffle, bucket files are created in a temporary directory

[jira] [Updated] (SPARK-2440) Enable HistoryServer to display lots of Application History

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2440: -- Fix Version/s: (was: 1.0.0) Enable HistoryServer to display lots of Application History

[jira] [Created] (SPARK-2457) Inconsistent description in README about build option

2014-07-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2457: - Summary: Inconsistent description in README about build option Key: SPARK-2457 URL: https://issues.apache.org/jira/browse/SPARK-2457 Project: Spark Issue

[jira] [Commented] (SPARK-2457) Inconsistent description in README about build option

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059571#comment-14059571 ] Kousuke Saruta commented on SPARK-2457: --- PR:

[jira] [Commented] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

2014-07-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059606#comment-14059606 ] Kousuke Saruta commented on SPARK-1667: --- PR:

[jira] [Commented] (SPARK-1667) Jobs never finish successfully once bucket file missing occurred

2014-07-15 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062774#comment-14062774 ] Kousuke Saruta commented on SPARK-1667: --- Can anyone review the patch or have

[jira] [Created] (SPARK-2539) ConnectionManager should handle Uncaught Exception

2014-07-16 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2539: - Summary: ConnectionManager should handle Uncaught Exception Key: SPARK-2539 URL: https://issues.apache.org/jira/browse/SPARK-2539 Project: Spark Issue

[jira] [Created] (SPARK-2542) Exit Code Class should be renamed and placed package properly

2014-07-16 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2542: - Summary: Exit Code Class should be renamed and placed package properly Key: SPARK-2542 URL: https://issues.apache.org/jira/browse/SPARK-2542 Project: Spark

[jira] [Commented] (SPARK-2542) Exit Code Class should be renamed and placed package properly

2014-07-17 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065528#comment-14065528 ] Kousuke Saruta commented on SPARK-2542: --- PR:

[jira] [Commented] (SPARK-2491) When an OOM is thrown,the executor does not stop properly.

2014-07-17 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065757#comment-14065757 ] Kousuke Saruta commented on SPARK-2491: --- Hi [~gq] I found the issue related to you

[jira] [Commented] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

2014-07-18 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067309#comment-14067309 ] Kousuke Saruta commented on SPARK-2583: --- PR:

[jira] [Commented] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

2014-07-21 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068814#comment-14068814 ] Kousuke Saruta commented on SPARK-2583: --- Hi [~pwendell], When I simulate disk fault

[jira] [Commented] (SPARK-2583) ConnectionManager cannot distinguish whether error occurred or not

2014-07-24 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073419#comment-14073419 ] Kousuke Saruta commented on SPARK-2583: --- I have added some test cases to my PR for

[jira] [Created] (SPARK-2670) FetchFailedException should be thrown when local fetch has failed

2014-07-24 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2670: - Summary: FetchFailedException should be thrown when local fetch has failed Key: SPARK-2670 URL: https://issues.apache.org/jira/browse/SPARK-2670 Project: Spark

[jira] [Created] (SPARK-2671) BlockObjectWriter should create parent directory when the directory doesn't exist

2014-07-24 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2671: - Summary: BlockObjectWriter should create parent directory when the directory doesn't exist Key: SPARK-2671 URL: https://issues.apache.org/jira/browse/SPARK-2671

[jira] [Created] (SPARK-2673) Improve Spark so that we can attach Debugger to Executors easily

2014-07-24 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2673: - Summary: Improve Spark so that we can attach Debugger to Executors easily Key: SPARK-2673 URL: https://issues.apache.org/jira/browse/SPARK-2673 Project: Spark

[jira] [Created] (SPARK-2677) BasicBlockFetchIterator#next can be wait forever

2014-07-24 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2677: - Summary: BasicBlockFetchIterator#next can be wait forever Key: SPARK-2677 URL: https://issues.apache.org/jira/browse/SPARK-2677 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

2014-07-24 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2677: -- Summary: BasicBlockFetchIterator#next can wait forever (was: BasicBlockFetchIterator#next can

[jira] [Commented] (SPARK-2681) Spark can hang when fetching shuffle blocks

2014-07-25 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075230#comment-14075230 ] Kousuke Saruta commented on SPARK-2681: --- Hey, I reported similar issue.

[jira] [Created] (SPARK-2896) Remove redundant error notification by results.put() when fetching from remote

2014-08-06 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2896: - Summary: Remove redundant error notification by results.put() when fetching from remote Key: SPARK-2896 URL: https://issues.apache.org/jira/browse/SPARK-2896

[jira] [Commented] (SPARK-2677) BasicBlockFetchIterator#next can wait forever

2014-08-10 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092355#comment-14092355 ] Kousuke Saruta commented on SPARK-2677: --- SPARK-2538 was resolved but there is still

[jira] [Created] (SPARK-2963) There no documentation for building about SparkSQL

2014-08-10 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2963: - Summary: There no documentation for building about SparkSQL Key: SPARK-2963 URL: https://issues.apache.org/jira/browse/SPARK-2963 Project: Spark Issue

[jira] [Updated] (SPARK-2963) There no documentation for building about SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use ThriftServer or CLI for SparkSQL, we need to use

[jira] [Updated] (SPARK-2963) There no documentation about building ThriftServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: There no documentation about building ThriftServer and CLI for SparkSQL (was: There

[jira] [Updated] (SPARK-2963) There no documentation about building to use HiveServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: There no documentation about building to use HiveServer and CLI for SparkSQL (was:

[jira] [Updated] (SPARK-2963) There no documentation about building to use HiveServer and CLI for SparkSQL

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use HiveServer or CLI for SparkSQL, we need to use

[jira] [Created] (SPARK-2964) Wrong silent option in spark-sql script

2014-08-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2964: - Summary: Wrong silent option in spark-sql script Key: SPARK-2964 URL: https://issues.apache.org/jira/browse/SPARK-2964 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092705#comment-14092705 ] Kousuke Saruta commented on SPARK-2970: --- I noticed it's not caused by the reason

[jira] [Updated] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2970: -- Description: When spark-sql script run with spark.eventLog.enabled set true, it ends with

[jira] [Commented] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092879#comment-14092879 ] Kousuke Saruta commented on SPARK-2970: --- [~liancheng] Thank you pointing my mistake.

[jira] [Commented] (SPARK-2963) The description about building to use HiveServer and CLI is imcomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092894#comment-14092894 ] Kousuke Saruta commented on SPARK-2963: --- I've updated this title and Github's one.

[jira] [Updated] (SPARK-2963) The description about building to use HiveServer and CLI is imcomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Description: Currently, if we'd like to use HiveServer or CLI for SparkSQL, we need to use

[jira] [Updated] (SPARK-2963) The description about building to use HiveServer and CLI is incomplete

2014-08-11 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: The description about building to use HiveServer and CLI is incomplete (was: The

[jira] [Created] (SPARK-2976) There are too many tabs in some source files

2014-08-11 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-2976: - Summary: There are too many tabs in some source files Key: SPARK-2976 URL: https://issues.apache.org/jira/browse/SPARK-2976 Project: Spark Issue Type:

[jira] [Updated] (SPARK-2964) Improve spark-sql and start-thriftserver to leverage bin/util.sh

2014-08-16 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Description: Now, we have bin/utils.sh, so let's improve spark-sql and start-thriftserver.sh

[jira] [Updated] (SPARK-2964) Improve spark-sql and start-thriftserver to leverage bin/util.sh

2014-08-16 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Summary: Improve spark-sql and start-thriftserver to leverage bin/util.sh (was: Wrong silent

[jira] [Created] (SPARK-3089) Make error message in ConnectionManager more meaningful

2014-08-17 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3089: - Summary: Make error message in ConnectionManager more meaningful Key: SPARK-3089 URL: https://issues.apache.org/jira/browse/SPARK-3089 Project: Spark

[jira] [Created] (SPARK-3090) Add shutdown hook to stop SparkContext for YARN Client mode

2014-08-17 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3090: - Summary: Add shutdown hook to stop SparkContext for YARN Client mode Key: SPARK-3090 URL: https://issues.apache.org/jira/browse/SPARK-3090 Project: Spark

[jira] [Created] (SPARK-3099) Add a shutdown hook for cleanup staging directory to ExecutorLauncher

2014-08-18 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3099: - Summary: Add a shutdown hook for cleanup staging directory to ExecutorLauncher Key: SPARK-3099 URL: https://issues.apache.org/jira/browse/SPARK-3099 Project: Spark

[jira] [Created] (SPARK-3101) Flag variable in ApplicationMaster should be declared as volatile

2014-08-18 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3101: - Summary: Flag variable in ApplicationMaster should be declared as volatile Key: SPARK-3101 URL: https://issues.apache.org/jira/browse/SPARK-3101 Project: Spark

[jira] [Created] (SPARK-3106) Suppress unwilling Exception and error messages caused by SendingConnection

2014-08-18 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3106: - Summary: Suppress unwilling Exception and error messages caused by SendingConnection Key: SPARK-3106 URL: https://issues.apache.org/jira/browse/SPARK-3106 Project:

[jira] [Updated] (SPARK-2964) Fix wrong option (-S, --silent), and improve spark-sql and start-thriftserver to leverage bin/util.sh

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Summary: Fix wrong option (-S, --silent), and improve spark-sql and start-thriftserver to

[jira] [Updated] (SPARK-2964) Fix wrong option (-S, --silent), and improve spark-sql and start-thriftserver to leverage bin/util.sh

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Description: In spark-sql script, they expect -s option but it's wrong. It's typo for -S

[jira] [Updated] (SPARK-3106) *Race Condition Issue* Fix the order of resources in Connection

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3106: -- Summary: *Race Condition Issue* Fix the order of resources in Connection (was: Suppress

[jira] [Updated] (SPARK-3106) *Race Condition Issue* Fix the order of resources in Connection

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3106: -- Description: Now, when we run Spark application, error message is appear on driver's log. The

[jira] [Updated] (SPARK-3106) *Race Condition Issue* Fix the order of closing resources when Connection is closed

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3106: -- Description: Now, when we run Spark application, error message is appear on driver's log. The

[jira] [Updated] (SPARK-3106) *Race Condition Issue* Fix the order of closing resources when Connection is closed

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3106: -- Summary: *Race Condition Issue* Fix the order of closing resources when Connection is closed

[jira] [Updated] (SPARK-3099) Staging Directory is never deleed when we run job with YARN Client Mode

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3099: -- Summary: Staging Directory is never deleed when we run job with YARN Client Mode (was: Add a

[jira] [Updated] (SPARK-3101) Missing volatile annotation in ApplicationMaster

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3101: -- Summary: Missing volatile annotation in ApplicationMaster (was: Flag variable in

[jira] [Updated] (SPARK-3099) Staging Directory is never deleted when we run job with YARN Client Mode

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3099: -- Summary: Staging Directory is never deleted when we run job with YARN Client Mode (was:

[jira] [Updated] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3090: -- Summary: Avoid not stopping SparkContext with YARN Client mode (was: Add shutdown hook to

[jira] [Updated] (SPARK-3089) Fix meaningless error message in ConnectionManager

2014-08-19 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3089: -- Summary: Fix meaningless error message in ConnectionManager (was: Make error message in

[jira] [Created] (SPARK-3149) Connection establishment information is not enough.

2014-08-20 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3149: - Summary: Connection establishment information is not enough. Key: SPARK-3149 URL: https://issues.apache.org/jira/browse/SPARK-3149 Project: Spark Issue

[jira] [Updated] (SPARK-2964) Fix wrong option (-S, --silent), and improve spark-sql and start-thriftserver to leverage bin/util.sh

2014-08-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Priority: Major (was: Minor) Fix wrong option (-S, --silent), and improve spark-sql and

[jira] [Updated] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2970: -- Priority: Critical (was: Major) spark-sql script ends with IOException when EventLogging is

[jira] [Updated] (SPARK-2976) Too many ugly tabs instead of white spaces

2014-08-20 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2976: -- Summary: Too many ugly tabs instead of white spaces (was: There are too many tabs in some

[jira] [Created] (SPARK-3171) Don't print meaningless information of SelectionKey

2014-08-20 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3171: - Summary: Don't print meaningless information of SelectionKey Key: SPARK-3171 URL: https://issues.apache.org/jira/browse/SPARK-3171 Project: Spark Issue

[jira] [Reopened] (SPARK-2963) The description about building to use HiveServer and CLI is incomplete

2014-08-21 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reopened SPARK-2963: --- The description about building to use HiveServer and CLI is incomplete

[jira] [Updated] (SPARK-2963) The description about how to build for using CLI and Thrift JDBC server is absent in proper document

2014-08-21 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2963: -- Summary: The description about how to build for using CLI and Thrift JDBC server is absent in

[jira] [Commented] (SPARK-3106) Fix the race condition issue about Connection and ConnectionManager

2014-08-23 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14107939#comment-14107939 ] Kousuke Saruta commented on SPARK-3106: --- I think one of the reason of this issue is

[jira] [Updated] (SPARK-2964) Remove duplicated code from spark-sql and start-thriftserver.sh

2014-08-23 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-2964: -- Summary: Remove duplicated code from spark-sql and start-thriftserver.sh (was: Fix wrong

[jira] [Created] (SPARK-3192) Some scripts have 2 space indentation but other scripts have 4 space indentation.

2014-08-23 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3192: - Summary: Some scripts have 2 space indentation but other scripts have 4 space indentation. Key: SPARK-3192 URL: https://issues.apache.org/jira/browse/SPARK-3192

[jira] [Created] (SPARK-3229) spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented

2014-08-26 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3229: - Summary: spark.shuffle.safetyFraction and spark.storage.safetyFraction is not documented Key: SPARK-3229 URL: https://issues.apache.org/jira/browse/SPARK-3229

[jira] [Created] (SPARK-3232) Backport SPARK-3006 into branch-1.0

2014-08-26 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3232: - Summary: Backport SPARK-3006 into branch-1.0 Key: SPARK-3232 URL: https://issues.apache.org/jira/browse/SPARK-3232 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3233) Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.

2014-08-26 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3233: - Summary: Executor never stop its SparnEnv, BlockManager, ConnectionManager etc. Key: SPARK-3233 URL: https://issues.apache.org/jira/browse/SPARK-3233 Project:

[jira] [Updated] (SPARK-3233) Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.

2014-08-26 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3233: -- Description: Executor never stop its SparnEnv. Because of this, following component in

[jira] [Created] (SPARK-3279) Remove useless field variable in ApplicationMaster

2014-08-28 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3279: - Summary: Remove useless field variable in ApplicationMaster Key: SPARK-3279 URL: https://issues.apache.org/jira/browse/SPARK-3279 Project: Spark Issue

[jira] [Created] (SPARK-3304) ApplicationMaster's Finish status is wrong when uncaught exception is thrown from ReporterThread

2014-08-29 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3304: - Summary: ApplicationMaster's Finish status is wrong when uncaught exception is thrown from ReporterThread Key: SPARK-3304 URL: https://issues.apache.org/jira/browse/SPARK-3304

[jira] [Created] (SPARK-3305) Remove unused import from UI classes.

2014-08-29 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3305: - Summary: Remove unused import from UI classes. Key: SPARK-3305 URL: https://issues.apache.org/jira/browse/SPARK-3305 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3090) Avoid not stopping SparkContext with YARN Client mode

2014-08-31 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117036#comment-14117036 ] Kousuke Saruta commented on SPARK-3090: --- It sound good idea that SparkContext

[jira] [Updated] (SPARK-3372) Scalastyle fails due to UTF-8 character contained in Gradient.scala

2014-09-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3372: -- Summary: Scalastyle fails due to UTF-8 character contained in Gradient.scala (was: Scalastyle

[jira] [Created] (SPARK-3377) codahale base Metrics data between applications can jumble up together

2014-09-03 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3377: - Summary: codahale base Metrics data between applications can jumble up together Key: SPARK-3377 URL: https://issues.apache.org/jira/browse/SPARK-3377 Project:

[jira] [Updated] (SPARK-3377) codahale base Metrics data between applications can jumble up together

2014-09-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3377: -- Description: I'm using codahale base MetricsSystem of Spark with JMX or Graphite, and I saw

[jira] [Commented] (SPARK-3304) ApplicationMaster's Finish status is wrong when uncaught exception is thrown from ReporterThread

2014-09-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14120030#comment-14120030 ] Kousuke Saruta commented on SPARK-3304: --- Actually, I didn't see exception causing

[jira] [Created] (SPARK-3378) Replace the word SparkSQL with right word Spark SQL

2014-09-03 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3378: - Summary: Replace the word SparkSQL with right word Spark SQL Key: SPARK-3378 URL: https://issues.apache.org/jira/browse/SPARK-3378 Project: Spark Issue

[jira] [Updated] (SPARK-3372) MLlib doesn't pass maven build / checkstyle due to multi-byte character contained in Gradient.scala

2014-09-03 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3372: -- Summary: MLlib doesn't pass maven build / checkstyle due to multi-byte character contained in

[jira] [Created] (SPARK-3399) Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

2014-09-04 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3399: - Summary: Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR Key: SPARK-3399 URL: https://issues.apache.org/jira/browse/SPARK-3399 Project: Spark

[jira] [Created] (SPARK-3401) Wrong usage of tee command in python/run-tests

2014-09-04 Thread Kousuke Saruta (JIRA)
Kousuke Saruta created SPARK-3401: - Summary: Wrong usage of tee command in python/run-tests Key: SPARK-3401 URL: https://issues.apache.org/jira/browse/SPARK-3401 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3377) Don't mix metrics from different applications

2014-09-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3377: -- Summary: Don't mix metrics from different applications (was: codahale base Metrics data

[jira] [Updated] (SPARK-3410) The priority of shutdownhook for ApplicationMaster should not be integer literal, rather than refer constant.

2014-09-04 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3410: -- Issue Type: Improvement (was: Bug) The priority of shutdownhook for ApplicationMaster should

[jira] [Updated] (SPARK-3410) The priority of shutdownhook for ApplicationMaster should not be integer literal

2014-09-05 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3410: -- Summary: The priority of shutdownhook for ApplicationMaster should not be integer literal

[jira] [Updated] (SPARK-3377) Don't mix metrics from different applications otherwise we cannot distinguish

2014-09-05 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-3377: -- Summary: Don't mix metrics from different applications otherwise we cannot distinguish (was:

  1   2   3   4   5   6   7   8   9   10   >