[jira] [Resolved] (SPARK-1890) add modify acls to the web UI for the "kill" button

2014-08-05 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1890. -- Resolution: Fixed > add modify acls to the web UI for the "kill" button > -

[jira] [Resolved] (SPARK-1680) Clean up use of setExecutorEnvs in SparkConf

2014-08-05 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1680. -- Resolution: Fixed > Clean up use of setExecutorEnvs in SparkConf > ---

[jira] [Created] (SPARK-2881) Snappy is now default codec - could lead to conflicts since uses /tmp

2014-08-06 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-2881: Summary: Snappy is now default codec - could lead to conflicts since uses /tmp Key: SPARK-2881 URL: https://issues.apache.org/jira/browse/SPARK-2881 Project: Spark

[jira] [Created] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2014-08-08 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-2930: Summary: clarify docs on using webhdfs with spark.yarn.access.namenodes Key: SPARK-2930 URL: https://issues.apache.org/jira/browse/SPARK-2930 Project: Spark

[jira] [Commented] (SPARK-2089) With YARN, preferredNodeLocalityData isn't honored

2014-08-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092881#comment-14092881 ] Thomas Graves commented on SPARK-2089: -- Sandy, just wondering if you have any ETA on

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

2014-08-12 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094211#comment-14094211 ] Thomas Graves commented on SPARK-2140: -- The commented out code was sufficient at the

[jira] [Commented] (SPARK-2140) yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM

2014-08-13 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095721#comment-14095721 ] Thomas Graves commented on SPARK-2140: -- ah it seems things have changed. Its now actu

[jira] [Created] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-15 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3072: Summary: Yarn AM not always properly exiting after unregistering from RM Key: SPARK-3072 URL: https://issues.apache.org/jira/browse/SPARK-3072 Project: Spark

[jira] [Commented] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098705#comment-14098705 ] Thomas Graves commented on SPARK-3072: -- Note that in yarn-cluster mode the client sid

[jira] [Commented] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-18 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101134#comment-14101134 ] Thomas Graves commented on SPARK-3072: -- Note that this also isn't an issue on hadoop

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

2014-08-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102199#comment-14102199 ] Thomas Graves commented on SPARK-3120: -- Can you please clarify this? You are trying

[jira] [Resolved] (SPARK-3072) Yarn AM not always properly exiting after unregistering from RM

2014-08-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3072. -- Resolution: Fixed Fix Version/s: 1.1.0 > Yarn AM not always properly exiting after unreg

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

2014-08-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102444#comment-14102444 ] Thomas Graves commented on SPARK-3120: -- If you want to change the local-dirs then you

[jira] [Resolved] (SPARK-3120) Local Dirs is not useful in yarn-client mode

2014-08-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3120. -- Resolution: Invalid > Local Dirs is not useful in yarn-client mode > --

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

2014-08-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102504#comment-14102504 ] Thomas Graves commented on SPARK-3129: -- A couple of random thoughts on this for yarn.

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

2014-08-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103859#comment-14103859 ] Thomas Graves commented on SPARK-3120: -- No. I am saying that you probably do have to

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

2014-08-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104077#comment-14104077 ] Thomas Graves commented on SPARK-3129: -- Yes that probably means using reflection. I

[jira] [Created] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-20 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3152: Summary: Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly Key: SPARK-3152 URL: https://issues.apache.org/jira/browse/SPARK-3152 Project: S

[jira] [Assigned] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-3152: Assignee: Thomas Graves > Yarn AM cluster mode doesn't cleanup staging directory when it ex

[jira] [Commented] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104430#comment-14104430 ] Thomas Graves commented on SPARK-3152: -- I'm hoping we can fix this in SPARK-2933 for

[jira] [Resolved] (SPARK-2742) The variable inputFormatInfo and inputFormatMap never used

2014-08-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2742. -- Resolution: Fixed Fix Version/s: 1.2.0 > The variable inputFormatInfo and inputFormatMap

[jira] [Commented] (SPARK-2742) The variable inputFormatInfo and inputFormatMap never used

2014-08-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106997#comment-14106997 ] Thomas Graves commented on SPARK-2742: -- https://github.com/apache/spark/pull/1614 >

[jira] [Resolved] (SPARK-1287) yarn alpha and stable Client calculateAMMemory routines are different

2014-08-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1287. -- Resolution: Duplicate duplicate of SPARK-2140 > yarn alpha and stable Client calculateAMMemory

[jira] [Commented] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2014-08-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110730#comment-14110730 ] Thomas Graves commented on SPARK-2541: -- So I haven't had time to full investigate thi

[jira] [Updated] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2014-08-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-2541: - Attachment: SPARK-2541-partial.patch > Standalone mode can't access secure HDFS anymore > ---

[jira] [Comment Edited] (SPARK-2541) Standalone mode can't access secure HDFS anymore

2014-08-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110730#comment-14110730 ] Thomas Graves edited comment on SPARK-2541 at 8/26/14 2:11 PM: -

[jira] [Resolved] (SPARK-2933) Cleanup unnecessary and duplicated code in Yarn module

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-2933. -- Resolution: Fixed Fix Version/s: 1.2.0 > Cleanup unnecessary and duplicated code in Yarn

[jira] [Resolved] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3152. -- Resolution: Duplicate > Yarn AM cluster mode doesn't cleanup staging directory when it exits cl

[jira] [Commented] (SPARK-3152) Yarn AM cluster mode doesn't cleanup staging directory when it exits cleanly

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112377#comment-14112377 ] Thomas Graves commented on SPARK-3152: -- this was fixed by SPARK-2933 in master. I do

[jira] [Updated] (SPARK-3260) Yarn - pass acls along with executor launch

2014-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3260: - Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0) > Yarn - pass acls along with executor launch >

[jira] [Created] (SPARK-3260) Yarn - pass acls along with executor launch

2014-08-27 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-3260: Summary: Yarn - pass acls along with executor launch Key: SPARK-3260 URL: https://issues.apache.org/jira/browse/SPARK-3260 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-12713) UI Executor page should keep links around to executors that died

2016-01-08 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-12713: - Summary: UI Executor page should keep links around to executors that died Key: SPARK-12713 URL: https://issues.apache.org/jira/browse/SPARK-12713 Project: Spark

[jira] [Commented] (SPARK-12713) UI Executor page should keep links around to executors that died

2016-01-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15089458#comment-15089458 ] Thomas Graves commented on SPARK-12713: --- I mean while the job is still running and

[jira] [Reopened] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-12654: --- > sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop > ---

[jira] [Resolved] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12654. --- Resolution: Fixed Assignee: Thomas Graves (was: Apache Spark) > sc.wholeTextFiles with

[jira] [Closed] (SPARK-12654) sc.wholeTextFiles with spark.hadoop.cloneConf=true fails on secure Hadoop

2016-01-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves closed SPARK-12654. - Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > sc.wholeTextFiles with s

[jira] [Commented] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2016-01-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091941#comment-15091941 ] Thomas Graves commented on SPARK-2930: -- I think we should still document this. Its a

[jira] [Reopened] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2016-01-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-2930: -- > clarify docs on using webhdfs with spark.yarn.access.namenodes > -

[jira] [Commented] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

2016-01-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091965#comment-15091965 ] Thomas Graves commented on SPARK-2930: -- I think simply putting a webhdfs url in the e

[jira] [Created] (SPARK-12784) Spark UI IndexOutOfBoundsException with dynamic allocation

2016-01-12 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-12784: - Summary: Spark UI IndexOutOfBoundsException with dynamic allocation Key: SPARK-12784 URL: https://issues.apache.org/jira/browse/SPARK-12784 Project: Spark

[jira] [Resolved] (SPARK-12716) Executor UI improvement suggestions - Totals

2016-01-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12716. --- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.0.0 > Executor UI i

[jira] [Updated] (SPARK-12149) Executor UI improvement suggestions - Color UI

2016-01-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12149: -- Assignee: Alex Bozarth > Executor UI improvement suggestions - Color UI > -

[jira] [Commented] (SPARK-3374) Spark on Yarn config cleanup

2016-01-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106766#comment-15106766 ] Thomas Graves commented on SPARK-3374: -- I think with 2.0 we should actually just remo

[jira] [Commented] (SPARK-1747) check for Spark on Yarn ApplicationMaster split brain

2016-01-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106784#comment-15106784 ] Thomas Graves commented on SPARK-1747: -- This should stay open. > check for Spark on

[jira] [Updated] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

2016-01-19 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-6166: - Assignee: (was: Shixiong Zhu) > Add config to limit number of concurrent outbound connections

[jira] [Created] (SPARK-12930) NullPointerException running hive query with array dereference in select and where clause

2016-01-20 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-12930: - Summary: NullPointerException running hive query with array dereference in select and where clause Key: SPARK-12930 URL: https://issues.apache.org/jira/browse/SPARK-12930

[jira] [Commented] (SPARK-12930) NullPointerException running hive query with array dereference in select and where clause

2016-01-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108871#comment-15108871 ] Thomas Graves commented on SPARK-12930: --- Note that change the query to remove the [

[jira] [Comment Edited] (SPARK-12930) NullPointerException running hive query with array dereference in select and where clause

2016-01-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108871#comment-15108871 ] Thomas Graves edited comment on SPARK-12930 at 1/20/16 4:41 PM: ---

[jira] [Updated] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

2016-01-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Parent Issue: SPARK-11806 (was: SPARK-3492) > Spark on Yarn remove deprecated configs for 2.0 > -

[jira] [Updated] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

2016-01-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-3374: - Summary: Spark on Yarn remove deprecated configs for 2.0 (was: Spark on Yarn config cleanup) > S

[jira] [Commented] (SPARK-11806) Spark 2.0 deprecations and removals

2016-01-20 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108961#comment-15108961 ] Thomas Graves commented on SPARK-11806: --- I added a task to remove the deprecated ya

[jira] [Commented] (SPARK-10911) Executors should System.exit on clean shutdown

2016-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115225#comment-15115225 ] Thomas Graves commented on SPARK-10911: --- see the pull request for comments and disc

[jira] [Commented] (SPARK-3611) Show number of cores for each executor in application web UI

2016-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115309#comment-15115309 ] Thomas Graves commented on SPARK-3611: -- I know the pull request was closed due to not

[jira] [Resolved] (SPARK-12149) Executor UI improvement suggestions - Color UI

2016-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12149. --- Resolution: Fixed Fix Version/s: 2.0.0 > Executor UI improvement suggestions - Color U

[jira] [Resolved] (SPARK-1832) Executor UI improvement suggestions

2016-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-1832. -- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.0.0 > Executor UI impr

[jira] [Commented] (SPARK-1832) Executor UI improvement suggestions

2016-01-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116057#comment-15116057 ] Thomas Graves commented on SPARK-1832: -- I think they meant the driver. I'll mark thi

[jira] [Resolved] (SPARK-10911) Executors should System.exit on clean shutdown

2016-01-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-10911. --- Resolution: Fixed Fix Version/s: 2.0.0 > Executors should System.exit on clean shutdow

[jira] [Assigned] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-01-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-1239: Assignee: Thomas Graves > Don't fetch all map output statuses at each reducer during shuffl

[jira] [Commented] (SPARK-13064) api/v1/application/jobs/attempt lacks "attempId" field for spark-shell

2016-01-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122077#comment-15122077 ] Thomas Graves commented on SPARK-13064: --- thanks [~vanzin] We can just have the res

[jira] [Commented] (SPARK-3374) Spark on Yarn remove deprecated configs for 2.0

2016-01-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123498#comment-15123498 ] Thomas Graves commented on SPARK-3374: -- The Drive and AM configs should not be combin

[jira] [Updated] (SPARK-10873) Change history table to use datatables to support sorting columns and searching

2016-01-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-10873: -- Summary: Change history table to use datatables to support sorting columns and searching (was:

[jira] [Resolved] (SPARK-10873) can't sort columns on history page

2016-01-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-10873. --- Resolution: Fixed Fix Version/s: 2.0.0 > can't sort columns on history page >

[jira] [Updated] (SPARK-10873) Change history to use datatables to support sorting columns and searching

2016-01-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-10873: -- Summary: Change history to use datatables to support sorting columns and searching (was: Chang

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12316: -- Assignee: SaintBacchus > Stack overflow with endless call of `Delegation token thread` when >

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-08 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137000#comment-15137000 ] Thomas Graves commented on SPARK-12316: --- you say "endless cycle call" do you mean t

[jira] [Commented] (SPARK-11316) isEmpty before coalesce seems to cause huge performance issue in setupGroups

2016-02-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15139658#comment-15139658 ] Thomas Graves commented on SPARK-11316: --- So we ran into this again, here is the sce

[jira] [Resolved] (SPARK-13163) Column width on new History Server DataTables not getting set correctly

2016-02-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13163. --- Resolution: Fixed Fix Version/s: 2.0.0 > Column width on new History Server DataTables

[jira] [Updated] (SPARK-13126) History Server page always has horizontal scrollbar

2016-02-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13126: -- Assignee: Zhuo Liu > History Server page always has horizontal scrollbar >

[jira] [Resolved] (SPARK-13126) History Server page always has horizontal scrollbar

2016-02-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13126. --- Resolution: Fixed Fix Version/s: 2.0.0 > History Server page always has horizontal scr

[jira] [Updated] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-13124: -- Assignee: Alex Bozarth > Adding JQuery DataTables messed up the Web UI css and js > ---

[jira] [Resolved] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13124. --- Resolution: Fixed Fix Version/s: 2.0.0 > Adding JQuery DataTables messed up the Web UI

[jira] [Commented] (SPARK-4224) Support group acls

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148627#comment-15148627 ] Thomas Graves commented on SPARK-4224: -- why did you close this? > Support group acls

[jira] [Reopened] (SPARK-4224) Support group acls

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reopened SPARK-4224: -- > Support group acls > -- > > Key: SPARK-4224 > URL:

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148662#comment-15148662 ] Thomas Graves commented on SPARK-12316: --- I'm not following how this ended up in an

[jira] [Commented] (SPARK-4224) Support group acls

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148682#comment-15148682 ] Thomas Graves commented on SPARK-4224: -- Ok, I understand closing older things but in

[jira] [Created] (SPARK-13343) speculative tasks that didn't commit shouldn't be marked as success

2016-02-16 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-13343: - Summary: speculative tasks that didn't commit shouldn't be marked as success Key: SPARK-13343 URL: https://issues.apache.org/jira/browse/SPARK-13343 Project: Spark

[jira] [Updated] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-11701: -- Description: I am using dynamic container allocation and speculation and am seeing issues with

[jira] [Resolved] (SPARK-11701) YARN - dynamic allocation and speculation active task accounting wrong

2016-02-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-11701. --- Resolution: Duplicate > YARN - dynamic allocation and speculation active task accounting wron

[jira] [Resolved] (SPARK-9439) ExternalShuffleService should be robust to NodeManager restarts in yarn

2015-08-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-9439. -- Resolution: Fixed Assignee: Imran Rashid Fix Version/s: 1.6.0 > ExternalShuffleS

[jira] [Commented] (SPARK-6951) History server slow startup if the event log directory is large

2015-08-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711226#comment-14711226 ] Thomas Graves commented on SPARK-6951: -- I'm running Spark 1.4.1 history server in pro

[jira] [Commented] (SPARK-6951) History server slow startup if the event log directory is large

2015-08-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711292#comment-14711292 ] Thomas Graves commented on SPARK-6951: -- Sorry I was wrong.. I just went and tested it

[jira] [Commented] (SPARK-4223) Support * (meaning all users) as part of the acls

2015-08-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711384#comment-14711384 ] Thomas Graves commented on SPARK-4223: -- [~srowen] [~rxin] do one of you have permissi

[jira] [Commented] (SPARK-6906) Improve Hive integration support

2015-08-25 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711843#comment-14711843 ] Thomas Graves commented on SPARK-6906: -- Can someone who did this work explain how thi

[jira] [Commented] (SPARK-6906) Improve Hive integration support

2015-08-27 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716616#comment-14716616 ] Thomas Graves commented on SPARK-6906: -- Thanks for the information. I'm trying to ge

[jira] [Created] (SPARK-10332) spark-submit to yarn doesn't fail if executors is 0

2015-08-28 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10332: - Summary: spark-submit to yarn doesn't fail if executors is 0 Key: SPARK-10332 URL: https://issues.apache.org/jira/browse/SPARK-10332 Project: Spark Issue T

[jira] [Commented] (SPARK-10332) spark-submit to yarn doesn't fail if executors is 0

2015-08-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720222#comment-14720222 ] Thomas Graves commented on SPARK-10332: --- Unless something else has changed that was

[jira] [Comment Edited] (SPARK-10332) spark-submit to yarn doesn't fail if executors is 0

2015-08-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720222#comment-14720222 ] Thomas Graves edited comment on SPARK-10332 at 8/28/15 4:52 PM: ---

[jira] [Created] (SPARK-10406) Document spark on yarn distributed cache symlink functionality

2015-09-01 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10406: - Summary: Document spark on yarn distributed cache symlink functionality Key: SPARK-10406 URL: https://issues.apache.org/jira/browse/SPARK-10406 Project: Spark

[jira] [Created] (SPARK-10432) spark.port.maxRetries documentation is unclear

2015-09-03 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10432: - Summary: spark.port.maxRetries documentation is unclear Key: SPARK-10432 URL: https://issues.apache.org/jira/browse/SPARK-10432 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9386) Feature flag for metastore partitioning

2015-09-03 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729085#comment-14729085 ] Thomas Graves commented on SPARK-9386: -- [~marmbrus] Were these just unit test failu

[jira] [Assigned] (SPARK-10432) spark.port.maxRetries documentation is unclear

2015-09-03 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-10432: - Assignee: Thomas Graves > spark.port.maxRetries documentation is unclear > -

[jira] [Commented] (SPARK-10432) spark.port.maxRetries documentation is unclear

2015-09-03 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729176#comment-14729176 ] Thomas Graves commented on SPARK-10432: --- https://github.com/apache/spark/pull/8585

[jira] [Commented] (SPARK-9924) checkForLogs and cleanLogs are scheduled at fixed rate and can get piled up

2015-09-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737416#comment-14737416 ] Thomas Graves commented on SPARK-9924: -- [~vanzin] Any reason this wasn't picked back

[jira] [Commented] (SPARK-9924) checkForLogs and cleanLogs are scheduled at fixed rate and can get piled up

2015-09-09 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737478#comment-14737478 ] Thomas Graves commented on SPARK-9924: -- Ok, thanks. wanted to make sure no known issu

[jira] [Commented] (SPARK-9924) checkForLogs and cleanLogs are scheduled at fixed rate and can get piled up

2015-09-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739115#comment-14739115 ] Thomas Graves commented on SPARK-9924: -- https://github.com/apache/spark/pull/8701 pr

[jira] [Created] (SPARK-10549) scala 2.11 spark on yarn with security - Repl doesn't work

2015-09-10 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10549: - Summary: scala 2.11 spark on yarn with security - Repl doesn't work Key: SPARK-10549 URL: https://issues.apache.org/jira/browse/SPARK-10549 Project: Spark

[jira] [Assigned] (SPARK-10549) scala 2.11 spark on yarn with security - Repl doesn't work

2015-09-11 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-10549: - Assignee: Thomas Graves > scala 2.11 spark on yarn with security - Repl doesn't work > -

[jira] [Created] (SPARK-10619) Can't sort columns on Executor Page

2015-09-15 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10619: - Summary: Can't sort columns on Executor Page Key: SPARK-10619 URL: https://issues.apache.org/jira/browse/SPARK-10619 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-10640: - Summary: Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied Key: SPARK-10640 URL: https://issues.apache.org/jira/browse/SPARK-10640 Project:

[jira] [Commented] (SPARK-10640) Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

2015-09-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790777#comment-14790777 ] Thomas Graves commented on SPARK-10640: --- looks like the jsonProtocol.taskEndReasonF

<    4   5   6   7   8   9   10   11   12   13   >