GitHub user nishkamravi2 opened a pull request:
https://github.com/apache/spark/pull/5672
[SPARK-6014] [core][hotfix] Revamp Spark shutdown hooks, fix shutdown races.
Add a try/catch block around removeShutDownHook else IllegalStateException
thrown in YARN cluster mode (see https://github.com/apache/spark/pull/4690)
cc @andrewor14, @srowen
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nishkamravi2/spark master_nravi
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5672.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5672
----
commit 681b36f5fb63e14dc89e17813894227be9e2324f
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-05-08T07:05:33Z
Fix for SPARK-1758: failing test
org.apache.spark.JavaAPISuite.wholeTextFiles
The prefix "file:" is missing in the string inserted as key in HashMap
commit 5108700230fd70b995e76598f49bdf328c971e77
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-03T22:25:22Z
Fix in Spark for the Concurrent thread modification issue (SPARK-1097,
HADOOP-10456)
commit 6b840f017870207d23e75de224710971ada0b3d0
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-03T22:34:02Z
Undo the fix for SPARK-1758 (the problem is fixed)
commit df2aeb179fca4fc893803c72a657317f5b5539d7
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-09T19:02:59Z
Improved fix for ConcurrentModificationIssue (Spark-1097, Hadoop-10456)
commit eb663ca20c73f9c467192c95fc528c6f55f202be
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-09T19:04:39Z
Merge branch 'master' of https://github.com/apache/spark
commit 5423a03ddf4d747db7261d08a64e32f44e8be95e
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-10T20:06:07Z
Merge branch 'master' of https://github.com/apache/spark
commit 3bf8fad85813037504189cf1323d381fefb6dfbe
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-16T05:47:00Z
Merge branch 'master' of https://github.com/apache/spark
commit 2b630f94079b82df3ebae2b26a3743112afcd526
Author: nravi <nr...@c1704.halxg.cloudera.com>
Date: 2014-06-16T06:00:31Z
Accept memory input as "30g", "512M" instead of an int value, to be
consistent with rest of Spark
commit efd688a4e15b79e92d162073035b03362fcf66f0
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-07-13T00:04:17Z
Merge branch 'master' of https://github.com/apache/spark
commit 2e69f112d1be59951cd32da4127d8b51bfa03338
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-21T23:17:15Z
Merge branch 'master' of https://github.com/apache/spark into master_nravi
commit ebcde10252e6c45169ea086e8426ec9997d46490
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-22T06:44:40Z
Modify default YARN memory_overhead-- from an additive constant to a
multiplier (redone to resolve merge conflicts)
commit 1cf2d1ef57ed6d783df06dad36b9505bc74329fb
Author: nishkamravi2 <nishkamr...@gmail.com>
Date: 2014-09-22T08:54:33Z
Update YarnAllocator.scala
commit f00fa311945c1eafa8957eae5c84719521761dcd
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-22T23:06:07Z
Improving logging for AM memoryOverhead
commit c726bd9f707ce182ec8d56ffecf9da87dcdb3091
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T01:19:32Z
Merge branch 'master' of https://github.com/apache/spark into master_nravi
Conflicts:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala
commit 362da5edfd04bd8bad990fb210a9e11b8494fa62
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T19:56:13Z
Additional changes for yarn memory overhead
commit 42c2c3d18862d3632c20931ecfe2c64883c5febf
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T20:02:49Z
Additional changes for yarn memory overhead issue
commit dac1047995c99f5a2670f934eb8d3a4ad9b532c8
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T21:20:38Z
Additional documentation for yarn memory overhead issue
commit 5ac2ec11629e19030ad5577da1eee2d135cc3d1c
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T21:25:44Z
Remove out
commit 35daa6498048cabb736316e2f19e565c99243b7e
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T21:59:22Z
Slight change in the doc for yarn memory overhead
commit 8f76c8b46379736aeb7dbe1a4d88729424a041f7
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-09-25T22:03:00Z
Doc change for yarn memory overhead
commit 636a9ffeb4a4ae0b941edd849dcbabf38821db53
Author: nishkamravi2 <nishkamr...@gmail.com>
Date: 2014-09-30T18:33:28Z
Update YarnAllocator.scala
commit 5f8f9ede0fda5c7a4f6a411c746a3d893f550524
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-11-19T01:46:58Z
Merge branch 'master' of https://github.com/apache/spark into master_nravi
Conflicts:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
commit 9f6583eb92b59e88b0994ef45dd0e1a418d349a6
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-11-19T03:42:25Z
Changes to maxResultSize code (improve error message and add condition to
check if maxResultSize > 0)
commit 3e1b616a56cedec0b963b7f813a8e0195c3417e8
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-11-19T06:56:55Z
Modify test for maxResultSize
commit 535295a371b259a4fb1eb2ef01ea9f396aed40dd
Author: nishkamravi2 <nishkamr...@gmail.com>
Date: 2014-11-19T21:56:18Z
Update TaskSetManager.scala
commit 5c9a4cb4eb12eb6cefbf9e9dda42b722abe1d3ed
Author: nishkamravi2 <nishkamr...@gmail.com>
Date: 2014-11-19T21:57:01Z
Update TaskSetManagerSuite.scala
commit b446edc5ce672139564bcbc98217862a2921783b
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2014-11-19T23:30:17Z
Merge branch 'master' of https://github.com/apache/spark into master_nravi
commit 79ea8b418f709232ec6e106d7f58bb11ba86fa4a
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2015-02-19T06:03:18Z
Merge branch 'master' of https://github.com/apache/spark into master_nravi
commit f0d12de3a2179d2ee32bf7ae049223b5ee246795
Author: Nishkam Ravi <nr...@cloudera.com>
Date: 2015-02-19T07:45:31Z
Workaround for IllegalStateException caused by recent changes to
BlockManager.stop
commit 3c5ddba51441b676391e43f3b09b9c59624ffef0
Author: nishkamravi2 <nishkamr...@gmail.com>
Date: 2015-02-19T09:31:30Z
Update DiskBlockManager.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
