Repository: spark
Updated Branches:
refs/heads/master 2d222fb39 - 80098109d
[SPARK-6314] [CORE] handle JsonParseException for history server
This is handled in the same way with
[SPARK-6197](https://issues.apache.org/jira/browse/SPARK-6197). The result of
this PR is that exception showed in
Repository: spark
Updated Branches:
refs/heads/branch-1.3 80ba82e8e - 37fb5a968
[SPARK-6314] [CORE] handle JsonParseException for history server
This is handled in the same way with
[SPARK-6197](https://issues.apache.org/jira/browse/SPARK-6197). The result of
this PR is that exception
Repository: spark
Updated Branches:
refs/heads/master 6fbeb82e1 - 199133733
[SPARK-5933] [core] Move config deprecation warnings to SparkConf.
I didn't find many deprecated configs after a grep-based search,
but the ones I could find were moved to the centralized location
in SparkConf.
While
Repository: spark
Updated Branches:
refs/heads/master 199133733 - d850b4bd3
[SPARK-6975][Yarn] Fix argument validation error
`numExecutors` checking is failed when dynamic allocation is enabled with
default configuration. Details can be seen is
Repository: spark
Updated Branches:
refs/heads/master c5ed51013 - 6fbeb82e1
[SPARK-6350][Mesos] Make mesosExecutorCores configurable in mesos
fine-grained mode
- Defined executorCores from spark.mesos.executor.cores
- Changed the amount of mesosExecutor's cores to executorCores.
- Added new
Repository: spark
Updated Branches:
refs/heads/master a452c5921 - c5ed51013
[SPARK-6703][Core] Provide a way to discover existing SparkContext's
I've added a static getOrCreate method to the static SparkContext object that
allows one to either retrieve a previously created SparkContext or to
Repository: spark
Updated Branches:
refs/heads/master 971738936 - 30a6e0dcc
[SPARK-5634] [core] Show correct message in HS when no incomplete apps f...
...ound.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5515 from vanzin/SPARK-5634 and squashes the following commits:
f74ecf1
Repository: spark
Updated Branches:
refs/heads/master 6de282e2d - 971738936
[SPARK-6890] [core] Fix launcher lib work with SPARK_PREPEND_CLASSES.
The fix for SPARK-6406 broke the case where sub-processes are launched
when SPARK_PREPEND_CLASSES is set, because the code now would only add
the
Repository: spark
Updated Branches:
refs/heads/branch-1.2 3c13936aa - 5845a6236
[SPARK-5634] [core] Show correct message in HS when no incomplete apps f...
...ound.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5515 from vanzin/SPARK-5634 and squashes the following commits:
f74ecf1
Repository: spark
Updated Branches:
refs/heads/master 320bca450 - f63b44a5c
[SPARK-6894]spark.executor.extraLibraryOptions =
spark.executor.extraLibraryPath
https://issues.apache.org/jira/browse/SPARK-6894
cc vanzin
Author: WangTaoTheTonic wangtao...@huawei.com
Closes #5506 from
Repository: spark
Updated Branches:
refs/heads/master f63b44a5c - dcf8a9f33
[CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled
due to previous stage failure
Fixed null check when all the dependent stages are cancelled due to previous
stage failure. This happens
Repository: spark
Updated Branches:
refs/heads/master 51b306b93 - 320bca450
[SPARK-6081] Support fetching http/https uris in driver runner.
Currently if passed uris such as http/https, it won't able to fetch them as it
only calls HadoopFs get.
This fix utilizes the existing util method to
Repository: spark
Updated Branches:
refs/heads/master dcf8a9f33 - 25998e4d7
[SPARK-2033] Automatically cleanup checkpoint
Author: GuoQiang Li wi...@qq.com
Closes #855 from witgo/cleanup_checkpoint_date and squashes the following
commits:
1649850 [GuoQiang Li] review commit
c0087e0
Repository: spark
Updated Branches:
refs/heads/master 25998e4d7 - 8f8dc45f6
SPARK-1706: Allow multiple executors per worker in Standalone mode
resubmit of https://github.com/apache/spark/pull/636 for a totally different
algorithm
https://issues.apache.org/jira/browse/SPARK-1706
In current
Repository: spark
Updated Branches:
refs/heads/master 8f8dc45f6 - b075e4b72
[SPARK-6700] [yarn] Re-enable flaky test.
Test runs have been successful on jenkins. So let's re-enable the test and look
out for any failures, and fix things appropriately.
Author: Marcelo Vanzin
Repository: spark
Updated Branches:
refs/heads/master 6adb8bcbf - 65774370a
[SPARK-5808] [build] Package pyspark files in sbt assembly.
This turned out to be more complicated than I wanted because the
layout of python/ doesn't really follow the usual maven conventions.
So some extra code is
Repository: spark
Updated Branches:
refs/heads/master 65774370a - 4d4b24927
[SPARK-6769][YARN][TEST] Usage of the ListenerBus in YarnClusterSuite is wrong
In YarnClusterSuite, a test case uses `SaveExecutorInfo` to handle
ExecutorAddedEvent as follows.
```
private class SaveExecutorInfo
' into
feature/spark-4848
044ca6f [Nathan Kronenfeld] Documentation and formatting as requested by by
andrewor14
d739640 [Nathan Kronenfeld] Move looping through instances from the master to
the workers, so that each worker respects its own number of instances and
web-ui port
Project: http://git-wip
Repository: spark
Updated Branches:
refs/heads/master c5602bdc3 - c4ab255e9
[SPARK-5931][CORE] Use consistent naming for time properties
I've added new utility methods to do the conversion from times specified as
e.g. 120s, 240ms, 360us to convert to a consistent internal representation.
Repository: spark
Updated Branches:
refs/heads/master ffe8cc9a2 - 14632b794
[SPARK-6688] [core] Always use resolved URIs in EventLoggingListener.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5340 from vanzin/SPARK-6688 and squashes the following commits:
ccfddd9 [Marcelo Vanzin]
Repository: spark
Updated Branches:
refs/heads/branch-1.3 0c1b78b72 - f17a2fe9b
[SPARK-6688] [core] Always use resolved URIs in EventLoggingListener.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5340 from vanzin/SPARK-6688 and squashes the following commits:
ccfddd9 [Marcelo Vanzin]
Repository: spark
Updated Branches:
refs/heads/branch-1.3 f17a2fe9b - 3366af67b
[SPARK-6700] disable flaky test
Author: Davies Liu dav...@databricks.com
Closes #5356 from davies/flaky and squashes the following commits:
08955f4 [Davies Liu] disable flaky test
(cherry picked from commit
Repository: spark
Updated Branches:
refs/heads/master 26b415e15 - 9b40c17ab
[SPARK-6700] disable flaky test
Author: Davies Liu dav...@databricks.com
Closes #5356 from davies/flaky and squashes the following commits:
08955f4 [Davies Liu] disable flaky test
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 0c1c0fb90 - 17ab6b0cf
[CORE] The descriptionof jobHistory config should be
spark.history.fs.logDirectory
The config option is spark.history.fs.logDirectory, not
spark.fs.history.logDirectory. So the descriptionof should be changed.
Repository: spark
Updated Branches:
refs/heads/branch-1.2 2991dd042 - f4a9c41b1
[CORE] The descriptionof jobHistory config should be
spark.history.fs.logDirectory
The config option is spark.history.fs.logDirectory, not
spark.fs.history.logDirectory. So the descriptionof should be changed.
Repository: spark
Updated Branches:
refs/heads/master 4b82bd730 - 8a0aa81ca
[CORE] The descriptionof jobHistory config should be
spark.history.fs.logDirectory
The config option is spark.history.fs.logDirectory, not
spark.fs.history.logDirectory. So the descriptionof should be changed.
Repository: spark
Updated Branches:
refs/heads/master 052dee070 - 45134ec92
[SPARK-6650] [core] Stop ExecutorAllocationManager when context stops.
This fixes the thread leak. I also changed the unit test to keep track
of allocated contexts and make sure they're closed after tests are
run;
Repository: spark
Updated Branches:
refs/heads/branch-1.3 2927af102 - 0ef46b2d8
[SPARK-6650] [core] Stop ExecutorAllocationManager when context stops.
This fixes the thread leak. I also changed the unit test to keep track
of allocated contexts and make sure they're closed after tests are
run;
Repository: spark
Updated Branches:
refs/heads/master 968408b34 - 883b7e903
[SPARK-6076][Block Manager] Fix a potential OOM issue when StorageLevel is
MEMORY_AND_DISK_SER
In
because of a subtle race condition explained in that PR.
Tested this locally with and without calling `sc.stop()`.
Author: Andrew Or and...@databricks.com
Closes #5177 from andrewor14/master-ui-cores and squashes the following commits:
62449d1 [Andrew Or] Freeze application state before finishing
Repository: spark
Updated Branches:
refs/heads/master 4fc4d0369 - d44a3362e
[SPARK-6079] Use index to speed up StatusTracker.getJobIdsForGroup()
`StatusTracker.getJobIdsForGroup()` is implemented via a linear scan over a
HashMap rather than using an index, which might be an expensive
Repository: spark
Updated Branches:
refs/heads/branch-1.3 de8b2d4be - e4db5a327
[SPARK-3570] Include time to open files in shuffle write time.
Opening shuffle files can be very significant when the disk is
contended, especially when using ext3. While writing data to
a file can avoid hitting
Repository: spark
Updated Branches:
refs/heads/branch-1.3 586e0d924 - de8b2d4be
[SPARK-6088] Correct how tasks that get remote results are shown in UI.
It would be great to fix this for 1.3. since the fix is surgical and it helps
understandability for users.
cc shivaram pwendell
Author:
Repository: spark
Updated Branches:
refs/heads/master 73348012d - 6948ab6f8
[SPARK-6088] Correct how tasks that get remote results are shown in UI.
It would be great to fix this for 1.3. since the fix is surgical and it helps
understandability for users.
cc shivaram pwendell
Author: Kay
Repository: spark
Updated Branches:
refs/heads/branch-1.3 f48c16d3b - dcf56aa8b
[SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load
classes (master branch PR)
ExecutorClassLoader does not ensure proper cleanup of network connections that
it opens. If it fails to
Repository: spark
Updated Branches:
refs/heads/master 6948ab6f8 - d8ccf655f
[SPARK-3570] Include time to open files in shuffle write time.
Opening shuffle files can be very significant when the disk is
contended, especially when using ext3. While writing data to
a file can avoid hitting disk
Revert [SPARK-5771] Number of Cores in Completed Applications of Standalone
Master Web Page always be 0 if sc.stop() is called
This reverts commit dd077abf2e2949fdfec31074b760b587f00efcf2.
Conflicts:
core/src/main/scala/org/apache/spark/deploy/master/ApplicationInfo.scala
Repository: spark
Updated Branches:
refs/heads/master d8ccf655f - dd907d1a9
Revert [SPARK-5771][UI][hotfix] Change Requested Cores into * if default cores
is not set
This reverts commit 12135e90549f957962899487cd5eb95badd8976d.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Repository: spark
Updated Branches:
refs/heads/master dd907d1a9 - 05c2214b4
[SPARK-6469] Improving documentation on YARN local directories usage
Clarify the local directories usage in YARN
Author: Christophe Préaud christophe.pre...@kelkoo.com
Closes #5165 from preaudc/yarn-doc-local-dirs
Repository: spark
Updated Branches:
refs/heads/branch-1.3 e4db5a327 - 6af9408a9
[SPARK-6469] Improving documentation on YARN local directories usage
Clarify the local directories usage in YARN
Author: Christophe Préaud christophe.pre...@kelkoo.com
Closes #5165 from
Repository: spark
Updated Branches:
refs/heads/master 94598653b - c14ddd97e
[SPARK-6515] update OpenHashSet impl
Though I don't see any bug in the existing code, the update in this PR makes it
read better. rxin
Author: Xiangrui Meng m...@databricks.com
Closes #5176 from mengxr/SPARK-6515
Repository: spark
Updated Branches:
refs/heads/branch-1.3 bfa4e3194 - 3fc74f45a
[SPARK-6144] [core] Fix addFile when source files are on hdfs:
The code failed in two modes: it complained when it tried to re-create a
directory that already existed, and it was placing some files in the wrong
Repository: spark
Updated Branches:
refs/heads/master f6773edce - 3a35a0dfe
[SPARK-6144] [core] Fix addFile when source files are on hdfs:
The code failed in two modes: it complained when it tried to re-create a
directory that already existed, and it was placing some files in the wrong
Repository: spark
Updated Branches:
refs/heads/master e359794ce - 9af001749
Revert [SPARK-5423][Core] Cleanup resources in DiskMapIterator.finalize to
ensure deleting the temp file
This reverts commit 90095bf3ce9304d09a32ceffaa99069079071b59.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.1 91d0effb3 - d70754df0
Revert [SPARK-5423][Core] Cleanup resources in DiskMapIterator.finalize to
ensure deleting the temp file
This reverts commit 36f3c499fd1ad53a68a084d6a16a2c68099e7049.
Project:
Repository: spark
Updated Branches:
refs/heads/master 9af001749 - e750a6bfd
SPARK-1911 [DOCS] Warn users if their assembly jars are not built with Java 6
Add warning about building with Java 7+ and running the JAR on early Java 6.
CC andrewor14
Author: Sean Owen so...@cloudera.com
Closes
and I suspect that this is the cause of many flaky tests, most
notably the `JavaAPISuite`. We were able to reproduce the failure locally
(though it is not deterministic and very hard to reproduce).
Author: Andrew Or and...@databricks.com
Closes #4869 from andrewor14/cleaner-masquerade
Repository: spark
Updated Branches:
refs/heads/branch-1.3 05d5a29eb - ee4929d1d
Revert [SPARK-5423][Core] Cleanup resources in DiskMapIterator.finalize to
ensure deleting the temp file
This reverts commit 25fae8e7e6c93b7817771342d370b73b40dcf92e.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.2 5226dc74d - eb30fc180
Revert [SPARK-5423][Core] Cleanup resources in DiskMapIterator.finalize to
ensure deleting the temp file
This reverts commit 61bde0049fac324b5004eadfa22b02cd76cf2187.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 ee4929d1d - 8446ad0eb
SPARK-1911 [DOCS] Warn users if their assembly jars are not built with Java 6
Add warning about building with Java 7+ and running the JAR on early Java 6.
CC andrewor14
Author: Sean Owen so...@cloudera.com
SparkListenerApplicationEnd(1425348445682)
```
Author: Andrew Or and...@databricks.com
Closes #4871 from andrewor14/sc-stop and squashes the following commits:
a14afc5 [Andrew Or] Move code after code
915db16 [Andrew Or] Move code into code
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git
SparkListenerApplicationEnd(1425348445682)
```
Author: Andrew Or and...@databricks.com
Closes #4871 from andrewor14/sc-stop and squashes the following commits:
a14afc5 [Andrew Or] Move code after code
915db16 [Andrew Or] Move code into code
(cherry picked from commit 6c20f35290e220e4a659a0222d62575ff959d703
this.
Author: Andrew Or and...@databricks.com
Closes #4882 from andrewor14/cleaner-interrupt and squashes the following
commits:
8652120 [Andrew Or] Just a hot fix
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d334bfbc
Tree: http
Repository: spark
Updated Branches:
refs/heads/master 6b348d90f - 26c1c56de
[SPARK-5522] Accelerate the Histroty Server start
When starting the history server, all the log files will be fetched and parsed
in order to get the applications' meta data e.g. App Name, Start Time,
Duration, etc.
Repository: spark
Updated Branches:
refs/heads/master 0375a413b - 8cd1692c9
[SPARK-6036][CORE] avoid race condition between eventlogListener and akka actor
system
For detail description, pls refer to
[SPARK-6036](https://issues.apache.org/jira/browse/SPARK-6036).
Author: Zhang, Liye
Repository: spark
Updated Branches:
refs/heads/master 12135e905 - 67595eb8f
[SPARK-5495][UI] Add app and driver kill function in master web UI
Add application kill function in master web UI for standalone mode. Details can
be seen in
Repository: spark
Updated Branches:
refs/heads/branch-1.3 6200f0709 - 485b91934
SPARK-2168 [Spark core] Use relative URIs for the app links in the History
Server.
As agreed in PR #1160 adding test to verify if history server generates
relative links to applications.
Author: Lukasz
Repository: spark
Updated Branches:
refs/heads/master 67595eb8f - 4a8a0a8ec
SPARK-2168 [Spark core] Use relative URIs for the app links in the History
Server.
As agreed in PR #1160 adding test to verify if history server generates
relative links to applications.
Author: Lukasz Jastrzebski
Repository: spark
Updated Branches:
refs/heads/master 7c99a014f - 0375a413b
fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode
jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033
In standalone deploy mode, the cleanup will only remove the stopped
Repository: spark
Updated Branches:
refs/heads/branch-1.3 485b91934 - b8db84c5b
fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode
jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033
In standalone deploy mode, the cleanup will only remove the
Repository: spark
Updated Branches:
refs/heads/branch-1.1 2785210fa - 814934da6
fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode
jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033
In standalone deploy mode, the cleanup will only remove the
Repository: spark
Updated Branches:
refs/heads/branch-1.0 14e042b65 - e751f8f26
fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode
jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033
In standalone deploy mode, the cleanup will only remove the
Repository: spark
Updated Branches:
refs/heads/branch-1.2 d0bf938ec - d4ce702c4
fix spark-6033, clarify the spark.worker.cleanup behavior in standalone mode
jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033
In standalone deploy mode, the cleanup will only remove the
Repository: spark
Updated Branches:
refs/heads/master 5e5ad6558 - 12135e905
[SPARK-5771][UI][hotfix] Change Requested Cores into * if default cores is not
set
cc andrewor14, srowen.
Author: jerryshao saisai.s...@intel.com
Closes #4800 from jerryshao/SPARK-5771 and squashes the following
Repository: spark
Updated Branches:
refs/heads/branch-1.2 2d83442f2 - 64e0cbc73
SPARK-4300 [CORE] Race condition during SparkWorker shutdown
Close appender saving stdout/stderr before destroying process to avoid
exception on reading closed input stream.
(This also removes a redundant
Repository: spark
Updated Branches:
refs/heads/branch-1.2 cc7313d09 - 602d5c1fc
SPARK-4704 [CORE] SparkSubmitDriverBootstrap doesn't flush output
Join on output threads to make sure any lingering output from process reaches
stdout, stderr before exiting
CC andrewor14 since I believe he
Repository: spark
Updated Branches:
refs/heads/master 7fa960e65 - cd5c8d7bb
SPARK-4704 [CORE] SparkSubmitDriverBootstrap doesn't flush output
Join on output threads to make sure any lingering output from process reaches
stdout, stderr before exiting
CC andrewor14 since I believe he created
Repository: spark
Updated Branches:
refs/heads/branch-1.2 602d5c1fc - 94faf4c49
Modify default value description for
spark.scheduler.minRegisteredResourcesRatio on docs.
The configuration is not supported in mesos mode now.
See https://github.com/apache/spark/pull/1462
Author: Li Zhihui
Repository: spark
Updated Branches:
refs/heads/master cd5c8d7bb - 10094a523
Modify default value description for
spark.scheduler.minRegisteredResourcesRatio on docs.
The configuration is not supported in mesos mode now.
See https://github.com/apache/spark/pull/1462
Author: Li Zhihui
Repository: spark
Updated Branches:
refs/heads/master 10094a523 - 8942b522d
[SPARK-3562]Periodic cleanup event logs
Author: xukun 00228947 xukun...@huawei.com
Closes #4214 from viper-kun/cleaneventlog and squashes the following commits:
7a5b9c5 [xukun 00228947] fix issue
31674ee [xukun
Repository: spark
Updated Branches:
refs/heads/master 8942b522d - aa63f633d
[SPARK-6027][SPARK-5546] Fixed --jar and --packages not working for KafkaUtils
and improved error message
The problem with SPARK-6027 in short is that JARs like the kafka-assembly.jar
does not work in python as the
Repository: spark
Updated Branches:
refs/heads/branch-1.3 62652dc5b - 731a997db
[SPARK-6027][SPARK-5546] Fixed --jar and --packages not working for KafkaUtils
and improved error message
The problem with SPARK-6027 in short is that JARs like the kafka-assembly.jar
does not work in python as
Repository: spark
Updated Branches:
refs/heads/master aa63f633d - 5f3238b3b
[SPARK-6018] [YARN] NoSuchMethodError in Spark app is swallowed by YARN AM
Author: Cheolsoo Park cheols...@netflix.com
Closes #4773 from piaozhexiu/SPARK-6018 and squashes the following commits:
2a919d5 [Cheolsoo
Repository: spark
Updated Branches:
refs/heads/branch-1.3 731a997db - fe7967483
[SPARK-6018] [YARN] NoSuchMethodError in Spark app is swallowed by YARN AM
Author: Cheolsoo Park cheols...@netflix.com
Closes #4773 from piaozhexiu/SPARK-6018 and squashes the following commits:
2a919d5
Repository: spark
Updated Branches:
refs/heads/branch-1.2 94faf4c49 - e21475d16
[SPARK-6018] [YARN] NoSuchMethodError in Spark app is swallowed by YARN AM
Author: Cheolsoo Park cheols...@netflix.com
Closes #4773 from piaozhexiu/SPARK-6018 and squashes the following commits:
2a919d5
Repository: spark
Updated Branches:
refs/heads/branch-1.3 fe7967483 - 297c3ef82
Add a note for context termination for History server on Yarn
The history server on Yarn only shows completed jobs. This adds a note
concerning the needed explicit context termination at the end of a spark job
Repository: spark
Updated Branches:
refs/heads/master c871e2dae - b38dec2ff
[SPARK-5951][YARN] Remove unreachable driver memory properties in yarn client
mode
Remove unreachable driver memory properties in yarn client mode
Author: mohit.goyal mohit.go...@guavus.com
Closes #4730 from
Repository: spark
Updated Branches:
refs/heads/branch-1.3 297c3ef82 - 5b426cb1f
[SPARK-5951][YARN] Remove unreachable driver memory properties in yarn client
mode
Remove unreachable driver memory properties in yarn client mode
Author: mohit.goyal mohit.go...@guavus.com
Closes #4730 from
Repository: spark
Updated Branches:
refs/heads/master 5f3238b3b - 3fb53c029
SPARK-4300 [CORE] Race condition during SparkWorker shutdown
Close appender saving stdout/stderr before destroying process to avoid
exception on reading closed input stream.
(This also removes a redundant `waitFor()`
Repository: spark
Updated Branches:
refs/heads/branch-1.0 f74bccbe3 - 14e042b65
Add a note for context termination for History server on Yarn
The history server on Yarn only shows completed jobs. This adds a note
concerning the needed explicit context termination at the end of a spark job
Repository: spark
Updated Branches:
refs/heads/branch-1.1 36f3c499f - 2785210fa
Add a note for context termination for History server on Yarn
The history server on Yarn only shows completed jobs. This adds a note
concerning the needed explicit context termination at the end of a spark job
Repository: spark
Updated Branches:
refs/heads/branch-1.2 64e0cbc73 - 58b3aa692
Add a note for context termination for History server on Yarn
The history server on Yarn only shows completed jobs. This adds a note
concerning the needed explicit context termination at the end of a spark job
Repository: spark
Updated Branches:
refs/heads/master 3fb53c029 - c871e2dae
Add a note for context termination for History server on Yarn
The history server on Yarn only shows completed jobs. This adds a note
concerning the needed explicit context termination at the end of a spark job
which
Repository: spark
Updated Branches:
refs/heads/master e60ad2f4c - fbc469473
SPARK-4579 [WEBUI] Scheduling Delay appears negative
Ensure scheduler delay handles unfinished task case, and ensure delay is never
negative even due to rounding
Author: Sean Owen so...@cloudera.com
Closes #4796
Repository: spark
Updated Branches:
refs/heads/master fbc469473 - 18f209843
[SPARK-5529][CORE]Add expireDeadHosts in HeartbeatReceiver
If a blockManager has not send heartBeat more than 120s,
BlockManagerMasterActor will remove it. But coarseGrainedSchedulerBackend can
only remove executor
Repository: spark
Updated Branches:
refs/heads/branch-1.2 71173de7a - 2c9d9659d
Revert [SPARK-4808] Removing minimum number of elements read before spill
check
This reverts commit 5cea859fd27dc6a216fa9d31d293c93407fbff01.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/master 201236628 - 64d2c01ff
[Spark-5967] [UI] Correctly clean JobProgressListener.stageIdToActiveJobIds
Patch should be self-explanatory
pwendell JoshRosen
Author: Tathagata Das tathagata.das1...@gmail.com
Closes #4741 from tdas/SPARK-5967 and
Repository: spark
Updated Branches:
refs/heads/branch-1.2 2c9d9659d - 3ad00ee1c
[Spark-5967] [UI] Correctly clean JobProgressListener.stageIdToActiveJobIds
Patch should be self-explanatory
pwendell JoshRosen
Author: Tathagata Das tathagata.das1...@gmail.com
Closes #4741 from tdas/SPARK-5967
Repository: spark
Updated Branches:
refs/heads/branch-1.3 e46096b1e - 28dd53b1b
[Spark-5967] [UI] Correctly clean JobProgressListener.stageIdToActiveJobIds
Patch should be self-explanatory
pwendell JoshRosen
Author: Tathagata Das tathagata.das1...@gmail.com
Closes #4741 from tdas/SPARK-5967
Or and...@databricks.com
Closes #4739 from andrewor14/user-jar-blocker and squashes the following
commits:
23c4a9e [Andrew Or] Use right argument
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6d2caa57
Tree: http://git-wip-us.apache.org
Or and...@databricks.com
Closes #4739 from andrewor14/user-jar-blocker and squashes the following
commits:
23c4a9e [Andrew Or] Use right argument
(cherry picked from commit 6d2caa576fcdc5c848d1472b09c685b3871e220e)
Signed-off-by: Andrew Or and...@databricks.com
Project: http://git-wip-us.apache.org/repos
Repository: spark
Updated Branches:
refs/heads/branch-1.3 eed7389cf - 4186dd3dd
Revert [SPARK-4808] Removing minimum number of elements read before spill
check
This reverts commit 0382dcc0a94f8e619fd11ec2cc0b18459a690c2b.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 932338eda - 76e3e6527
[SPARK-5937][YARN] Fix ClientSuite to set YARN mode, so that the correct class
is used in t...
...ests.
Without this SparkHadoopUtil is used by the Client instead of
YarnSparkHadoopUtil.
Author: Hari
Repository: spark
Updated Branches:
refs/heads/master d3cbd38c3 - 7138816ab
[SPARK-5937][YARN] Fix ClientSuite to set YARN mode, so that the correct class
is used in t...
...ests.
Without this SparkHadoopUtil is used by the Client instead of
YarnSparkHadoopUtil.
Author: Hari Shreedharan
Repository: spark
Updated Branches:
refs/heads/master e15532471 - d3cbd38c3
SPARK-5841 [CORE] [HOTFIX 2] Memory leak in DiskBlockManager
Continue to see IllegalStateException in YARN cluster mode. Adding a simple
workaround for now.
Author: Nishkam Ravi nr...@cloudera.com
Author:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 b9a6c5c84 - 932338eda
SPARK-5841 [CORE] [HOTFIX 2] Memory leak in DiskBlockManager
Continue to see IllegalStateException in YARN cluster mode. Adding a simple
workaround for now.
Author: Nishkam Ravi nr...@cloudera.com
Author:
Repository: spark
Updated Branches:
refs/heads/branch-1.2 61bde0049 - 856fdcb65
[SPARK-5825] [Spark Submit] Remove the double checking instance name when
stopping the service
`spark-daemon.sh` will confirm the process id by fuzzy matching the class name
while stopping the service, however,
Repository: spark
Updated Branches:
refs/heads/branch-1.3 25fae8e7e - fe00eb66e
[SPARK-5825] [Spark Submit] Remove the double checking instance name when
stopping the service
`spark-daemon.sh` will confirm the process id by fuzzy matching the class name
while stopping the service, however,
Repository: spark
Updated Branches:
refs/heads/master 90095bf3c - 94cdb05ff
[SPARK-5825] [Spark Submit] Remove the double checking instance name when
stopping the service
`spark-daemon.sh` will confirm the process id by fuzzy matching the class name
while stopping the service, however, it
Repository: spark
Updated Branches:
refs/heads/branch-1.1 651ceaeb3 - 36f3c499f
[SPARK-5423][Core] Cleanup resources in DiskMapIterator.finalize to ensure
deleting the temp file
This PR adds a `finalize` method in DiskMapIterator to clean up the resources
even if some exception happens
901 - 1000 of 1431 matches
Mail list logo
