This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 309638eeefb [MINOR][DOCS] Reviews and updates the doc links for
running-on-yarn
309638eeefb is described below
commit 309638eeefbfb13dae8dbded0279bf44390389ee
Author: panbingkun <pbk1...@gmail.com>
AuthorDate: Fri Oct 7 20:38:43 2022 -0500
[MINOR][DOCS] Reviews and updates the doc links for running-on-yarn
### What changes were proposed in this pull request?
The pr aim to reviews and updates the doc links for running-on-yarn.
### Why are the changes needed?
Improve docs.
After SPARK-39863, hadoop has been upgraded to version 3.3.4, but docs
still retains the old link.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually verified.
Closes #38129 from panbingkun/fix_doc_for_running-on-yarn.
Authored-by: panbingkun <pbk1...@gmail.com>
Signed-off-by: Sean Owen <sro...@gmail.com>
---
docs/running-on-yarn.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index 03179da115b..ea117f31357 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -163,7 +163,7 @@ To use a custom metrics.properties for the application
master and executors, upd
Amount of resource to use for the YARN Application Master in client mode.
In cluster mode, use
<code>spark.yarn.driver.resource.<resource-type>.amount</code> instead.
Please note that this feature can be used only with YARN 3.0+
- For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/r3.0.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
+ For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
<p/>
Example:
To request GPU resources from YARN, use:
<code>spark.yarn.am.resource.yarn.io/gpu.amount</code>
@@ -185,7 +185,7 @@ To use a custom metrics.properties for the application
master and executors, upd
<td>
Amount of resource to use for the YARN Application Master in cluster mode.
Please note that this feature can be used only with YARN 3.0+
- For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/r3.0.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
+ For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
<p/>
Example:
To request GPU resources from YARN, use:
<code>spark.yarn.driver.resource.yarn.io/gpu.amount</code>
@@ -198,7 +198,7 @@ To use a custom metrics.properties for the application
master and executors, upd
<td>
Amount of resource to use per executor process.
Please note that this feature can be used only with YARN 3.0+
- For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/r3.0.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
+ For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
<p/>
Example:
To request GPU resources from YARN, use:
<code>spark.yarn.executor.resource.yarn.io/gpu.amount</code>
@@ -956,7 +956,7 @@ Or
```
The two `spark-*-config` directories each contain one file,
`spark-shuffle-site.xml`. These are XML
-files in the [Hadoop Configuration
format](https://hadoop.apache.org/docs/r3.2.2/api/org/apache/hadoop/conf/Configuration.html)
+files in the [Hadoop Configuration
format](https://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html)
which each contain a few configurations to adjust the port number and metrics
name prefix used:
```xml
<configuration>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
