dongjoon-hyun commented on code in PR #58757:
URL: https://github.com/apache/spark/pull/58757#discussion_r4001831028
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala:
##########
@@ -329,21 +329,21 @@ private[yarn] class YarnAllocator(
// to request YARN containers with extra resources without Spark
scheduling on
// them, the user can specify resources via the
<code>spark.yarn.executor.resource.</code>
// config. Those configs are only used in the base default profile
though and do
- // not get propogated into any other custom ResourceProfiles. This is
because
+ // not get propagated into any other custom ResourceProfiles. This is
because
// there would be no way to remove them if you wanted a stage to not
have them.
// This results in your default profile getting custom resources defined
in
// <code>spark.yarn.executor.resource.</code> plus spark defined
resources of
// GPU or FPGA. Spark converts GPU and FPGA resources into the YARN
built in
// types <code>yarn.io/gpu</code>) and <code>yarn.io/fpga</code>, but
does not
Review Comment:
nit. There is an unmatched `)` here.
```suggestion
// types <code>yarn.io/gpu</code> and <code>yarn.io/fpga</code>, but
does not
```
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala:
##########
@@ -329,21 +329,21 @@ private[yarn] class YarnAllocator(
// to request YARN containers with extra resources without Spark
scheduling on
// them, the user can specify resources via the
<code>spark.yarn.executor.resource.</code>
// config. Those configs are only used in the base default profile
though and do
- // not get propogated into any other custom ResourceProfiles. This is
because
+ // not get propagated into any other custom ResourceProfiles. This is
because
// there would be no way to remove them if you wanted a stage to not
have them.
// This results in your default profile getting custom resources defined
in
// <code>spark.yarn.executor.resource.</code> plus spark defined
resources of
// GPU or FPGA. Spark converts GPU and FPGA resources into the YARN
built in
// types <code>yarn.io/gpu</code>) and <code>yarn.io/fpga</code>, but
does not
// know the mapping of any other resources. Any other Spark custom
resources
- // are not propogated to YARN for the default profile. So if you want
Spark
+ // are not propagated to YARN for the default profile. So if you want
Spark
// to schedule based off a custom resource and have it requested from
YARN, you
// must specify it in both YARN
(<code>spark.yarn.{driver/executor}.resource.</code>)
// and Spark (<code>spark.{driver/executor}.resource.</code>) configs.
Leave the Spark
// config off if you only want YARN containers with the extra resources
but Spark not to
// schedule using them. Now for custom ResourceProfiles, it doesn't
currently have a way
// to only specify YARN resources without Spark scheduling off of them.
This means for
- // custom ResourceProfiles we propogate all the resources defined in the
ResourceProfile
+ // custom ResourceProfiles we propagate all the resources defined in the
ResourceProfile
// to YARN. We still convert GPU and FPGA to the YARN build in types as
well. This requires
Review Comment:
nit. `build in` -> `built in` (like line 336)
```suggestion
// to YARN. We still convert GPU and FPGA to the YARN built in types
as well. This requires
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
