shubhluck opened a new pull request, #58078:
URL: https://github.com/apache/spark/pull/58078
…nd deploy-mode=cluster
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR:
https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a
faster review.
7. If you want to add a new configuration, please read the guideline first
for naming configurations in
'common/utils/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
8. If you want to add or modify an error type or message, please read the
guideline first in
'common/utils/src/main/resources/error/README.md'.
-->
### What changes were proposed in this pull request?
Fix `SecurityManager.initializeAuth()` so that in standalone cluster mode it
accepts the auth secret from the `_SPARK_AUTH_SECRET` environment variable in
addition to the `spark.authenticate.secret `config key.
Today, the non-YARN / non-local / non-K8s branch of `initializeAuth()` only
checks `sparkConf.contains(SPARK_AUTH_SECRET_CONF)`:
```
case _ =>
require(sparkConf.contains(SPARK_AUTH_SECRET_CONF),
s"A secret key must be specified via the $SPARK_AUTH_SECRET_CONF
config.")
return
```
This is inconsistent with `getSecretKey()` a few lines above, which already
reads from both sources — conf and `_SPARK_AUTH_SECRET` env var — before
failing. The check is broadened to match:
```
require(sparkConf.contains(SPARK_AUTH_SECRET_CONF) ||
sparkConf.getenv(ENV_AUTH_SECRET) != null,
s"A secret key must be specified via the $SPARK_AUTH_SECRET_CONF config.")
```
No other code path, master type, wire format, or secret-propagation
mechanism is touched. The actual secret used for the SASL handshake still comes
from the same source it always has (`getSecretKey()` — unchanged).
<!--
Please clarify what changes you are proposing. The purpose of this section
is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster
reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class
hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other
DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<p style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif; font-size:
13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align:
start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;">This fixes SPARK-25078 (<a
href="https://issues.apache.org/jira/browse/SPARK-25078"; target="_blank"
rel="noopener noreferrer" style="color: rgb(77, 170, 252); text-decoration:
rgb(77, 170, 252);">https://issues.apache.org/jira/browse/SPARK-25078</a>).
Submitting a driver in standalone cluster mode with authentication enabled
fails immediately even when everything is
configured correctly:</p><div class="codeBlockWrapper_-a7MRw" style="position:
relative; margin: 8px 0px; color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; font-style: normal; font-variant-ligatures: normal;
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans:
2; text-align: start; text-indent: 0px; text-transform: none; widows: 2;
word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal;
background-color: rgb(31, 31, 31); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;"><button
class="copyButton_CEmTFw copyButton_-a7MRw" title="Copy code" aria-label="Copy
code to clipboard" style="color: rgb(204, 204, 204); font-family:
-apple-system, "system-ui", "Segoe UI", Roboto, sans-serif;
font-size: 13px; background: none 0% 0% / auto repeat scroll padding-box
border-box rgb(31, 31, 31); border-co
lor: rgb(69, 69, 69); border-style: solid; border-width: 1px; border-image:
none 100% / 1 / 0 stretch; cursor: pointer; opacity: 1; display: flex;
border-radius: 4px; justify-content: center; align-items: center; padding: 4px;
transition: opacity 0.15s, background 0.15s; position: absolute; top: 4px;
right: 4px;"><svg xmlns="http://www.w3.org/2000/svg"; viewBox="0 0 20 20"
fill="currentColor" aria-hidden="true" data-slot="icon"
class="copyIcon_CEmTFw"><path fill-rule="evenodd" d="M15.988 3.012A2.25 2.25 0
0 1 18 5.25v6.5A2.25 2.25 0 0 1 15.75 14H13.5v-3.379a3 3 0 0
0-.879-2.121l-3.12-3.121a3 3 0 0 0-1.402-.791 2.252 2.252 0 0 1
1.913-1.576A2.25 2.25 0 0 1 12.25 1h1.5a2.25 2.25 0 0 1 2.238 2.012ZM11.5
3.25a.75.75 0 0 1 .75-.75h1.5a.75.75 0 0 1 .75.75v.25h-3v-.25Z"
clip-rule="evenodd"></path><path d="M3.5 6A1.5 1.5 0 0 0 2 7.5v9A1.5 1.5 0 0 0
3.5 18h7a1.5 1.5 0 0 0 1.5-1.5v-5.879a1.5 1.5 0 0 0-.44-1.06L8.44 6.439A1.5 1.5
0 0 0 7.378 6H3.5Z"></path></svg></button><pre style="overflow-x:
auto; white-space: pre; box-sizing: border-box; border-radius: 4px;
max-width: 100%; margin: 0px; padding: 8px;"><code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 0px; border-radius: 3px; word-break: break-word; font-size: 0.9em;">$
spark-submit \
--class org.apache.spark.examples.SparkPi \
--master spark://host:7077 \
--deploy-mode cluster \
--conf spark.authenticate=true \
--conf spark.authenticate.secret=secret \
/path/to/spark-examples.jar 100
Driver stderr:
java.lang.IllegalArgumentException:
A secret key must be specified via the spark.authenticate.secret config.
</code></pre></div><p style="white-space: pre-wrap; margin-top: 0.1em;
margin-bottom: 0.2em; unicode-bidi: plaintext; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400;
letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width:
0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;">The
reason:</p><ol style="padding-inline-start: 2em; color: rgb(204, 204, 204);
font-family: -apple-system, "system-ui", "Segoe UI",
Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400;
letter-spacing: normal; orphans: 2; text-align: start; text-ind
ent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(31,
31, 31); text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;"><li style="unicode-bidi: plaintext;"><code
style="font-family: monospace; color: rgb(208, 208, 208); background-color:
rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px; word-break: break-word;
font-size: 0.9em;">spark-submit</code><span> </span>deliberately strips<span>
</span><code style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size:
0.9em;">spark.authenticate.secret</code><span> </span>from the
SparkConf/javaOpts before the driver process is launched — see<span>
</span><code style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-brea
k: break-word; font-size: 0.9em;">CommandUtils.buildCommandSeq</code><span>
</span>in<span> </span><code style="font-family: monospace; color: rgb(208,
208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius:
3px; word-break: break-word; font-size:
0.9em;">core/src/main/scala/org/apache/spark/deploy/worker/CommandUtils.scala</code>,
which both:<ul style="padding-inline-start: 2em;"><li style="unicode-bidi:
plaintext;">filters<span> </span><code style="font-family: monospace; color:
rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px;
border-radius: 3px; word-break: break-word; font-size:
0.9em;">-Dspark.authenticate.secret=…</code><span> </span>out of<span>
</span><code style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size: 0.9em;">javaOpts</code>, and</li><li
style="unicode-bidi: plaintext;">injects the master/worker-side se
cret into the driver's environment as<span> </span><code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">_SPARK_AUTH_SECRET</code>.</li></ul></li><li style="unicode-bidi:
plaintext;">So by design, in standalone cluster mode the driver receives the
secret via the env var, not via SparkConf.</li><li style="unicode-bidi:
plaintext;"><code style="font-family: monospace; color: rgb(208, 208, 208);
background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius: 3px;
word-break: break-word; font-size: 0.9em;">initializeAuth()</code><span>
</span>only inspects the conf and throws before ever consulting the env var —
even though<span> </span><code style="font-family: monospace; color: rgb(208,
208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius:
3px; word-break: break-word; font-size: 0.9em;">getSecretKey()</code><span>
</span>later
would have found it there.</li></ol><p style="white-space: pre-wrap;
margin-top: 0.1em; margin-bottom: 0.2em; unicode-bidi: plaintext; color:
rgb(204, 204, 204); font-family: -apple-system, "system-ui",
"Segoe UI", Roboto, sans-serif; font-size: 13px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400;
letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width:
0px; background-color: rgb(31, 31, 31); text-decoration-thickness: initial;
text-decoration-style: initial; text-decoration-color: initial;">Standalone
cluster mode + authentication has been unusable out-of-the-box since the
introduction of this validation. The only known workaround is programmatically
injecting a dummy config value inside the driver code (<code
style="font-family: monospace; color: rgb(208, 208, 208); background-color:
rgb(60, 60, 60); padding: 2px 4px;
border-radius: 3px; word-break: break-word; font-size:
0.9em;">conf.setIfMissing("spark.authenticate.secret", "placeholder")</code>)
purely to satisfy the misplaced check; the real secret from the env var is used
for the actual handshake anyway. That workaround requires modifying every
application, which is neither discoverable nor viable for prebuilt jars.</p><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif; font-size:
13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align:
start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;">There i
s no security implication: the fix does not introduce a new secret source,
does not change what travels over the network, and does not weaken the require
— it only aligns the pre-check with the source <code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">getSecretKey()</code> was already trusting.</p><h3 style="unicode-bidi:
plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif; font-style:
normal; font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px;
text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width:
0px; white-space: normal; background-color: rgb(31, 31, 31);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;">Does this PR intro
duce<span> </span><em>any</em><span> </span>user-facing change?</h3><p
style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif; font-size:
13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align:
start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;">Yes — but only in the fix direction. Behavior
matrix for the standalone (<code style="font-family: monospace; color: rgb(208,
208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px; border-radius:
3px; word-break: break-word; font-size: 0.9em;">spark://…</code>) branch of
<code style="fon
t-family: monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60,
60); padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">initializeAuth()</code> with <code style="font-family: monospace;
color: rgb(208, 208, 208); background-color: rgb(60, 60, 60); padding: 2px 4px;
border-radius: 3px; word-break: break-word; font-size:
0.9em;">spark.authenticate=true</code>:</p>
Secret source at driver startup | Before | After
-- | -- | --
spark.authenticate.secret in SparkConf | pass | pass
_SPARK_AUTH_SECRET env var only (standalone cluster mode) | throws
IllegalArgumentException | passes
Both present | pass | pass
Neither present | throws | throws
<p style="white-space: pre-wrap; margin-top: 0.1em; margin-bottom: 0.2em;
unicode-bidi: plaintext; color: rgb(204, 204, 204); font-family: -apple-system,
"system-ui", "Segoe UI", Roboto, sans-serif; font-size:
13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align:
start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(31, 31, 31);
text-decoration-thickness: initial; text-decoration-style: initial;
text-decoration-color: initial;">Net effect: <code style="font-family:
monospace; color: rgb(208, 208, 208); background-color: rgb(60, 60, 60);
padding: 2px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">spark-submit --master spark://… --deploy-mode cluster</code> with <code
style="font-family: monospace; color: rgb(208, 208, 208); background-color:
rgb(60, 60, 60); padding: 2
px 4px; border-radius: 3px; word-break: break-word; font-size:
0.9em;">spark.authenticate=true</code> now works without the in-code
dummy-secret workaround. No configuration change is required from users.</p>
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### How was this patch tested?
Two new unit tests in
core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala:
`SPARK-25078: standalone cluster mode accepts secret from env var `— sets
master to spark://…, provides the secret only via _SPARK_AUTH_SECRET (using the
existing SparkConfWithEnv helper), and asserts that initializeAuth() succeeds
and getSecretKey() returns the env-var value.
`SPARK-25078: standalone cluster mode still fails when no secret is provided
`— regression guard confirming that with neither the conf key nor the env var
set, initializeAuth() still throws IllegalArgumentException.
Run locally:
`./build/sbt "core/testOnly org.apache.spark.SecurityManagerSuite"`
Also manually validated on a standalone cluster: `spark-submit --deploy-mode
cluster --conf spark.authenticate=true --conf spark.authenticate.secret=`…
succeeds after the patch and fails before it, matching the reproducer above.
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
If benchmark tests were added, please run the benchmarks in GitHub Actions
for the consistent environment, and the instructions could accord to:
https://spark.apache.org/developer-tools.html#github-workflow-benchmarks.
-->
### Was this patch authored or co-authored using generative AI tooling?
No
<!--
If generative AI tooling has been used in the process of authoring this
patch, please include the
phrase: 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
