Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99634/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99634 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99634/testReport)**
for PR 23174 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99629/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99629 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99629/testReport)**
for PR 23174 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5691/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5691/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99634 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99634/testReport)**
for PR 23174 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
I looked at the test failure, but the logs weren't super useful. This
passed locally, but let me retrigger here.
---
-
To
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
Ok that's fine. Will merge to master if there are no further comments in
the near future.
---
-
To unsubscribe, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> It matters because we're discussing direction
I'm not, you guys are. I'm adding a missing feature with one particular
implementation. If you want to add other implementations that enable
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
It matters because we're discussing direction - that is, what opinion Spark
wants to take regarding how to set up security on K8s. It's not obvious from
our discussion on SPARK-26239 that we agree
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> with the caveat that we merge the subsequent optionality soon
Again, and sorry for pounding on that key, but why does that matter? It has
zero effect on the feature being added here. If
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5687/
---
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
Ok that's fine, with the caveat that we merge the subsequent optionality
soon. I'll work on the file-based secret authentication and encryption this
week. I'm very concerned that we'll ship with
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
I don't understand what you want.
Without this change, auth does not work, period.
With this, users at least have one choice.
If you want to add another choice, you're free
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
It's just to have the assurance that we will have some way to bypass this
for auth at least for 3.x. I'd like to concretely determine this before merging
if possible. But I hope that the suggestion
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
As I suggested before, any alternative method can be added later. I don't
see why does it need to block this PR.
---
-
To
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
I think as long as we have one alternate mechanism proposed in SPARK-26239
this is ok to merge. I proposed one in [this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5687/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99629 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99629/testReport)**
for PR 23174 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
So, can we move forward with this and let any future new feature be handled
in SPARK-26239?
---
-
To unsubscribe, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
I filed SPARK-26239.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> A proposed scheme is to have
spark.authenticate.k8s.secret.provider=autok8ssecret
If you're going to add a different way to get the auth secret later, then
you can introduce that option
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
The trouble is the API proposed here and how it would have to change for
future features. If we wanted to add the optionality to support authentication
via mounted files later, then what's the API
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> The way it's written now
Code can change after it's written...
> If this change is merged into 3.x without any other changes, users will
be forced to use the K8s secret based
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
> There doesn't need to be a single solution. This patch going in does not
preclude adding more features later, one of which might be reading this from a
pre-defined secret.
The way it's
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> while leaving it an exercise for the reader to understand how to properly
run spark such that the secrets are actually secured.
I don't think that's an exercise for the user, but for the
Github user gdearment commented on the issue:
https://github.com/apache/spark/pull/23174
The issue with requiring the use of secrets is quite a bit of work must be
done in order to secure a cluster to ensure that the secrets are themselves
secured. Most of the high level concerns are
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
(In fact, env variables don't even show up in the UI or event logs, as far
as I can see. Other configs - Spark config, system properties, e.g. - do show
up, and are redacted to mask secrets.)
---
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> if the secret would be listed under the environment variables in the
Spark UI
Secrets are redacted in the UI and event logs. We already use env variables
in other contexts (e.g.
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
> Why? And how are mounted files better?
Environment variables leak far more easily than file contents. One can
accidentally `printenv` in a shell attached to the and get the secret
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/23174
> via a mounted file
> Also the user should be able to specify their own mounted file
The point is that the user shouldn't need to set this at all. You enable
auth, Spark takes care of
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/23174
Would it be possible to also provide support for passing this via a mounted
file? Some users would prefer to avoid propagating sensitive information via
environment variables. Also the user should
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99402 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99402/testReport)**
for PR 23174 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99402/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5476/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23174
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5476/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23174
**[Test build #99402 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99402/testReport)**
for PR 23174 at commit
48 matches
Mail list logo
