Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
I am observing some weird behaviour when i am trying to respond to the
comments. Hence i am adding the resposes to comments as below.
Following are the responses for the comments
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
Can some body pls merge this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user suryag10 reopened a pull request:
https://github.com/apache/spark/pull/22433
[SPARK-25442][SQL][K8S] Support STS to run in k8s deployments with spark
deployment mode as cluster.
## What changes were proposed in this pull request?
Code is enhanced to allow
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
Can some body pls merge this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> If possible, there should be some basic integration testing. Run a thrift
server command against the minishift cluster used by the other testing.
Will add this a separate
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> In the scenario of a cluster-mode submission, what is the command-line
behavior? Does the thrift-server script "block" until the thrift server pod is
shut down?
By defau
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> In the scenario of a cluster-mode submission, what is the command-line
behavior? Does the thrift-server script "block" until the thrift server pod is
shut down?
By defau
Github user suryag10 closed the pull request at:
https://github.com/apache/spark/pull/22433
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user suryag10 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22433#discussion_r226898234
--- Diff: docs/running-on-kubernetes.md ---
@@ -340,6 +340,43 @@ RBAC authorization and how to configure Kubernetes
service accounts for pods, pl
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
@liyinan926
>>The script may be run from a client machine outside a k8s cluster. In
this case, there's not even a pod. >>I would suggest separating the explanation
of the user
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
@liyinan926
>>The script may be run from a client machine outside a k8s cluster. In
this case, there's not even a pod. >>I would suggest separating the explanation
of the user
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/21669
> like it, but we could also first support cluster mode and add client mode
after.
Thats the reason i said "Point
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/21669
Hi Ilan,
Point to note, this kerberos support for spark k8s is working only for the
cluster mode deployment. As client mode is also supported now in spark K8S, we
should plan for supporting
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
can somebody pls review and merge?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user suryag10 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22433#discussion_r219476048
--- Diff: docs/running-on-kubernetes.md ---
@@ -340,6 +340,39 @@ RBAC authorization and how to configure Kubernetes
service accounts for pods, pl
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
@mridulm @liyinan926 @jacobdr @ifilonenko
code check for space,"/" handling is already present at
https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> > Agreed with @mridulm that the naming restriction is specific to k8s and
should be handled in a k8s specific way, e.g., somewhere around
https://github.com/apache/spark/blob/master/re
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> Agreed with @mridulm that the naming restriction is specific to k8s and
should be handled in a k8s specific way, e.g., somewhere around
https://github.com/apache/spark/blob/master/resou
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/20272
> If this can get a rebase, and maybe a few sentences in the k8s docs, I'll
merge.
Hi, I had rebased this patch to the latest master and this patch was
missing another fix to run the
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> I'm wondering, is there some reason this isn't supported in cluster mode
for yarn & mesos? Or put another way, what is the rationale for k8s being added
as an exception to this rule?
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> It is an implementation detail of k8s integration that application name
is expected to be DNS compliant ... spark does not have that requirement; and
yarn/mesos/standalone/local work with
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> Does it fail in k8s or does spark k8s code error out ?
> If former, why not fix ânameâ handling in k8s to replace unsupported
characters ?
Following is the error seen w
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/22433
> Thank you for your first contribution, @suryag10 .
>
> * Could you file a SPARK JIRA issue since this is a code change?
Sure.
> * Could you update the PR title like t
GitHub user suryag10 opened a pull request:
https://github.com/apache/spark/pull/22433
Support STS to run in k8s cluster mode
## What changes were proposed in this pull request?
Code is enhanced to allow the STS run in kubernetes deployment with spark
deploy mode
Github user suryag10 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r215338063
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/hadoopsteps/HadoopBootstrapUtil.scala
---
@@ -0,0 +1,186
Github user suryag10 commented on the issue:
https://github.com/apache/spark/pull/21669
Hi Ilan,
I was able to make work the Kerberos with one work around(which I am trying
to do a full fix) and one fix.
Fix is the one which i had commented earlier and is as follows
Github user suryag10 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r213408857
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/hadoopsteps/HadoopBootstrapUtil.scala
---
@@ -0,0 +1,186
28 matches
Mail list logo
