[
https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599258#comment-16599258
]
Stavros Kontopoulos edited comment on SPARK-24434 at 8/31/18 8:47 PM:
----------------------------------------------------------------------
[~eje] Personally I will just stick to the facts (the ones I am aware of):
1) Several weeks ago I asked if this feature (in one of the meetings) should
go in 2.4 and you responded that this cannot be the case, as it needs testing
etc. I had no objection. It seems now that Palantir is pushing this for their
own reasons and will be marked as experimental. The PR created though is not
that big (not complete yet if you ask me). Given that I suspect we had time
back then even with the old dates of the 2.4 cut to make a similar PR. Next
time will push harder.
2) Before I leave on vacations I left a comment on our slack channel, not to
mention the explicit comment in this Jira above:
Stavros [3:27 PM]
@liyinan926 @eje I am working on the pod template PR but i will be off for a
couple of weeks, work more on that after.
eje [5:04 PM]
@Stavros thanks!
liyinan926 [7:29 PM]
@Stavros Thanks for working on that!
As you see you agreed that im working on it no?
In any other Jira I have seen before, people just need to state that they are
working on something. They dont need to create a WIP PR AFAIK (Next time I will
just commit a few lines of code to declare assignment ).
3) Copying again from the meeting notes:
[https://docs.google.com/document/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit]
15th of August
* Custom YAML
* Stavros on vacation
* Complete it without him?
* Palantir will look at completing
I missed that part sorry. I was expecting some update on the Jira ticket,
because I only checked emails on my vacations for good or bad. You could have
pinged me though not just do it without me. I created the design doc dont you
think I want to finish the work?
4) I almost always join the meetings and im active on the project. But nobody
me pinged AFAIK. Fine.
5) Palantir guys didnt update the Jira so all people (outside meetings) know
the status of things, also in the minutes doc I dont see any decision about who
is going to do the PR.
I think the reasonable thing to do is ask what I have done, so people dont do
double effort.
If the whole thing looks ok in terms of collaboration, then fine, my
misunderstanding then, will adapt. It is not awkward at all, people on the call
decided to assign it to Palantir guys without me knowing anything, that's all.
Nobody is obliged to inform me about anything, but I took it for granted that
this is the case when collaborating in a healthy community.
was (Author: skonto):
[~eje] Personally I will just stick to the facts (the ones I am aware of):
1) Several weeks ago I asked if this feature (in one of the meetings) should
go in 2.4 and you responded that this cannot be the case, as it needs testing
etc. I had no objection. It seems now that Palantir is pushing this for their
own reasons and will be marked as experimental. The PR created though is not
that big (not complete yet if you ask me). Given that I suspect we had time
back then even with the old dates of the 2.4 cut to make a similar PR. Next
time will push harder.
2) Before I leave on vacations I left a comment on our slack channel, not to
mention the explicit comment in this Jira above:
Stavros [3:27 PM]
@liyinan926 @eje I am working on the pod template PR but i will be off for a
couple of weeks, work more on that after.
eje [5:04 PM]
@Stavros thanks!
liyinan926 [7:29 PM]
@Stavros Thanks for working on that!
As you see you agreed that im working on it no?
In any other Jira I have seen before, people just need to state that they are
working on something. They dont need to create a WIP PR AFAIK (Next time I will
just commit a few lines of code to declare assignment ).
3) Copying again from the meeting notes:
[https://docs.google.com/document/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit]
15th of August
* Custom YAML
* Stavros on vacation
* Complete it without him?
* Palantir will look at completing
I missed that part sorry. I was expecting some update on the Jira ticket,
because I only checked emails on my vacations for good or bad. You could have
pinged me though not just do it without me. I created the design doc dont you
think I want to finish the work?
4) I almost always join the meetings and im active on the project. But nobody
me pinged AFAIK. Fine.
5) Palantir guys didnt update the Jira so all people (outside meetings) know
the status of things, also in the minutes doc I dont see any decision about who
is going to do the PR.
I think the reasonable thing to do is ask what I have done, so people dont do
double effort.
If the whole thing looks ok in terms of collaboration, then fine, my
misunderstanding then, will adapt. It is not awkward at all, people on the call
decided to assign it to Palantir guys without me knowing anything, that's all.
Nobody is obliged to inform me about anything, but I thought that is normal
when collaborate in a healthy community.
> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>
> Key: SPARK-24434
> URL: https://issues.apache.org/jira/browse/SPARK-24434
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Yinan Li
> Priority: Major
>
> With more requests for customizing the driver and executor pods coming, the
> current approach of adding new Spark configuration options has some serious
> drawbacks: 1) it means more Kubernetes specific configuration options to
> maintain, and 2) it widens the gap between the declarative model used by
> Kubernetes and the configuration model used by Spark. We should start
> designing a solution that allows users to specify pod templates as central
> places for all customization needs for the driver and executor pods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
