Re: Removing references to slave (and maybe in the future master)

I've created SPARK-32036 and SPARK-32037 for changes related to "blacklist"/"whitelist" terminology, the latter of which focuses on the blacklisting feature. I invite all of you to participate

Re: Removing references to slave (and maybe in the future master)

Thanks for getting this started! I think it will be worth the effort, and it's great to get started early in the 3.x release line to give the most time to prepare for this. On Thu, Jun 18, 2020 at 3:44 PM Russell Spitzer wrote: > I really dislike the use of "worker" in the code base since it

Re: Removing references to slave (and maybe in the future master)

I really dislike the use of "worker" in the code base since it describes a process which doesn't actually do work, but I don't think it's in the scope for this ticket. I would definitely prefer we use "agent" instead of "worker" (or some other name) and have master switched to something like

Re: Removing references to slave (and maybe in the future master)

Thanks a lot for proposing this, Holden. I'd be curious to know how others feel about also tackling the word blacklist -- while I think most would agree it is not as egregious as master/slave, it seems to be an appropriate time to use the momentum to really a make a best effort at removing any

Re: Removing references to slave (and maybe in the future master)

So I think using Worker everywhere would be a bit confusing since the relationship between worker and blockmanager replica is complex, also in the current PR `AgentLost` is not `WorkerLost` because it doesn't necessarily mean the worker is lost (there's a flag for if the worker has been lost). On

Re: Removing references to slave (and maybe in the future master)

Yup, it would be great to do this. FWIW, I would propose using “worker” everywhere instead unless it already means something in that context, just to have a single word for this (instead of multiple words such as agent, replica, etc), but I haven’t looked into whether that would make anything

Re: Removing references to slave (and maybe in the future master)

Thank you. I agree being careful with API comparability is important. I think in situations where the terms are exposed in our API we can introduce alternatives and deprecate the old ones to allow for a smooth migration. On Thu, Jun 18, 2020 at 12:28 PM Reynold Xin wrote: > Thanks for doing

Re: Removing references to slave (and maybe in the future master)

Thanks for doing this. I think this is a great thing to do. But we gotta be careful with API compatibility. On Thu, Jun 18, 2020 at 11:32 AM, Holden Karau < hol...@pigscanfly.ca > wrote: > > Hi Folks, > > > I've started working on cleaning up the Spark code to remove references to > slave

Removing references to slave (and maybe in the future master)

Hi Folks, I've started working on cleaning up the Spark code to remove references to slave since the word has a lot of negative connotations and we can generally replace it with more accurate/descriptive words in our code base. The PR is at https://github.com/apache/spark/pull/28864 (I'm a little