On Fri, Feb 28, 2020 at 12:03 PM Holden Karau <hol...@pigscanfly.ca> wrote: >> 1. Could you estimate how many revert commits are required in >> `branch-3.0` for new rubric?
Fair question about what actual change this implies for 3.0? so far it seems like some targeted, quite reasonable reverts. I don't think anyone's suggesting reverting loads of changes. >> 2. Are you going to revert all removed test cases for the deprecated >> ones? > This is a good point, making sure we keep the tests as well is important > (worse than removing a deprecated API is shipping it broken),. (I'd say, yes of course! which seems consistent with what is happening now) >> 3. Does it make any delay for Apache Spark 3.0.0 release? >> (I believe it was previously scheduled on June before Spark Summit >> 2020) > > I think if we need to delay to make a better release this is ok, especially > given our current preview releases being available to gather community > feedback. Of course these things block 3.0 -- all the more reason to keep it specific and targeted -- but nothing so far seems inconsistent with finishing in a month or two. >> Although there was a discussion already, I want to make the following tough >> parts sure. >> 4. We are not going to add Scala 2.11 API, right? > I hope not. >> >> 5. We are not going to support Python 2.x in Apache Spark 3.1+, right? > I think doing that would be bad, it's already end of lifed elsewhere. Yeah this is an important subtext -- the valuable principles here could be interpreted in many different ways depending on how much you weight value of keeping APIs for compatibility vs value in simplifying Spark and pushing users to newer APIs more forcibly. They're all judgment calls, based on necessarily limited data about the universe of users. We can only go on rare direct user feedback, on feedback perhaps from vendors as proxies for a subset of users, and the general good faith judgment of committers who have lived Spark for years. My specific interpretation is that the standard is (correctly) tightening going forward, and retroactively a bit for 3.0. But, I do not think anyone is advocating for the logical extreme of, for example, maintaining Scala 2.11 compatibility indefinitely. I think that falls out readily from the rubric here: maintaining 2.11 compatibility is really quite painful if you ever support 2.13 too, for example. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
