There's only one other item on my radar, which is considering updating Jackson to 2.9 in branch-2.4 to get security fixes. Pros: it's come up a few times now that there are a number of CVEs open for 2.6.7. Cons: not clear they affect Spark, and Jackson 2.6->2.9 does change Jackson behavior non-trivially. That said back-porting the update PR to 2.4 worked out OK locally. Any strong opinions on this one?
On Wed, Apr 17, 2019 at 7:49 PM Wenchen Fan <cloud0...@gmail.com> wrote: > > I volunteer to be the release manager for 2.4.2, as I was also going to > propose 2.4.2 because of the reverting of SPARK-25250. Is there any other > ongoing bug fixes we want to include in 2.4.2? If no I'd like to start the > release process today (CST). > > Thanks, > Wenchen > > On Thu, Apr 18, 2019 at 3:44 AM Sean Owen <sro...@gmail.com> wrote: >> >> I think the 'only backport bug fixes to branches' principle remains sound. >> But what's a bug fix? Something that changes behavior to match what is >> explicitly supposed to happen, or implicitly supposed to happen -- implied >> by what other similar things do, by reasonable user expectations, or simply >> how it worked previously. >> >> Is this a bug fix? I guess the criteria that matches is that behavior >> doesn't match reasonable user expectations? I don't know enough to have a >> strong opinion. I also don't think there is currently an objection to >> backporting it, whatever it's called. >> >> >> Is the question whether this needs a new release? There's no harm in another >> point release, other than needing a volunteer release manager. One could >> say, wait a bit longer to see what more info comes in about 2.4.1. But given >> that 2.4.1 took like 2 months, it's reasonable to move towards a release >> cycle again. I don't see objection to that either (?) >> >> >> The meta question remains: is a 'bug fix' definition even agreed, and being >> consistently applied? There aren't correct answers, only best guesses from >> each person's own experience, judgment and priorities. These can differ even >> when applied in good faith. >> >> Sometimes the variance of opinion comes because people have different info >> that needs to be surfaced. Here, maybe it's best to share what about that >> offline conversation was convincing, for example. >> >> I'd say it's also important to separate what one would prefer from what one >> can't live with(out). Assuming one trusts the intent and experience of the >> handful of others with an opinion, I'd defer to someone who wants X and will >> own it, even if I'm moderately against it. Otherwise we'd get little done. >> >> In that light, it seems like both of the PRs at issue here are not _wrong_ >> to backport. This is a good pair that highlights why, when there isn't a >> clear reason to do / not do something (e.g. obvious errors, breaking public >> APIs) we give benefit-of-the-doubt in order to get it later. >> >> >> On Wed, Apr 17, 2019 at 12:09 PM Ryan Blue <rb...@netflix.com.invalid> wrote: >>> >>> Sorry, I should be more clear about what I'm trying to say here. >>> >>> In the past, Xiao has taken the opposite stance. A good example is PR >>> #21060 that was a very similar situation: behavior didn't match what was >>> expected and there was low risk. There was a long argument and the patch >>> didn't make it into 2.3 (to my knowledge). >>> >>> What we call these low-risk behavior fixes doesn't matter. I called it a >>> bug on #21060 but I'm applying Xiao's previous definition here to make a >>> point. Whatever term we use, we clearly have times when we want to allow a >>> patch because it is low risk and helps someone. Let's just be clear that >>> that's perfectly fine. >>> >>> On Wed, Apr 17, 2019 at 9:34 AM Ryan Blue <rb...@netflix.com> wrote: >>>> >>>> How is this a bug fix? >>>> >>>> On Wed, Apr 17, 2019 at 9:30 AM Xiao Li <lix...@databricks.com> wrote: >>>>> >>>>> Michael and I had an offline discussion about this PR >>>>> https://github.com/apache/spark/pull/24365. He convinced me that this is >>>>> a bug fix. The code changes of this bug fix are very tiny and the risk is >>>>> very low. To avoid any behavior change in the patch releases, this PR >>>>> also added a legacy flag whose default value is off. >>>>> --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
