Hi Sean, Thanks for the valuable feedback. Good point on not using dependency classes in public API parameters. One example is HADOOP-15502 <https://issues.apache.org/jira/browse/HADOOP-15502> (blame me for breaking the API)
>From what I know, the biggest risk is that downstreamers include dependencies from Hadoop implicitly. Therefore, if Hadoop updates a dependency that has a breaking change, the downstream application breaks, sometimes during compile time, sometimes at runtime. Jetty is probably not the best example. But take Guava as an example, when we updated Guava from 11.0 to 27.0, it breaks downstreamers like crazy -- Hive, Tez, Pheonix, Oozie all have to make changes. Probably not Hadoop's responsibility that they don't use shaded Hadoop client artifacts, but maybe we can add release note and state potential breaking changes. On Tue, Oct 22, 2019 at 7:43 AM Sean Busbey <bus...@cloudera.com.invalid> wrote: > speaking with my HBase hat on instead of my Hadoop hat, when the > Hadoop project publishes that there's a CVE but does not include a > maintenance release that mitigates it for a given minor release line, > we assume that means the Hadoop project is saying that release line is > EOM and should be abandoned. > > I don't know if that's an accurate interpretation in all cases. > > With my Hadoop hat on, I think downstream projects should use the > interfaces we say are safe to use and those interfaces should not > include dependencies where practical. I don't know how often a CVE > comes along for things like our logging API dependency, for example. > But downstream folks should definitely not rely on dependencies we use > for internal service, so I'm surprised that a version change for Jetty > would impact downstream. > > > On Mon, Oct 21, 2019 at 12:33 PM Wei-Chiu Chuang <weic...@apache.org> > wrote: > > > > Hi Hadoop developers, > > > > I've always had this question and I don't know the answer. > > > > For the last few months I finally spent time to deal with the > vulnerability > > reports from our internal dependency check tools. > > > > Say in HADOOP-16152 <https://issues.apache.org/jira/browse/HADOOP-16152> > > we update Jetty from 9.3.27 to 9.4.20 because of CVE-2019-16869, should I > > cherrypick the fix into all lower releases? > > This is not a trivial change, and it breaks downstreams like Tez. On the > > other hand, it doesn't seem reasonable if I put this fix only in trunk, > and > > left older releases vulnerable. What's the expectation of downstream > > applications w.r.t breaking compatibility vs fixing security issues? > > > > Thoughts? > > > > -- > busbey > > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
