I'd agree with #1 or #2. Deprecation now seems fine. Perhaps this should be raised on the user list also?
And perhaps it makes sense to look at moving the Flume support into Apache Bahir if there is interest (I've cc'ed Bahir dev list here)? That way the current state of the connector could keep going for those users who may need it. As for examples, for the Kinesis connector the examples now live in the subproject (see e.g. KinesisWordCountASL under external/kinesis-asl). So we don't have to completely remove the examples, just move them (this may not solve the doc issue but at least the examples are still there for anyone who needs them). On Mon, 2 Oct 2017 at 06:36 Mridul Muralidharan <mri...@gmail.com> wrote: > I agree, proposal 1 sounds better among the options. > > Regards, > Mridul > > > On Sun, Oct 1, 2017 at 3:50 PM, Reynold Xin <r...@databricks.com> wrote: > > Probably should do 1, and then it is an easier transition in 3.0. > > > > On Sun, Oct 1, 2017 at 1:28 AM Sean Owen <so...@cloudera.com> wrote: > >> > >> I tried and failed to do this in > >> https://issues.apache.org/jira/browse/SPARK-22142 because it became > clear > >> that the Flume examples would have to be removed to make this work, too. > >> (Well, you can imagine other solutions with extra source dirs or > modules for > >> flume examples enabled by a profile, but that doesn't help the docs and > is > >> nontrivial complexity for little gain.) > >> > >> It kind of suggests Flume support should be deprecated if it's put > behind > >> a profile. Like with Kafka 0.8. (This is why I'm raising it again to the > >> whole list.) > >> > >> Any preferences among: > >> 1. Put Flume behind a profile, remove examples, deprecate > >> 2. Put Flume behind a profile, remove examples, but don't deprecate > >> 3. Punt until Spark 3.0, when this integration would probably be removed > >> entirely (?) > >> > >> On Tue, Sep 26, 2017 at 10:36 AM Sean Owen <so...@cloudera.com> wrote: > >>> > >>> Not a big deal, but I'm wondering whether Flume integration should at > >>> least be opt-in and behind a profile? it still sees some use (at least > on > >>> our end) but not applicable to the majority of users. Most other > third-party > >>> framework integrations are behind a profile, like YARN, Mesos, Kinesis, > >>> Kafka 0.8, Docker. Just soliciting comments, not arguing for it. > >>> > >>> (Well, actually it annoys me that the Flume integration always fails to > >>> compile in IntelliJ unless you generate the sources manually) > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
