CCing user@ Yeah good point about perhaps moving the examples into the module itself. Actually removing it would be a long way off, no matter what.
On Mon, Oct 2, 2017 at 8:35 AM Nick Pentreath <nick.pentre...@gmail.com> wrote: > I'd agree with #1 or #2. Deprecation now seems fine. > > Perhaps this should be raised on the user list also? > > And perhaps it makes sense to look at moving the Flume support into Apache > Bahir if there is interest (I've cc'ed Bahir dev list here)? That way the > current state of the connector could keep going for those users who may > need it. > > As for examples, for the Kinesis connector the examples now live in the > subproject (see e.g. KinesisWordCountASL under external/kinesis-asl). So we > don't have to completely remove the examples, just move them (this may not > solve the doc issue but at least the examples are still there for anyone > who needs them). > > On Mon, 2 Oct 2017 at 06:36 Mridul Muralidharan <mri...@gmail.com> wrote: > >> I agree, proposal 1 sounds better among the options. >> >> Regards, >> Mridul >> >> >> On Sun, Oct 1, 2017 at 3:50 PM, Reynold Xin <r...@databricks.com> wrote: >> > Probably should do 1, and then it is an easier transition in 3.0. >> > >> > On Sun, Oct 1, 2017 at 1:28 AM Sean Owen <so...@cloudera.com> wrote: >> >> >> >> I tried and failed to do this in >> >> https://issues.apache.org/jira/browse/SPARK-22142 because it became >> clear >> >> that the Flume examples would have to be removed to make this work, >> too. >> >> (Well, you can imagine other solutions with extra source dirs or >> modules for >> >> flume examples enabled by a profile, but that doesn't help the docs >> and is >> >> nontrivial complexity for little gain.) >> >> >> >> It kind of suggests Flume support should be deprecated if it's put >> behind >> >> a profile. Like with Kafka 0.8. (This is why I'm raising it again to >> the >> >> whole list.) >> >> >> >> Any preferences among: >> >> 1. Put Flume behind a profile, remove examples, deprecate >> >> 2. Put Flume behind a profile, remove examples, but don't deprecate >> >> 3. Punt until Spark 3.0, when this integration would probably be >> removed >> >> entirely (?) >> >> >> >> On Tue, Sep 26, 2017 at 10:36 AM Sean Owen <so...@cloudera.com> wrote: >> >>> >> >>> Not a big deal, but I'm wondering whether Flume integration should at >> >>> least be opt-in and behind a profile? it still sees some use (at >> least on >> >>> our end) but not applicable to the majority of users. Most other >> third-party >> >>> framework integrations are behind a profile, like YARN, Mesos, >> Kinesis, >> >>> Kafka 0.8, Docker. Just soliciting comments, not arguing for it. >> >>> >> >>> (Well, actually it annoys me that the Flume integration always fails >> to >> >>> compile in IntelliJ unless you generate the sources manually) >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >>
