Probably should do 1, and then it is an easier transition in 3.0. On Sun, Oct 1, 2017 at 1:28 AM Sean Owen <so...@cloudera.com> wrote:
> I tried and failed to do this in > https://issues.apache.org/jira/browse/SPARK-22142 because it became clear > that the Flume examples would have to be removed to make this work, too. > (Well, you can imagine other solutions with extra source dirs or modules > for flume examples enabled by a profile, but that doesn't help the docs and > is nontrivial complexity for little gain.) > > It kind of suggests Flume support should be deprecated if it's put behind > a profile. Like with Kafka 0.8. (This is why I'm raising it again to the > whole list.) > > Any preferences among: > 1. Put Flume behind a profile, remove examples, deprecate > 2. Put Flume behind a profile, remove examples, but don't deprecate > 3. Punt until Spark 3.0, when this integration would probably be removed > entirely (?) > > On Tue, Sep 26, 2017 at 10:36 AM Sean Owen <so...@cloudera.com> wrote: > >> Not a big deal, but I'm wondering whether Flume integration should at >> least be opt-in and behind a profile? it still sees some use (at least on >> our end) but not applicable to the majority of users. Most other >> third-party framework integrations are behind a profile, like YARN, Mesos, >> Kinesis, Kafka 0.8, Docker. Just soliciting comments, not arguing for it. >> >> (Well, actually it annoys me that the Flume integration always fails to >> compile in IntelliJ unless you generate the sources manually) >> >
