Re: Should Flume integration be behind a profile?

I agree, proposal 1 sounds better among the options. Regards, Mridul On Sun, Oct 1, 2017 at 3:50 PM, Reynold Xin wrote: > Probably should do 1, and then it is an easier transition in 3.0. > > On Sun, Oct 1, 2017 at 1:28 AM Sean Owen wrote: >> >> I

Re: Should Flume integration be behind a profile?

Probably should do 1, and then it is an easier transition in 3.0. On Sun, Oct 1, 2017 at 1:28 AM Sean Owen wrote: > I tried and failed to do this in > https://issues.apache.org/jira/browse/SPARK-22142 because it became clear > that the Flume examples would have to be removed

Re: [discuss] Data Source V2 write path

The main entries of data source inside Spark is the SQL API and `DataFrameReader/Writer`. For SQL API, I think the semantic is well defined, the data and metadata operations are separated. E.g., INSERT INTO means write data into an existing table, CREATE TABLE means only create the metadata. But

Re: Should Flume integration be behind a profile?

I tried and failed to do this in https://issues.apache.org/jira/browse/SPARK-22142 because it became clear that the Flume examples would have to be removed to make this work, too. (Well, you can imagine other solutions with extra source dirs or modules for flume examples enabled by a profile, but