Hi all, should we do this refactoring for the 1.2 release? If yes, I'll prepare a PR for that.
Cheers, Fabian 2016-09-26 13:55 GMT+02:00 Fabian Hueske <fhue...@gmail.com>: > Thanks everybody for your comments. > > I opened FLINK-4676 [1] for merging the connector modules. > > [1] https://issues.apache.org/jira/browse/FLINK-4676 > > 2016-09-26 13:17 GMT+02:00 Robert Metzger <rmetz...@apache.org>: > >> +1 good suggestion. >> >> On Mon, Sep 26, 2016 at 1:03 PM, Stephan Ewen <se...@apache.org> wrote: >> >> > The module would have both dependencies, but both are provided anyways, >> so >> > that would not be much of an issue, I think. >> > >> > On Mon, Sep 26, 2016 at 12:25 PM, Till Rohrmann <trohrm...@apache.org> >> > wrote: >> > >> > > I think this only holds true for modules which depend on the batch or >> > > streaming counter part, respectively. We could refactor these modules >> by >> > > pulling out common types which are independent of streaming/batch and >> are >> > > used by the batch and streaming module. >> > > >> > > Cheers, >> > > Till >> > > >> > > On Fri, Sep 23, 2016 at 11:15 AM, Aljoscha Krettek < >> aljos...@apache.org> >> > > wrote: >> > > >> > > > I don't think it's that easy. The streaming connectors have >> > > flink-streaming >> > > > as dependency while the batch connectors have the batch >> dependencies. >> > > > >> > > > Combining them would mean that users always have all dependencies, >> > right? >> > > > >> > > > On Thu, 22 Sep 2016 at 15:41 Stephan Ewen <se...@apache.org> wrote: >> > > > >> > > > > +1 for Fabian's suggestion >> > > > > >> > > > > On Thu, Sep 22, 2016 at 3:25 PM, Swapnil Chougule < >> > > > the.swapni...@gmail.com >> > > > > > >> > > > > wrote: >> > > > > >> > > > > > +1 >> > > > > > It will be good to have one module flink-connectors (union of >> > > streaming >> > > > > and >> > > > > > batch connectors). >> > > > > > >> > > > > > Regards, >> > > > > > Swapnil >> > > > > > >> > > > > > On Thu, Sep 22, 2016 at 6:35 PM, Fabian Hueske < >> fhue...@gmail.com> >> > > > > wrote: >> > > > > > >> > > > > > > Hi everybody, >> > > > > > > >> > > > > > > right now, we have two separate Maven modules for batch and >> > > streaming >> > > > > > > connectors (flink-batch-connectors and >> > flink-streaming-connectors) >> > > > that >> > > > > > > contain modules for the individual external systems and >> storage >> > > > formats >> > > > > > > such as HBase, Cassandra, Avro, Elasticsearch, etc. >> > > > > > > >> > > > > > > Some of these systems can be used in streaming as well as >> batch >> > > jobs >> > > > as >> > > > > > for >> > > > > > > instance HBase, Cassandra, and Elasticsearch. However, due to >> the >> > > > > > separate >> > > > > > > main modules for streaming and batch connectors, we currently >> > need >> > > to >> > > > > > > decide where to put a connector. For example, the >> > > > > > flink-connector-cassandra >> > > > > > > module is located in flink-streaming-connectors but includes a >> > > > > > > CassandraInputFormat and CassandraOutputFormat (i.e., a batch >> > > source >> > > > > and >> > > > > > > sink). >> > > > > > > >> > > > > > > In my opinion, it would be better to just merge >> > > > flink-batch-connectors >> > > > > > and >> > > > > > > flink-streaming-connectors into a joint flink-connectors >> module. >> > > > > > > >> > > > > > > This would be only an internal restructuring of code and not >> be >> > > > visible >> > > > > > to >> > > > > > > users (unless we change the module names of the individual >> > > connectors >> > > > > > which >> > > > > > > is not necessary, IMO). >> > > > > > > >> > > > > > > What do others think? >> > > > > > > >> > > > > > > Best, Fabian >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
