Hi Abhishek, For #b, I confirm this contribution will be under ASF2.0 license, and all code files have proper license headers.
Regards, Chris > Hi Chris, > > Thanks for starting this thread, and for your contribution. My thoughts: > 1. I like the idea of keeping Gobblin's connectors on a separate sub-repo, > because these are fairly independent pluggable connectors. > 2. They can bring in a significant amount of dependencies, which are better > kept isolated from core. > > As for the next steps: > a. I request Gobblin committers and PMCs to review this repo to ensure it's > in a shape to add to Gobblin (including code quality, structure, > dependencies it brings in, etc.) > b. Chris, please confirm that this contribution will be under Apache 2.0 > license and the code already has the right license headers. > > Once, we have an affirmative answer for (a), and (b). I will coordinate > with Apache Infra to set up a sub-repo for this in Gobblin, and work with > Chris to bring it in. > > Thanks, > Abhishek > > > On Mon, Mar 22, 2021 at 5:41 PM Chris Li <[email protected]> wrote: > Proposal: > > DIL (LinkedIn internal project name) is a generic multi-stage Gobblin > connector library. The code can be accessed here: > https://github.com/linkedin/gobblin-connectors. Its core features and > high level descriptions are shared here: > https://engineering.linkedin.com/blog/2021/data-integration-library. > > Per initial discussion with members of Gobblin community, we are here > proposing a separate sub-repo for this library. > > Why: > Some thoughts/justifications of a sub-repo vs. a module in the > main Gobblin repo. > > > 1. Gobblin connectors are important part of Gobblin ecosystem, but the > development of connectors is relatively independent of Gobblin core. > 2. Gobblin connector is where open source communities can contribute > the most, and it will be growing much faster than Gobblin core. > 3. The new connector library is a comprehensive package of unique > design patterns. This is where the data integration diversity challenge > will be addressed. The importance of this code base grows by day as more > integration scenarios are becoming supported. > 4. The new connector library evolves and replaces many prior Gobblin > connectors under the “gobblin-modules” module. A separate repo will help > avoid confusion. > 5. Separating core and ecosystem modules can help improve isolation and > reduce the number of defects. > > Regards, > Chris > > > > >
