I would think that github would be a better option for the Spark Geode Connector. That way it's not tightly coupled to the Geode release cycle.
I don't see why it's desirable to bloat Geode with every single script, tool, or connector that might interact with Geode. Another reason to consider separating projects is testing. Do we really want the Geode build to run unit/integration/e2e tests for Geode, JVSD, Spark connector, etc? If someone isn't working on the Spark connector, should they be forced to execute the tests for it before committing? I think the Geode tests alone are going to push the limits of what ASF allows in Jenkins. -Kirk On Mon, Jul 6, 2015 at 11:22 PM, Qihong Chen <qc...@pivotal.io> wrote: > The problem is caused by multiple major dependencies and different release > cycles. Spark Geode Connector depends on two products: Spark and Geode (not > counting other dependencies), and Spark moves much faster than Geode, and > some features/code are not backward compatible. > > Our initial connector implementation depends on Spark 1.2 in before the > last week of March 15. Then Spark 1.3 was released on the last week of > March, and some connector feature doesn't work with Spark 1.3, then we > moved on, and now support Spark 1.3 (but not 1.2 any more, we did create > tag). Two weeks ago, Spark 1.4 was released, and it breaks our connector > code again. > > Therefore, for each Geode release, we probably need multiple Connector > releases, and probably need to maintain last 2 or 3 Connector releases, for > example, we need to support both Spark 1.3 and 1.4 with the current Geode > code. > > The question is how to support this with single source repository? > > Thanks, > Qihong >
