> *for each Geode release, we probably need multiple Connector releases, and probably need to maintain last 2 or 3 Connector releases, for example, we need to support both Spark1.3 and 1.4 with the current Geode code.*
Exactly my point for maintaining the GemFire/Geode Spark Connector as a separate, individually releasable artifact with it's own repo. *Spring Data GemFire* is no different and you would be *mistaken* if you think *GemFire/Geode* is the only dependency *Spring Data GemFire* has. In fact, SDG depends on the core *Spring Framework* as well as *Spring Data Commons* from the *Spring* ecosystem as well as many other 3rd party dependencies, even some that overlap with GemFire/Geode (e.g. Jackson). So what I "mean" when I say, "**How do you know* which version of Geode, or any other dependency (e.g. Apache Spark) for that matter, the GemFire/Geode Spark Connector depends on*"... you need to look at the Maven POM or the Gradle dependency declarations (e.g. SDG gradle.properties <https://github.com/spring-projects/spring-data-gemfire/blob/master/gradle.properties> [0]). The "declared" dependency of Geode is the version upon which a user is able to run the connector, that version and that version alone unless the sub-project states otherwise (.e.g. Geode version 1.0 to 1.5 supported, Spark 1.2 and 1.3 supported). If you try to bundle all the various modules (Spark Connector, Redis, Memcached, etc) you are going to have a dependency management nightmare and you will stifle any individual module's (.e.g. Spark Connectors) ability to support newer versions of important dependencies (i.e. Spark) beyond Geode, because the module will be tied Geode's release cycle, where now the module will only be able to be released when Geode is ready. My $0.02. Thanks, John [0] - https://github.com/spring-projects/spring-data-gemfire/blob/master/gradle.properties On Mon, Jul 6, 2015 at 11:22 PM, Qihong Chen <qc...@pivotal.io> wrote: > The problem is caused by multiple major dependencies and different release > cycles. Spark Geode Connector depends on two products: Spark and Geode (not > counting other dependencies), and Spark moves much faster than Geode, and > some features/code are not backward compatible. > > Our initial connector implementation depends on Spark 1.2 in before the > last week of March 15. Then Spark 1.3 was released on the last week of > March, and some connector feature doesn't work with Spark 1.3, then we > moved on, and now support Spark 1.3 (but not 1.2 any more, we did create > tag). Two weeks ago, Spark 1.4 was released, and it breaks our connector > code again. > > Therefore, for each Geode release, we probably need multiple Connector > releases, and probably need to maintain last 2 or 3 Connector releases, for > example, we need to support both Spark 1.3 and 1.4 with the current Geode > code. > > The question is how to support this with single source repository? > > Thanks, > Qihong > -- -John 503-504-8657 john.blum10101 (skype)
