Re: Reoganizing drivers
On 13 July 2011 00:47, Jonathan Ellis jbel...@gmail.com wrote:
On Sat, Jul 9, 2011 at 8:29 PM, Rick Shaw wfs...@gmail.com wrote:
On Jul 7, 2011, at 10:53 AM, Eric Evans wrote:
On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote:
- the git mirror won't pick up anything under drivers/
Has there been any effort made to have INFRA add it?
Aaron asked, they said they only do standard svn trunk + branches + tags.
But I think what they mean is that you'd need to tweak things a little, e.g.
svn mkdir http://svn.apache.org/repos/asf/cassandra/server
svn mv http://svn.apache.org/repos/asf/cassandra/trunk
http://svn.apache.org/repos/asf/cassandra/tags
http://svn.apache.org/repos/asf/cassandra/branches
http://svn.apache.org/repos/asf/cassandra/server/
svn mkdir http://svn.apache.org/repos/asf/cassandra/drivers/trunk
svn mv http://svn.apache.org/repos/asf/cassandra/drivers/java
http://svn.apache.org/repos/asf/cassandra/drivers/py
http://svn.apache.org/repos/asf/cassandra/drivers/trunk/
and basically end up with at the
http://svn.apache.org/repos/asf/cassandra/ a set of directories with
trunk/tags/branches underneath.
In fact I suspect you would not even have to move the main branches
(which would be better for svn mergeinfo) as they should be able to
retain the git repo for that as is and just by structuring the drivers
directory with a trunk/tags/branches (and sure keep branches empty if
you want) they could give you
git://git.apache.org/cassandra-drivers.git for the drivers and
git://git.apache.org/cassandra.git remains in place and as is for the
core/server
if they give you any hassle just point out that they do this already
for mina, myfaces, maven, etc... and if it is a big issue you can
always rename cassandra to massandra as they seem to do it for any
project beinging with m ;-)
What do you mean by but not have multiple versions for 0.8 branch?
I mean it would live in trunk but only in trunk -- there would be no
branches/0.8/drivers or branches/1.0/drivers.
Can't we keep the /drivers code in the trunk and just have separate Ant
tasks for building the driver parts independent of the tasks for for the
server?
Right, this feels ideal to me. Otherwise the right way to handle it
is to download a Cassandra stuff-the-driver-needs jar from the maven
repo.
I will be providing a patch that will allow deploying -SNAPSHOTs to
the apache snapshot repository once I get some releases off the decks
at Maven, that will make life easier for the Hector guys etc (as they
can follow trunk more easily, and push -SNAPSHOTs themselves to allow
for easier user testing) but a side-effect is that it would make life
easier if you went with this approach... and remember that the maven
repo in this case does not even need to go off your machine as you can
just install the core deps in your local repo by doing a build of core
locally.
I'd rather just have {cassandra} and {driver} build targets
personally, from the same tree, rather than introducing this
intermediate dependency.
It does bring up versioning... the tag for drivers releases will
include cassandra stuff too... I have the feeling that you cannot have
your cake and eat it too
Another thought would be to keep it in the separate tree as it is now, and
dumb down the driver build to just build the JAR artifacts (binary, source,
and javadoc). and do all testing and integration work in the server-side
trunk by using just the Driver jar as a dependency for the tests in the
server tree?
Then you have the reverse problem, that running the test suite
requires this other checkout to be around somewhere.
But the driver jar is supposed to be more stable, and have a much
slower release cadence. I see much less of an issue pulling the driver
jar from the (local/remote) maven repo than pulling the core stuff.
I would see this option as being closer to the way you guys seem to
want to work...
But I will raise this one question
The tension seems to be between whether to re-release a drivers jar
with every core release, or have it with its own release cadence. To
my mind, from the Maven Pom point-of-view as long as the drivers
depend on core classes (and that includes thrift) it needs to at least
match the release cadence of core releases (i.e. you would need one
release for each release of core, and if it needs its own bug-fixes
they would be extra releases)
The major pain for releasing the drivers jar is that you have to test
it... building it is no big shakes but testing is where the effort
lies. Now the testing effort has to happen anyway at the main core
release cadence, because even if you think you don't need to rebuild
drivers for the core 0.8.2 release, you do need to test that the
drivers jar you tested against 0.8.1 still works [or at least do some
investigation to say that the changes cannot affect it working...
chaos butterflies will still want testing though*] and that is
ignoring the Maven Pom issue§
So here is the
Re: Reoganizing drivers
On Tue, 2011-07-12 at 16:47 -0700, Jonathan Ellis wrote:
What do you mean by but not have multiple versions for 0.8
branch?
I mean it would live in trunk but only in trunk -- there would be no
branches/0.8/drivers or branches/1.0/drivers.
Maybe I'm missing some svn-fu here, but how would you even do this?
Delete the directory after branching? Wouldn't merging forward to trunk
try to remove it there again?
Can't we keep the /drivers code in the trunk and just have separate
Ant tasks for building the driver parts independent of the tasks for
for the server?
Right, this feels ideal to me. Otherwise the right way to handle it
is to download a Cassandra stuff-the-driver-needs jar from the maven
repo. I'd rather just have {cassandra} and {driver} build targets
personally, from the same tree, rather than introducing this
intermediate dependency.
The JDBC driver seems to be the only reason this is being brought up,
there wouldn't be much to discuss if you removed that from the equation.
None of the original reasons for moving the drivers in the first place
have changed.
I don't think the story here is any different than it would be for any
other project that depended on Cassandra like this, (i.e. it would be
unnecessarily difficult for them as well). Of course it will never be
as easy as treating it all like one big monolithic project, but it could
be a whole lot easier (for anyone) and if it makes sense that they be
treated separately (I feel strongly that it does), then I'd rather we
fix it the right way.
I realize that implicitly means that I've volunteered. :)
--
Eric Evans
eev...@rackspace.com
Re: Reoganizing drivers
On Wed, Jul 13, 2011 at 10:15 AM, Eric Evans eev...@rackspace.com wrote: I don't think the story here is any different than it would be for any other project that depended on Cassandra like this, (i.e. it would be unnecessarily difficult for them as well). Of course it will never be as easy as treating it all like one big monolithic project, but it could be a whole lot easier (for anyone) and if it makes sense that they be treated separately (I feel strongly that it does), then I'd rather we fix it the right way. I realize that implicitly means that I've volunteered. :) Works for me, then. -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Reoganizing drivers
On Sat, Jul 9, 2011 at 8:29 PM, Rick Shaw wfs...@gmail.com wrote:
On Jul 7, 2011, at 10:53 AM, Eric Evans wrote:
On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote:
- the git mirror won't pick up anything under drivers/
Has there been any effort made to have INFRA add it?
Aaron asked, they said they only do standard svn trunk + branches + tags.
What do you mean by but not have multiple versions for 0.8 branch?
I mean it would live in trunk but only in trunk -- there would be no
branches/0.8/drivers or branches/1.0/drivers.
Can't we keep the /drivers code in the trunk and just have separate Ant tasks
for building the driver parts independent of the tasks for for the server?
Right, this feels ideal to me. Otherwise the right way to handle it
is to download a Cassandra stuff-the-driver-needs jar from the maven
repo. I'd rather just have {cassandra} and {driver} build targets
personally, from the same tree, rather than introducing this
intermediate dependency.
Another thought would be to keep it in the separate tree as it is now, and
dumb down the driver build to just build the JAR artifacts (binary, source,
and javadoc). and do all testing and integration work in the server-side
trunk by using just the Driver jar as a dependency for the tests in the
server tree?
Then you have the reverse problem, that running the test suite
requires this other checkout to be around somewhere.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Reoganizing drivers
munch munch munch... this popcorn is nice... having a hard time keeping stum... :-) - Stephen --- Sent from my Android phone, so random spelling mistakes, random nonsense words and other nonsense are a direct result of using swype to type on the screen On 10 Jul 2011 04:30, Rick Shaw wfs...@gmail.com wrote: On Jul 7, 2011, at 10:53 AM, Eric Evans wrote: On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote: - the git mirror won't pick up anything under drivers/ Has there been any effort made to have INFRA add it? It was separated out to make it look/act like an independent project. Cant we get an additional GIT project for drivers? - building the Java drivers is fragile and complicated, and there's a lot of duplication with the main ant build Fragile how so? Because of the build-dependency on Cassandra (and/or how it is satisfied)? Yes. What duplication are you referring to? I don't see much beyond all of the boilerplate you'd see between any two ant projects. Wow I would hardly call the current build for C* boilerplate. It is quite comprehensive but very elaborate and complicate. A lot of clever expertise has been applied. Duplicating the necessary functionality and reaching into another existing source tree is just complicated and error prone. Not impossible, but fragile seems a good description. - patches that affect both Cassandra and JDBC are cumbersome since they have to be committed separately (e.g. https://issues.apache.org/jira/browse/CASSANDRA-2857) Well, the idea of moving it was to be able to treat it as a separate project (more or less), so it follows that you'd have to independently patch anything using AbstractCassandraDaemon. This is a problem that cant be helped if they are separate projects. RIght? Well, it follows that if we change an API that any project using it will need to be updated as well. Since the idea behind moving the drivers was to be able to treat them as separate projects, it follows that we'd have to do it here as well. I'm inclined to think we should move it back to trunk (but not have multiple versions for 0.8 branch). We can still tag/branch separately from there. What do you mean by but not have multiple versions for 0.8 branch? I like the idea of it being being built as a separate project. But I also like it being in the main trunk. Is it possible to have both? Can't we keep the /drivers code in the trunk and just have separate Ant tasks for building the driver parts independent of the tasks for for the server? Another thought would be to keep it in the separate tree as it is now, and dumb down the driver build to just build the JAR artifacts (binary, source, and javadoc). and do all testing and integration work in the server-side trunk by using just the Driver jar as a dependency for the tests in the server tree? -- Eric Evans eev...@rackspace.com
Re: Reoganizing drivers
On Sat, 2011-07-09 at 23:29 -0400, Rick Shaw wrote: On Jul 7, 2011, at 10:53 AM, Eric Evans wrote: On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote: - building the Java drivers is fragile and complicated, and there's a lot of duplication with the main ant build Fragile how so? Because of the build-dependency on Cassandra (and/or how it is satisfied)? Yes. The issues with the former are something that need to be solved either way. Fixing that would all but take care of the latter, but there are things that could be done there in the meantime. It seems like the biggest problem is that there seems to be very little progress here at all recently; I'll try and peel off some time in the next few days to help. -- Eric Evans eev...@rackspace.com
Re: Reoganizing drivers
On Jul 7, 2011, at 10:53 AM, Eric Evans wrote: On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote: - the git mirror won't pick up anything under drivers/ Has there been any effort made to have INFRA add it? It was separated out to make it look/act like an independent project. Cant we get an additional GIT project for drivers? - building the Java drivers is fragile and complicated, and there's a lot of duplication with the main ant build Fragile how so? Because of the build-dependency on Cassandra (and/or how it is satisfied)? Yes. What duplication are you referring to? I don't see much beyond all of the boilerplate you'd see between any two ant projects. Wow I would hardly call the current build for C* boilerplate. It is quite comprehensive but very elaborate and complicate. A lot of clever expertise has been applied. Duplicating the necessary functionality and reaching into another existing source tree is just complicated and error prone. Not impossible, but fragile seems a good description. - patches that affect both Cassandra and JDBC are cumbersome since they have to be committed separately (e.g. https://issues.apache.org/jira/browse/CASSANDRA-2857) Well, the idea of moving it was to be able to treat it as a separate project (more or less), so it follows that you'd have to independently patch anything using AbstractCassandraDaemon. This is a problem that cant be helped if they are separate projects. RIght? Well, it follows that if we change an API that any project using it will need to be updated as well. Since the idea behind moving the drivers was to be able to treat them as separate projects, it follows that we'd have to do it here as well. I'm inclined to think we should move it back to trunk (but not have multiple versions for 0.8 branch). We can still tag/branch separately from there. What do you mean by but not have multiple versions for 0.8 branch? I like the idea of it being being built as a separate project. But I also like it being in the main trunk. Is it possible to have both? Can't we keep the /drivers code in the trunk and just have separate Ant tasks for building the driver parts independent of the tasks for for the server? Another thought would be to keep it in the separate tree as it is now, and dumb down the driver build to just build the JAR artifacts (binary, source, and javadoc). and do all testing and integration work in the server-side trunk by using just the Driver jar as a dependency for the tests in the server tree? -- Eric Evans eev...@rackspace.com
Re: Reoganizing drivers
On Wed, 2011-07-06 at 13:33 -0500, Jonathan Ellis wrote: - the git mirror won't pick up anything under drivers/ Has there been any effort made to have INFRA add it? - building the Java drivers is fragile and complicated, and there's a lot of duplication with the main ant build Fragile how so? Because of the build-dependency on Cassandra (and/or how it is satisfied)? What duplication are you referring to? I don't see much beyond all of the boilerplate you'd see between any two ant projects. - patches that affect both Cassandra and JDBC are cumbersome since they have to be committed separately (e.g. https://issues.apache.org/jira/browse/CASSANDRA-2857) Well, the idea of moving it was to be able to treat it as a separate project (more or less), so it follows that you'd have to independently patch anything using AbstractCassandraDaemon. Well, it follows that if we change an API that any project using it will need to be updated as well. Since the idea behind moving the drivers was to be able to treat them as separate projects, it follows that we'd have to do it here as well. I'm inclined to think we should move it back to trunk (but not have multiple versions for 0.8 branch). We can still tag/branch separately from there. What do you mean by but not have multiple versions for 0.8 branch? -- Eric Evans eev...@rackspace.com
Re: Reoganizing drivers
I don't think this is working out as well as hoped. - the git mirror won't pick up anything under drivers/ - building the Java drivers is fragile and complicated, and there's a lot of duplication with the main ant build - patches that affect both Cassandra and JDBC are cumbersome since they have to be committed separately (e.g. https://issues.apache.org/jira/browse/CASSANDRA-2857) I'm inclined to think we should move it back to trunk (but not have multiple versions for 0.8 branch). We can still tag/branch separately from there. On Tue, Jun 7, 2011 at 3:08 AM, Eric Evans eev...@rackspace.com wrote: Sylvain and I have been discussing release issues while here at buzzwords, and some of the issues are related to drivers. Not surprising since that's a new concept for us, and there wasn't much thought given to the current organization. Because the CQL drivers are independently versioned and capable of releasing on their own timelines, the current location in SVN is suboptimal. There are a number of reasons why, not least of which is that it sets the expectation that the correct version of a driver is whatever corresponds to the release version of Cassandra. So, we'd like to move the drivers sub-directory up one level, making it look something like the following: |- branches |- tags |- site |- drivers | |- java | |- py | |- txpy |- trunk There are a few additional implied changes here as well, for example the JDBC driver will need its own build, and Cassandra's will need some minor changes as well (JDBC driver tests, release artifacts, etc). Does anyone object to this? -- Eric Evans eev...@rackspace.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Reoganizing drivers
Asked on #asfinfra and was told the only things mirrored on git are trunk /
tags / branches .
git-svn it is.
Cheers
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 20 Jun 2011, at 16:28, Jonathan Ellis wrote:
Maybe the non-standard path is giving the git mirror fits.
On Sun, Jun 19, 2011 at 11:26 PM, aaron morton aa...@thelastpickle.com
wrote:
I can see the drivers have moved to
http://svn.apache.org/repos/asf/cassandra/drivers/
Just wondering where that path is available on
git://git.apache.org/cassandra.git
These are the remote branches I can find
$ git ls-remote | grep drivers
From git://git.apache.org/cassandra.git
20635cec24389d83b146af51fa902fcf2d21491brefs/remotes/tags/drivers
dd06878fa6b143dbff1e1e338087041b1b230d48refs/tags/drivers
20635cec24389d83b146af51fa902fcf2d21491brefs/tags/drivers^{}
Thanks
A
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 8 Jun 2011, at 05:01, Eric Evans wrote:
On Tue, 2011-06-07 at 18:40 +0200, Sylvain Lebresne wrote:
On Tue, Jun 7, 2011 at 3:18 PM, Jonathan Ellis jbel...@gmail.com
wrote:
Sounds fine as far as it goes, but don't we want some concept of
branches/tags for driver releases too?
Our idea so far (Eric can correct me if I'm wrong :)) was to consider
the drivers directory as the 'trunk' for drivers, and create branches
and tags for them alongside the cassandra ones.
Yup. In fact, I already tagged the Python and Java drivers as
tags/drivers/lang/version during the last release (neither of those
driver artifacts corresponded to the same SVN rev, nor did they
correspond to the rev for 0.8.0).
Truth is, I even think that consider the drivers as a whole is not
granular enough. It's unlikely the different drivers will move at the
same pace.
As far as I know, there is no reason that a tag (say
tags/drivers/py/1.1.1) can't point to a subdirectory of drivers/ (i.e.
drivers/py). In fact, that's how the tags mentioned above were done
(except those pointed to branches/cassandra-0.8.0/drivers/lang). I
think it just boils down to a matter convention.
*But*, we believe that moving the drivers up one level is at least a
first step towards something better than the status quo.
Yeah, even if we decide to do something different later on, this is an
improvement over what we have now.
--
Eric Evans
eev...@rackspace.com
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Reoganizing drivers
Maybe the non-standard path is giving the git mirror fits.
On Sun, Jun 19, 2011 at 11:26 PM, aaron morton aa...@thelastpickle.com wrote:
I can see the drivers have moved to
http://svn.apache.org/repos/asf/cassandra/drivers/
Just wondering where that path is available on
git://git.apache.org/cassandra.git
These are the remote branches I can find
$ git ls-remote | grep drivers
From git://git.apache.org/cassandra.git
20635cec24389d83b146af51fa902fcf2d21491b refs/remotes/tags/drivers
dd06878fa6b143dbff1e1e338087041b1b230d48 refs/tags/drivers
20635cec24389d83b146af51fa902fcf2d21491b refs/tags/drivers^{}
Thanks
A
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com
On 8 Jun 2011, at 05:01, Eric Evans wrote:
On Tue, 2011-06-07 at 18:40 +0200, Sylvain Lebresne wrote:
On Tue, Jun 7, 2011 at 3:18 PM, Jonathan Ellis jbel...@gmail.com
wrote:
Sounds fine as far as it goes, but don't we want some concept of
branches/tags for driver releases too?
Our idea so far (Eric can correct me if I'm wrong :)) was to consider
the drivers directory as the 'trunk' for drivers, and create branches
and tags for them alongside the cassandra ones.
Yup. In fact, I already tagged the Python and Java drivers as
tags/drivers/lang/version during the last release (neither of those
driver artifacts corresponded to the same SVN rev, nor did they
correspond to the rev for 0.8.0).
Truth is, I even think that consider the drivers as a whole is not
granular enough. It's unlikely the different drivers will move at the
same pace.
As far as I know, there is no reason that a tag (say
tags/drivers/py/1.1.1) can't point to a subdirectory of drivers/ (i.e.
drivers/py). In fact, that's how the tags mentioned above were done
(except those pointed to branches/cassandra-0.8.0/drivers/lang). I
think it just boils down to a matter convention.
*But*, we believe that moving the drivers up one level is at least a
first step towards something better than the status quo.
Yeah, even if we decide to do something different later on, this is an
improvement over what we have now.
--
Eric Evans
eev...@rackspace.com
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Reoganizing drivers
Sounds fine as far as it goes, but don't we want some concept of branches/tags for driver releases too? On Tue, Jun 7, 2011 at 3:08 AM, Eric Evans eev...@rackspace.com wrote: Sylvain and I have been discussing release issues while here at buzzwords, and some of the issues are related to drivers. Not surprising since that's a new concept for us, and there wasn't much thought given to the current organization. Because the CQL drivers are independently versioned and capable of releasing on their own timelines, the current location in SVN is suboptimal. There are a number of reasons why, not least of which is that it sets the expectation that the correct version of a driver is whatever corresponds to the release version of Cassandra. So, we'd like to move the drivers sub-directory up one level, making it look something like the following: |- branches |- tags |- site |- drivers | |- java | |- py | |- txpy |- trunk There are a few additional implied changes here as well, for example the JDBC driver will need its own build, and Cassandra's will need some minor changes as well (JDBC driver tests, release artifacts, etc). Does anyone object to this? -- Eric Evans eev...@rackspace.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Reoganizing drivers
On Tue, Jun 7, 2011 at 3:18 PM, Jonathan Ellis jbel...@gmail.com wrote: Sounds fine as far as it goes, but don't we want some concept of branches/tags for driver releases too? Our idea so far (Eric can correct me if I'm wrong :)) was to consider the drivers directory as the 'trunk' for drivers, and create branches and tags for them alongside the cassandra ones. Truth is, I even think that consider the drivers as a whole is not granular enough. It's unlikely the different drivers will move at the same pace. *But*, we believe that moving the drivers up one level is at least a first step towards something better than the status quo. -- Sylvain On Tue, Jun 7, 2011 at 3:08 AM, Eric Evans eev...@rackspace.com wrote: Sylvain and I have been discussing release issues while here at buzzwords, and some of the issues are related to drivers. Not surprising since that's a new concept for us, and there wasn't much thought given to the current organization. Because the CQL drivers are independently versioned and capable of releasing on their own timelines, the current location in SVN is suboptimal. There are a number of reasons why, not least of which is that it sets the expectation that the correct version of a driver is whatever corresponds to the release version of Cassandra. So, we'd like to move the drivers sub-directory up one level, making it look something like the following: |- branches |- tags |- site |- drivers | |- java | |- py | |- txpy |- trunk There are a few additional implied changes here as well, for example the JDBC driver will need its own build, and Cassandra's will need some minor changes as well (JDBC driver tests, release artifacts, etc). Does anyone object to this? -- Eric Evans eev...@rackspace.com -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Reoganizing drivers
On Tue, 2011-06-07 at 18:40 +0200, Sylvain Lebresne wrote: On Tue, Jun 7, 2011 at 3:18 PM, Jonathan Ellis jbel...@gmail.com wrote: Sounds fine as far as it goes, but don't we want some concept of branches/tags for driver releases too? Our idea so far (Eric can correct me if I'm wrong :)) was to consider the drivers directory as the 'trunk' for drivers, and create branches and tags for them alongside the cassandra ones. Yup. In fact, I already tagged the Python and Java drivers as tags/drivers/lang/version during the last release (neither of those driver artifacts corresponded to the same SVN rev, nor did they correspond to the rev for 0.8.0). Truth is, I even think that consider the drivers as a whole is not granular enough. It's unlikely the different drivers will move at the same pace. As far as I know, there is no reason that a tag (say tags/drivers/py/1.1.1) can't point to a subdirectory of drivers/ (i.e. drivers/py). In fact, that's how the tags mentioned above were done (except those pointed to branches/cassandra-0.8.0/drivers/lang). I think it just boils down to a matter convention. *But*, we believe that moving the drivers up one level is at least a first step towards something better than the status quo. Yeah, even if we decide to do something different later on, this is an improvement over what we have now. -- Eric Evans eev...@rackspace.com
