[
https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031406#comment-17031406
]
Alex Petrov edited comment on CASSANDRA-15539 at 2/6/20 9:32 AM:
-----------------------------------------------------------------
I would say we should split the process in two tree logical parts: first one is
to commit API extraction, while leaving all the tests in-tree, then committing
tests to a separate repo, and finally removing the tests. This allows us to
work around the chicken-and-egg problem of two repositories sort of depending
on each other.
I'm also thinking about how to make it fully frictionless for PR builds that
change both the library and tests, especially with snapshot builds. This means
that we might end up with tests relying not on the _latest_ (SNAPSHOT) version
of the API, but rather relying on the _latest published_ (RELEASE) version of
the API. Let's say we have published {{in-jvm-dtest-api}} of version {{0.0.2}}
and started using it in Cassandra. If we first commit the part that runs in-jvm
dtests out of tree, it will try to run tests for all branches against 0.0.2,
even though Cassandra with 0.0.2 couldn't have been built by then, since 0.0.2
wasn't published yet. So we will have to first commit the API changes, then
bump version of API dep in Cassandra, and only then bump the version of API dep
in the library that runs in-jvm-tests out of tree.
To be completely fair, I think that using this for PR builds is fine, as long
as we do not bump the version of the API. As soon as we bump the version, we
end up with a deadlock of dependencies and have to commit things step-by-step,
which might be not the worst thing to do, especially if we know it upfront and
write reasonable build failure error messages instead of weird linkage errors.
In other words, we'll never even attempt to run incompatible versions.
To summarize:
1. Create a new library that publishes a new in-jvm-dtest API out-of-tree.
2. Publish artifact for a version 1.0.0 of API library
3. Bring API library to be used in Cassandra, without migrating the tests out
4. Create a patch that migrates tests out of tree, that uses 1.0.0 version of
the API library
5. Run tests out of tree, make sure everything passes
6. Remove overlapping tests from the tree
When making changes to API library:
1. Locally, test that all branches work with the new version of the library
2. Publish a new version of the API library (say, 1.0.1)
3. Publish Cassandra branches that use 1.0.1 version of the library
4. Update version of the library used in out-of-tree test runner to 1.0.1
(since after before 4 is done, we'll be failing tests that try to link
cassandra against 1.0.1 since is built against 1.0.0, we can as well publish
the out-of-tree test runner during step 2, and simply wait until Cassandra
itself is published)
When making changes to Cassandra that do not change API library, we'll simlpy
trigger a in-jvm-dtest-runner on PR build, and do the same on commit.
When making changes to in-jvm-dtest-runner, we'll run tests for all branches,
assuming their version matches the expected one.
was (Author: ifesdjeen):
I would say we should split the process in two tree logical parts: first one is
to commit API extraction, while leaving all the tests in-tree, then committing
tests to a separate repo, and finally removing the tests. This allows us to
work around the chicken-and-egg problem of two repositories sort of depending
on each other.
I'm also thinking about how to make it fully frictionless for PR builds that
change both the library and tests, especially with snapshot builds. This means
that we might end up with tests relying not on the _latest_ (SNAPSHOT) version
of the API, but rather relying on the _latest published_ (RELEASE) version of
the API. Let's say we have published {{in-jvm-dtest-api}} of version {{0.0.2}}
and started using it in Cassandra. If we first commit the part that runs in-jvm
dtests out of tree, it will try to run tests for all branches against 0.0.2,
even though Cassandra with 0.0.2 couldn't have been built by then, since 0.0.2
wasn't published yet. So we will have to first commit the API changes, then
bump version of API dep in Cassandra, and only then bump the version of API dep
in the library that runs in-jvm-tests out of tree.
To be completely fair, I think that using this for PR builds is fine, as long
as we do not bump the version of the API. As soon as we bump the version, we
end up with a deadlock of dependencies and have to commit things step-by-step.
To summarize:
1. Create a new library that publishes a new in-jvm-dtest API out-of-tree.
2. Publish artifact for a version 1.0.0 of API library
3. Bring API library to be used in Cassandra, without migrating the tests out
4. Create a patch that migrates tests out of tree, that uses 1.0.0 version of
the API library
5. Run tests out of tree, make sure everything passes
6. Remove overlapping tests from the tree
When making changes to API library:
1. Locally, test that all branches work with the new version of the library
2. Publish a new version of the API library (say, 1.0.1)
3. Publish Cassandra branches that use 1.0.1 version of the library
4. Update version of the library used in out-of-tree test runner to 1.0.1
(since after before 4 is done, we'll be failing tests that try to link
cassandra against 1.0.1 since is built against 1.0.0, we can as well publish
the out-of-tree test runner during step 2, and simply wait until Cassandra
itself is published)
When making changes to Cassandra that do not change API library, we'll simlpy
trigger a in-jvm-dtest-runner on PR build, and do the same on commit.
When making changes to in-jvm-dtest-runner, we'll run tests for all branches,
assuming their version matches the expected one.
> Extract in-jvm API and tests out of Cassandra and into a separate repository
> ----------------------------------------------------------------------------
>
> Key: CASSANDRA-15539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15539
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest
> Reporter: Alex Petrov
> Assignee: Alex Petrov
> Priority: Normal
>
> Extract in-jvm DTest _API_ and tests into a separate repository that is
> shared between Cassandra branches. Tests themselves should be buildable using
> just API, which is not the case now, since cluster creation relies on impl
> package, since we do not have factories / constructors in API.
> Main goals we’re trying to achieve:
> 1. We should be able to fail a build on API incompatibility between versions
> 2. Make it as easy as possible to detect break APIs between versions.
> 3. Make development of _tests_ based on in-jvm framework simpler
> 4. Reduce surface area of impact when making modifications to tests
> Potentially, we’d also like to use a plugin to detect API incompatibilities
> between in-jvm DTest API and in-branch implementations, and start running
> tests using shared in-jvm test repository with each existing implementation
> in the branch. This entails both running tests for all branches whenever
> there’s a change in tests jar and running tests for a specific branch
> whenever the branch has changed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
