[
https://issues.apache.org/jira/browse/CASSANDRA-12050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Russ Hatch updated CASSANDRA-12050:
-----------------------------------
Description:
Coverage data offers a unique opportunity to build metadata about tests and
related Cassandra code.
Using the jacoco coverage api we should be able to build a simple index of
tests (dtest, unit) to the Cassandra code they touch (and the reverse).
When a new patch is introduced, we do a lookup in that index, based on java
source files touched (and possibly lines within those files), and use that to
infer most relevant dtests or unit tests. Patch authors can then run that small
test subset as a first testing pass.
In this way we can build small, focused, test suites that are custom to each
patch. Once this small custom smoke test appears successful things would of
course need to be vetted across a more complete test run on CI.
I think the best interface would simply be ant targets. One target would be
used to build/refresh the test:source code index (run occasionally and saved
somewhere; index building would be time consuming since it will require full
job runs). A second target looks at files touched and does the index lookups,
then outputs a list of tests to run.
The dev user experience might looking something like this:
{noformat}
ant get-custom-smoke -Dcoverage_index=./trunk_coverage_index_SHA_foo.idx
Generating test lists....
Please run the following two scripts to vet your changes:
./custom_smoke_junit.sh
./custom_smoke_dtest.sh
{noformat}
was:
Coverage data offers a unique opportunity to build metadata about tests and
related Cassandra code.
Using the jacoco coverage api we should be able to build a simple index of
tests (dtest, unit) to the Cassandra code they touch (and the reverse).
When a new patch is introduced, we do a lookup in that index, based on java
source files touched (and possibly lines within those files), and use that to
infer most relevant dtests or unit tests. Patch authors can then run that small
test subset as a first testing pass.
In this way we can build small, focused, test suites that are custom to each
patch. Once this small custom smoke test appears successful things would of
course need to be vetted across a more complete test run on CI.
I think the best interface would simply be ant targets. One target would be
used to build/refresh the test:source code index (run occasionally and saved
somewhere (index building would be time consuming since it will require full
job runs). A second target looks at files touched and does the index lookups,
then outputs a list of tests to run.
The dev user experience might looking something like this:
{noformat}
ant get-custom-smoke -Dcoverage_index=./trunk_coverage_index_SHA_foo.idx
Generating test lists....
Please run the following two scripts to vet your changes:
./custom_smoke_junit.sh
./custom_smoke_dtest.sh
{noformat}
> per-patch smoke suites as an early/fast testing tier
> ----------------------------------------------------
>
> Key: CASSANDRA-12050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12050
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Russ Hatch
> Assignee: Russ Hatch
>
> Coverage data offers a unique opportunity to build metadata about tests and
> related Cassandra code.
> Using the jacoco coverage api we should be able to build a simple index of
> tests (dtest, unit) to the Cassandra code they touch (and the reverse).
> When a new patch is introduced, we do a lookup in that index, based on java
> source files touched (and possibly lines within those files), and use that to
> infer most relevant dtests or unit tests. Patch authors can then run that
> small test subset as a first testing pass.
> In this way we can build small, focused, test suites that are custom to each
> patch. Once this small custom smoke test appears successful things would of
> course need to be vetted across a more complete test run on CI.
> I think the best interface would simply be ant targets. One target would be
> used to build/refresh the test:source code index (run occasionally and saved
> somewhere; index building would be time consuming since it will require full
> job runs). A second target looks at files touched and does the index lookups,
> then outputs a list of tests to run.
> The dev user experience might looking something like this:
> {noformat}
> ant get-custom-smoke -Dcoverage_index=./trunk_coverage_index_SHA_foo.idx
> Generating test lists....
> Please run the following two scripts to vet your changes:
> ./custom_smoke_junit.sh
> ./custom_smoke_dtest.sh
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
