[
https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206637#comment-17206637
]
Michael Semb Wever edited comment on CASSANDRA-16079 at 10/3/20, 9:46 AM:
--------------------------------------------------------------------------
bq. I have started on a PoC for the caching of bootstrapped ccm, to help
identify whether (and how much) time can be saved.
This approach did not bear much fruit.
To cache bootstrapped ccm clusters it meant that those ccm clusters had to be
bootstrapped, then stopped, then the directory copied, and then started up
again for the test in question. This added ~2 seconds per node. When a
bootstrapped ccm cluster was cached, it still had to have its directory copied
(back into place of the current dtest ccm directory), and only saved up to ~1
second. These numbers are pre- CASSANDRA-13701. While there might have been
some improvement post 13701, it wouldn't have been as much as hoped as each
dtest split in jenkins is only running ~20 dtest methods and the re-use of
bootstrapped ccm clusters would have only been a fraction of that.
This approach could be better utilised if dtests could be run in a "bootstrap
and cache all ccm clusters" mode, and the resulting cache zipped and re-used
between runs (e.g. stored and downloaded from nightlies.a.o).
was (Author: michaelsembwever):
bq. I have started on a PoC for the caching of bootstrapped ccm, to help
identify whether (and how much) time can be saved.
This approach did not bear much fruit.
To cache bootstrapped ccm clusters it meant that those ccm clusters had to be
bootstrapped, then stopped, then the directory copied, and then started up
again for the test in question. This added ~2 seconds per node. When a
bootstrapped ccm cluster was cached, it still had to have its directory copied
(back into place of the current dtest ccm directory), and only saved up to ~1
second. These numbers are pre- CASSANDRA-13701. While there might have been
some improvement post 13701, it wouldn't have been as much as hoped as each
dtest split in jenkins is only running ~20 dtest methods and the re-use of
bootstrapped ccm clusters would have only been a fraction of that.
> Improve dtest runtime
> ---------------------
>
> Key: CASSANDRA-16079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16079
> Project: Cassandra
> Issue Type: Improvement
> Components: CI
> Reporter: Adam Holmberg
> Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
> Attachments: Screenshot 2020-09-19 at 12.32.21.png
>
>
> A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a
> [30% increase in run
> time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html].
> While that change was accepted, we wanted to spin out a ticket to optimize
> dtests in an attempt to gain back some of that runtime.
> At this time we don't have concrete improvements in mind, so the first order
> of this ticket will be to analyze the state of things currently, and try to
> ascertain some valuable optimizations. Once the problems are understood, we
> will break down subtasks to divide the work.
> Some areas to consider:
> * cluster reuse
> * C* startup optimizations
> * Tests that should be ported to in-JVM dtest or even unit tests
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
