Dmitry Konstantinov created CASSANDRA-21805:
-----------------------------------------------
Summary: CCM: download releases from dlcdn.apache.org before
archive.apache.org, and time out stalled download
Key: CASSANDRA-21805
URL: https://issues.apache.org/jira/browse/CASSANDRA-21805
Project: Apache Cassandra
Issue Type: Improvement
Components: Tool/CCM
Reporter: Dmitry Konstantinov
ccm downloads release tarballs only from
http://archive.apache.org/dist/cassandra and calls urlopen() without a timeout.
archive.apache.org is a single throttled host. When many CI splits download
from it at once, downloads slow down, and a stalled connection blocks until
pytest kills the test.
In Cassandra-trunk #2641 (
https://ci-cassandra.apache.org/job/Cassandra-trunk/2641), every upgrade split
downloaded 6.0-alpha1 from archive.apache.org. The median speed per split was
188 KB/s, 14 splits got stuck for more than 900s, and 20 upgrade tests from
current_6_0_x failed with "Timeout >900.0s", most of them while still in
repository.__download().
Proposed change:
* Try https://dlcdn.apache.org, then https://downloads.apache.org, then
archive.apache.org. dlcdn and downloads only keep current releases; a 404 moves
on to the next mirror immediately. [repositories] cassandra in ~/.ccm/config
still overrides the list.
* Use a 30s socket timeout, so a stalled connection fails instead of hanging.
* Retry each URL up to 3 times on timeouts, connection errors, truncated
responses and HTTP 5xx/408/429. Do not retry other 4xx responses, certificate
errors or file:// URLs.
To be used by dtests, the cassandra-test tag has to be moved after the merge.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
