[
https://issues.apache.org/jira/browse/GEODE-8899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320373#comment-17320373
]
ASF subversion and git services commented on GEODE-8899:
--------------------------------------------------------
Commit 75bb0d297b0624120fd62bea2ee9d5dfc400a18f in geode's branch
refs/heads/feature/GEODE-7665 from Dale Emery
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=75bb0d2 ]
GEODE-8899: Upgrade to Gradle v6.8.3 (#6280)
* GEODE-8899: Upgrade to Gradle v6.8.3
Updated Geode's build system to use Gradle 6.8.3:
- Updated the gradle wrapper to use Gradle 6.8.3.
- Updated geode-assembly/build.gradle to make the ivy "repository"
compatible with Gradle 6.8.3.
- Changed ./gradle.properties to specify 'minimumGradleVersion=6.8'.
- Updated the expected-pom.xml file to the format that Gradle 6.8.3
produces.
- Updated all Gradle plugins to the latest versions.
- In buildSrc/build.gradle, added groovy source sets for testing, to
allow running tests of plugin code.
- Updated common "test isolation" code to be compatible with Gradle
6.8.3. See below for details.
- Updated the RepeatTest task to be compatible with Gradle 6.8.3. See
below for details.
- Rewrote the Dockerized test plugin to be compatible with Gradle 6.8.3.
See below for details.
- Added a new 'dunitDockerJVM' project property to allow developers run
tests in Docker on macOS. To do this, use -PtestJVM to identify the
mac's JVM, and use -PdunitDockerJVM to specify the JVM for the test
worker process running in Docker.
- Updated the BUILDING.md file for Gradle 6.8.3. As part of this I added
two versions of the Apache copyright notice to the etc dir: A plain
text version and an xml version that IntelliJ IDEA can import into its
copyright settings.
===== Test isolation code =====
Some code for isolating tests is used by the RepeatTest task and the
Dockerized test plugin. This code was updated to be compatible with
Gradle v6.8.3, and to somewhat reduce its dependence on Gradle internal
implementation details:
- WorkingDirectoryIsolator adjusts a given ProcessBuilder to give it a
unique working directory.
- AdjustableProcessLauncher applies a given Consumer (such as the
isolator, above) to adjust each ProcessBuilder before launching the
process.
- LauncherProxyWorkerProcessBuilder and
LauncherProcessWorkerProcessFactory forcefully update each Gradle exec
handle to use a given ProcessLauncher to launch test worker processes.
- Executers and Workers offer convenience methods for constructing
TestExecuters and ProcessWorker builders and factories.
===== RepeatTest task update =====
Geode's RepeatTest task uses a custom TestExecuter that was copied from
Gradle 5.5 (or earlier) and modified. This custom class is now updated
to be compatible with Gradle 6.8.3, and renamed from
OverriddenTestExecutor to RepeatableTestExecuter.
===== Dockerized test plugin rewrite =====
Removed the modified copy of pedjak's Dockerized test plugin.
Configuration
-------------
- Rewrote gradle/docker.gradle and renamed it as
gradle/multi-process-test.gradle.
- Moved the code that translates project properties into configuration
settings from the plugin to the config class. Renamed the config class
as DockerTestWorkerConfig.
- Moved code that configures command lines and environments for Docker
test workers based on project properties. Thise code was defiend as a
closure in a build script. It is now a method in
DockerTestWorkerConfig.
- Changed command line options that CI scripts pass to Gradle to run
tests in Docker:
- Specify the --max-workers Gradle option to configure the maximum
number of workers.
- Specify the 'testMaxParallelForks' project property to configure
the maximum number of forks for each test task.
- Remove the 'dunitParallelForks' option.
Implementation
--------------
The Dockerized test plugin uses these classes to launches processes in
Docker containers:
- DockerProcess overrides java.lang.Process to represent a test worker
process running in Docker.
- DockerProcessLauncher launches each test worker process in a Docker
container, and creates a DockerProcess to represent and manage it.
- Other classes described in the "Test isolation code" section, above.
The plugin overrides several of Gradle's internal classes to allow test
worker processes running in Docker containers to communicate with
Gradle:
- DockerConnectionAcceptor implements Gradle's ConnectionAcceptor, which
(among other responsibilities), produces a "multi-choice address" that
a test worker process can use to connect to Gradle's messaging server.
Gradle's default implementation produces multi-choice addresses that
processes in Docker containers cannot use.
- DockerMessagingServer implements Gradle's MessagingServer using
DockerConnectionAcceotpr.
- WildcardBindingInetAddressFactory overrides Gradle's
InetAddressFactory to instruct messaging servers to listen on an
address that processes in Dockerized containers can connect to.
Timeouts
--------
Due to a recent surge of CI problems due to delays in Docker operations,
I rewrote the plugins's timeout mechanism to make it configurable and
much more robust:
- Added a 'dockerTimeout' project property, defaulting to 5 minutes.
Each Docker operation will throw an exception if its duration exceeds
this value.
- Log a warning if any Docker operation takes longer than 1 minute. This
can help us to see how often significant delays occur in CI, while
still allowing tests to proceed.
- These exceptions and warnings identify the project for which test
worker process is being launched. This can help us to see whether
there are project-specific factors that affect the delays.
* Fix problems identified by rhoughton-pivot
> Update dockerized test plugin
> -----------------------------
>
> Key: GEODE-8899
> URL: https://issues.apache.org/jira/browse/GEODE-8899
> Project: Geode
> Issue Type: Test
> Components: tests
> Reporter: Dale Emery
> Assignee: Dale Emery
> Priority: Major
> Labels: GeodeOperationAPI, pull-request-available
>
> *Problem:* The dockerized test plugin includes modified copies of internal
> Gradle code. This code depends on internal Gradle classes and method
> signatures as defined in Gradle 5.8. These dependencies prevent us from
> updating Gradle.
> *Solution:* Update the dockerized test plugin code to be compatible with the
> latest Gradle, and to take advantage of recent Gradle features (such as the
> "worker lease" mechanism) that make some plugin details unnecessary.
> Where possible, change the plugin code to use public Gradle APIs.
> Where the plugin must depend on internal Gradle implementation details,
> comment the code to indicate:
> * Which internal Gradle implementation details the code depends on
> * The latest version of Gradle known to satisfy these dependencies
> Where the plugin must use modified copies of internal Gradle code, comment
> the code to indicate:
> * Which Gradle class was copied
> * Which version of Gradle the class was copied from
> * What modifications were made in the copy, and why
> These comments will help identify implementation details in the plugin that
> risk incompatibility with future versions of Gradle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
