This is an automated email from the ASF dual-hosted git repository.
zstan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 2933efc56a1 IGNITE-27445 [ducktests] Fix permission mismatch in
ducktests local environment (#12637)
2933efc56a1 is described below
commit 2933efc56a1ea7f15e82377e119a67236f5f6011
Author: Maksim Davydov <[email protected]>
AuthorDate: Tue Jul 7 20:11:38 2026 +0300
IGNITE-27445 [ducktests] Fix permission mismatch in ducktests local
environment (#12637)
---
modules/ducktests/README.md | 497 ++++++++++++++-------------
modules/ducktests/tests/docker/Dockerfile | 166 ++++++---
modules/ducktests/tests/docker/ducker-ignite | 16 +-
modules/ducktests/tests/tox.ini | 40 +--
4 files changed, 398 insertions(+), 321 deletions(-)
diff --git a/modules/ducktests/README.md b/modules/ducktests/README.md
index e539db2a3dc..3d56ca33f29 100644
--- a/modules/ducktests/README.md
+++ b/modules/ducktests/README.md
@@ -1,311 +1,312 @@
-# Overview
-The `ignitetest` framework provides basic functionality and services
-to write integration tests for Apache Ignite. This framework bases on
-the `ducktape` test framework, for information about it check the links:
-- https://github.com/confluentinc/ducktape - source code of the `ducktape`.
-- https://ducktape.readthedocs.io/en/latest/index.html - documentation to the
`ducktape`.
-
-Structure of the `tests` directory is:
-- `./ignitetest/services` contains basic services functionality.
-- `./ignitetest/utils` contains utils for testing.
-- `./ignitetest/tests` contains tests.
-- `./checks` contains unit tests of utils, tests' decorators etc.
-
-Some tests (like the CDC replication ones) require modules maintained in the
-separate [ignite-extensions](https://github.com/apache/ignite-extensions)
repository.
-
-To run these tests the `ignite-extensions` working directory should be checked
out
-to the same filesystem level next to the `ignite` one. So it should be the
below structure
-of directories:
-- `./ignite` working directory checked out from the ignite repository
(mentioned as `${IGNITE_HOME}` below)
-- `./ignite-extensions` working directory checked out from the
ignite-extensions repository
-
-The needed extension module should be built before tests run.
-
-For example for the CDC replication tests the `cdc-ext` module should be built
as:
-```
-cd ${IGNITE_HOME}\..\ignite-extensions
-mvn clean package -pl :ignite-cdc-ext -Pskip-docs -DskipTests
-```
+# Apache Ignite Integration Test Framework - Ducktests
+
+The `ignitetest` framework provides basic functionality and services to write
integration tests for Apache Ignite. This framework is built on top of the
**ducktape** test framework.
+* For core concepts, see the [ducktape source
code](https://github.com/confluentinc/ducktape).
+* For framework details, see the [ducktape
documentation](https://ducktape.readthedocs.io/en/latest/index.html).
+
+### Repository Structure
+All paths below are relative to `${IGNITE_HOME}/modules/ducktests/tests`:
+* `./ignitetest/services`: Contains basic services and cluster orchestration
logic.
+* `./ignitetest/utils`: Contains testing helper utilities.
+* `./ignitetest/tests`: Contains the actual integration test scenarios.
+* `./checks`: Contains internal framework unit tests and style decorators.
+
+---
+
+## Quick Start (Local Docker Run)
+
+Docker is used to emulate a distributed multi-node cluster environment where
each individual container acts as a running cluster node.
+
+### 1. Prerequisites
+* **Docker** installed and running on your host system.
+* **Python >= 3.8** installed on your host system (required only for local
environment scripts and development).
+
+### 2. Prepare the Environment & Code
+Execute these preparation steps from the root directory of your project:
+
+> **WARNING:** Building and compiling the Apache Ignite project is strictly
mandatory. You must run `mvn clean install` to generate the local Maven
artifacts. Skipping this step will cause the Ignite service startup to fail
with a `ClassNotFoundException`.
+
+```bash
+# 1. Change your current directory to the Ignite root
+cd \${IGNITE_HOME}
-# Local run
-Docker is used to emulate distributed environment. Single container represents
-a running node.
+# 2. Build and install Apache Ignite artifacts locally
+mvn clean install -DskipTests
-## Requirements
-To just start tests locally the only requirement is preinstalled `docker`.
-For development process requirements are `python` >= 3.7.
+# 3. Build the Apache Ignite ducktests modules
+./scripts/build-module.sh ducktests
-## Run tests
-- Change a current directory to`${IGNITE_HOME}`
-- Build Apache IGNITE invoking `${IGNITE_HOME}/scripts/build-module.sh
ducktests`
-- (Optionally) Build the needed extension modules in the
`${IGNITE_HOME}\..\ignite-extensions` directory
-- Change a current directory to `${IGNITE_HOME}/modules/ducktests/tests`
-- Run tests in docker containers using a following command:
+# 4. Navigate into the ducktests directory
+cd modules/ducktests/tests
```
-./docker/run_tests.sh
+
+### 3. Preparing the Local Environment
+Run the following commands from your host system's shell inside
`${IGNITE_HOME}/modules/ducktests/tests`:
+```bash
+# Create and activate an isolated development virtual environment
+python3 -m venv ~/.virtualenvs/ignite-ducktests-dev
+source ~/.virtualenvs/ignite-ducktests-dev/bin/activate
+
+# Install core framework testing requirements and editable dependencies
+pip install -r docker/requirements-dev.txt
+pip install -e .
```
-- For detailed help and instructions, use a following command:
+
+> If your environment is configured to look only at internal or restricted
artifact registries, `pip install` may fail to find specific package versions.
+>
+> To resolve this, append the public PyPI mirror to your installation command:
+
+```bash
+pip install -r docker/requirements-dev.txt --extra-index-url
https://pypi.org/simple
```
-./docker/run_tests.sh --help
+
+### 4. Run a Smoke Test
+Run the test runner script by pointing it directly to a specific smoke test
target. The script will automatically build the required container images,
bring up the necessary nodes, and run the test scenario:
+
+```bash
+./docker/run_tests.sh -t
./ignitetest/tests/smoke_test.py::SmokeServicesTest.test_ignite_start_stop -n 3
--global-json '{"cluster_size": 2}'
```
-- Test reports, including service logs, are located in the
`${IGNITE_HOME}/results` directory.
-## Runned tests management
-- Tear down all the currently active ducker-ignite nodes using a following
command:
+### 5. What a Successful Run Looks Like
+When the test runs successfully, your terminal output will display discovery
logs followed by a clean execution matrix report:
+
+```text
+[INFO]: Discovered 1 tests to run
+[INFO]: starting test run with session id 2026-06-25--004...
+[INFO]: running 1 tests...
+[INFO]: RunnerClient: ... SmokeServicesTest.test_ignite_start_stop: Running...
+[INFO]: RunnerClient: ... SmokeServicesTest.test_ignite_start_stop: PASS
+================================================================================
+SESSION REPORT (ALL TESTS)
+ducktape version: 0.13.0
+run time: 6.554 seconds
+tests run: 1
+passed: 1
+failed: 0
+================================================================================
+test_id: ...SmokeServicesTest.test_ignite_start_stop
+status: PASS
```
+
+### 6. Managing and Cleaning Containers
+Always clean up and tear down active background nodes after your test runs
finish to free up local resources:
+```bash
+# Display detailed help options and arguments
+./docker/run_tests.sh --help
+
+# Stop and remove all currently active ducker-ignite cluster nodes
./docker/clean_up.sh
```
-# Real environment run
-[Ducktape](https://ducktape.readthedocs.io/en/latest/index.html) allow runs on
-Custom cluster, Vagrant, K8s, Mesos, Docker, cloud providers, etc.
-
-## Requirements
-- Set up the cluster.
- See `./docker/Dockerfile` for servers setup hints.
-
-## Run tests
-- Change a current directory to`${IGNITE_HOME}`
-- Build Apache IGNITE invoking `${IGNITE_HOME}/scripts/build-module.sh
ducktests`
-- (Optionally) Build the needed extension modules in the
`${IGNITE_HOME}\..\ignite-extensions` directory
-- Run tests using
[Ducktape](https://ducktape.readthedocs.io/en/latest/run_tests.html). \
- For example:
- ```
- ducktape --results-root=./results --cluster-file=./cluster.json --repeat 1
--max-parallel 16 ./modules/ducktests/tests/ignitetest
- ```
-# Custom Ignites (forks) testing
-## Run all tests
-### Setup
-Any version of Apache Ignite, or it's fork, can be tested.
-Binary releases supported as well as compiled sources.
-
-- Binary releases should be located at `/opt` directory, eg.
`/opt/ignite-2.11.0`.
-- Source releases also should be located at `/opt` directory, but should be
compiled before the first use.\
- Use the following command to compile sources properly:
- ```
- ./scripts/build-module.sh ducktests
- ```
-You may replace `/opt` with custom directory by setting `install_root` globals
param. \
-For example, `--globals-json, eg: {"install_root": "/dir42"}`
+---
-### Execution
-You may set versions (products) using `@ignite_versions` decorator at code
-```
-@ignite_versions(str(DEV_BRANCH), str(LATEST))
-```
-or passing versions set via globals during the execution
-```
---global-json, eg: {"ignite_versions":["2.8.1", "dev"]}
-```
-You may also specify product prefix by `project` param at globals, for example:
-```
---global-json, eg: {"project": "fork" ,"ignite_versions": ["ignite-2.8.1",
"2.8.1", "dev"]}
-```
-will execute tests on `ignite-2.8.1, fork-2.8.1, fork-dev`
+## Local Development & Code Checks
-## Run tests from the external source/repository
-TBD
+To modify framework logic or contribute features locally without depending on
Docker container environments, isolate your development runtime dependencies.
-# Special runs
-## Run with FlightRecorder (JFR)
-To run ignite with flight recorder you should enable `jfr_enabled` through
globals, for example:
-```
---global-json, eg: {"jfr_enabled":true}
-```
-## Run with safepoints logging
-Safepoint logging is disabled by default, to enable it, you need to pass
`true` for `safepoint_log_enabled` for example:
-```
---global-json, eg: {"safepoint_log_enabled":true}
-```
-## Run with enabled security
-### Run with SSL enabled
-To enable ssl it is only required to pass `enabled` for `ssl` in globals:
-```
---global-json, eg: {"ssl":{"enabled":true}}
+### 1. Activate the Local Environment
+Run the following commands from your host system's shell inside
`${IGNITE_HOME}/modules/ducktests/tests`:
+```bash
+# Activate an isolated development virtual environment
+source ~/.virtualenvs/ignite-ducktests-dev/bin/activate
```
-In this case, all ssl params will be set to default values, and will be
written to ignite config.
-These values correspond to the keystores that are generated (and you shouldn't
worry about it).
-Default keystores for these services are generated automatically on creating
environment.
-If you want, you could override these values through globals, for example:
-```
- {"ssl": {
- "enabled": true,
- "params": {
- "server": {
- "key_store_jks": "server.jks",
- "key_store_password": "123456",
- "trust_store_jks": "truststore.jks",
- "trust_store_password": "123456"
- },
- "client": {
- "key_store_jks": "client.jks",
- "key_store_password": "123456",
- "trust_store_jks": "truststore.jks",
- "trust_store_password": "123456"
- },
- "admin": {
- "key_store_jks": "admin.jks",
- "key_store_password": "123456",
- "trust_store_jks": "truststore.jks",
- "trust_store_password": "123456"
- }
- }
- }
- }
+### 2. Running Linters and Unit Tests
+```bash
+# Execute local framework utility unit tests
+pytest
+
+# Enforce uniform codebase style rules
+flake8
```
-Where:
-Server, client and admin are three possible interactions with a cluster in a
ducktape, each of them has its own alias,
-which corresponds to keystore:
-* Ignite(clientMode = False) - server
-* Ignite(clientMode = True) - client
-* ControlUtility - admin
+### 3. Testing Across Multiple Python Versions (Optional)
+To locally simulate validation matrices across distinct target runtimes (e.g.,
Python 3.8, 3.9) using `tox`:
+1. Install [pyenv](https://github.com/pyenv/pyenv#installation) onto your
machine.
+2. Download target runtimes and initialize your runtime shell profile context:
+ ```bash
+ pyenv install 3.8
+ pyenv install 3.9
+ pyenv shell 3.8 3.9
+ ```
+3. Install `tox` and run the validation suite:
+ ```bash
+ pip install tox
+ tox
+ tox -r -e codestyle,py3
+ ```
+
+---
-And options `key_store_jks` and `trust_store_jks` are paths to keys.
-If you start it with `/` it will be used as an absolute path.
-Otherwise, it will be a relative path that starts from `/mnt/service/shared/`.
+## Testing with Ignite Extensions
-And if you need to specify values only for one configuration, you can skip
other configurations, for example:
+Some integration tests (such as CDC replication scenarios) require additional
modules maintained in the separate
[ignite-extensions](https://github.com/apache/ignite-extensions) repository.
+### Setup Directory Structure
+To run these tests, the `ignite-extensions` working directory **must** be
checked out at the exact same filesystem level as your main `ignite` repository:
+```text
+ your-development-folder/
+├── ignite/ <- Checked out from the main ignite repository
(${IGNITE_HOME})
+└── ignite-extensions/ <- Checked out from the ignite-extensions repository
```
- {"ssl": {
- "enabled": true,
- "params": {
- "server": {
- "key_store_jks": "server.jks",
- "key_store_password": "123456",
- "trust_store_jks": "truststore.jks",
- "trust_store_password": "123456"
- }
- }
- }
- }
+
+### Building the Extensions
+The target extension module must be compiled on your host before running the
test suite. For example, to prepare the `cdc-ext` module:
+```bash
+cd ${IGNITE_HOME}/../ignite-extensions
+mvn clean package -pl :ignite-cdc-ext -Pskip-docs -DskipTests
```
+*Note: The local docker startup script automatically handles binding this
directory inside the containers if the folder structure matches.*
-For more information about ssl in ignite you can check this link: [SSL in
ignite](https://ignite.apache.org/docs/latest/security/ssl-tls)
+---
-### Run with build-in authentication enabled
-Via this option you could overwrite default login and password options in
tests with authentication.
-```
- {"authentication":{
- "enabled": true,
- "username": "username",
- "password": "password"
- }
- }
-```
+## Global Parameters & Special Runs
-## Run with metrics export enabled
+You can modify test environments at execution time using global flags injected
through the `--global-json` parameter.
-To let Ignite nodes export metrics in different formats during test run use
the `metrics` globals parameter
-as described in following sections.
+#### Test Configuration
-You would need to install and configure metrics collecting software (like
prometheus or zabbix) by yourselves.
+| Global Parameter Key | Definition | Example Configuration |
+|---------------------|------------|----------------------|
+| **cluster_size** | Controls the cluster size for tests. Overrides the
default num_nodes value passed in the @cluster annotation. Default value is
determined by the test framework. | ```{"cluster_size": 13}``` |
+| **ignite_versions** | List of Ignite versions for testing. Tests run for
each version in the list. Values are folder names in /opt/ (e.g., "ignite-dev"
for master branch, "ignite-2.17.0" for release). Default is determined by test
annotations. | ```{"ignite_versions": ["ignite-dev", "ignite-2.17.0"]}``` |
+| **failure_detection_timeout** | Timeout in milliseconds for failure
detection in discovery. Used to detect node failures in the cluster. Default
value is 10000 ms (10 seconds). | ```{"failure_detection_timeout": 20000}``` |
-Below is the sample `prometheus.yml` config file which may be used to collect
metrics from tests running in docker:
+#### Security & Authentication
-```yaml
-global:
- scrape_interval: 1s
- evaluation_interval: 5s
+| Global Parameter Key | Definition | Example Configuration |
+|---------------------|------------|----------------------|
+| **ssl** | SSL configuration with nested parameters. Enabled flag controls
SSL globally, params contains keystore configurations for different aliases
(server, client, admin). *Paths beginning with a `/` resolve as absolute paths.
Otherwise, files are read relative to `/mnt/service/shared/`.* | ```{"ssl":
{"enabled": true, "params": {"client": {"key_store_jks": "client.jks",
"key_store_password": "pwd", "trust_store_jks": "truststore.jks",
"trust_store_password": "pwd"}}}}``` |
+| **authentication** | Authentication configuration with nested parameters.
Enabled flag controls authentication globally, username and password specify
credentials. Default username is "ignite" and password is "ignite". |
```{"authentication": {"enabled": true, "username": "admin", "password":
"secret"}}``` |
-scrape_configs:
- - job_name: 'ducktests'
- static_configs:
- - targets:
- - 172.20.0.2:8082
- - 172.20.0.3:8082
- - 172.20.0.4:8082
- - 172.20.0.5:8082
- - 172.20.0.6:8082
- - 172.20.0.7:8082
- - 172.20.0.8:8082
- - 172.20.0.9:8082
- - 172.20.0.10:8082
- - 172.20.0.11:8082
- - 172.20.0.12:8082
- - 172.20.0.13:8082
- - 172.20.0.14:8082
+#### Performance & Monitoring
+
+| Global Parameter Key | Definition | Example Configuration |
+|---------------------|------------|----------------------|
+| **jfr_enabled** | Boolean flag to enable Java Flight Recorder for
performance profiling. Default is False. | ```{"jfr_enabled": true}``` |
+| **safepoint_log_enabled** | Boolean flag to enable safepoint logging for
debugging JVM behavior. Default is False. | ```{"safepoint_log_enabled":
true}``` |
+| **jmx_remote** | JMX remote monitoring configuration with nested parameters.
Enabled flag controls remote JMX access, port specifies the listening port
(default is 1098). | ```{"jmx_remote": {"enabled": true, "port": 1099}}``` |
+| **metrics** | Metrics configuration with nested parameters. JMX metrics can
be enabled, and OpenCensus metrics can be configured with period and port
settings. Default period is 1000ms and default port is 8082. | ```{"metrics":
{"jmx": {"enabled": true}, "opencensus": {"enabled": true, "period": 2000,
"port": 8083}}}``` |
+
+*Note on OpenCensus Telemetry:* To separate metrics from different tests, the
`iin` label is injected into the configuration of each Ignite node. It can be
used in downstream software like Grafana for precise filtration. The label
contains a detailed dot-concatenated string representing the full namespace of
the current test runner context (e.g.,
`discovery_test.DiscoveryTest.test_nodes_fail_not_sequential_zk.nodes_to_kill.2.load_type.ClusterLoad.ATOMIC.ignite_version.ignite-2.11.0`).
+
+#### Framework Configuration
+
+| Global Parameter Key | Definition | Example Configuration |
+|---------------------|------------|----------------------|
+| **NodeSpec** | Specifies the class to use for node specifications in Ignite
services. Controls how Ignite nodes are configured and started. |
```{"NodeSpec": "myapp.services.MyNodeSpec"}``` |
+| **AppSpec** | Specifies the class to use for application specifications in
Ignite applications. Controls how Ignite applications are configured and
started. | ```{"AppSpec": "myapp.services.MyAppSpec"}``` |
+| **IgniteTestContext** | Class name for the test context implementation.
Allows customization of test context behavior. | ```{"IgniteTestContext":
"myapp.context.CustomTestContext"}``` |
+| **project** | Project/fork name for version handling (e.g., "ignite",
"fork"). Used to distinguish between different Ignite variants. Default is
"ignite". | ```{"project": "fork"}``` |
+
+#### Paths & Directories
+
+| Global Parameter Key | Definition | Example Configuration |
+|---------------------|------------|----------------------|
+| **persistent_root** | Root directory for persistent storage in tests. Used
for storing test data and logs. Default is typically the test working
directory. | ```{"persistent_root": "/opt/ignite/test-data"}``` |
+| **install_root** | Root directory for Ignite installation. Points to the
base directory where Ignite is installed for testing. Default is typically
"/opt/ignite". | ```{"install_root": "/opt/ignite-testing"}``` |
+
+### Custom Ignites & Forks Testing
+You can test arbitrary binary releases or custom compiled forks by overriding
the execution path:
+* **Binary releases:** Must be extracted to the `/opt` directory (e.g.,
`/opt/ignite-2.11.0`).
+* **Source forks:** Must be located at `/opt` and compiled using
`./scripts/build-module.sh ducktests`.
+
+To use a custom directory path entirely, pass the `install_root` global
configuration variable:
+```bash
+--global-json '{"install_root": "/custom_directory_path"}'
```
-In case of docker you might want ask it to assign the same IP addresses to
containers each time it is invoked.
-This way you will be able to create a static config file for your metrics
scraper. To do so use
-the `--subnet` parameter for the `./docker/run_tests.sh` script as
+You can target specific cross-product version compatibility combinations
inside test suites using the `@ignite_versions` decorator, or override them
dynamically during running execution:
+
+```bash
+# Runs tests only against versions '2.8.1' and 'dev'
+--global-json '{"ignite_versions":["2.8.1", "dev"]}'
+# Specifies a custom product prefix ('fork') to execute tests across
'ignite-2.8.1', 'fork-2.8.1', and 'fork-dev'
+--global-json '{"project": "fork", "ignite_versions": ["ignite-2.8.1",
"2.8.1", "dev"]}'
```
-./docker/run_tests.sh --subnet 170.20.0.0/16
+
+
+### Diagnostics & Performance Utilities
+```bash
+# Enable Java Flight Recorder (JFR) tracing
+--global-json '{"jfr_enabled": true}'
+
+# Enable JVM Safepoints performance logging
+--global-json '{"safepoint_log_enabled": true}'
```
-### Run with the Prometheus metrics exporter
+### Security Settings
+```bash
+# Enable built-in authentication overrides
+--global-json '{"authentication": {"enabled": true, "username": "custom_user",
"password": "custom_password"}}'
-To export metrics in the Prometheus format via the HTTP use the following
`--globals-json` parameters:
+# Enable default SSL/TLS configurations
+--global-json '{"ssl": {"enabled": true}}'
+```
+To supply precise keystores for explicit communication roles, use detailed
layout arrays:
```json
{
- "metrics": {
- "opencensus": {
- "enabled":true,
- "port": 8082
+ "ssl": {
+ "enabled": true,
+ "params": {
+ "server": { "key_store_jks": "server.jks", "key_store_password": "pwd",
"trust_store_jks": "truststore.jks", "trust_store_password": "pwd" }
}
}
}
```
+*Paths beginning with a `/` resolve as absolute paths. Otherwise, files are
read relative to `/mnt/service/shared/`.*
-To separate metrics from different tests the `iin` label is used. It may be
used in software like grafana for filtration.
+For more information about ssl in ignite you can check this link: [SSL in
ignite](https://ignite.apache.org/docs/latest/security/ssl-tls)
-Label contains the *test id* string: dot concatenated test package name, test
class name, test method name and set of
-name/value pairs for parameters, like:
+### Prometheus & JMX Metrics Exporters
+To configure external scraping agents (like Prometheus or Zabbix) to
systematically track container telemetry, apply tracking labels.
-`discovery_test.DiscoveryTest.test_nodes_fail_not_sequential_zk.nodes_to_kill.2.load_type.ClusterLoad.ATOMIC.ignite_version.ignite-2.11.0`
+In case of docker you might want ask it to assign the same IP addresses to
containers each time it is invoked. This way you will be able to create a
static config file for your metrics scraper. To do so use the `--subnet`
parameter for the `./docker/run_tests.sh` script as:
-**Implementation note.** The `ignite-opencensus` metrics module uses the `iin`
label to expose the `IgniteInstanceName`
-cluster configuration parameter. So the `ducktests` engine exploits this fact
and puts the *test id* to
-configuration of each ignite node started in scope of particular test.
+```bash
+./docker/run_tests.sh --subnet 170.20.0.0/16
+```
-### Run with the JMX metrics exporter
-To have Ignite nodes export metrics via the JMX use below global parameters.
Note that you also might want to enable the
-JMХ remote access (running on 1098 port by default) to let external tools like
zabbix collect metrics.
+Pass the following telemetry properties to open standard scrape ports:
```json
{
- "jmx_remote": {
- "enabled": true,
- "port": 1098
- },
+ "jmx_remote": { "enabled": true, "port": 1098 },
"metrics": {
- "jmx": {
- "enabled":true
- }
+ "jmx": { "enabled": true },
+ "opencensus": { "enabled": true, "port": 8082 }
}
}
```
-
-# Development
-## Preparing development environment
-- Create a virtual environment and activate it using following commands:
-```
-python3 -m venv ~/.virtualenvs/ignite-ducktests-dev
-source ~/.virtualenvs/ignite-ducktests-dev/bin/activate
-```
-- Change a current directory to `${IGNITE_HOME}/modules/ducktests/tests`. We
refer to it as `${DUCKTESTS_DIR}`.
-- Install requirements and `ignitetests` as editable using following commands:
-```
-pip install -r docker/requirements-dev.txt
-pip install -e .
+To separate metrics from different tests, the `iin` label is used. It can be
utilized in downstream software like Grafana for precise filtration.
+
+The label contains a detailed **test id** string composed of the
dot-concatenated test package name, test class name, test method name, and
name/value pairs for all parameters:
+
+`discovery_test.DiscoveryTest.test_nodes_fail_not_sequential_zk.nodes_to_kill.2.load_type.ClusterLoad.ATOMIC.ignite_version.ignite-2.11.0`
+
+**Implementation note:** The `ignite-opencensus` metrics module uses the `iin`
label to expose the `IgniteInstanceName` cluster configuration parameter. The
`ducktests` engine exploits this behavior by injecting the exact *test id*
directly into the configuration of each Ignite node started within that test
scope.
+
+```yaml
+# Sample host prometheus.yml scrape job configuration
+scrape_configs:
+ - job_name: 'ducktests'
+ static_configs:
+ - targets: ['172.20.0.2:8082', '172.20.0.3:8082', '172.20.0.4:8082']
```
+
---
-- For running unit tests invoke `pytest` in `${DUCKTESTS_DIR}`.
-- For checking codestyle invoke `flake8` in `${DUCKTESTS_DIR}`.
+## Distributed Remote Environments
-#### Run checks over multiple python's versions using tox (optional)
-All commits and PR's are checked against multiple python's version, namely
3.6, 3.7 and 3.8.
-If you want to check your PR as it will be checked on Travis CI, you should do
following steps:
+The underlying `ducktape` execution orchestrator allows running these exact
suites on production clusters, managed infrastructure, Kubernetes (K8s),
Vagrant, Mesos, or Cloud environments.
-- Install `pyenv`, see installation instruction
[here](https://github.com/pyenv/pyenv#installation).
-- Install different versions of python (recommended versions are `3.7.9` and
`3.8.5`)
-- Activate them with a command `pyenv shell 3.7.9 3.8.5`
-- Install `tox` by invoking a command `pip install tox`
-- Change a current directory to `${DUCKTESTS_DIR}` and invoke `tox`
+### Requirements
+* Ensure target remote cluster node configurations align with base OS images.
Refer to `./docker/Dockerfile` for underlying software, storage routing setup,
and platform software prerequisites.
+
+### Execution
+Trigger remote integration test execution loops via explicit command wrappers:
+```bash
+ducktape --results-root=./results --cluster-file=./cluster.json --repeat 1
--max-parallel 16 ./modules/ducktests/tests/ignitetest
+```
diff --git a/modules/ducktests/tests/docker/Dockerfile
b/modules/ducktests/tests/docker/Dockerfile
index a88d0a969e9..59ed67463d5 100644
--- a/modules/ducktests/tests/docker/Dockerfile
+++ b/modules/ducktests/tests/docker/Dockerfile
@@ -13,90 +13,164 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-ARG jdk_version=eclipse-temurin:17
-FROM $jdk_version
-
-MAINTAINER Apache Ignite [email protected]
-VOLUME ["/opt/ignite-dev"]
-
-# Set the timezone.
+# =====================================================================
+# GLOBAL ARGS (Must be declared at the very top, before any FROM)
+# =====================================================================
+ARG python_version="3.13"
+ARG jdk_version="eclipse-temurin:17"
+
+# ---------------------------------------------------------------------
+# Stage 1: JDK source (Temurin JDKs are self-contained and relocatable,
+# so copying the JDK across images is safe -- unlike copying CPython,
+# whose _ssl extension is linked against the source image's OpenSSL)
+# ---------------------------------------------------------------------
+FROM $jdk_version AS jdk_source
+
+# ---------------------------------------------------------------------
+# Stage 2: Main Image Build
+# ---------------------------------------------------------------------
+FROM python:${python_version}-slim
+
+# Re-declare global args here to make them accessible inside this specific
stage
+ARG python_version
+ARG jdk_version
+
+# ---------------------------------------------------------------------
+# 1. Environment & Global System Configurations
+# ---------------------------------------------------------------------
ENV TZ=Europe/Moscow
-RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
+ENV DEBIAN_FRONTEND=noninteractive
-# Do not ask for confirmations when running apt-get, etc.
-ENV DEBIAN_FRONTEND noninteractive
+RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
-# Set the ducker.creator label so that we know that this is a ducker image.
This will make it
-# visible to 'ducker purge'. The ducker.creator label also lets us know what
UNIX user built this
-# image.
+# Set up ducker labels
ARG ducker_creator=default
LABEL ducker.creator=$ducker_creator
-# Update Linux and install necessary utilities.
-RUN cat /etc/apt/sources.list | sed
's/http:\/\/deb.debian.org/https:\/\/deb.debian.org/g' >
/etc/apt/sources.list.2 && mv /etc/apt/sources.list.2 /etc/apt/sources.list
-RUN apt update && apt install -y sudo netcat-traditional iptables rsync unzip
wget curl jq coreutils openssh-server net-tools vim python3-pip python3-dev
python3-venv libffi-dev libssl-dev cmake pkg-config libfuse-dev iperf
traceroute mc git && apt-get -y clean
-
+# Fix package mirror protocol from http to https
+# (Debian trixie uses the deb822 sources file; fall back to sources.list if
present)
+RUN if [ -f /etc/apt/sources.list.d/debian.sources ]; then \
+ sed -i 's|http://deb.debian.org|https://deb.debian.org|g'
/etc/apt/sources.list.d/debian.sources; \
+ elif [ -f /etc/apt/sources.list ]; then \
+ sed -i 's|http://deb.debian.org|https://deb.debian.org|g'
/etc/apt/sources.list; \
+ fi
+
+# ---------------------------------------------------------------------
+# 2. System Utilities Installation
+# ---------------------------------------------------------------------
+RUN apt-get update && apt-get install -y \
+ sudo \
+ netcat-traditional \
+ iptables \
+ rsync \
+ unzip \
+ wget \
+ curl \
+ jq \
+ coreutils \
+ openssh-server \
+ net-tools \
+ vim \
+ libffi-dev \
+ libssl-dev \
+ cmake \
+ pkg-config \
+ libfuse-dev \
+ iperf \
+ traceroute \
+ mc \
+ git \
+ build-essential \
+ && apt-get -y clean \
+ && rm -rf /var/lib/apt/lists/*
+
+# ---------------------------------------------------------------------
+# 3. JDK Installation (copied from the Temurin image)
+# ---------------------------------------------------------------------
+ENV JAVA_HOME=/opt/java/openjdk
+COPY --from=jdk_source $JAVA_HOME $JAVA_HOME
+ENV PATH="$JAVA_HOME/bin:$PATH"
+
+# Confirm both runtimes are healthy
+RUN java -version
+
+# ---------------------------------------------------------------------
+# 4. Python Virtual Environment Setup & Requirements
+# ---------------------------------------------------------------------
+# Standardize path routing via the virtual environment block
RUN python3 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
+
+# Print the active python version and verify ssl is importable
+RUN python3 --version && python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"
+
COPY ./requirements.txt /root/requirements.txt
-RUN pip3 install -r /root/requirements.txt
-# Set up ssh
+# Install requirements
+RUN pip3 install \
+ --default-timeout=100 \
+ --no-cache-dir \
+ -r /root/requirements.txt
+
+# ---------------------------------------------------------------------
+# 5. SSH & Security Settings
+# ---------------------------------------------------------------------
COPY ./ssh-config /root/.ssh/config
-# NOTE: The paramiko library supports the PEM-format private key, but does not
support the RFC4716 format.
RUN ssh-keygen -m PEM -q -t rsa -N '' -f /root/.ssh/id_rsa && cp -f
/root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
RUN echo 'PermitUserEnvironment yes' >> /etc/ssh/sshd_config
+# ---------------------------------------------------------------------
+# 6. Apache Component Stack Dependencies (Ignite, Zookeeper, Kafka, JMX)
+# ---------------------------------------------------------------------
ARG APACHE_MIRROR="https://apache-mirror.rbc.ru/pub/apache/";
ARG APACHE_ARCHIVE="https://archive.apache.org/dist/";
-# Install binary test dependencies.
+# Install Ignite
RUN for v in "2.7.6" "2.17.0"; \
do cd /opt; \
curl -O $APACHE_ARCHIVE/ignite/$v/apache-ignite-$v-bin.zip;\
- unzip apache-ignite-$v-bin.zip && mv /opt/apache-ignite-$v-bin
/opt/ignite-$v;\
- done
-
-RUN rm /opt/apache-ignite-*-bin.zip
+ unzip apache-ignite-$v-bin.zip && mv /opt/apache-ignite-$v-bin
/opt/ignite-$v; \
+ done \
+ && rm /opt/apache-ignite-*-bin.zip
-#Install zookeeper.
+# Install Zookeeper
ARG ZOOKEEPER_VERSION="3.5.8"
ARG ZOOKEEPER_NAME="zookeeper-$ZOOKEEPER_VERSION"
ARG ZOOKEEPER_RELEASE_NAME="apache-$ZOOKEEPER_NAME-bin"
ARG ZOOKEEPER_RELEASE_ARTIFACT="$ZOOKEEPER_RELEASE_NAME.tar.gz"
-RUN echo $APACHE_ARCHIVE/zookeeper/$ZOOKEEPER_NAME/$ZOOKEEPER_RELEASE_ARTIFACT
RUN cd /opt && curl -O
$APACHE_ARCHIVE/zookeeper/$ZOOKEEPER_NAME/$ZOOKEEPER_RELEASE_ARTIFACT \
- && tar xvf $ZOOKEEPER_RELEASE_ARTIFACT && rm $ZOOKEEPER_RELEASE_ARTIFACT
-RUN mv /opt/$ZOOKEEPER_RELEASE_NAME /opt/$ZOOKEEPER_NAME
+ && tar xvf $ZOOKEEPER_RELEASE_ARTIFACT && rm $ZOOKEEPER_RELEASE_ARTIFACT \
+ && mv /opt/$ZOOKEEPER_RELEASE_NAME /opt/$ZOOKEEPER_NAME
-#Install kafka
+# Install Kafka
ARG KAFKA_VERSION="3.9.1"
ARG KAFKA_NAME="kafka"
ARG KAFKA_RELEASE_NAME="${KAFKA_NAME}_2.13-$KAFKA_VERSION"
ARG KAFKA_RELEASE_ARTIFACT="$KAFKA_RELEASE_NAME.tgz"
-RUN echo $APACHE_ARCHIVE/kafka/$KAFKA_VERSION/$KAFKA_RELEASE_ARTIFACT
RUN cd /opt && curl -O
$APACHE_ARCHIVE/kafka/$KAFKA_VERSION/$KAFKA_RELEASE_ARTIFACT \
- && tar xvf $KAFKA_RELEASE_ARTIFACT && rm $KAFKA_RELEASE_ARTIFACT
-RUN mv /opt/$KAFKA_RELEASE_NAME /opt/$KAFKA_NAME-$KAFKA_VERSION
-
-# The version of Kibosh to use for testing.
-# If you update this, also update vagrant/base.sh
-ARG KIBOSH_VERSION="8841dd392e6fbf02986e2fb1f1ebf04df344b65a"
-
-# Install Kibosh
-RUN apt-get install fuse
-RUN cd /opt && git clone -q https://github.com/confluentinc/kibosh.git && cd
"/opt/kibosh" && git reset --hard $KIBOSH_VERSION && mkdir "/opt/kibosh/build"
&& cd "/opt/kibosh/build" && ../configure && make -j 2
+ && tar xvf $KAFKA_RELEASE_ARTIFACT && rm $KAFKA_RELEASE_ARTIFACT \
+ && mv /opt/$KAFKA_RELEASE_NAME /opt/$KAFKA_NAME-$KAFKA_VERSION
-#Install jmxterm
+# Install Jmxterm
ARG JMXTERM_NAME="jmxterm"
ARG JMXTERM_VERSION="1.0.1"
ARG JMXTERM_ARTIFACT="$JMXTERM_NAME-$JMXTERM_VERSION-uber.jar"
RUN cd /opt && curl -OL
https://github.com/jiaqi/jmxterm/releases/download/v$JMXTERM_VERSION/$JMXTERM_ARTIFACT
\
- && mv $JMXTERM_ARTIFACT $JMXTERM_NAME.jar
+ && mv $JMXTERM_ARTIFACT $JMXTERM_NAME.jar
+# ---------------------------------------------------------------------
+# 7. User Isolation Permissions (Ducker Execution Account Setup)
+# ---------------------------------------------------------------------
# Set up the ducker user.
-RUN userdel -r ubuntu \
- && useradd -ms /bin/bash ducker \
+ARG USER_UID=1000
+ARG USER_GID=1000
+
+# Remove conflicting default user if it exists
+RUN getent passwd ubuntu &>/dev/null && userdel -r ubuntu || true
+
+# Provision ducker system environment and security configurations
+RUN groupadd -g $USER_GID ducker \
+ && useradd -u $USER_UID -g $USER_GID -ms /bin/bash ducker \
&& mkdir -p /home/ducker/ \
&& rsync -aiq /root/.ssh/ /home/ducker/.ssh \
&& chown -R ducker /home/ducker/ /mnt/ /var/log/ /opt \
@@ -106,9 +180,9 @@ RUN userdel -r ubuntu \
&& echo "JAVA_HOME=$JAVA_HOME" >> /home/ducker/.ssh/environment \
&& echo 'PATH=$PATH:'"$JAVA_HOME/bin" >> /home/ducker/.profile \
&& echo 'ducker ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
+
USER ducker
CMD sudo service ssh start && tail -f /dev/null
-# Container port exposure
EXPOSE 11211 47100 47500 49112 10800 8080 2888 3888 2181 1098 8082 8083
diff --git a/modules/ducktests/tests/docker/ducker-ignite
b/modules/ducktests/tests/docker/ducker-ignite
index da6cc205c98..6285f91fb3f 100755
--- a/modules/ducktests/tests/docker/ducker-ignite
+++ b/modules/ducktests/tests/docker/ducker-ignite
@@ -44,6 +44,9 @@ default_num_nodes=4
# The default OpenJDK base image.
default_jdk="openjdk:17"
+# The default Python version.
+default_python="3.13"
+
# The default ducker-ignite image name.
default_image_name="ducker-ignite"
@@ -60,12 +63,15 @@ Usage: ${script_path} [command] [options]
help|-h|--help
Display this help message
-build [-j|--jdk JDK] [-c|--context] [image-name]
+build [-j|--jdk JDK] [-p|--python PYTHON_VERSION] [-c|--context] [image-name]
Build a docker image that represents ducker node. Image is tagged with
specified ${image_name}.
If --jdk is specified then we will use this argument as base image for
ducker docker images.
Otherwise ${default_jdk} is used.
+ If --python is specified, its value will be used to extract the Python
runtime for the Docker images.
+ Otherwise, ${default_python} will be used.
+
If --context is specified then build docker image from this path. Context
directory must contain Dockerfile.
up [-n|--num-nodes NUM_NODES] [-f|--force] [docker-image]
@@ -255,8 +261,10 @@ ducker_build_image() {
must_pushd "${ducker_dir}"
# Tip: if you are scratching your head for some dependency problems that
are referring to an old code version
# (for example java.lang.NoClassDefFoundError), add --no-cache flag to the
build shall give you a clean start.
- must_do -v -o docker build --memory="${docker_build_memory_limit}" \
- --build-arg "ducker_creator=${user_name}" --build-arg
"jdk_version=${jdk_version}" -t "${image_name}" \
+ must_do -v -o docker build --network=host
--memory="${docker_build_memory_limit}" \
+ --build-arg USER_UID=$(id -u) --build-arg USER_GID=$(id -g) \
+ --build-arg "ducker_creator=${user_name}" --build-arg
"jdk_version=${jdk_version}" \
+ --build-arg "python_version=${python_version}" -t "${image_name}" \
-f "${docker_context}/Dockerfile" -- "${docker_context}"
docker_status=$?
must_popd
@@ -279,12 +287,14 @@ ducker_build() {
while [[ $# -ge 1 ]]; do
case "${1}" in
-j|--jdk) set_once jdk_version "${2}" "the OpenJDK base image";
shift 2;;
+ -p|--python) set_once python_version "${2}" "the Python version";
shift 2;;
-c|--context) docker_context="${2}"; shift 2;;
*) set_once image_name "${1}" "docker image name"; shift;;
esac
done
[[ -n "${jdk_version}" ]] || jdk_version="${default_jdk}"
+ [[ -n "${python_version}" ]] || python_version="${default_python}"
[[ -n "${image_name}" ]] ||
image_name="${default_image_name}-${jdk_version/:/-}"
[[ -n "${docker_context}" ]] || docker_context="${ducker_dir}"
diff --git a/modules/ducktests/tests/tox.ini b/modules/ducktests/tests/tox.ini
index 56d89ca0081..e0fa4a74df8 100644
--- a/modules/ducktests/tests/tox.ini
+++ b/modules/ducktests/tests/tox.ini
@@ -13,46 +13,38 @@
# See the License for the specific language governing permissions and
# limitations under the License.
[tox]
-envlist = codestyle, py38, py39
+envlist = codestyle, py{38,39,310,311,312,313}
skipsdist = True
[testenv]
-install_command = pip install --extra-index-url https://pypi.org/simple {opts}
{packages}
-envdir = {toxworkdir}/.virtualenvs/ignite-ducktests-{envname}
-deps =
- -r ./docker/requirements-dev.txt
usedevelop = True
-commands =
- pytest {env:PYTESTARGS:} {posargs}
+envdir = {toxworkdir}/.virtualenvs/ignite-ducktests-{envname}
+deps = -r{toxinidir}/docker/requirements-dev.txt
+install_command = pip install --extra-index-url https://pypi.org/simple {opts}
{packages}
+commands = pytest {env:PYTESTARGS:} {posargs}
[testenv:codestyle]
basepython = python3
-commands =
- flake8
-
-[testenv:py38]
-envdir = {toxworkdir}/.virtualenvs/ignite-ducktests-py38
-
-[testenv:py39]
-envdir = {toxworkdir}/.virtualenvs/ignite-ducktests-py39
+deps = flake8
+commands = flake8
[BASIC]
-min-public-methods=0
-good-names=i, j, k, x, y, ex, pk, tx
+min-public-methods = 0
+good-names = i, j, k, x, y, ex, pk, tx
[SIMILARITIES]
-ignore-imports=yes
+ignore-imports = yes
[FORMAT]
-max-line-length=120
+max-line-length = 120
[DESIGN]
-max-parents=10
+max-parents = 10
[flake8]
-max-line-length=120
+max-line-length = 120
[pytest]
-python_files=check_*.py
-python_classes=Check
-python_functions=check_*
+python_files = check_*.py
+python_classes = Check
+python_functions = check_*
