This is an automated email from the ASF dual-hosted git repository.
sbp pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-trusted-releases.git
The following commit(s) were added to refs/heads/main by this push:
new d8bb33d Add a page about running and contributing tests to the
documentation
d8bb33d is described below
commit d8bb33d9a6a7877fd04a7f112001bcedbac3db49
Author: Sean B. Palmer <[email protected]>
AuthorDate: Tue Oct 14 10:19:37 2025 +0100
Add a page about running and contributing tests to the documentation
---
atr/docs/build-processes.html | 2 +-
atr/docs/build-processes.md | 2 +-
atr/docs/code-conventions.html | 6 ++--
atr/docs/code-conventions.md | 6 ++--
atr/docs/developer-guide.html | 5 ++--
atr/docs/developer-guide.md | 5 ++--
atr/docs/how-to-contribute.html | 6 ++--
atr/docs/how-to-contribute.md | 6 ++--
atr/docs/index.html | 5 ++--
atr/docs/index.md | 5 ++--
atr/docs/running-and-creating-tests.html | 28 ++++++++++++++++++
atr/docs/running-and-creating-tests.md | 49 ++++++++++++++++++++++++++++++++
12 files changed, 103 insertions(+), 22 deletions(-)
diff --git a/atr/docs/build-processes.html b/atr/docs/build-processes.html
index f7a270c..82d8c93 100644
--- a/atr/docs/build-processes.html
+++ b/atr/docs/build-processes.html
@@ -1,7 +1,7 @@
<h1 id="build-processes">3.6. Build processes</h1>
<p><strong>Up</strong>: <code>3.</code> <a href="developer-guide">Developer
guide</a></p>
<p><strong>Prev</strong>: <code>3.5.</code> <a href="user-interface">User
interface</a></p>
-<p><strong>Next</strong>: <code>3.7.</code> <a href="code-conventions">Code
conventions</a></p>
+<p><strong>Next</strong>: <code>3.7.</code> <a
href="running-and-creating-tests">Running and creating tests</a></p>
<p><strong>Sections</strong>:</p>
<ul>
<li><a href="#documentation-build-script">Documentation build script</a></li>
diff --git a/atr/docs/build-processes.md b/atr/docs/build-processes.md
index c5bb313..c73de00 100644
--- a/atr/docs/build-processes.md
+++ b/atr/docs/build-processes.md
@@ -4,7 +4,7 @@
**Prev**: `3.5.` [User interface](user-interface)
-**Next**: `3.7.` [Code conventions](code-conventions)
+**Next**: `3.7.` [Running and creating tests](running-and-creating-tests)
**Sections**:
diff --git a/atr/docs/code-conventions.html b/atr/docs/code-conventions.html
index 0a5b59b..b09fb1f 100644
--- a/atr/docs/code-conventions.html
+++ b/atr/docs/code-conventions.html
@@ -1,7 +1,7 @@
-<h1 id="code-conventions">3.7. Code conventions</h1>
+<h1 id="code-conventions">3.8. Code conventions</h1>
<p><strong>Up</strong>: <code>3.</code> <a href="developer-guide">Developer
guide</a></p>
-<p><strong>Prev</strong>: <code>3.6.</code> <a href="build-processes">Build
processes</a></p>
-<p><strong>Next</strong>: <code>3.8.</code> <a href="how-to-contribute">How to
contribute</a></p>
+<p><strong>Prev</strong>: <code>3.7.</code> <a
href="running-and-creating-tests">Running and creating tests</a></p>
+<p><strong>Next</strong>: <code>3.9.</code> <a href="how-to-contribute">How to
contribute</a></p>
<p><strong>Sections</strong>:</p>
<ul>
<li><a href="#python-code">Python code</a></li>
diff --git a/atr/docs/code-conventions.md b/atr/docs/code-conventions.md
index 8ec5b99..32df343 100644
--- a/atr/docs/code-conventions.md
+++ b/atr/docs/code-conventions.md
@@ -1,10 +1,10 @@
-# 3.7. Code conventions
+# 3.8. Code conventions
**Up**: `3.` [Developer guide](developer-guide)
-**Prev**: `3.6.` [Build processes](build-processes)
+**Prev**: `3.7.` [Running and creating tests](running-and-creating-tests)
-**Next**: `3.8.` [How to contribute](how-to-contribute)
+**Next**: `3.9.` [How to contribute](how-to-contribute)
**Sections**:
diff --git a/atr/docs/developer-guide.html b/atr/docs/developer-guide.html
index 7127f58..967ca82 100644
--- a/atr/docs/developer-guide.html
+++ b/atr/docs/developer-guide.html
@@ -10,8 +10,9 @@
<li><code>3.4.</code> <a href="storage-interface">Storage interface</a></li>
<li><code>3.5.</code> <a href="user-interface">User interface</a></li>
<li><code>3.6.</code> <a href="build-processes">Build processes</a></li>
-<li><code>3.7.</code> <a href="code-conventions">Code conventions</a></li>
-<li><code>3.8.</code> <a href="how-to-contribute">How to contribute</a></li>
+<li><code>3.7.</code> <a href="running-and-creating-tests">Running and
creating tests</a></li>
+<li><code>3.8.</code> <a href="code-conventions">Code conventions</a></li>
+<li><code>3.9.</code> <a href="how-to-contribute">How to contribute</a></li>
</ul>
<p><strong>Sections</strong>:</p>
<ul>
diff --git a/atr/docs/developer-guide.md b/atr/docs/developer-guide.md
index fe7667d..aec6007 100644
--- a/atr/docs/developer-guide.md
+++ b/atr/docs/developer-guide.md
@@ -14,8 +14,9 @@
* `3.4.` [Storage interface](storage-interface)
* `3.5.` [User interface](user-interface)
* `3.6.` [Build processes](build-processes)
-* `3.7.` [Code conventions](code-conventions)
-* `3.8.` [How to contribute](how-to-contribute)
+* `3.7.` [Running and creating tests](running-and-creating-tests)
+* `3.8.` [Code conventions](code-conventions)
+* `3.9.` [How to contribute](how-to-contribute)
**Sections**:
diff --git a/atr/docs/how-to-contribute.html b/atr/docs/how-to-contribute.html
index 6bb556c..bc31425 100644
--- a/atr/docs/how-to-contribute.html
+++ b/atr/docs/how-to-contribute.html
@@ -1,6 +1,6 @@
-<h1 id="how-to-contribute">3.8. How to contribute</h1>
+<h1 id="how-to-contribute">3.9. How to contribute</h1>
<p><strong>Up</strong>: <code>3.</code> <a href="developer-guide">Developer
guide</a></p>
-<p><strong>Prev</strong>: <code>3.7.</code> <a href="code-conventions">Code
conventions</a></p>
+<p><strong>Prev</strong>: <code>3.8.</code> <a href="code-conventions">Code
conventions</a></p>
<p><strong>Next</strong>: (none)</p>
<p><strong>Sections</strong>:</p>
<ul>
@@ -31,7 +31,7 @@
<p><strong>Create a branch.</strong> <a
href="https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-and-deleting-branches-within-your-repository";>Create
a new branch</a> for your work. Use a descriptive name that indicates what you
are working on, such as <code>fix-typo-in-docs</code> or
<code>improve-error-messages</code>.</p>
</li>
<li>
-<p><strong>Make your changes.</strong> Implement your fix or feature,
following our <a href="code-conventions">code conventions</a>. If you are
changing code, ensure that your changes do not break existing functionality. If
you are adding a new feature, consider whether tests are appropriate.</p>
+<p><strong>Make your changes.</strong> Implement your fix or feature,
following our <a href="code-conventions">code conventions</a>. If you are
changing code, ensure that your changes do not break existing functionality.
Whenever you change code, and especially if you are adding a new feature,
consider <a href="running-and-creating-tests">adding a test</a>.</p>
</li>
<li>
<p><strong>Commit your changes.</strong> Write clear, concise commit messages
following <a href="#commit-message-style">our commit message style</a>. Each
commit should represent a logical unit of work, but we are not particularly
strict about this.</p>
diff --git a/atr/docs/how-to-contribute.md b/atr/docs/how-to-contribute.md
index c1834a9..d40df40 100644
--- a/atr/docs/how-to-contribute.md
+++ b/atr/docs/how-to-contribute.md
@@ -1,8 +1,8 @@
-# 3.8. How to contribute
+# 3.9. How to contribute
**Up**: `3.` [Developer guide](developer-guide)
-**Prev**: `3.7.` [Code conventions](code-conventions)
+**Prev**: `3.8.` [Code conventions](code-conventions)
**Next**: (none)
@@ -38,7 +38,7 @@ Once you have identified something to work on, the process of
contributing is as
3. **Create a branch.** [Create a new
branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-and-deleting-branches-within-your-repository)
for your work. Use a descriptive name that indicates what you are working on,
such as `fix-typo-in-docs` or `improve-error-messages`.
-4. **Make your changes.** Implement your fix or feature, following our [code
conventions](code-conventions). If you are changing code, ensure that your
changes do not break existing functionality. If you are adding a new feature,
consider whether tests are appropriate.
+4. **Make your changes.** Implement your fix or feature, following our [code
conventions](code-conventions). If you are changing code, ensure that your
changes do not break existing functionality. Whenever you change code, and
especially if you are adding a new feature, consider [adding a
test](running-and-creating-tests).
5. **Commit your changes.** Write clear, concise commit messages following
[our commit message style](#commit-message-style). Each commit should represent
a logical unit of work, but we are not particularly strict about this.
diff --git a/atr/docs/index.html b/atr/docs/index.html
index 702cfe5..dd9c88b 100644
--- a/atr/docs/index.html
+++ b/atr/docs/index.html
@@ -13,8 +13,9 @@
<li><code>3.4.</code> <a href="storage-interface">Storage interface</a></li>
<li><code>3.5.</code> <a href="user-interface">User interface</a></li>
<li><code>3.6.</code> <a href="build-processes">Build processes</a></li>
-<li><code>3.7.</code> <a href="code-conventions">Code conventions</a></li>
-<li><code>3.8.</code> <a href="how-to-contribute">How to contribute</a></li>
+<li><code>3.7.</code> <a href="running-and-creating-tests">Running and
creating tests</a></li>
+<li><code>3.8.</code> <a href="code-conventions">Code conventions</a></li>
+<li><code>3.9.</code> <a href="how-to-contribute">How to contribute</a></li>
</ul>
</li>
</ul>
diff --git a/atr/docs/index.md b/atr/docs/index.md
index 28bb016..0fd3015 100644
--- a/atr/docs/index.md
+++ b/atr/docs/index.md
@@ -15,5 +15,6 @@ NOTE: This documentation is a work in progress.
* `3.4.` [Storage interface](storage-interface)
* `3.5.` [User interface](user-interface)
* `3.6.` [Build processes](build-processes)
- * `3.7.` [Code conventions](code-conventions)
- * `3.8.` [How to contribute](how-to-contribute)
+ * `3.7.` [Running and creating tests](running-and-creating-tests)
+ * `3.8.` [Code conventions](code-conventions)
+ * `3.9.` [How to contribute](how-to-contribute)
diff --git a/atr/docs/running-and-creating-tests.html
b/atr/docs/running-and-creating-tests.html
new file mode 100644
index 0000000..1f0a2f0
--- /dev/null
+++ b/atr/docs/running-and-creating-tests.html
@@ -0,0 +1,28 @@
+<h1 id="running-and-creating-tests">3.7. Running and creating tests</h1>
+<p><strong>Up</strong>: <code>3.</code> <a href="developer-guide">Developer
guide</a></p>
+<p><strong>Prev</strong>: <code>3.6.</code> <a href="build-processes">Build
processes</a></p>
+<p><strong>Next</strong>: <code>3.8.</code> <a href="code-conventions">Code
conventions</a></p>
+<p><strong>Sections</strong>:</p>
+<ul>
+<li><a href="#running-tests">Running tests</a></li>
+<li><a href="#creating-tests">Creating tests</a></li>
+</ul>
+<h2 id="running-tests">Running tests</h2>
+<p>We currently only have end-to-end browser tests, but we plan to expand
these as part of <a
href="https://github.com/apache/tooling-trusted-releases/issues/209";>Issue
#209</a>. Meanwhile, these browser tests serve as a simple consistency check
when developing ATR.</p>
+<p>To run the tests, you will need Docker. Other OCI runtimes should work, but
you will need to edit the <a href="/ref/Makefile"><code>Makefile</code></a> or
run your own command. The simple way to run the tests is:</p>
+<pre><code class="language-shell">make build-playwright && make
run-playwright
+</code></pre>
+<p>Where the two <code>make</code> invocations correspond to:</p>
+<pre><code class="language-shell">docker build -t atr-playwright -f
tests/Dockerfile.playwright playwright
+docker run --net=host -it atr-playwright python3 test.py --skip-slow
+</code></pre>
+<p>In other words, we build <a
href="/ref/tests/Dockerfile.playwright"><code>tests/Dockerfile.playwright</code></a>,
and then run <a
href="/ref/playwright/test.py"><code>playwright/test.py</code></a> inside that
container. The container is called <code>atr-playwright</code>; if you want to
give the container a different name, then you'll need to run the manual
<code>docker</code> commands. Replace <code>docker</code> with the name of your
Docker-compatible OCI runtime to use an alternati [...]
+<p>The tests should, as of 14 Oct 2025, take about 20 to 25 seconds to run.
The last line should be <code>Tests finished successfully</code>, and if the
tests do not complete successfully there should be an obvious Python
backtrace.</p>
+<h2 id="creating-tests">Creating tests</h2>
+<p>You can add tests to <code>playwright/test.py</code>. If you're feeling
particularly adventurous, you can add separate unit tests etc., but it's okay
to add tests only to the Playwright test script until <a
href="https://github.com/apache/tooling-trusted-releases/issues/209";>Issue
#209</a> is resolved.</p>
+<h3 id="how-the-tests-work">How the tests work</h3>
+<p>The browser tests use <a href="https://playwright.dev/";>Playwright</a>,
which is a cross-browser, cross-platform web testing framework. It's a bit like
the older <a href="https://en.wikipedia.org/wiki/PhantomJS";>PhantomJS</a>, now
discontinued, which allows you to operate a browser through scripting.
Playwright took the same concept and improved the user experience by adding
better methods for polling browser state. Most interactions with a browser take
some time to complete, and in P [...]
+<p>We use the official Playwright OCI container, install a few dependencies
(<code>apt-get</code> is available in the container), and then run
<code>test.py</code>.</p>
+<p>The <code>test.py</code> script calls <a
href="/ref/playwright/test.py:run_tests"><code>run_tests</code></a> from its
<code>main</code>, which sets up all the context, but the main action takes
place in <a href="/ref/playwright/test.py:test_all"><code>test_all</code></a>.
This function removes any state accidentally left over from a previous run,
then runs tests of certain components. Because ATR is stateful, the order of
the tests is important. When adding a test, please be careful t [...]
+<p>We want to make it more clear which Playwright tests depend on which, and
have more isolated tests. Reusing context, however, helps to speed up the
tests.</p>
+<p>The actual test cases themselves tend to use helpers such as <a
href="/ref/playwright/test.py:go_to_path"><code>go_to_path</code></a> and <a
href="/ref/playwright/test.py:wait_for_path"><code>wait_for_path</code></a>,
and then call <a
href="https://docs.python.org/3/library/logging.html#logging.info";><code>logging.info</code></a>
to print information to the console. Try to keep logging messages terse and
informative.</p>
diff --git a/atr/docs/running-and-creating-tests.md
b/atr/docs/running-and-creating-tests.md
new file mode 100644
index 0000000..a407a75
--- /dev/null
+++ b/atr/docs/running-and-creating-tests.md
@@ -0,0 +1,49 @@
+# 3.7. Running and creating tests
+
+**Up**: `3.` [Developer guide](developer-guide)
+
+**Prev**: `3.6.` [Build processes](build-processes)
+
+**Next**: `3.8.` [Code conventions](code-conventions)
+
+**Sections**:
+
+* [Running tests](#running-tests)
+* [Creating tests](#creating-tests)
+
+## Running tests
+
+We currently only have end-to-end browser tests, but we plan to expand these
as part of [Issue
#209](https://github.com/apache/tooling-trusted-releases/issues/209).
Meanwhile, these browser tests serve as a simple consistency check when
developing ATR.
+
+To run the tests, you will need Docker. Other OCI runtimes should work, but
you will need to edit the [`Makefile`](/ref/Makefile) or run your own command.
The simple way to run the tests is:
+
+```shell
+make build-playwright && make run-playwright
+```
+
+Where the two `make` invocations correspond to:
+
+```shell
+docker build -t atr-playwright -f tests/Dockerfile.playwright playwright
+docker run --net=host -it atr-playwright python3 test.py --skip-slow
+```
+
+In other words, we build
[`tests/Dockerfile.playwright`](/ref/tests/Dockerfile.playwright), and then run
[`playwright/test.py`](/ref/playwright/test.py) inside that container. The
container is called `atr-playwright`; if you want to give the container a
different name, then you'll need to run the manual `docker` commands. Replace
`docker` with the name of your Docker-compatible OCI runtime to use an
alternative runtime.
+
+The tests should, as of 14 Oct 2025, take about 20 to 25 seconds to run. The
last line should be `Tests finished successfully`, and if the tests do not
complete successfully there should be an obvious Python backtrace.
+
+## Creating tests
+
+You can add tests to `playwright/test.py`. If you're feeling particularly
adventurous, you can add separate unit tests etc., but it's okay to add tests
only to the Playwright test script until [Issue
#209](https://github.com/apache/tooling-trusted-releases/issues/209) is
resolved.
+
+### How the tests work
+
+The browser tests use [Playwright](https://playwright.dev/), which is a
cross-browser, cross-platform web testing framework. It's a bit like the older
[PhantomJS](https://en.wikipedia.org/wiki/PhantomJS), now discontinued, which
allows you to operate a browser through scripting. Playwright took the same
concept and improved the user experience by adding better methods for polling
browser state. Most interactions with a browser take some time to complete, and
in PhantomJS the developer ha [...]
+
+We use the official Playwright OCI container, install a few dependencies
(`apt-get` is available in the container), and then run `test.py`.
+
+The `test.py` script calls [`run_tests`](/ref/playwright/test.py:run_tests)
from its `main`, which sets up all the context, but the main action takes place
in [`test_all`](/ref/playwright/test.py:test_all). This function removes any
state accidentally left over from a previous run, then runs tests of certain
components. Because ATR is stateful, the order of the tests is important. When
adding a test, please be careful to ensure that you use the correct state and
that you try not to modif [...]
+
+We want to make it more clear which Playwright tests depend on which, and have
more isolated tests. Reusing context, however, helps to speed up the tests.
+
+The actual test cases themselves tend to use helpers such as
[`go_to_path`](/ref/playwright/test.py:go_to_path) and
[`wait_for_path`](/ref/playwright/test.py:wait_for_path), and then call
[`logging.info`](https://docs.python.org/3/library/logging.html#logging.info)
to print information to the console. Try to keep logging messages terse and
informative.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
