This is an automated email from the ASF dual-hosted git repository.
sbp pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-trusted-releases.git
The following commit(s) were added to refs/heads/main by this push:
new 0a67178 Add more documentation about understanding the code
0a67178 is described below
commit 0a671785ccc85386e97de774a0c1ced61f45703f
Author: Sean B. Palmer <[email protected]>
AuthorDate: Wed Oct 8 17:25:26 2025 +0100
Add more documentation about understanding the code
---
atr/docs/developer-guide.html | 21 ++++++++++++++++++---
atr/docs/developer-guide.md | 36 +++++++++++++++++++++++++++++++++---
2 files changed, 51 insertions(+), 6 deletions(-)
diff --git a/atr/docs/developer-guide.html b/atr/docs/developer-guide.html
index 83f426b..b4b504b 100644
--- a/atr/docs/developer-guide.html
+++ b/atr/docs/developer-guide.html
@@ -37,8 +37,23 @@ make serve-local
<p><a
href="https://127.0.0.1:8080/";><code>https://127.0.0.1:8080/</code></a></p>
<p>Pick one or the other, because logging into the site on one host does not
log you in to the site on any other host.</p>
<h2 id="understanding-the-code">Understanding the code</h2>
-<p>Once you have the server running, you can test it. At this point, it is
useful to understand how the ATR works in general.</p>
-<p>ATR is an <a
href="https://github.com/apache/infrastructure-asfquart";>ASFQuart</a>
application, running in <a
href="https://hypercorn.readthedocs.io/en/latest/index.html";>Hypercorn</a>. The
entry point for Hypercorn is <a
href="/ref/atr/server.py:app"><code>atr.server:app</code></a>, which is then
called with a few standard arguments <a
href="https://asgi.readthedocs.io/en/latest/specs/main.html#overview";>as per
the ASGI specification</a>. We create the <code>app</code> object using t [...]
+<p>Once you have the server running, you can test it. At this point, it is
useful to understand how the ATR works in general. References to symbols in
this section are given without their <code>atr.</code> prefix, for brevity, and
are linked to the respective source code. You should understand e.g. <a
href="/ref/atr/server.py:app"><code>server.app</code></a> to mean
<code>atr.server.app</code>.</p>
+<h3 id="hypercorn-and-asgi">Hypercorn and ASGI</h3>
+<p>ATR is an <a
href="https://github.com/apache/infrastructure-asfquart";>ASFQuart</a>
application, running in <a
href="https://hypercorn.readthedocs.io/en/latest/index.html";>Hypercorn</a>. The
entry point for Hypercorn is <a
href="/ref/atr/server.py:app"><code>server.app</code></a>, which is then called
with a few standard arguments <a
href="https://asgi.readthedocs.io/en/latest/specs/main.html#overview";>as per
the ASGI specification</a>. We create the <code>app</code> object using the f
[...]
<pre><code class="language-python">app = create_app(config.get())
</code></pre>
-<p>The <a href="/ref/atr/server.py:create_app"><code>create_app</code></a>
function performs a lot of setup, and if you're interested in how the server
works then you can read it and the functions it calls to understand the process
further. In general, however, when developing ATR we do not work at the
ASFQuart, Quart, and Hypercorn levels very often.</p>
+<p>The <a
href="/ref/atr/server.py:create_app"><code>server.create_app</code></a>
function performs a lot of setup, and if you're interested in how the server
works then you can read it and the functions it calls to understand the process
further. In general, however, when developing ATR we do not make modifications
at the ASFQuart, Quart, and Hypercorn levels very often.</p>
+<h3 id="routes-and-database">Routes and database</h3>
+<p>Users request ATR pages over HTTPS, and the ATR server processes those
requests in route handlers. Most of those handlers are in <a
href="/ref/atr/routes/"><code>routes</code></a>, but not all. What each handler
does varies, of course, from handler to handler, but most perform at least one
access to the ATR SQLite database.</p>
+<p>The path of the SQLite database is configured in <a
href="/ref/atr/config.py:SQLITE_DB_PATH"><code>config.AppConfig.SQLITE_DB_PATH</code></a>
by default, and will usually appear as <code>state/atr.db</code> with related
<code>shm</code> and <code>wal</code> files. We do not expect ATR to have so
many users that we need to scale beyond SQLite.</p>
+<p>We use <a href="https://sqlmodel.tiangolo.com/";>SQLModel</a>, an ORM
utilising <a href="https://docs.pydantic.dev/latest/";>Pydantic</a> and <a
href="https://www.sqlalchemy.org/";>SQLAlchemy</a>, to create Python models for
the ATR database. The core models file is <a
href="/ref/atr/models/sql.py"><code>models.sql</code></a>. The most important
individual SQLite models in this module are <a
href="/ref/atr/models/sql.py:Committee"><code>Committee</code></a>, <a
href="/ref/atr/models/sql. [...]
+<p>It is technically possible to interact with SQLite directly, but we do not
do that in the ATR source. We use various interfaces in <a
href="/ref/atr/db/__init__.py"><code>db</code></a> for reads, and interfaces in
<a href="/ref/atr/storage/"><code>storage</code></a> for writes. We plan to
move the <code>db</code> code into <code>storage</code> too eventually, because
<code>storage</code> is designed to have read components and write components.
There is also a legacy <a href="/ref/atr [...]
+<p>These three interfaces, <a href="/ref/atr/routes/"><code>routes</code></a>,
<a href="/ref/atr/models/sql.py"><code>models.sql</code></a>, and <a
href="/ref/atr/storage/"><code>storage</code></a>, are where the majority of
activity happens when developing ATR.</p>
+<h3 id="user-interface">User interface</h3>
+<p>ATR provides a web interface for users to interact with the platform, and
the implementation of that interface is split across several modules. The web
interface uses server-side rendering almost entirely, where HTML is generated
on the server and sent to the browser.</p>
+<p>The template system in ATR is <a
href="https://jinja.palletsprojects.com/";>Jinja2</a>, always accessed through
the ATR <a href="/ref/atr/template.py"><code>template</code></a> module.
Template files in Jinja2 syntax are stored in <a
href="/ref/atr/templates/"><code>templates/</code></a>, and route handlers
render them using the asynchronous <a
href="/ref/atr/template.py:render"><code>template.render</code></a>
function.</p>
+<p>Template rendering can be slow if templates are loaded from disk on every
request. To address this, we use <a
href="/ref/atr/preload.py"><code>preload</code></a> to load all templates into
memory before the server starts serving requests. The <a
href="/ref/atr/preload.py:setup_template_preloading"><code>preload.setup_template_preloading</code></a>
function registers a startup hook that finds and caches every template
file.</p>
+<p>The ATR user interface includes many HTML forms. We use <a
href="https://wtforms.readthedocs.io/";>WTForms</a> for form handling, accessed
through the ATR <a href="/ref/atr/forms.py"><code>forms</code></a> module. The
<a href="/ref/atr/forms.py:Typed"><code>forms.Typed</code></a> base class
extends the standard <code>QuartForm</code> class in <a
href="https://quart-wtf.readthedocs.io/";>Quart-WTF</a>. Each form field is
created using helper functions such as <a href="/ref/atr/forms.py:s [...]
+<p>In addition to templates, we sometimes need to generate HTML
programmatically in Python. For this we use <a
href="https://htpy.dev/";>htpy</a>, another third party library, for building
HTML using Python syntax. The ATR <a
href="/ref/atr/htm.py"><code>htm</code></a> module extends htpy with a <a
href="/ref/atr/htm.py:Block"><code>Block</code></a> class that makes it easier
to build complex HTML structures incrementally. Using htpy means that we get
type checking for our HTML generation [...]
+<h3 id="other-important-interfaces">Other important interfaces</h3>
+<p>The server configuration in <a
href="/ref/atr/config.py"><code>config</code></a> determines a lot of global
state, and the <a href="/ref/atr/util.py"><code>util</code></a> module contains
lots of useful code which is used throughout ATR.</p>
diff --git a/atr/docs/developer-guide.md b/atr/docs/developer-guide.md
index d587add..73e3919 100644
--- a/atr/docs/developer-guide.md
+++ b/atr/docs/developer-guide.md
@@ -63,12 +63,42 @@ Pick one or the other, because logging into the site on one
host does not log yo
## Understanding the code
-Once you have the server running, you can test it. At this point, it is useful
to understand how the ATR works in general.
+Once you have the server running, you can test it. At this point, it is useful
to understand how the ATR works in general. References to symbols in this
section are given without their `atr.` prefix, for brevity, and are linked to
the respective source code. You should understand e.g.
[`server.app`](/ref/atr/server.py:app) to mean `atr.server.app`.
-ATR is an [ASFQuart](https://github.com/apache/infrastructure-asfquart)
application, running in
[Hypercorn](https://hypercorn.readthedocs.io/en/latest/index.html). The entry
point for Hypercorn is [`atr.server:app`](/ref/atr/server.py:app), which is
then called with a few standard arguments [as per the ASGI
specification](https://asgi.readthedocs.io/en/latest/specs/main.html#overview).
We create the `app` object using the following code:
+### Hypercorn and ASGI
+
+ATR is an [ASFQuart](https://github.com/apache/infrastructure-asfquart)
application, running in
[Hypercorn](https://hypercorn.readthedocs.io/en/latest/index.html). The entry
point for Hypercorn is [`server.app`](/ref/atr/server.py:app), which is then
called with a few standard arguments [as per the ASGI
specification](https://asgi.readthedocs.io/en/latest/specs/main.html#overview).
We create the `app` object using the following code:
```python
app = create_app(config.get())
```
-The [`create_app`](/ref/atr/server.py:create_app) function performs a lot of
setup, and if you're interested in how the server works then you can read it
and the functions it calls to understand the process further. In general,
however, when developing ATR we do not work at the ASFQuart, Quart, and
Hypercorn levels very often.
+The [`server.create_app`](/ref/atr/server.py:create_app) function performs a
lot of setup, and if you're interested in how the server works then you can
read it and the functions it calls to understand the process further. In
general, however, when developing ATR we do not make modifications at the
ASFQuart, Quart, and Hypercorn levels very often.
+
+### Routes and database
+
+Users request ATR pages over HTTPS, and the ATR server processes those
requests in route handlers. Most of those handlers are in
[`routes`](/ref/atr/routes/), but not all. What each handler does varies, of
course, from handler to handler, but most perform at least one access to the
ATR SQLite database.
+
+The path of the SQLite database is configured in
[`config.AppConfig.SQLITE_DB_PATH`](/ref/atr/config.py:SQLITE_DB_PATH) by
default, and will usually appear as `state/atr.db` with related `shm` and `wal`
files. We do not expect ATR to have so many users that we need to scale beyond
SQLite.
+
+We use [SQLModel](https://sqlmodel.tiangolo.com/), an ORM utilising
[Pydantic](https://docs.pydantic.dev/latest/) and
[SQLAlchemy](https://www.sqlalchemy.org/), to create Python models for the ATR
database. The core models file is [`models.sql`](/ref/atr/models/sql.py). The
most important individual SQLite models in this module are
[`Committee`](/ref/atr/models/sql.py:Committee),
[`Project`](/ref/atr/models/sql.py:Project), and
[`Release`](/ref/atr/models/sql.py:Release).
+
+It is technically possible to interact with SQLite directly, but we do not do
that in the ATR source. We use various interfaces in
[`db`](/ref/atr/db/__init__.py) for reads, and interfaces in
[`storage`](/ref/atr/storage/) for writes. We plan to move the `db` code into
`storage` too eventually, because `storage` is designed to have read components
and write components. There is also a legacy
[`db.interaction`](/ref/atr/db/interaction.py) module which we plan to migrate
into `storage`.
+
+These three interfaces, [`routes`](/ref/atr/routes/),
[`models.sql`](/ref/atr/models/sql.py), and [`storage`](/ref/atr/storage/), are
where the majority of activity happens when developing ATR.
+
+### User interface
+
+ATR provides a web interface for users to interact with the platform, and the
implementation of that interface is split across several modules. The web
interface uses server-side rendering almost entirely, where HTML is generated
on the server and sent to the browser.
+
+The template system in ATR is [Jinja2](https://jinja.palletsprojects.com/),
always accessed through the ATR [`template`](/ref/atr/template.py) module.
Template files in Jinja2 syntax are stored in
[`templates/`](/ref/atr/templates/), and route handlers render them using the
asynchronous [`template.render`](/ref/atr/template.py:render) function.
+
+Template rendering can be slow if templates are loaded from disk on every
request. To address this, we use [`preload`](/ref/atr/preload.py) to load all
templates into memory before the server starts serving requests. The
[`preload.setup_template_preloading`](/ref/atr/preload.py:setup_template_preloading)
function registers a startup hook that finds and caches every template file.
+
+The ATR user interface includes many HTML forms. We use
[WTForms](https://wtforms.readthedocs.io/) for form handling, accessed through
the ATR [`forms`](/ref/atr/forms.py) module. The
[`forms.Typed`](/ref/atr/forms.py:Typed) base class extends the standard
`QuartForm` class in [Quart-WTF](https://quart-wtf.readthedocs.io/). Each form
field is created using helper functions such as
[`forms.string`](/ref/atr/forms.py:string),
[`forms.select`](/ref/atr/forms.py:select), and [`forms.submit`] [...]
+
+In addition to templates, we sometimes need to generate HTML programmatically
in Python. For this we use [htpy](https://htpy.dev/), another third party
library, for building HTML using Python syntax. The ATR
[`htm`](/ref/atr/htm.py) module extends htpy with a
[`Block`](/ref/atr/htm.py:Block) class that makes it easier to build complex
HTML structures incrementally. Using htpy means that we get type checking for
our HTML generation, and can compose HTML elements just like any other Python
[...]
+
+### Other important interfaces
+
+The server configuration in [`config`](/ref/atr/config.py) determines a lot of
global state, and the [`util`](/ref/atr/util.py) module contains lots of useful
code which is used throughout ATR.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
