This is looking good. One other thing we talked about is that the
autopkgtest-cloud-worker units actually pull from the amqp server and inputs
push items onto the queue.
We also talked about using debci-$release-$arch as those are well understood
variables. It might also be good to add the other types of queues e.g.
debci-ppa-$release-$arch, debci-upstream-$release-$arch,
debci-huge-$release-$arch.
I've also a few in-line comments.
Diff comments:
> diff --git a/docs/architecture.rst b/docs/architecture.rst
> index 4a31322..2f4c446 100644
> --- a/docs/architecture.rst
> +++ b/docs/architecture.rst
> @@ -11,18 +11,22 @@ Overview
> subgraph cluster_autopkgtestcloud {
> subgraph cluster_prodstack {
> node [shape=Mrecord]
> - web [label="{autopkgtest-web web UI|results browser for
> developers|shows which tests are running|receives requests to run tests from
> developers or GitHub PRs|can be scaled out}"]
> + web [label="{autopkgtest-web web UI|* results browser for
> developers\n* shows which tests are running\n* receives requests to run tests
> from developers or GitHub PRs\n* can be scaled out}"]
> rabbitmq [label="{RabbitMQ AMQP
> server|debci-focal-amd64|...}"]
> - cloudworker [label="{cloud worker|call autopkgtest with
> ssh+nova (openstack) runner}"]
> - lxdworker [label="{cloud worker (lxd)|call autopkgtest with
> lxd runner}"]
> + cloudworker [label="{autopkgtest-cloud-worker|call
> autopkgtest with ssh+nova (openstack) runner}"]
> + lxdworker [label="{autopkgtest-cloud-worker (lxd)|call
> autopkgtest with lxd runner}"]
Let's use the application name here which is autopkgtest-lxd-worker.
> swift [label="{OpenStack Swift object store (test
> results)|autopkgtest-focal|...}"]
> - haproxyweb [label="{autopkgtest.ubuntu.com haproxy|provides
> SSL termination|https://autopkgtest.ubuntu.com}";]
> - haproxylxd [label="{haproxy-armhf-lxd|mediates access to lxd
> cluster to avoid SPOF}"]
> + swiftprivate [label="{OpenStack Swift object store|private
> PPA test results, embargoed CVEs}"]
> + haproxyweb [label="{HAproxy|* provides SSL termination\n*
> https://autopkgtest.ubuntu.com}";]
> + runner [label="{OpenStack Instances|dynamically allocated
> test runners|amd64, arm64, i386, ppc64el, s390x}"]
Only the amd64 instances (which run i386) tests are actually in prodstack5. All
port architectures are in scalingstack at this point in time.
>
> + "cloudworker" -> "runner"
> "rabbitmq" -> "cloudworker"
> "rabbitmq" -> "lxdworker"
> "cloudworker" -> "swift"
> + "cloudworker" -> "swiftprivate"
> "lxdworker" -> "swift"
> + "lxdworker" -> "swiftprivate"
> "swift" -> "web" [label="download new results into database"]
> "web" -> "swift" [label="link to logs and artifacts"
> style="dotted"]
> "haproxyweb" -> "web" [label="one proxy, many backends"]
> @@ -32,10 +36,8 @@ Overview
> }
> subgraph cluster_scalingstack {
> node [shape=Mrecord]
> - runner [label="{dynamically allocated test runners|amd64,
> arm64, i386, ppc64el, s390x|openstack instances}"]
> - static [label="{statically provisioned lxd runners|arm64
> host|armhf tests|many of these|running lxd in an lxd cluster}"]
> + static [label="{statically provisioned lxd runners|arm64
> host|armhf tests|all of these running lxd in an lxd cluster}"]
I'm pretty sure that these are not in a lxd cluster there is some documentation
which indicates the lxd cluster is unused.
>
> - "cloudworker" -> "runner"
> "lxdworker" -> "static"
>
> label="'scalingstack' workload OpenStack cloud"
> @@ -46,9 +48,19 @@ Overview
> subgraph cluster_external {
> node [shape=Mrecord]
> archive [label="{Ubuntu developers|packages in -proposed}"]
> - britney [label="{britney|gates pkgs from devel-proposed to
> release pocket|completed builds|installability|autopkgtests|generates
> update_excuses.html/update_output.txt}"]
> + britney [label="{britney|* gates pkgs from -proposed to release
> pocket\n* completed builds\n* installability\n* autopkgtests\n* generates
> update_excuses.html/update_output.txt}"]
"non-regressing(?) autopkgtests" something like that would be clarify that not
all autopkgtests need to pass.
> +
> "archive" -> "britney" [label="Ubuntu developers upload
> packages"]
> "britney" -> "rabbitmq" [label="send AMQP requests for tests"]
> +
> + archivesecure [label="{Ubuntu security team|packages in ???}"]
??? == PPAs
> + britneysecure [label="{britney-secure}"]
It's actually referred to as security-britney.
> + "archivesecure" -> "britneysecure" [label="Ubuntu security team
> upload packages"]
> + "britneysecure" -> "rabbitmq" [label="send AMQP requests for
> tests"]
> +
> + thirdinput [label="{??? Third input}"]
Ubuntu developers can upload packages to PPA and then request that tests be run
using the packages in those PPAs. The tests are actually requested via the web
UI and then go to amqp but that seems overly complicated.
A fourth input is packages being sync'ed from Debian to Ubuntu and britney ends
up queuing tests for those package syncs.
> + "thirdinput" -> "rabbitmq" [label="send AMQP requests for tests"]
> +
>
> color=green
> label="external inputs"
--
https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/456376
Your team Canonical's Ubuntu QA is requested to review the proposed merge of
~hyask/autopkgtest-cloud:skia/docs_archi_diagram into autopkgtest-cloud:master.
--
Mailing list: https://launchpad.net/~canonical-ubuntu-qa
Post to : [email protected]
Unsubscribe : https://launchpad.net/~canonical-ubuntu-qa
More help : https://help.launchpad.net/ListHelp
