Hello everyone, I am Robin Jarry. Among other things, I maintain the aerc email client (https://sr.ht/~rjarry/aerc/) and I have been a Patchwork user for a long time. Over the past couple of months, I have been working on a ground-up rewrite of Patchwork with the support of Stephen Finucane. The code lives under the official organization:
https://github.com/getpatchwork/patchwork-next https://getpatchwork.github.io/patchwork-next/ I know "rewrite from scratch" triggers alarm bells, so let me explain what motivated this and where things stand. The problem ----------- Patchwork works. It has served the community well for years. But some long-standing issues are getting harder to ignore: The REST API is slow. This is not news to anyone running a public instance. The patchwork.ozlabs.org and patches.dpdk.org deployments have both experienced outages when too many users or CI systems hit the API concurrently. The codebase itself acknowledges this: multiple fields in the API serializers are marked "too slow to calculate", patch tags are disabled entirely because they are not performant, and offset-based pagination becomes prohibitively expensive on large projects. Several past releases have listed performance fixes as their headline feature, yet the issues keep coming back because they are rooted in the framework layer, not in the application logic. Deployment is heavy. A production setup requires assembling a Python virtualenv, a WSGI server (uWSGI or Gunicorn), a reverse proxy (nginx or Apache), a mail retriever (getmail), systemd units for each, and a cron job. The installation guide is over 700 lines. Each component has its own configuration surface and failure modes. Administration depends on django-admin, which is powerful but opaque to anyone who is not a Django developer. The development toolchain is painful. Anyone who has dealt with Python packaging knows the drill: virtualenvs, pip, setuptools, tox, multiple requirements files, version pinning, transitive dependency conflicts. The CI matrix tests 6 combinations of Python and database versions. Getting a working dev environment takes non-trivial effort and the experience varies across platforms. What I built ------------ Patchwork 4.0 compiles to a single static binary. Deployment means one binary, one TOML config file, one systemd unit. It embeds everything: HTTP server, REST API (versions 1.0 through 1.4), server-side rendered web UI, SMTP/LMTP mail ingress, schema migrations, and a full admin CLI. No reverse proxy, no virtualenv, no mail retriever daemon needed. The choice of implementation language was driven by practical concerns. Go has a standardized, batteries-included toolchain: one command to build, test, format, lint, and manage dependencies. No virtualenv, no tox, no pip, no setuptools. The standard library provides production-grade HTTP servers, and the ecosystem has excellent email libraries (go-message, go-smtp, go-mbox from Simon Ser) that I already use in aerc. It is a natural fit for a network service that needs to handle concurrent HTTP and SMTP traffic efficiently. The whole application is about 15,000 lines of code (comparable to the Django version excluding tests and migrations) and compiles in seconds. It supports PostgreSQL, MySQL, and SQLite. The SQLite support is particularly interesting: with WAL mode, it makes Patchwork viable for small deployments without running a database server at all. The REST API versions 1.0 through 1.4 are compatible with the existing Django-based implementation. Existing clients (git-pw, snowpatch, or custom scripts) should work without modification. The legacy XML-RPC API has been removed entirely. A few more things worth highlighting: - The built-in SMTP/LMTP server receives mail directly, eliminating the getmail/pipe/management-command chain. On busy lists, not spawning a full interpreter with the whole django framework overhead per message makes a real difference. - API versioning is handled with struct tags instead of duplicated serializer classes. All five versions share the same code path. - Webhook delivery with HMAC signatures is built in, something that has been requested for years. - Mutating HTTP requests are automatically wrapped in database transactions; read-only requests skip the overhead. Bridging email workflows and forges ------------------------------------ Some of you may remember the email I sent back in March 2024 about bidirectional synchronization between mailing lists and code forges: https://www.mail-archive.com/[email protected]/msg07327.html This idea gained traction at the DPDK summit in Stockholm this past May, where we observed first-hand how the lack of a bridge between email-based workflows and git forges is a real barrier for new contributors. I have started working on this as a companion project: https://github.com/getpatchwork/pwforge pwforge is a service that listens to Patchwork webhooks and forge events to automatically create pull requests from mailing list patch series, mirror patch comments as PR reviews, and send forge comments back to the mailing list as properly threaded email replies. The webhook system in Patchwork 4.0 was designed with exactly this use case in mind. Migration path -------------- The new version uses a different database schema, but ships with a migration tool. `pw db export` reads directly from an existing Django database and produces SQL for the new schema, handling the structural differences automatically. This has been tested with databases that include local, non-standard customizations, so existing deployments should be able to migrate without losing data. The original copyrights and license (GPL-2.0-or-later) have been preserved, and all contributors to the Django codebase are acknowledged in the CONTRIBUTORS file. What is next ------------ I am not proposing to deprecate the Django version tomorrow. The rewrite needs more eyes, more testing, and feedback from people running real instances. Stephen has been involved from the start and developers from the DPDK community have expressed interest in contributing. If you run a Patchwork instance, I would love to hear what matters most to you. If you want to try it out, the documentation should get you started. Bug reports, feature requests, and patches are all welcome. If you are curious, I have taken an anonymized export of the DPDK.org patchwork database and deployed both implementations for comparison purposes: https://pw3.jarry.cc/ https://pw4.jarry.cc/ Cheers, Robin _______________________________________________ Patchwork mailing list [email protected] https://lists.ozlabs.org/listinfo/patchwork
