Hello everyone,

*TL;DR; Just wanted to make you aware that we have now some "refresh" of
the https://airflow.apache.org <https://airflow.apache.org> to make it more
in-line with the policies and expectations of the ASF*

If you want to read more about it and some details - feel free to dive
deeper:

# Changes deployed recently

* Important change ! We only show the latest patchlevel version of airflow,
providers, task0-sdk, helm-chart  - you can still get to any version with
version in URL or when you have links to old documentation.
Version selector is **finally** usable.

* We are linking directly to the Privacy Policy of the ASF from the footer
(as required by the ASF)

* We are linking to "https://events.apache.org"; - also as expected by the
ASF policies

* Our footers are now (hopefully) fully consistent across all 250K (!!!)
pages we have - including all versions of packages

* We are using CSP builder for Content-Security-Policy
https://infra.apache.org/tools/csp.html  - which is a new way how we can
embed external sources (youtube videos) - I am still in the process of
verifying if particular use of it in 'Meetups" page is OK With ASF privacy
policy, but I think we are pretty close if not "perfect" when it comes to
compliance with ASF expectations.

* We will also have Airflow CTL documentation linked once we release the
1.0.0 version (the voting is in progress)

# Comment on doc infra tooling

Just a comment - that was all rather easy with the "infrastructure" we
worked on with Pavan mostly - and it has proven to be working really well.
We have a Git-archive where we can manipulate all historical versions where
we can manipulate all the "versioned" docs, we have workflows in CI that
sync changes back/forth and finally I could close the issue
https://github.com/apache/airflow/issues/44373  - which has 17 subtasks
completed since 26 th Novemeber last year where I raised the "we need to
finally modernise our docs infra".

Now I think we are fully ready to support anyone who would like to take any
steps with design changes or structure changes to the docs of ours - it
should be easier than ever and we have the right tooling in place to make
even very complex changes. We have fully automated workflows that are used
by release managers, including staging site generation for RC candidates,
automated cache invalidation when needed etc. etc. If anyone would like to
take a stab at restructuring some of our docs, improving the content,
navigation etc, we are ready to support that rather easily from the
"infrastructure" side - including testing things on staging end-2-end
before releasing it to the public.

This is thanks to many people - Pavan, Amogh, Shahar, Ankit, Ash, Kaxil.
myself - and likely a number of other people with smaller changes - spanned
over quite a long time - but done in parallel to all our Airflow 3.0 and
3.1 efforts without disrupting those. You can see the result of it in our
documentation building architecture
https://github.com/apache/airflow/tree/main/docs

# LLM support:

Also another shout-out to AI and LLMS helping in such efforts. For the big
"consistency" change I used (while in train to Vienna)  GH Copilot + Claude
Sonet 4.5 to make the bulk changes on all historical variants of the
footers we had for 250K files. It turned out to be really helpful when
asked to synchronize all the footers - it took almost "0" effort from my
side.

I literally just overlooked what it was doing when it was plowing through
250K pages to fix after getting the right instructions (the AI even wrote a
temporary python script that I reviewed to apply the fix that it deleted
afterwards).

Copilot / Claude even used interesting ways of verifying if there is more
work to do, It did not only run all the changes but it actually attempted
to find and fix all other similar cases - all on its own - including
various ways of checking it for 250K files - like getting some subselection
of files to look at in various sequences an random sampling of pages.
Pretty neat when you consider that it took me just a few minutes to write
(long and comprehensive though) prompt/instructions.

I hope it worked well ;) but from what I checked, it did. The diff of the
change is so huge that it does not display nicely in GitHub but if you are
courious -
https://github.com/apache/airflow-site-archive/commit/64ff61087317802f76cbb4e3f44789a557d5e331


J.

Reply via email to