Hello community,
While working on the documentation for the GCP operators, my team at
Polidea encountered some confusion related to the structure of the
documentation.
Short story:
We want to rewrite the `integration.rst` file so that it does not contain
duplicates from `code.rst ' (API Reference). In the next step, introduce
the reference API generation based on the source code that will replace the
`code.rst` file.
Long story:
Currently, the documentation contains two places where the description of
classes related to operators is included. They are `code.rst` and
`integration.rst` files.
The `integration.rst` file contains information about integration, in
particular for Azure: Microsoft Azure, AWS: Amazon Web Services,
Databricks, GCP: Google Cloud Platform, Qubole. Other integrations,
however, do not have descriptions.
The `code.rst` file contains “API Reference” which contains information
about *all* classes including those included in the file `integration.rst`.
Such duplication, however, is problematic for several reasons:
1.
Users may feel lost and may not know which section they should look into.
2.
Changes must be made in many places which leads to desynchronization.
Most often, changes are made only in the source code, so they do not appear
in the generated documentation.
3.
Linking to classes using the `class` directive for Sphinx is
inconclusive - if the code is embedded both in `integration.rst` and
`code.rst` using the `autoclass` directive, we’re not sure where the user
will be navigated.
There are several solutions::
1.
Leave it as is. Then we need to agree on which `autoclass` directive
should have the `no-index` flags.
2.
Delete duplicates from the `code.rst` file and add a note about the
`integration.rst` file in the `code.rst` file.
3.
Delete duplicates from the `integration.rst` file and add a note about
the `code.rst` file in the `integration.rst` file.
4.
Delete information from both files and generate the API documentation
always based only on the source code. This solution means that we would
have to write less documentation.
There are ready tools that we can use:
1.
epydoc - http://epydoc.sourceforge.net/ ;
2.
autoapi extension for Sphinx - https://github.com/rtfd/sphinx-autoapi
;
3.
other - https://wiki.python.org/moin/DocumentationTools
The first, second, third solution does not solve all problems. In
particular, we still need to complete the `code.rst` and `integration.rst`
files. The fourth solution solves all problems, but is the most complex. It
is worth noting that mixing solutions is possible. For example, we can
delete information from the file `integration.rst` as short term solution
and start working on creating another form of documentation as a long term
solution. This is the best option in our opinion.
I’ve recently done a few activities related to this topic.
First, I added the noindex flag to autoclass directives for all operators
in `integration.rst` file. In rare cases (If any), this caused links that
were previously directed to the file `integration.rst` to be redirected to
the `code.rst` file. Elements had to be linked using `:class:` instead of a
section link. Each operator is included in the new section in this file.
PR: https://github.com/apache/airflow/pull/4585
<https://github.com/apache/airflow/pull/4585/files>
Second, I completed the `code.rst` file with the missing classes.
PR: https://github.com/apache/airflow/pull/4644
I would like to ask which solution is the best in your opinion? What steps
should we take to make the documentation more enjoyable?
Greetings
Kamil Breguła
