I am in favor of this change. However, do I correctly understand that providers.transfer will need many other providers (GCS=gcp, S3=aws, etc)?
T. On Sat, May 30, 2020 at 2:28 PM Kamil Breguła <kamil.breg...@polidea.com> wrote: > > Hello, > > We currently have transfer operators and other operators in one > package - operators. > This causes some packages to be large and will grow all the time. For > example: airflow.providers.google.cloud.operators have 52 modules > For this reason, the idea arose to move these operators to the new package. > As a result, we will have the following provider package structure. > > airflow.providers.*.example_dags > airflow.providers.*.hooks > airflow.providers.*.operators > airflow.providers.*.secrets > airflow.providers.*.sensors > airflow.providers.*.transfer > airflow.providers.*.utils > > Such a division already exists in the documentation. We distinguish > operators, transfer operators, and sensors. > https://airflow.readthedocs.io/en/latest/_api/index.html#operators > https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html > In my opinion, this change has many advantages: > * the documentation and the user habits would more suit the structure > of the code, > * would facilitate browsing of operators, > * would allow more comfortable documentation generation based on the > directory structure. It cannot be determined now whether > cloud_speech_to_text is transfer operators or a regular operator based > on the module name. > > The only minus is the need to change users' habits. Users will have to > start looking for operators in the new package, but if they find this > package, they will find the module faster. > > Best regards, > Kamil
