TL;DR; We are looking for volunteers to help with testing backported providers package using airflow 1.10, so that we can release them. This way users will be able to use the latest and greatest Airflow providers in 1.10 without waiting for 2.0.
We are just about to finish system-testing the backported google package (and few others) and we are looking for volunteers who would like to help with testing the other packages. Just to remind - the basic idea is that we will release some of the provider's packages (those that we test) as apache-airflow-providers-PACKAGE and make it available for use for 1.10.* series. Some of those provider's operators are already improved vs. the operators that were released in 1.10 releases. By releasing the packages officially, users will have an easy way to install and start using and testing those Airflow 2.0 providers even before we release Airflow 2.0 and it will provide an easier migration path (as you will be able to change your DAGs to use the new operators without migrating to 2.0 even after we release it). We will release the packages after we test them - so if you are interested in using some provider in 1.10 - the best way is to help us to test it :D. What are we asking for? Three things: 1) Mark yourself as "maintainer" for the provider package that you would like to test: https://cwiki.apache.org/confluence/display/AIRFLOW/Backported+providers+packages+for+Airflow+1.10.*+series 2) Adding "System tests" in packages that do not have them. Basically it is all about turning the example_dags (that are mostly already there) into actually executable DAGs that use those operators. 3) Run those tests and confirming that they work fine for 1.10.9 at least (1.10.9 + 1.10.6 ideally), This might require your own system where you can run it on. We have an easy way to run those example dags as pytest tests - both using master and any airflow 1.10.* released version: https://github.com/apache/airflow/blob/master/TESTING.rst#airflow-system-tests Then you need to record status in https://cwiki.apache.org/confluence/display/AIRFLOW/Backported+providers+packages+for+Airflow+1.10.*+series I hope we can have help from you - and test as many providers as possible. Also, the system tests you will write and use will be then used to automate the whole process and run it in CI environment (following AIP-4 - Automation of system tests proposal) The most important (we think) packages that we need maintainers are: azure, aws, salesforce, snowflake, apache-spark, http but if you would like to help with other packages - feel free. You can join #system-tests channel on slack if you need any help or guidance. Looking forward to your help here! J. -- Jarek Potiuk Polidea | Principal Software Engineer M: +48 660 796 129
