Fixed :). Shift + Refresh On Fri, May 29, 2020 at 12:26 PM Jarek Potiuk <[email protected]> wrote:
> Sha of sha .... Wow.. That's something like master level of security :). > > On Fri, May 29, 2020 at 10:48 AM Kaxil Naik <[email protected]> wrote: > >> I think we have some extra files at >> >> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/ >> (maybe >> erroneously generated by the Bash scripts) >> >> Examples: >> >> - >> >> >> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512 >> < >> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.asc.sha512.sha512 >> > >> - >> >> >> apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512 >> < >> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-amazon-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512 >> > >> - >> >> >> apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512 >> < >> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/apache-airflow-backport-providers-apache-cassandra-2020.5.20rc3-bin.tar.gz.sha512.sha512.sha512 >> > >> >> >> Can we remove those first, please, they are not a blocker to the release >> but makes it difficult to view the actual packages? >> >> Regards, >> Kaxil >> >> On Thu, May 28, 2020 at 6:22 PM Jarek Potiuk <[email protected]> >> wrote: >> >> > Looking forward to some PMC votes :) But happy to hear more feedback >> from >> > others! >> > >> > Just a kind reminder (not sure if visible to everyone but let me try): >> > >> > [image: majestic_rabbit.jpg] >> > >> > Fri May 29 22:42:38 CEST 2020. >> > >> > J. >> > >> > >> > On Tue, May 26, 2020 at 11:47 PM Jarek Potiuk <[email protected] >> > >> > wrote: >> > >> >> Indeed What an oversight :). Thanks for +1 - even non-binding is super >> >> important :) Anyone else trying the backport packages is great - I >> think >> >> it's super important for the Airflow community and great step to ease >> >> migration to 2.0 :D >> >> >> >> J. >> >> >> >> >> >> >> >> On Tue, May 26, 2020 at 11:12 PM Bas Harenslak >> >> <[email protected]> wrote: >> >> >> >>> Thanks for the compliment but I’m no PMC so here’s my non-binding +1 >> :-) >> >>> >> >>> Bas >> >>> >> >>> On 26 May 2020, at 22:43, Jarek Potiuk <[email protected] >> <mailto: >> >>> [email protected]>> wrote: >> >>> >> >>> Hey all, >> >>> >> >>> I have cut Airflow Backport Providers 2020.5.20rc3. This email is >> >>> calling a >> >>> vote on the release, >> >>> which will last for 72 hours - which means that it will end on Fri >> May 29 >> >>> 22:42:38 CEST 2020. >> >>> >> >>> Consider this my (binding) +1. >> >>> >> >>> The changes vs. rc2 are explained at the end of the email. >> >>> >> >>> Airflow Backport Providers 2020.5.20rc3 are available at: >> >>> >> >>> >> https://dist.apache.org/repos/dist/dev/airflow/backport-providers/2020.5.20rc3/ >> >>> >> >>> *apache-airflow-backport-providers-2020.5.20rc3-source.tar.gz* is a >> >>> source >> >>> release that comes >> >>> with INSTALL instructions. >> >>> >> >>> *apache-airflow-backport-providers-<PROVIDER>-2020.5.20rc3-bin.tar.gz* >> >>> are >> >>> the binary >> >>> Python "sdist" release. >> >>> >> >>> Public keys are available at: >> >>> https://dist.apache.org/repos/dist/release/airflow/KEYS >> >>> >> >>> Please vote accordingly: >> >>> >> >>> [ ] +1 approve >> >>> [ ] +0 no opinion >> >>> [ ] -1 disapprove with the reason >> >>> >> >>> >> >>> Only votes from PMC members are binding, but members of the community >> are >> >>> encouraged to test the release and vote with "(non-binding)". >> >>> >> >>> Please note that the version number excludes the 'rcX' string, so it's >> >>> now >> >>> simply 2020.5.20. This will allow us to rename the artifact without >> >>> modifying >> >>> the artifact checksums when we actually release it. >> >>> >> >>> Each of the packages contains a detailed changelog. Here is the list >> of >> >>> links to >> >>> the released packages and changelogs: >> >>> >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-amazon/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-cassandra/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-druid/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-hdfs/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-hive/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-livy/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-pig/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-pinot/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-spark/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-apache-sqoop/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-celery/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-cloudant/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-databricks/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-datadog/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-dingding/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-discord/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-docker/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-elasticsearch/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-email/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-exasol/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-facebook/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-ftp/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-google/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-grpc/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-hashicorp/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-http/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-imap/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-jdbc/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-jenkins/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-jira/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-microsoft-azure/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-microsoft-mssql/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-microsoft-winrm/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-mongo/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-mysql/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-odbc/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-openfaas/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-opsgenie/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-oracle/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-pagerduty/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-postgres/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-presto/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-qubole/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-redis/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-salesforce/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-samba/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-segment/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-sftp/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-singularity/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-slack/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-snowflake/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-sqlite/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-ssh/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-vertica/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-yandex/2020.5.20rc3/ >> >>> >> >>> >> https://pypi.org/project/apache-airflow-backport-providers-zendesk/2020.5.20rc3/ >> >>> >> >>> J. >> >>> >> >>> >> >>> Appendix: The changes vs. RC2: >> >>> >> >>> * During testing we found a few (20 out of 537) inconsistencies in the >> >>> names of the classes vs. the agreed AIP-21 standard. We fixed them all >> >>> and >> >>> also introduced automated tests that will fail PR in case someone >> adds an >> >>> inconsistently named class. >> >>> >> >>> * Several packages failed to import cleanly in 1.10. That was a great >> >>> idea >> >>> from one of the PMCs (Bas Hanrenslak) to automatically import all the >> >>> classes from all the providers and see if they are importable in >> 1.10. We >> >>> identified a few operators in a few providers (papermill, odbc, hive, >> >>> and a >> >>> few others) that would not import properly on 1.10. This is all now >> fixed >> >>> (except papermill - see below). The good thing is that we also added >> >>> automated tests for this: so we will be testing the importability of >> >>> backported packages with every PR. This means that the backport >> packages >> >>> will continue to be importable in the future and we check it with >> every >> >>> single PR! >> >>> >> >>> Of course, importability does not guarantee that the operators will >> work >> >>> with !00% certainty - this can only be assured by automated system >> tests >> >>> - >> >>> this is something we will be working on the coming months. >> >>> >> >>> * Papermill operator has proven to be problematic. It uses the Airflow >> >>> 2.0 >> >>> lineage functionality that is not compatible with 1.10. That's why we >> >>> decided to remove papermill from the released backport packages and >> >>> remove >> >>> the previous RCs for it. >> >>> >> >>> * The most important reason why we needed RC3. We noticed that while >> the >> >>> README package instructions were all fine, the CHANGELOG and INSTALL >> in >> >>> source packages contained changelog/Install instructions for Airflow >> 2.0. >> >>> We fixed that and the INSTALL and CHANGELOG in the packages contain >> >>> instructions/changelog for the backport packages only, not for the >> whole >> >>> Airflow. >> >>> >> >>> Cheers, >> >>> >> >>> Jarek >> >>> >> >>> -- >> >>> >> >>> Jarek Potiuk >> >>> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >>> >> >>> M: +48 660 796 129 <+48660796129> >> >>> [image: Polidea] <https://www.polidea.com/> >> >>> >> >>> >> >> >> >> -- >> >> >> >> Jarek Potiuk >> >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> >> >> M: +48 660 796 129 <+48660796129> >> >> [image: Polidea] <https://www.polidea.com/> >> >> >> >> >> > >> > -- >> > >> > Jarek Potiuk >> > Polidea <https://www.polidea.com/> | Principal Software Engineer >> > >> > M: +48 660 796 129 <+48660796129> >> > [image: Polidea] <https://www.polidea.com/> >> > >> > >> > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
