Hi Folks, We probably need to adjust our release process as can be observed in the IPMC thread. As we are packaging a “sdist” it does not pass license checks and one cannot verify the validity of what we are doing. It was suggested by one of the maintainers of another python project to create 3 different packages:
1. A source tarball which is essentially a snapshot of the repository 2. A sdist 3. A bdist 1 should then be the canonical Apache release. It should be accompanied by build instructions and it should pass RAT checks. This is the package we will vote on. 2 is what we have voted upon until now. It should contain (it does) LICENSE, NOTICE, and DISCLAIMER 3 bdist, wheel package. Same as 2. Not really required, but more a convenience package as is 2 2 and 3 can be published to PyPi. Max: can you take care of this? We need to vote on 1. Build instruction could be added to an INSTALL file or just to the README.md file? See for inspiration the GitHub page of ariatosca: https://github.com/apache/incubator-ariatosca <https://github.com/apache/incubator-ariatosca> In the meantime I am figuring out an issue with one of the dependencies of nvd3 which might be or have been GPL3 which is incompatible with the APL, which we are distributing together with our source. Ideally, we should move to a “yarn/webpack” build which will resolve those issues automatically as these become runtime dependencies then in case of 1,2,3. Cheers Bolke > On 6 Jul 2017, at 05:20, Sumit Maheshwari <sumeet.ma...@gmail.com> wrote: > > Awesome.. thanks a lot Max for taking the RM responsibility.. > > On Jul 5, 2017 11:10 PM, "Chris Riccomini" <criccom...@apache.org> wrote: > >> Nice, thanks Max! >> >> On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin < >> maximebeauche...@gmail.com> wrote: >> >>> Hello, >>> >>> Apache Airflow (incubating) 1.8.2 (RC2) has been accepted. >>> >>> 5 “+1” votes received, 4 out of which are "binding": >>> >>> - Maxime Beauchemin (binding) >>> - Bolke de Bruin (binding) >>> - Chris Riccomini (binding) >>> - Sumit Maheshwari (binding) >>> >>> - Kengo Seki (non-binding) >>> >>> Vote thread (start): >>> *http://mail-archives.apache.org/mod_mbox/incubator- >>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc-- >>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E >>> <http://mail-archives.apache.org/mod_mbox/incubator- >>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc-- >>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E>* >>> >>> Next steps: >>> 1) will start the voting process at the IPMC mailinglist. I don’t expect >>> changes. >>> 2) Only after the positive voting on the IPMC and finalisation I will >>> rebrand the RC to Release. >>> 3) I will upload it to the incubator release page, then the tar ball >> needs >>> to propagate to the mirrors. >>> 4) Update the website >>> 5) Finally, will upload to Pypi. >>> >>> Cheers, >>> >>> Max >>> >>> On Mon, Jul 3, 2017 at 12:11 PM, Bolke de Bruin <bdbr...@gmail.com> >> wrote: >>> >>>> Ping? >>>> >>>> >>>>> On 30 Jun 2017, at 14:21, Bolke de Bruin <bdbr...@gmail.com> wrote: >>>>> >>>>> Max, >>>>> >>>>> I think you can close the vote? >>>>> >>>>> Bolke >>>>> >>>>>> On 27 Jun 2017, at 02:45, Kengo Seki <sek...@apache.org> wrote: >>>>>> >>>>>> +1 (non-binding) >>>>>> >>>>>> - verified signatures and checksums >>>>>> - ran scheduler and webserver, confirmed they worked fine >>>>>> - confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included >>>>>> >>>>>> Kengo Seki <sek...@apache.org> >>>>>> >>>>>> >>>>>> 2017-06-27 8:53 GMT+09:00 Chris Riccomini <criccom...@apache.org>: >>>>>>> +1 (binding) >>>>>>> >>>>>>> Been running in our dev env, and everything looks good. >>>>>>> >>>>>>> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel < >> alex.guz...@airbnb.com >>> . >>>> invalid >>>>>>>> wrote: >>>>>>> >>>>>>>> Yeah that makes sense. It pages by default at 500 so it explains >> why >>>> we saw >>>>>>>> it. >>>>>>>> >>>>>>>> On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini < >>>> criccom...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> In 1.8.1, the "DAGs" page has "Show <blank> entries". In 1.8.2, >> it >>>> has >>>>>>>>> "Show <25> entries". So it looks like prior to 1.8.2, the >>> pagination >>>> was >>>>>>>>> broken in the sense that it defaulted to the whole list. We have >>> 479 >>>> DAGs >>>>>>>>> in one env, and it shows them all. It looks like someone fixed >> the >>>> entry >>>>>>>> to >>>>>>>>> default to 25 now, which exposed the problem for our >> environments. >>>>>>>>> >>>>>>>>> On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel < >>> alex.guz...@airbnb.com >>>> . >>>>>>>>> invalid >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> We're running 1.8.0 + some extras, and none of us added >> pagination >>>>>>>>>> recently, and our homepage is paginated. Are you sure it's not >> the >>>>>>>> number >>>>>>>>>> of dags crossing the threshold? Maybe it's some Flask version >>> thing? >>>>>>>>>> >>>>>>>>>> On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini < >>>>>>>> criccom...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Yes, I did the 1.8.1 release. >>>>>>>>>>> >>>>>>>>>>> On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel < >>>> alex.guz...@airbnb.com >>>>>>>> . >>>>>>>>>>> invalid >>>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> There's no pagination in 1.8.1? Are you sure? >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini < >>>>>>>>>> criccom...@apache.org> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> It's not happening on 1.8.1 (since there's no pagination in >>> that >>>>>>>>>>>> version), >>>>>>>>>>>>> so I'd count this as a regression. I wouldn't say it's >>> blocking, >>>>>>>>> but >>>>>>>>>>> it's >>>>>>>>>>>>> pretty ugly. >>>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel < >>>>>>>>> alex.guz...@airbnb.com >>>>>>>>>> . >>>>>>>>>>>>> invalid >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I'm not so sure this is a new issue. I think we've seen it >> on >>>>>>>> our >>>>>>>>>>>>>> production for quite a while. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini < >>>>>>>>>>>> criccom...@apache.org> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I am seeing a strange UI behavior on 1.8.2.RC2. I've >> opened a >>>>>>>>>> JIRA >>>>>>>>>>>>> here: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1348 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Has anyone else seen this? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari < >>>>>>>>>>>>>> sumeet.ma...@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> +1, binding. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin < >>>>>>>>>>> bdbr...@gmail.com >>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We have been running it for the last couple of days. No >>>>>>>>>> issues >>>>>>>>>>>> and >>>>>>>>>>>>>>> seems >>>>>>>>>>>>>>>>> more responsive. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> +1, binding >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Bolke >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 25 Jun 2017, at 01:10, Maxime Beauchemin < >>>>>>>>>>>>>>>> maximebeauche...@gmail.com> >>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Dear all, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1.8.2 RC2 is baked and available at: >>>>>>>>>>>>>>>>>> https://dist.apache.org/repos/ >>>>>>>> dist/dev/incubator/airflow >>>>>>>>> , >>>>>>>>>>>> public >>>>>>>>>>>>>>> keys >>>>>>>>>>>>>>>>>> are available >>>>>>>>>>>>>>>>>> at https://dist.apache.org/repos/ >>>>>>>>>>>> dist/release/incubator/airflow. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Note that RC1 was the first RC (skipped RC0) and was >>>>>>>>> never >>>>>>>>>>>>>> announced >>>>>>>>>>>>>>>>> since >>>>>>>>>>>>>>>>>> it had issues coming out of the oven, so RC2 is the >>>>>>>> first >>>>>>>>>>>> public >>>>>>>>>>>>>> RC. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 1.8.2 RC2 is build on to of 1.8.1 with these listed >>>>>>>>>>> "cherries" >>>>>>>>>>>> on >>>>>>>>>>>>>>> top. >>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>> added the JIRAs that were identified blockers and >>>>>>>>> targeted >>>>>>>>>>>>> 1.8.2. I >>>>>>>>>>>>>>>>>> attempted to bring in all of the JIRAs that targeted >>>>>>>>> 1.8.2 >>>>>>>>>>> but >>>>>>>>>>>>>> bailed >>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>> the ones that were generating merge conflicts. I also >>>>>>>>> added >>>>>>>>>>> all >>>>>>>>>>>>> of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> JIRAs that we've been running in production at Airbnb. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Issues fixed: >>>>>>>>>>>>>>>>>> 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ >>>>>>>>> ColumnOperator >>>>>>>>>>> When >>>>>>>>>>>>>>> Testing >>>>>>>>>>>>>>>>>> Booleans >>>>>>>>>>>>>>>>>> 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all >>>>>>>>> downstream >>>>>>>>>>>> tasks >>>>>>>>>>>>>>>>>> 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop >>>>>>>>>> components >>>>>>>>>>>>>>>>>> 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific >>>>>>>>>>> version >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> create >>>>>>>>>>>>>>>>>> writable warehouse dir >>>>>>>>>>>>>>>>>> 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask >>>>>>>>>>>>>>>>>> 570b2ed [AIRFLOW-1294] Backfills can loose tasks to >>>>>>>>> execute >>>>>>>>>>>>>>>>>> 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files >>>>>>>> to >>>>>>>>>>> match >>>>>>>>>>>>> ASF >>>>>>>>>>>>>>>>>> requirements >>>>>>>>>>>>>>>>>> 69bd269 [AIRFLOW-1160] Update Spark parameters for >>>>>>>> Mesos >>>>>>>>>>>>>>>>>> 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom >>>>>>>>>> filters >>>>>>>>>>>> in >>>>>>>>>>>>>>> Jinja2 >>>>>>>>>>>>>>>>>> templates >>>>>>>>>>>>>>>>>> 6de5330 [AIRFLOW-1119] Fix unload query so headers are >>>>>>>> on >>>>>>>>>>> first >>>>>>>>>>>>>> row[] >>>>>>>>>>>>>>>>>> b4e9eb8 [AIRFLOW-1089] Add Spark application arguments >>>>>>>>>>>>>>>>>> a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old >>>>>>>>>> flask >>>>>>>>>>>>>>> versions >>>>>>>>>>>>>>>>>> 7a02841 [AIRFLOW-1074] Don't count queued tasks for >>>>>>>>>>> concurrency >>>>>>>>>>>>>>> limits >>>>>>>>>>>>>>>>>> a2c18a5 [AIRFLOW-1064] Change default sort to job_id >>>>>>>> for >>>>>>>>>>>>>>>>>> TaskInstanceModelView >>>>>>>>>>>>>>>>>> d1c64ab [AIRFLOW-1038] Specify celery serialization >>>>>>>>> options >>>>>>>>>>>>>>> explicitly >>>>>>>>>>>>>>>>>> b4ee88a [AIRFLOW-1036] Randomize exponential backoff >>>>>>>>>>>>>>>>>> 9fca409 [AIRFLOW-993] Update date inference logic >>>>>>>>>>>>>>>>>> 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook >>>>>>>>>>>>> modification >>>>>>>>>>>>>>>> time >>>>>>>>>>>>>>>>>> c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google >>>>>>>>>>> BigQuery >>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>> acd0166 [AIRFLOW-1263] Dynamic height for charts >>>>>>>>>>>>>>>>>> 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis >>>>>>>>>>>>>>>>>> fc33c04 [AIRFLOW-1290] set docs author to 'Apache >>>>>>>>> Airflow' >>>>>>>>>>>>>>>>>> 2e9eee3 [AIRFLOW-1282] Fix known event column sorting >>>>>>>>>>>>>>>>>> 2389a8a [AIRFLOW-1166] Speed up >>>>>>>>>>> _change_state_for_tis_without_ >>>>>>>>>>>>>> dagrun >>>>>>>>>>>>>>>>>> bf966e6 [AIRFLOW-1192] Some enhancements to >>>>>>>>> qubole_operator >>>>>>>>>>>>>>>>>> 57d5bcd [AIRFLOW-1281] Sort variables by key field by >>>>>>>>>> default >>>>>>>>>>>>>>>>>> 802fc15 [AIRFLOW-1244] Forbid creation of a pool with >>>>>>>>> empty >>>>>>>>>>>> name >>>>>>>>>>>>>>>>>> 1232b6a [AIRFLOW-1243] DAGs table has no default >>>>>>>> entries >>>>>>>>> to >>>>>>>>>>>> show >>>>>>>>>>>>>>>>>> b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs >>>>>>>>> view >>>>>>>>>>>>>>>>>> c406652 [AIRFLOW-1226] Remove empty column on the Jobs >>>>>>>>> view >>>>>>>>>>>>>>>>>> 51a83cc [AIRFLOW-1199] Fix create modal >>>>>>>>>>>>>>>>>> cac7d4c [AIRFLOW-1200] Forbid creation of a variable >>>>>>>> with >>>>>>>>>> an >>>>>>>>>>>>> empty >>>>>>>>>>>>>>> key >>>>>>>>>>>>>>>>>> 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by >>>>>>>>>>>>>> execution_date >>>>>>>>>>>>>>>>>> f446c08 [AIRFLOW-1145] Fix closest_date_partition >>>>>>>>> function >>>>>>>>>>> with >>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>> set >>>>>>>>>>>>>>>>>> to True If we're looking for the closest date before, >>>>>>>> we >>>>>>>>>>> should >>>>>>>>>>>>>> take >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> latest date in the list of date before. >>>>>>>>>>>>>>>>>> 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for >>>>>>>>>>>>>> test_csrf_rejection >>>>>>>>>>>>>>>>>> bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts >>>>>>>>>>> parameters >>>>>>>>>>>>>>>>> separately >>>>>>>>>>>>>>>>>> 093b2f0 [AIRFLOW-1150] Fix scripts execution in >>>>>>>> sparksql >>>>>>>>>>> hook[] >>>>>>>>>>>>>>>>>> 777f181 [AIRFLOW-1168] Add closing() to all connections >>>>>>>>> and >>>>>>>>>>>>> cursors >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Max >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>>> >>>> >>> >>
