Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

2017-08-01 Thread Bolke de Bruin
Yes I think so. 

Bolke

Sent from my iPhone

> On 1 Aug 2017, at 00:59, Maxime Beauchemin  wrote:
> 
> Alright so here's the INSTALL file:
> https://github.com/apache/incubator-airflow/pull/2492
> 
> I'm thinking I'll cherry pick this in the 1.8-test branch and tar the whole
> repo, crank up the RC number and publish to the same location as before.
> 
> Am I on the right track?
> 
> Max
> 
> On Thu, Jul 27, 2017 at 11:12 AM, Chris Riccomini 
> wrote:
> 
>> Oky.. then I guess we can address the feedback above. Owe you some beers,
>> Max.
>> 
>> On Thu, Jul 27, 2017 at 10:56 AM, Bolke de Bruin 
>> wrote:
>> 
>>> Yep
>>> 
>>> Sent from my iPhone
>>> 
 On 27 Jul 2017, at 19:00, Maxime Beauchemin <
>> maximebeauche...@gmail.com>
>>> wrote:
 
 I need to re-package it with build instructions. I'm pretty sure this
>>> means
 another vote. I have time carved up to work on this today/tomorrow.
 
 Max
 
 On Thu, Jul 27, 2017 at 9:18 AM, Chris Riccomini <
>> criccom...@apache.org>
 wrote:
 
> IMO, given the level of effort for 1.8.2, and how long it's taken, we
> should not be re-voting right now unless something horrific happened
>> to
>>> the
> release.
> 
> On Tue, Jul 25, 2017 at 9:54 PM, Pavel Martynov 
> wrote:
> 
>> Hi, folks!
>> 
>> AIRFLOW-935 issue marked as resolved and fix version is 1.8.2, but
>> this
>> commit contained in master branch only and not tagged by 1.8.2rc2.
>> Can fix of this issue be released in 1.8.2?
>> 
>> 2017-07-26 2:27 GMT+03:00 George Leslie-Waksman <
>> geo...@cloverhealth.com.invalid>:
>> 
>>> I've checked and we are no longer relying on the previous
>>> LatestOnlyOperator behavior for any of our DAGs.
>>> 
>>> This is not a dealbreaker (though I will need to keep it in mind).
>>> 
>>> Thanks for asking,
>>> --George
>>> 
>>> On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
>>> maximebeauche...@gmail.com> wrote:
>>> 
 [AIRFLOW-1296] is part of 1.8.2.
 
 Is this a dealbreaker for 1.8.2?
 
 Max
 
 On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
 geo...@cloverhealth.com.invalid> wrote:
 
> I hope that it's not too late for me to chime in but there is a
>>> breaking
> change in the behavior of LatestOnlyOperator.
> 
> The change was introduced in
> PR: https://github.com/apache/incubator-airflow/pull/2365
> Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
>>> tasks
> 
> Prior to this change, the LatestOnlyOperator would skip direct
>>> downstream
> but not indirect downstream; now it skips indirect downstream.
> 
> This breaks the use of LatestOnlyOperator with TriggerRules that
>> do
>> not
> propagate skips.
> 
> --George
> 
> On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
> 
>> I think I'm gathering a good picture of what is expected here.
> I'll
>>> try
> to
>> update the Confluence page as I go.
>> 
>> I'm hoping to get started tomorrow and package it early next
> week.
>> 
>> Max
>> 
>> On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand <
>> san...@apache.org>
>> wrote:
>> 
>>> FYI, can anyone pictorially describe the release process (and
>> post
>>> it
> on
>>> the apache airflow wiki)? I think that would eliminate a lot of
> confusion
>>> in the future and avoid a rehash of this email thread on the
> next
>> release.
>>> 
>>> -s
>>> 
>>> On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah <
> hit...@apache.org
>>> 
> wrote:
>>> 
 To add, the main source tarball should have instructions to
 generate
>> the
 sdist and bdist versions. Additionally, as part of the
> release
> process
>> if
 the plan is to publish to pypi (after the IPMC vote
> succeeds),
>>> then
> the
 appropriate bits also need to be verified/voted upon. There
> are
>>> not
>>> exactly
 counted as the official release bits but they do need to be
 verified
> as
 part of the voting process to ensure that the bits do indeed
>> map
>>> to
> the
 source release, license/notice files are correct, etc.
 
 thanks
 -- Hitesh
 
 
 On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
 bdbr...@gmail.com>
 wrote:
 
> Thanks Hitesh. We discussed it with John Ament on the IPMC.
>

[VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC3

2017-08-01 Thread Maxime Beauchemin
1.8.2 RC3 is baked and available at:
https://dist.apache.org/repos/dist/dev/incubator/airflow, public keys
are available
at https://dist.apache.org/repos/dist/release/incubator/airflow.

This is a source release that comes with INSTALL instructions.

1.8.2 RC3 is build upon 1.8.1 with the commits listed bellow on top of it.
I added the JIRAs that were identified blockers and targeted 1.8.2. I
attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed on
the ones that were generating merge conflicts. I also added all of the
JIRAs that we've been running in production at Airbnb.

Issues fixed:
9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing
Booleans
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and create
writable warehouse dir
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
requirements
69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2
templates
6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
TaskInstanceModelView
d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
b4ee88a [AIRFLOW-1036] Randomize exponential backoff
9fca409 [AIRFLOW-993] Update date inference logic
272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
acd0166 [AIRFLOW-1263] Dynamic height for charts
7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
2e9eee3 [AIRFLOW-1282] Fix known event column sorting
2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
57d5bcd [AIRFLOW-1281] Sort variables by key field by default
802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
51a83cc [AIRFLOW-1199] Fix create modal
cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set
to True If we're looking for the closest date before, we should take the
latest date in the list of date before.
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
777f181 [AIRFLOW-1168] Add closing() to all connections and cursors

As part of the process I updated the release intructions here:
https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow

Max