The fix is in master and should work across all DST changes. It will be
included in 1.10.1.
B.
Sent from my iPhone
> On 5 Nov 2018, at 19:54, Dave Fisher wrote:
>
>
>
>> On 2018/10/28 00:09:05, Bolke de Bruin wrote:
>> I wonder how to treat this:
>>
>> This is what I think happens (nee
On 2018/10/28 00:09:05, Bolke de Bruin wrote:
> I wonder how to treat this:
>
> This is what I think happens (need to verify more, but I am pretty sure) the
> specified DAG should run every 5 minutes. At DST change (3AM -> 2AM)
FYI - In the US the DST change is 2AM -> 1AM. Yes, TZ is hard s
They are :-)
I might have to adjust some tests, let's see. Will do that tomorrow if required.
B.
> On 30 Oct 2018, at 21:53, Ash Berlin-Taylor wrote:
>
> Fair :) Timezones are _hard_
>
> Giving it a look now.
>
> -ash
>
>> On 30 Oct 2018, at 20:50, Bolke de Bruin wrote:
>>
>> The reason f
Fair :) Timezones are _hard_
Giving it a look now.
-ash
> On 30 Oct 2018, at 20:50, Bolke de Bruin wrote:
>
> The reason for not passing a TZ aware object is, is that many libraries make
> mistakes (pytz, arrow etc) when doing transitions hence to use of pendulum
> which seem most complete.
The reason for not passing a TZ aware object is, is that many libraries make
mistakes (pytz, arrow etc) when doing transitions hence to use of pendulum
which seem most complete. I don’t know what croniter is relying on and I don’t
want to find out ;-).
B.
> On 30 Oct 2018, at 21:13, Ash Berlin
Patch available at:
https://github.com/apache/incubator-airflow/pull/4117
please test.
B.
> On 30 Oct 2018, at 21:14, Bolke de Bruin wrote:
>
> We should just pass it the UTC date (we should never use local time except at
> the user interface). I’m testing a patch right now.
>
> B.
>
>> On
I think if we give croniter a tz-aware DT in the local tz it will deal with DST
(i.e. will give 2:55 CEST followed by 2:00 CET) and then we convert it to UTC
for return - but right now we are giving it a TZ-unaware local time.
I think.
Ash
On 30 October 2018 19:40:27 GMT, Bolke de Bruin wrote
We should just pass it the UTC date (we should never use local time except at
the user interface). I’m testing a patch right now.
B.
> On 30 Oct 2018, at 21:13, Ash Berlin-Taylor wrote:
>
> I think if we give croniter a tz-aware DT in the local tz it will deal with
> DST (i.e. will give 2:55
I think we should use the UTC date for cron instead of the naive local date
time. I will check of croniter implements this so we can rely on that.
B.
> On 28 Oct 2018, at 02:09, Bolke de Bruin wrote:
>
> I wonder how to treat this:
>
> This is what I think happens (need to verify more, but I
I think DAGs will start running by themselves again as soon as the interval has
passed that it has already seen. So depending your schedule this can take a
while.
B.
> On 30 Oct 2018, at 19:53, a...@apache.org wrote:
>
> 1.10.1 isn't out yet, so 1.10.0 :)
>
> I think this only affects days wi
I wonder how to treat this:
This is what I think happens (need to verify more, but I am pretty sure) the
specified DAG should run every 5 minutes. At DST change (3AM -> 2AM) we
basically hit a schedule that we have already seen. 2AM -> 3AM has already
happened. Obviously the intention is to run
1.10.1 isn't out yet, so 1.10.0 :)
I think this only affects days with a schedule interval that is one hour or
more frequently, and where the dag timezone is not UTC.
For anyone stuck on this in prod I think the fix is to manually create (via
backfill or trigger) a dag run after the tz change o
Hi Airflow Devs:
Is this timezone issue in Airflow version 1.10.0 or only in 1.10.1?
Thanks.
Regards,
David
On Tue, Oct 30, 2018 at 11:11 AM Bolke de Bruin wrote:
> we specifically remove timezone info to determine the next schedule. Ie.
> cron sets exact date times so tz info should not make
we specifically remove timezone info to determine the next schedule. Ie. cron
sets exact date times so tz info should not make sense. I’m going to have a
look now.
> On 30 Oct 2018, at 19:01, Ash Berlin-Taylor wrote:
>
> I've done a bit more digging - the issue is of our tz-aware handling ins
I've done a bit more digging - the issue is of our tz-aware handling inside
following_schedule (and previous schedule) - causing it to loop.
This section of the croniter docs seems relevant
https://github.com/kiorky/croniter#about-dst
Be sure to init your croniter instance with a TZ aware d
Oh that’s a great environment to start digging. Thanks. I’ll have a look.
B.
Verstuurd vanaf mijn iPad
> Op 30 okt. 2018 om 18:25 heeft Ash Berlin-Taylor het
> volgende geschreven:
>
> This line in airflow.jobs (line 874 in my checkout) is causing the loop:
>
>last_run = dag.get_
This line in airflow.jobs (line 874 in my checkout) is causing the loop:
last_run = dag.get_last_dagrun(session=session)
if last_run and next_run_date:
while next_run_date <= last_run.execution_date:
next_run_date = dag.following_schedule
Hi, kaczors on gitter has produced a minmal reproduction case:
https://github.com/kaczors/airflow_1_10_tz_bug
Rough repro steps: In a VM, with time syncing disabled, and configured with
system timezone of Europe/Zurich (or any other CEST one) run
- `date 10280250.00`
- initdb, start scheduler,
Can this be confirmed? Then I can have a look at it. Preferably with dag
definition code.
On the licensing requirements:
1. Indeed licensing header for markdown documents. It was suggested to use html
comments. I’m not sure how that renders with others like PDF though.
2. The licensing notifica
I was going to make a start on the release, but two people have reported that
there might be an issue around non-UTC dags and the scheduler changing over
from Summer time.
> 08:45 Emmanuel> Hi there, we are currently experiencing a very strange issue
> : we have hourly DAGs with a start_date in
I agree with that, but I would favor time based releases instead. We are again
at the point that a release takes so much time that the gap is getting really
big again. @ash why not start releasing now and move the remainder to 1.10.2? I
dont think there are real blockers (although we might find
I was really hoping that https://github.com/apache/incubator-airflow/pull/4069
will be merged into 1.10.1
Deleting dags was a highly requested feature for 1.10 - this can fix the
problem with it.
‐‐‐ Original Message ‐‐‐
On Friday, October 26, 2018 6:12 PM, Bolke de Bruin wrote:
> Hey
Hey, yeah I've been working (a bit slowly) on it - we had a few test failures
on the v1-10-test branch (conflicts/bad cherry-picks) to unravel to get green
builds, but we're there now.
A rough summary of where we are (wide columns, won't look good if it has to
line-wrap, https://gist.github.com
23 matches
Mail list logo
