Re: UI not working on Master

f `airflow webserver` but this is quite easy to miss if you aren't looking for it) -ash Daniel Imberman wrote on 18/12/2018 23:23: Well locally I do this (which I'm hoping to PR soon once tests are passing): https://github.com/apache/incubator-airflow/pull/4326. This allows me to connec

Re: Please add me to the mailing list for airflow

Please send an email to dev-subscr...@airflow.apache.org Ryan Riopelle wrote on 18/12/2018 20:12: Hello, Please add me to the mailing list for airflow. Thanks, Ryan Ryan Riopelle Data Engineer San Francisco Office figure.com This message and its contents are confiden

Re: Configuring/sharing Airflow github repo security alerts

We're not admins of the repo - only the ASF Infra team are, so we'll have to open an ticket against the INFRA queue in jira asking for this (I haven't done this. Not on large device right now) -a Feng Lu wrote on 18/12/2018 08:01: Hi all, Looks like GitHub now adds a new "Security Alert" fea

Re: Call for fixes for Airflow 1.10.2

Looks good, thanks for the reminder. Merged to master and cherry-picked to the release branch. Kaxil Naik wrote on 15/12/2018 11:21: Sure, I am waiting for final comments from Ash on it, if he has any. If not I will pick it up and merge to master + cherry-pick to the test branch On Thu, Dec

Re: Recommended backend metastore for Airflow

Postgres. Friends don't let friends use MySQL is my personal rule. (I can get in to the reasons if you'd like, but the short version is I find Postgres has more compliant behaviour with SQL standard, and a much better query planner.) -ash > On 10 Dec 2018, at 15:10, ramandu

Re: Refactor models.py

things to? Are you thinking we have airflow.models.connection module containing just a Connection class for example? -ash > On 6 Dec 2018, at 11:35, Driesprong, Fokko wrote: > > Hi All, > > I think it is time to refactor the infamous models.py. This file is far too > big, a

Call for Graduation press releases and testimonials

ith me and Sally Khudairi off list. Time is a bit short on this, so if you are interested please get in touch! Thanks, Ash

Re: [RESULT] Graduate Apache Airflow as a TLP

BE IT RESOLVED, that a Project Management >>>>>> Committee (PMC), to be known as the "Apache Airflow Project", >>>>>> be and hereby is established pursuant to Bylaws of the >>>>>> Foundation; and be it further >>>>>>

Re: Call for fixes for Airflow 1.10.2

I'd like to get https://issues.apache.org/jira/browse/AIRFLOW-3422 in if we can - but we need a fix first (Bolke: it's our favourite! DST time zones in next_schedule!) I'll take a look at this ... soon. -ash > On 28 Nov 2018, at 22:40, Kaxil Naik wrote: > > Hi everyon

Re: [DISCUSS] Apache Airflow graduation from the incubator

RFLOW-3431 -- I'll make a start on this -ash > On 30 Nov 2018, at 22:06, Bolke de Bruin wrote: > > Thanks Jakob! > > Verstuurd vanaf mijn iPad > >> Op 30 nov. 2018 om 22:49 heeft Jakob Homan het volgende >> geschreven: >> >> I've finished the

Re: [VOTE] Graduate the Apache Airflow as a TLP

;responsibility of the Apache Airflow Project; and be it further > >RESOLVED, that the persons listed immediately below be and >hereby are appointed to serve as the initial members of the >Apache Airflow Project: > >* Alex Guziel >* Alex Van Boxel >* Arthur Wiedmer >* Ash Be

Re: Will Airflow 2.0.0 support Python 2.7?

This came up previously, and no firm conclusion was released, but given Python 2.7 is still maintained for another year, yes, probably. > On 29 Nov 2018, at 08:48, airflowuser > wrote: > > Are there plans to drop support for Python 2.7 - if so when ?

Re: programmatically creating and airflow quirks

Challenge though. (Oh, except no to microservices. Airflow is hard enough to operator right now without splitting things in to even more daemons) -ash > On 26 Nov 2018, at 03:06, soma dhavala wrote: > > > >> On Nov 26, 2018, at 7:50 AM, Maxime Beauchemin >> wrote: &g

Re: [DISCUSS] Apache Airflow graduation from the incubator

team CD30: probably good as it is, we may want to do https://issues.apache.org/jira/browse/AIRFLOW-3400 <https://issues.apache.org/jira/browse/AIRFLOW-3400> to remove the last niggle of the GPL env var at install time (but not a hard requirement, just nice) -ash > On 26 Nov 2018,

Airflow 1.10.1 is released

Dear Airflow community, I'm happy to announce that Airflow 1.10.1 was just released. The source release as well as the binary "sdist" release are available here: https://dist.apache.org/repos/dist/release/incubator/airflow/1.10.1-incubating/ We also made this version available on PyPi for c

[RESULT][VOTE] Release Airflow 1.10.1 (incubating)

The vote to release Airflow 1.10.1-incubating, having been open for 3 days is now closed. There were three binding +1s and no -1 votes. +1 (binding): Hitesh Shah Jakob Homan Justin Mclean The release is approved. Thanks to all those who voted. Cheers, Ash > On 21 Nov 2018, at 04

[RESULT][VOTE] Airflow 1.10.1 RC2

Hello, Apache Airflow (incubating) 1.10.1 (based on RC2) has been accepted. 3 “+1” binding votes received: - Ash Berlin-Taylor (binding) - Kaxil Naik (binding) - Fokko Driesprong (binding) 2 "+1" non-binding votes received: - Deng Xiaodong (non-binding) - Ikar Pohorsky (non-bin

Re: Apache Airflow Setup Issue

The error is related to python 3.7. `pip install apache-airflow==1.10.1rc2` should fix this. (The release candidate fixes this, so make sure you ask for that explicit version until the release is made) Note the new name - apache-airflow is the new package name. On 17 November 2018 13:03:10 GMT,

Re: [VOTE] Airflow 1.10.1 RC2

. (I feel Bolke's pain :) I'm now moderately annoyed at the Apache release process and how long it takes, it means each release ends up getting big) -ash > On 17 Nov 2018, at 01:01, Naik Kaxil wrote: > > -1 (binding) . Tested it on Python 2.7.14, got expected result but had

Re: [VOTE] Airflow 1.10.1 RC2

Friendly reminder for people (and especially committers) to test this out and vote on it please! -ash > On 14 Nov 2018, at 22:31, Ash Berlin-Taylor wrote: > > Hey all, > > I have cut Airflow 1.10.1 RC2. This email is calling a vote on the release, > which will last for

Re: reg airflow on kubernetes

(Not that is "closed" but the problem still persists) -ash > On 15 Nov 2018, at 14:09, manojbabu...@gmail.com wrote: > > Hi, > I was following the below steps to run airflow on kubernetes and getting > below error. > Can any one share thoughts or point to detailed

Re: Moving Airflow Config to Database.

> problem with > this approach is these env variables wont behave correctly when we > subshells Can you explain what you mean by this? -ash > On 15 Nov 2018, at 12:03, Sai Phanindhra wrote: > > Hi deng, > I am currently using env variables for few airflow config variabl

[VOTE] Airflow 1.10.1 RC2

Hey all, I have cut Airflow 1.10.1 RC2. This email is calling a vote on the release, which will last for 72 hours. Consider this my (binding) +1. Airflow 1.10.1 RC2 is available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.1rc2/ apache-airflow-1.10.1rc2+incubating-source.t

Re: [VOTE CANCELED] Airflow 1.10.1rc1

publishing it there makes it easier for people to test. (Pip won't install beta or rc versions when doing `pip install apache-airflow`, you have to add `==1.10.1b1`, so it's "safe" in that regard.) -ash > On 13 Nov 2018, at 15:59, Ash Berlin-Taylor wrote: > > CORRECTI

Re: [VOTE] Airflow 1.10.1rc1

CORRECTION: Correct URLs are https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.1rc1 <https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.1rc1> Copy-and-paste fail > On 13 Nov 2018, at 15:29, Ash Berlin-Taylor wrote: > > Hey all, > > I have cut Air

[VOTE] Airflow 1.10.1rc1

Hey all, I have cut Airflow 1.10.1 RC1. This email is calling a vote on the release, which will last for 72 hours. Consider this my (binding) +1. Airflow 1.10.1 RC1 is available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/ apache-airflow-1.10.1rc1+incubating-sour

Re: Airflow 1.10.1b1 release available - PLEASE TEST

this easily was worth the small risk of confusion. Possibly something I should have asked (voted on?) first, or before we do this in the future. -ash > On 12 Nov 2018, at 20:32, Hitesh Shah wrote: > > Hello Ash > > For someone who is not familiar with the beta notation or folk

Re: CSS issue on Airflow 1.10 Tree view UI

Screenshots (attachments) don't come through on the list - could you post a link to it somewhere? Does this only apply when the list of tasks is particularly tall or wide? A simple case I can't reproduce this on 1.10.1b1 (but I don't think anythere has changed recently.) -ash

Airflow 1.10.1b1 release available - PLEASE TEST

n `pip install apache-airflow` will still get 1.10.0) Thanks, Ash Included below is the changelog of this release: New features: [AIRFLOW-2524] Airflow integration with AWS Sagemaker [AIRFLOW-2657] Add ability to delete DAG from web ui [AIRFLOW-2780] Adds IMAP Hook to interact with a

Re: what is error[111] and how to deal with it on sending the email notification?

Errno 111 is a connection refused socket-level error, and it's saying that the server where your airflow scheduler is running cannot reach smtp.live.com <http://smtp.live.com/> on port 587. First thing to look at would be your firewall and networking settings. -ash > On 1 Nov

Re: 1.10.1 Release?

Fair :) Timezones are _hard_ Giving it a look now. -ash > On 30 Oct 2018, at 20:50, Bolke de Bruin wrote: > > The reason for not passing a TZ aware object is, is that many libraries make > mistakes (pytz, arrow etc) when doing transitions hence to use of pendulum > which see

Re: 1.10.1 Release?

I think if we give croniter a tz-aware DT in the local tz it will deal with DST (i.e. will give 2:55 CEST followed by 2:00 CET) and then we convert it to UTC for return - but right now we are giving it a TZ-unaware local time. I think. Ash On 30 October 2018 19:40:27 GMT, Bolke de Bruin

Re: 1.10.1 Release?

hange over time. This should unblock the scheduler. Ash On 30 October 2018 18:43:30 GMT, David Klosowski wrote: >Hi Airflow Devs: > >Is this timezone issue in Airflow version 1.10.0 or only in 1.10.1? > >Thanks. > >Regards, >David > >On Tue, Oct 30, 2018 at 11:11

Re: 1.10.1 Release?

7:35, Bolke de Bruin wrote: > > Oh that’s a great environment to start digging. Thanks. I’ll have a look. > > B. > > Verstuurd vanaf mijn iPad > >> Op 30 okt. 2018 om 18:25 heeft Ash Berlin-Taylor het >> volgende geschreven: >> >> This line in airf

Re: 1.10.1 Release?

ate = dag.following_schedule(next_run_date) > On 30 Oct 2018, at 17:20, Ash Berlin-Taylor wrote: > > Hi, kaczors on gitter has produced a minmal reproduction case: > https://github.com/kaczors/airflow_1_10_tz_bug > > Rough repro steps: In a VM, with time syncing disabled, and configured with

Re: 1.10.1 Release?

ick hacky debug print shows something is stuck in an infinite loop. -ash > On 29 Oct 2018, at 17:59, Bolke de Bruin wrote: > > Can this be confirmed? Then I can have a look at it. Preferably with dag > definition code. > > On the licensing requirements: > > 1. In

Re: 1.10.1 Release?

f I can get to the bottom of it. Bolke: are there any outstanding tasks/issues that you know of that might slow down the vote for a 1.10.1? (i.e. did we sort of out all the licensing issues that were asked of us? I thought I read something about license declarations in markdown files?) -ash >

Re: 1.10.1 Release?

ore fixed to cherry-pick to do on the branch, and there are a couple of open tasks that still need fixes writing or merging - (3033, 3172) > On 26 Oct 2018, at 16:12, Bolke de Bruin wrote: > > Hey Ash, > > I was wondering if you are picking up the 1.10.1 release? Master is speeding > ahead and you were tracking fixes for 1.10.1 right? > > B.

Re: [IE] Re: [External] RE: [IE] airflow ui not showing logs

I added a check for this that will be in 1.10.1 so Airflow will 1) warn you about this, and 2) try to correct it automatically https://github.com/apache/incubator-airflow/blob/a1e922fe6d7ac3aa19848a1ad34836b61fccf24d/airflow/logging_config.py#L79-L106

Re: taking an assigned Jira ticket

Usually comment on the Jira ticket itself seems to be the best way. If it's been a while then it's also probably safe to assume they aren't actively working on it. -ash > On 23 Oct 2018, at 16:10, matthew wrote: > > Hey all, > > There are a collection of do

Re: Using Too Many Aiflow Variables in Dag is Good thing ?

ce the latency and >> decrease the number of connections to main database. This whole assumption >> need to be benchmarked to prove the point. I feel like its worth a try. >> >> On Mon 22 Oct, 2018, 15:47 Ash Berlin-Taylor, wrote: >> >>> Cache them where? When wou

Re: Using Too Many Aiflow Variables in Dag is Good thing ?

does it with just one connection rather than N. -ash > On 22 Oct 2018, at 11:11, Sai Phanindhra wrote: > > Who don't we cache variables? We can fairly assume that variables won't get > changed very frequently(not as frequent as scheduler DAG run time). We can > keep def

Re: explicit_defaults_for_timestamp for mysql

This sounds sensible and would mean we could also run on GCP's MySQL offering too. This would need someone to try out and check that timezones behave sensibly with this change made. Any volunteers? -ash > On 19 Oct 2018, at 17:32, Deng Xiaodong wrote: > > Wondering if there

Re: Pinning dependencies for Apache Airflow

echo 'pandas==2.1.3' > constraints.txt pip install -c constraints.txt apache-airflow[pandas] That will ignore what ever we specify in setup.py and use 2.1.3. https://pip.pypa.io/en/latest/user_guide/#constraints-files (sorry for the brief message) > On 19 Oct 2018, at 17:02, Maxime Beauchemin

Re: Ingest daily data, but delivery is always delayed by two days

That would work for some of our other uses cases (and has been an idea in our backlog for months) but not this case as we're reading from someone else's bucket so can't set up notifications etc. :( -ash > On 12 Oct 2018, at 11:57, Bolke de Bruin wrote: > > S3

Ingest daily data, but delivery is always delayed by two days

and which files it picks up, so I've ruled out this option I could use a Time(Delta)Sensor to just delay the start of the checking. I guess with the new change in master to make sensors yield their execution slots that's not a terrible plan. Does anyone else have any other idea, including possible things we could add to Airflow itself. -ash

Re: Pinning dependencies for Apache Airflow

ported a way of saying "this dep is for py2.7 only": > Since version 6.0, pip also supports specifiers containing environment > markers like so: > >SomeProject ==5.4 ; python_version < '2.7' >SomeProject; sys_platform == 'win32' Ash

Re: Pinning dependencies for Apache Airflow

maybe something to be said about pinning for releases. The down side to that is that if there are updates to a module that we want then we have to make a point release to let people get it Both methods have draw-backs -ash > On 4 Oct 2018, at 17:13, Arthur Wiedmer wrote: > > Hi Jarek, >

Re: Flask-AppBuilder has pinned versions of Click & Flask-Login in 1.10.0

x27;t ready for real-world use yet? (I'm guessing and extrapolating, but I haven't used it myself so don't trust my word on this) -ash > On 4 Oct 2018, at 16:38, Kyle Hamlin wrote: > > If I remove the Flask-AppBuild pinning to 1.11.0 then it uncovers a Jinja2 > conflict whi

Re: Flask-AppBuilder has pinned versions of Click & Flask-Login in 1.10.0

We've committed a fix for this to master and will include it in a 1.10.1 https://github.com/apache/incubator-airflow/commit/fb5ffd146a5a33820cfa7541e5ce09098f3d541a For installing in the mea time pin `Flask-AppBuilder=1.11.0' > On 4 Oct 2018, at 00:41, Kyle Hamlin wrote: > > Hi, > > Today I

Re: Slides from London Airflow Meetup #1

t; Cheers, >> Kevin Y >> >> On Tue, Sep 25, 2018 at 3:36 AM Sumit Maheshwari >> wrote: >> >>> Thanks a lot, Ash. >>> >>> >>> >>> On Tue, Sep 25, 2018 at 3:47 PM Ash Berlin-Taylor >> wrote: >>> >&g

Re: execution_date - can we stop the confusion?

What about (aliasing) execution_date to period_start, and next_execution_date to period_end? Would this help any do we think? (Though things like ds and ts might still be confusing? This is probably where the OP got the idea for run_stamped from? One step at a time.) Ash On 27 September 2018

Re: Solved: suppress PendingDeprecationWarning messages in airflow logs

Sounds good for your use, certainly. I mainly wanted to make sure other people knew before blindly equipping a foot-canon :) -ash > On 29 Sep 2018, at 00:09, Sean Carey wrote: > > Thanks, Ash. I understand what you're saying. The warnings are coming from > the Qubole op

Re: Solved: suppress PendingDeprecationWarning messages in airflow logs

versions of Airflow (read: you might suddenly find that things stop working because you turned of the warnings.) -ash > On 28 Sep 2018, at 22:52, Sean Carey wrote: > > Hello, > > I’ve been looking for a way to suppress the PendingDeprecationWarning > messages cluttering our

Re: Travis CI tests failing in master

Looks like someone has beaten you to the punch https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413 <https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413> :) -ash > On 28 Sep 2018, at 09:59, Kaxil Naik wrote: > > Th

Re: Fundamental change - Separate DAG name and id.

> On 24 Sep 2018, at 23:12, Alex Tronchin-James 949-412-7220 > wrote: > > Re: [Brian Greene] "How does filename matter? Frankly I wish the filename > was REQUIRED to be the dag name so people would quit confusing themselves > by mismatching them !" > > FWIW in the Facebook predecessor to airf

Slides from London Airflow Meetup #1

Hi everyone, Slides from the first London Airflow Meetup are available in this this Google drive folder: https://drive.google.com/drive/folders/1wiSkrg_1rvqGrmbYN7rNFQaqW0Ty10Vk Sorry, we didn't get them captured on vid

Re: Airflow: Apache Graduation

as drafting a Charter (we would probably start with a "stock" Apache one) -ash > On 20 Sep 2018, at 18:22, Maxime Beauchemin > wrote: > > Yeah let's make it happen! I'm happy to set some time aside to help with > the final push. > > Max >

Re: It's very hard to become a committer on the project

> Remember my basic question: I want to contribute - how on earth I can find a > ticket that is suitable for first time committer? Can you show me? There aren't that many feature requests in Jira, so looking there for easy tickets, is as you have probably found a fruitless exercise. I'd recommen

Re: Connection Management in Multi-tenancy Scenario

would allow for much more granular permissions, and provides a pre-existing management UI for it to. I don't know if this would make the work dependent on the (in progress?) DAG-level access controls. -ash > On 19 Sep 2018, at 15:00, Deng Xiaodong wrote: > > Hi folks, > >

Re: It's very hard to become a committer on the project

> On 18 Sep 2018, at 13:07, Ash Berlin-Taylor wrote: > > Somewhat annoyingly you can't (or we don't have permission to) set the fix > version on a closed Jira ticket. We can probably ask for permission to edit > closed/resolved Jira tickets in the AIRFLOW project

Re: It's very hard to become a committer on the project

et. We can probably ask for permission to edit closed/resolved Jira tickets in the AIRFLOW project, which would remove some of the pain here. I've asked for that https://issues.apache.org/jira/browse/INFRA-17033 Thanks for finding the issues to close btw, it's helpful. -ash > On 18 S

Re: Guidelines on Contrib vs Non-contrib

bits which a simple python import can't manage - which I think is only anything that needs to be registered with another system, such as custom routes in the web UI. I'll draft an AIP for this soon. -ash > On 18 Sep 2018, at 00:50, George Leslie-Waksman wrote: > > Given

Re: Database referral integrity

Ooh good spot. Yes I would be in favour of adding these, but as you say we need to thing about how we might migrate old data. Doing this at 2.0.0 and providing a cleanup script (or doing it as part of the migration?) is probably the way to go. -ash- > On 17 Sep 2018, at 19:56, Ste

Re: It's very hard to become a committer on the project

h, I do dislike Jira, but I don't care enough to think about changing.) Hope this helps explain the situation. -ash > On 16 Sep 2018, at 14:29, airflowuser > wrote: > > Hello all, > > I'm struggling finding tickets to address and while discussing it on chat >

Re: Call for fixes for Airflow 1.10.1

Thanks Kaxil! I'll get on with finishing off the 1.10.1 release next week after my talk at the London Meetup. -ash > On 15 Sep 2018, at 16:03, Kaxil Naik wrote: > > I have cherry-picked the fix for this issue on top of 1.10-test branch > along-with 16 other commits (the ones

Re: [External] Dynamic tasks in a dag?

This isn't needed as the tasks are added to the dag when specified, so they DAG object keeps track of the tasks. -ash > On 14 Sep 2018, at 18:30, Alex Tronchin-James 949-412-7220 > wrote: > > Don't you need to preserve the task objects? Your implementation overwrites &

Re: [External] Dynamic tasks in a dag?

This isn't needed as the tasks are added to the dag when specified, so they DAG object keeps track of the tasks. -ash > On 14 Sep 2018, at 18:30, Alex Tronchin-James 949-412-7220 > wrote: > > Don't you need to preserve the task objects? Your implementation overwrites &

Re: Call for fixes for Airflow 1.10.1

targeted with a fix version of 1.10.1 (i.e. if you want it in 1.10.1, merge the pr into master and also mark the Jira as fix in 1.10.1 and I'll work on cherry-picking the fixes. If they can be. If it is diffucult/has other things to cherry pick in I might change the fix version on you.) -ash

Re: Call for fixes for Airflow 1.10.1

That is a good idea! Though it would involve some work right now as I don't think anyone (committers or contributors) have been particularly careful of what type of Jira issue they create. Or at least not uniformly. -ash > On 5 Sep 2018, at 14:08, airflowuser > wrote: > >

Re: Add git tag for 1.10

It is above the heading of Airflow 1.10, i.e. in the Airflow Master section already. -ash > On 3 Sep 2018, at 13:53, Robin Edwards wrote: > > I am not sure if anyone's aware of this, the 1.10.0 tag and the PyPi upload > dont contain the 'BashTaskRunner' -&g

Call for fixes for Airflow 1.10.1

Hi everyone, I'm starting the process of gathering fixes for a 1.10.1. So far the list of issues I have that we should pull in are https://issues.apache.org/jira/issues/?jql=project%20%3D%20AIRFLOW%20AND%20fixVersion%20%3D%201.10.1%20ORDER%20BY%20key%20ASC (reproduces below) I will start pushi

Re: Apache Spark Interfering with Airflow Jira/PRs ??

checking what settings Spark have for their Gitbox integration Jira and copying that? -ash > On 2 Sep 2018, at 21:57, Holden Karau wrote: > > Really sorry for the noise on JIRA, I've shut down the app and I'll try and > figure out how it ended up doings this. > > On

Re: Retiring Airflow Gitter?

install https://slack.com/apps/A7DL60U5D-irccloud/ <https://blog.irccloud.com/slack-integration/> I won't object ;) (I am not a fan of Gitter either, but I am constantly logged in via their IRC gateway. I'm usually the only PPMC responding in there, Bolke pops up from time-to-time too)

Re: Missing operators in the docs

tensions/autodoc.html?highlight=autodoc_mock_imports#confval-autodoc_mock_imports> for some of the harder-to-install modules? -ash > On 29 Aug 2018, at 21:53, Kaxil Naik wrote: > > I have fixed the issue on https://airflow.apache.org/ , added a comment on > confluence a

Re: Python 3.6 Support for Airflow 1.10.0

t; >>>>>>> On Tue, Aug 28, 2018 at 9:24 AM Naik Kaxil >>> wrote: >>>>>>> >>>>>>> We should definitely support 3.7. I left comments on the PR >>> @tedmiston >>>>>>> regarding the same. Python 2.7

Re: Running unit tests against SLUGIFY_USES_TEXT_UNIDECODE and AIRFLOW_GPL_UNIDECODE (also is this broken?)

I don't think we strictly care about running the tests in both these circumstances - it is a flag that control which dep is installed two or three levels down as you say, and the project has it's own tests. I'd rather we spent time on replacing python-nvd with something that means we don't hav

Re: Airflow variables and data profiling hidden

e this helps. Ash > On 29 Aug 2018, at 07:15, Shubham Gupta wrote: > > Hi, > > Airflow variables and data profiling tabs are hidden in the UI. Can someone > suggest how to unhide them? If I try to access through '/admin/variable/', > the result is > >>

Re: Python 3.6 Support for Airflow 1.10.0

of `async` as a keyword. both in A perhaps bigger question: What are people's thoughts on dropping support for Python2? This wouldn't happen before 2.0 at the earliest if we did it. Probably something to raise an AIP for. -ash > On 28 Aug 2018, at 16:56, Taylor Edmiston wrote: >

Re: [RESULT][VOTE] Release Airflow 1.10.0

t;>>> >>>>>> >>>>> >>>>> Kaxil Naik >>>>> >>>>> Data Reply >>>>> 2nd Floor, Nova South >>>>> 160 Victoria Street, Westminster >>>>> London SW1E 5LB - UK >>>>>

Cloudera Hue in License

g for hue doesn't turn up anything likely looking. -ash

Re: [RESULT][VOTE] Release Airflow 1.10.0

Could you push the git tag too please Fokko/Bolke? -ash > On 22 Aug 2018, at 08:16, Driesprong, Fokko wrote: > > Thanks Max, > > My PyPI ID is Fokko > > Cheers, Fokko > > 2018-08-21 22:49 GMT+02:00 Maxime Beauchemin : > >> I can, what's your PyPI I

Re: Plan to change type of dag_id from String to Number?

The performance of SQLite doesn't matter is it is restricted to a single worker anyway -- it's definitely not recommended for running in production. -ash > On 16 Aug 2018, at 08:33, George Leslie-Waksman wrote: > > These performance characteristics are metadata database bac

Re: apache-airflow v1.10.0 on PyPi?

ttps://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow>) I suspect the tags were in Bolke's local checkout. I hope I got the commits right -ash > On 15 Aug 2018, at 17:34, James Meickle > wrote: > > Can we make it a policy going forward to push GH tags for al

Re: [VOTE] Airflow 1.10.0rc4

If we can't score fractions then, yes +1 :) (And this time sent from the correct email address. I'm really bad at driving a Mail client it turns out.) -ash > On 9 Aug 2018, at 19:22, Bolke de Bruin wrote: > > 0.5?? Can we score fractions :-) ? Sorry I missed this Ash. I th

Re: [VOTE] Airflow 1.10.0rc4

a PyPi I'd guess? -ash > On 8 Aug 2018, at 19:21, Bolke de Bruin wrote: > > Hey all, > > I have cut Airflow 1.10.0 RC4. This email is calling a vote on the release, > which will last for 72 hours. Consider this my (binding) +1. > > Airflow 1.10.0 RC 4 is availabl

Re: Plan to change type of dag_id from String to Number?

Since this is a big change that would touch much of the code base, before we do this we need to see some hard numbers - timing or benchmarks of queries etc. Also how often do we actually do such a join etc? -ash > On 9 Aug 2018, at 13:04, vardangupta...@gmail.com > <mailto:va

Re: [VOTE] Airflow 1.10.0rc4

a PyPi I'd guess? -ash > On 8 Aug 2018, at 19:21, Bolke de Bruin wrote: > > Hey all, > > I have cut Airflow 1.10.0 RC4. This email is calling a vote on the release, > which will last for 72 hours. Consider this my (binding) +1. > > Airflow 1.10.0 RC 4 is availabl

Re: Plan to change type of dag_id from String to Number?

Absolutely - there will still need to be a human-readable DAG id, even we end up with an auto-icrementing integer ID column internally and for table join performance reasons. -ash > On 9 Aug 2018, at 12:35, Юли Волкова wrote: > > How will you understand what your DAG 2 doing en

Re: [VOTE] Airflow 1.10.0rc3

Could you upgrading to 1.9 first? And see if that helps? -ash > On 8 Aug 2018, at 00:07, George Leslie-Waksman <mailto:waks...@gmail.com>> wrote: > > We just tried to upgrade a 1.8.1 install to 1.10rc3 and ran into a critical > error on alembic migration execution. I have

Re: Multiple hosts for a single connection

g set multiple by default as it is often not what people want.) -ash > On 7 Aug 2018, at 15:17, Deng Xiaodong wrote: > > Hi Ben, > > If you would like to set multiple connections with the same *conn_id*, you > can only do that in Web UI. That is, you need to build multiple e

CVE-2017-12614 XSS Vulnerability in Airflow < 1.9

CVE-2017-12614: Apache Reflected Reflected XSS Vulnerability Vendor: The Apache Software Foundation: Versions Affected: < 1.9 Description: It was noticed an XSS in certain 404 pages that could be exploited to perform an XSS attack. Chrome will detect this as a reflected XSS attempt and prevent

Re: [VOTE] Airflow 1.10.0rc3

Yup, just worked out the same thing. I think as "punishment" for me finding bugs so late in two RCs (this, and 1.9) I should run the release for the next release. -ash > On 5 Aug 2018, at 22:05, Bolke de Bruin wrote: > > Yeah I figured it out. Originally i wa

Re: [VOTE] Airflow 1.10.0rc3

3 00:00:00+00 (1 row) postgres=# select '2018-08-03 01:00:00+01'::timestamp with time zone; timestamptz 2018-08-03 00:00:00+00 (1 row) -ash > On 5 Aug 2018, at 21:28, Bolke de Bruin wrote: > > This is the issue: > > [2018-08-05 22:08

Re: Airflow committers (a list)

- https://github.com/orgs/apache/teams/airflow-committers/members This one at least is populated automatically via a 30-minutely cron job. > On 5 Aug 2018, at 21:35, Sid Anand wrote: > > Committers/Mentors, > We have several

Re: [VOTE] Airflow 1.10.0rc3

change depending on how the server is configured. -ash > On 5 Aug 2018, at 20:58, Bolke de Bruin wrote: > > Digging in a bit further. > > ti.dag_id / ti.task_id / ts / try_number .log > > is the format > > ts = execution_date.isofo

Re: [VOTE] Airflow 1.10.0rc3

> On 5 Aug 2018, at 18:01, Bolke de Bruin wrote: > > Hi Ash, > > Thanks a lot for the proper review, obviously I would have liked that these > issues (I think just one) are popping up at rc3 but I understand why it > happened. Yeah, sorry I didn't couldn

Re: [VOTE] Airflow 1.10.0rc3

t. 3) Do we need to set the TZ of the connection to UTC in SQLAlchemy to have consistent behaviour? Is this possible some how? I don't know SQLAlchemy that well. -ash > On 5 Aug 2018, at 16:01, Ash Berlin-Taylor > wrote: > > 1.) Missing UPDATING note about change of task_lo

Re: [VOTE] Airflow 1.10.0rc3

s - either conditionally enable k8s:// support if the import works, or (less preferred) add kube-client to the emr deps (which I like less) Sorry - this is the first time I've been able to test it. I will install this dep manually and continue testing. -ash (Normally no time at home

Re: [VOTE] Airflow 1.10.0rc3

in 1.10) has a timezone in it, and the date is in local time, before it was UTC: before: 2018-07-23T00:00:00/1.log after: 2018-07-23T01:00:00+01:00/1.log We can possibly get away with an updating note about this to set a custom log_filename_template. Testing this now. > On 5 Aug 2018, a

  1   2   3   >