Re: Lock wait timeout exceeded

I just looked quickly and it looks like no index is defined on the XCom model beyond the PK. We should add a composite btree index on dag_id, task_id and execution_date. I don't think MySQL will let you include `key` in there as it's too long (512). The 3 other fields are plenty selective so that w

Re: Merging the experimental API Framework

It works very well. For Superset (aka Caravel) we're mostly in the React / async world while using FAB as an async REST backend, and still leveraging the CRUD interface where it makes sense. It's nice to be able to define roles as collection of perms that affect both the REST api and the UI consist

Re: Merging the experimental API Framework

Max, How well does FAB support building pages that use XHR to consume a REST api, instead of doing templating on the backend? We already have some ad-hoc XHR in airflow, and backend templating is the reason large DAG graph/tree views don't render (and probably other things I missed). It might be

Re: Merging the experimental API Framework

Glad to see this! On my side I've been playing around trying to use Flask App Builder (just obtained committer status on the project) which covers some out of the box CRUD REST API for models and a authentication/role/permission framework that would allow for a multi-tenant UI / API with arbitrary

Re: Merging the experimental API Framework

Just wanted to say this is very exciting, thank you Bolke :). Big +1 to that. Thanks, Bolke! ap

Re: Merging the experimental API Framework

There is some work on the wiki. However, I found out that it needs quite a lot of experimenting to found out the right way to approach things. Hence marking this as experimental. I think it needs a bit more experimenting, before we get to a stabilized API design doc. I’m also asking the communit

Re: Merging the experimental API Framework

Bolke, Thanks for kicking this off. Is there already a design document for this? If not, can you create one? It makes sense to have a design document for this to connect multiple PRs. You can also add the information above to the same wiki -- The mailing list is not always super friendly for histo

Re: November 16 SF Bay Airflow meetup

Thx Chris. I've added it to the announcements as well. : https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements#Announcements-Nov28,2016 On Mon, Nov 28, 2016 at 11:13 AM, Chris Riccomini wrote: > Hey all, > > A recording of the meetup is now available here: > > https://wepayinc.app.b

Re: Merging the experimental API Framework

Just wanted to say this is very exciting, thank you Bolke :). On Mon, Nov 28, 2016 at 10:50 AM, Bolke de Bruin wrote: > All, > > After a few weeks of work I have finalized the implementation of a Rest > API Framework. Out of the box it supports Kerberos authentication, which is > now fully end t

Re: November 16 SF Bay Airflow meetup

Hey all, A recording of the meetup is now available here: https://wepayinc.app.box.com/s/1183ra3z8gxf8fridysu4wbjckg1s05v Cheers, Chris On Tue, Nov 1, 2016 at 10:14 AM, Chris Riccomini wrote: > Hey all, > > Just a gentle reminder that the next Airflow meetup is happening on > November 16, 2016

Merging the experimental API Framework

All, After a few weeks of work I have finalized the implementation of a Rest API Framework. Out of the box it supports Kerberos authentication, which is now fully end to end tested on Travis’ with a working KDC. You can also switch the CLI to use the API endpoints when available. Currently, onl

Re: Python 3

I thought it would be worth following up on this given our experience of testing Python 3 compatibility. We are using Celery and RabbitMQ. Celery uses librabbitmq for interaction with RabbitMQ but librabbitmq is not compatible with Python 3 (see https://github.com/celery/librabbitmq/issues/13)

Cold case PR cleanup deadline

Is this week, Nov 30. Friendly reminder to all maintainers.

Lock wait timeout exceeded

Hey all, Starting to see this with some frequency: (_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND xcom.dag_id = %s'] [parameters: (u'return_

Re: Python 3

Thank you Dmitriy and Li. Luke Maycock OLIVER WYMAN luke.mayc...@affiliate.oliverwyman.com www.oliverwyman.com From: Li Xuan Ji Sent: 28 November 2016 12:20 To: dev@airflow.incubator.ap

Re: Python 3

Airflow is supposed to be Python 3 compatible and any incompatibilities are bugs on our side. I haven't heard of anyone running airflow on windows, but I'm not sure what the official stance is. On 28 November 2016 at 07:01, Dmitriy Krasnikov wrote: > I have been using it and developing plugins f

RE: Python 3

I have been using it and developing plugins for it in Python 3.5 with no problems. -Original Message- From: Maycock, Luke [mailto:luke.mayc...@affiliate.oliverwyman.com] Sent: Monday, November 28, 2016 6:58 AM To: dev@airflow.incubator.apache.org Subject: Python 3 Hi All, I have been

Python 3

Hi All, I have been using Airflow for a while now and have a couple of questions that I am hoping someone knows the answer to: 1. Is Airflow now Python 3 compatible? The documentation used to state that Airflow was only compatible with Python 2.7 (