Trigger dag fails if multiple dags runs are triggered at once

2019-02-24 Thread prem
Hi all, we are using postgres with airflow 1.10.1 on python2, if i submit multiple dag runs some of them fail with "NoResultFound: No row was found for one()" i found a similar issue here: https://stackoverflow.com/questions/54412780/airflow-1-10-sqlalchemy-orm-exc-noresultfound-no-row-was-found

aip access

2019-02-24 Thread Joshua Carp
Hi, I'd like to get access to post a new AIP. My username on confluence is jm.carp. Thanks! Josh

Re: Voting on AIPs - what's our process?

2019-02-24 Thread Tao Feng
Based on my observation of Kafka KIP, KIP usually could be very comprehensive which is almost like a tech spec including: 1. motivation; 2. what new public interface it tries to propose; 3. what are the rejected alternatives with what reason; 4. what could be the failure scenarios etc. And it see

Re: [DISCUSS] AIP-3: Drop Python2 support in Airflow 2.0

2019-02-24 Thread Bas Harenslak
Following the comments to warn users about dropping Python 2, I went ahead and added the deprecation library to Airflow in this PR: https://github.com/apache/airflow/pull/4763, and added the first @deprecated annotation. I checked a few major packages and all use some self-made annotation for

Re: Voting on AIPs - what's our process?

2019-02-24 Thread Deng Xiaodong
Great discussions! Two minor points from me: - For the workflow Ash summarised: may I remind that we may want to decide how long the vote on an AIP should be open as well (3, 5,or 7 days?). This can be decided later when the workflow is formalised. - For the process description Fokko proposed:

Re: Voting on AIPs - what's our process?

2019-02-24 Thread Kevin Yang
+1 for having AIPs backed by PRs. About having multiple PR for an AIP. I feels like it might cause overturn on an accepted AIP. Say idea in AIP and phase 1 PR all look good but in phase 2 we encountered blockers and have to employ creative solutions that is against the AIP. I think if an AIP is la

Re: Voting on AIPs - what's our process?

2019-02-24 Thread Driesprong, Fokko
An AIP should be a couple of PR's. I think splitting up major code changes into multiple PR's is a good thing to maintain progress and you don't need to resolve the conflicts all the time. We could also look at different Apache projects. For example, for Kafka you need to open a KIP if you're chan

Re: Voting on AIPs - what's our process?

2019-02-24 Thread Ash Berlin-Taylor
My thinking on AIPs that have a PR is that a vote on the AIP is "is this feature/design goal a good idea" but discussion about the code or merging the pr can happen on GitHub as usual. For example AIP-12 is an excellent Idea but there are a few questions to answer about the design. (The author

Re: [VOTE] Accept AIP-13: OpenAPI 3 based API definition

2019-02-24 Thread Kevin Yang
Finally! +1 (non-binding) On Thu, Feb 21, 2019 at 5:40 PM abhishek sharma wrote: > +1(binding) > > > On Thu, Feb 21, 2019 at 2:09 PM Arthur Wiedmer > wrote: > > > +1 (binding) > > > > This is the direction we should be taking. > > > > On Thu, Feb 21, 2019 at 9:28 AM Ash Berlin-Taylor > wrote:

Re: [DISCUSS] AIP-12 Persist DAG into DB

2019-02-24 Thread Kevin Yang
Sorry about being late to this. Love this change. Will look deep into this in the next couple of days. Thank you, Kevin Y On Sat, Feb 23, 2019 at 10:54 PM Tao Feng wrote: > I manage to test out the latest version of the branch which I think all my > previous concerns don't exist anymore. Great