Not seeing this, but I am not running master in my staging or production
envs currently.
-s
On Mon, Nov 14, 2016 at 12:38 PM, Ben Tallman wrote:
> We are seeing an issue when running Master where tasks sometimes never run.
> It seems that once they get marked as Dependencies Not met because the
If your requirement is to skip a portion of tasks in a DagRun based on some
state encountered while executing that DagRun, that is what
BranchPythonOperator or ShortCircruitOperator (optionally paired with a
Trigger Rule specified on a downstream task) is made for.
These operators take a custom Py
Today is the 6-week deadline to close or reduce our Cold Case PR (Pull
Request) backlog.
6 weeks ago today, we had 110+ open pull requests (PRs) -- right now, we
have 47.
https://cwiki.apache.org/confluence/display/AIRFLOW/Cold-Case+PR+Resolution
Having so many open PRs is a case of broken wind
Sorry, that one was my bad.
On Mon, Nov 14, 2016 at 8:02 PM, siddharth anand wrote:
> FYI,
> I needed to revert a commit made roughly 6 hours ago.
>
>
> https://github.com/apache/incubator-airflow/commit/c6dd4d4578918364da1cd3d5655a8d41a65871b5
>
>
> If you submitted any PRs in the last 6 hours
correct
On Tue, Nov 15, 2016 at 1:11 AM wrote:
Thanks siddharth for your answer. I'm going to look at the extra "failure
processing branch" as you suggested.
>It is intended that a DAGRun be deemed successful in all cases except for
> failure. So, skipped nodes F and G, would result in a Succ
Thank you for taking the time to respond. This is a great approach if you know
at the time of creating the DAG which tasks you expect to need to skip.
However, I don't think this is exactly the use case I have. For example, I may
be expecting a file to arrive in an FTP folder for loading into a
Thanks siddharth for your answer. I'm going to look at the extra "failure
processing branch" as you suggested.
>It is intended that a DAGRun be deemed successful in all cases except for
> failure. So, skipped nodes F and G, would result in a Successful DagRun.
A DAGRun is deemed succesful or fai
https://github.com/apache/incubator-airflow/blob/master/README.md
If you haven't yet, make sure to add your company to the list of companies
using Airflow! Submit a PR to add your company to the list.
-s
Hi Gerard,
I have them both under my name on the following wiki though anyone from the
community is welcome to help with the review and test it out as well.
https://cwiki.apache.org/confluence/display/AIRFLOW/Cold-Case+PR+Resolution
About 6 weeks ago, we had over 110 open PRs, many of which were v
