I would like to challenge the notion that "execution_date" is well
documented. Looking at airflow.apache.org right now and searching for all
references to "execution_date", I find that the only definition of
execution_date is, "The execution date of the DAG". There are some other
passing
Right I see where you're at, I looked through some of our operators and I
do see some logic that's repeated (converting to JSON and writing to temp
files). We also do what looks like pipes.clear_keys with some of the files
we get using the SSHHook. Now that I notice that, I'm tempted to pull it
Second use of “inane” on this subject. Brilliant, less combative response
Chris.
There’s another point.. left bound makes sense to some people, right bound to
others.
There’s no way to know or measure how “hard” this is to new users, so even if
the change was made - new name, use right
While taking a step back makes some sense, we also need to identify what
the issue is. Simply saying 'execution_date behavior is confusing to new
users' isn't good enough. What is confusing about it? Is it what it
represents, or just the name itself?
There are a number of different timestamps
Hi Michael, thank you for your comment.
XCom is for sharing state between tasks , and you are right in stating that
it won't be wise to pass datasets via it.
I'm suggesting to refactor already exists code in operators (that each
operator implemented separately) . if we move some logic to hooks
I also believe Chris is correct that it's not quite possible to be that
dynamic today.
If you can find a workaround like only running 1 DAG run for each DAG at a
time and reusing the pool, or perhaps it might work to create the pool
based on the DagRun's execution date instead of its id?
*Taylor
Ruiqin - Re: backwards compatibility - I'm not sure, but my guess is that
the major versions have breaking schema changes that aren't simultaneously
backwards compatible.
Matt - Here's the offline mode support in Airflow and the Alembic docs.
-
I see what your looking for and I think this is the purpose of XCom. We've
used xcom in some of our custom operators to get this type of
functionality.
Though, we tend to avoid putting a lot of data into xcom, I believe
somewhere in the docs it talks about how that's an anti pattern. The
pattern
I see also that the solution is on it's way for the first problem as well
:). https://github.com/apache/incubator-airflow/pull/3962
On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk
wrote:
> Seem that the second problem is already solved in this PR:
>
Seem that the second problem is already solved in this PR:
https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will be
merged soon :).
On Thu, Sep 27, 2018 at 9:57 AM airflowuser
wrote:
> Also I see this a lot:
> 1) ERROR: Failure: ProgrammingError
Also I see this a lot:
1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError)
(1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM
task_instance WHERE task_instance.dag_id = %s'] [parameters: ('unit_tests',)])
Sent with ProtonMail Secure Email.
‐‐‐
Hello Everyone,
Seems that since yesterday the builds started to fail in Travis CI badly.
And we need some urgent actions to fix it otherwise everyone developing
Airflow is affected now.
I am a bit fresh in Airflow so I am not sure how to handle those problems
in an "emergency" way (and I am not
12 matches
Mail list logo
