Hi David.
The only “out of box” way to share data/information between tasks is XCom (
https://airflow.apache.org/docs/stable/concepts.html?highlight=xcom#xcoms).
For you case, the quick suggestion I can share is
- either merging your tasks
- or persisting your Pandas Dataframes somewhere then lo
Hello maintainers
A programmatically generated DAG of mine is exploding (the bomb
text-picture) when triggered with following stack-trace
File
"/home/admin/.pyenv/versions/3.7.3/lib/python3.7/site-packages/airflow/models/__init__.py",
line 2392, in has_dag
return getattr(self, '_dag', None)
Hi,
Excuse me, I am new to this and maybe this topic has already been treated.
I would like to know if there is a way to "share/pass" pandas dataframes
between tasks in airflow.
Any help would be appreciated.
Thank you!!!
David.
Hello,
I created a spreadsheet with all the changes
https://docs.google.com/spreadsheets/d/1Z5jqsGz9QjmVDQAeI6NK8Aa-v6SQv8jfV8QzIZ4-q4w/edit
What do you think about it? The document has active comments if anyone has
questions. I want to move all the files based on it. I have an automatic
script th
+1 (binding)
Tested on Python3.7 with Local and Celery executors.
I did find one minor (non-critical) bug relating to removal of "logger" in CLI.
In the worker output we see:
> Running %s on host %s 2018-01-01T00:15:00+00:00 [queued]> themisto-2.localdomain
That's missing the `%` operator to
Kaxil and I are planning on tackling versioning (for display only right now, as
it's the first step on this journey) as part of AIP-24.
However the issue with versioning the _entire_ DAG code/environment is
sometimes you want to run in the old env, but sometimes you want to run in a
new/latest
I have a concrete proposal that we can start with. It's not a final set of
markers we might want to have but one that we can start with and make an
immediate use of.
I would like to adapt our tests to be immediately usable in Breeze (and
tied with it) and follow this approach:
*Proposed Breeze ch
+1 (non-binding)
Cheers,
Nathan
On 20/12/2019, 12:39, "Kaxil Naik" wrote:
Hey all,
I have cut Airflow 1.10.7 RC3. This email is calling a vote on the release,
which will last for 96 hours, until Tuesday, December 24nd at 12:30 UTC.
(I've extended the vote by a day to giv
