I am sharing the problem that we face very frequently. When we go a
particular dag and click the logs for a particular task_instance we end up
with following error. The logs become available after the task finished i.e
when it fetch from external source like s3.
*** Log file isn't local.
*** Fetc
I’m not sure package structure based on whether major providers will fund
development is the right approach. My $.02
Sent from a device with less than stellar autocorrect
> On Jan 7, 2019, at 3:44 PM, Tim Swast wrote:
>
> In general it’s easier for cloud providers to fund development of opera
In general it’s easier for cloud providers to fund development of operators
that bring data in. I’d say if there is overlap, put the operator in the
target system’s repo.
On Mon, Jan 7, 2019 at 2:17 PM Maxime Beauchemin
wrote:
> Something to think about is how data transfer operators like the
>
I would like to mention that there is a 2 pull requests that introduces
plugin support installed by pip
https://github.com/apache/airflow/pull/4412
https://github.com/apache/airflow/pull/730
These PRs should be analyzed to meet all our requirements
It is worth mentioning that this change not only
Something to think about is how data transfer operators like the
MysqlToHiveOperator usually rely on 2 hooks. With a package-specific
approach that may mean something like an `airflow-hive`, `airflow-mysql`
and `airflow-mysql-hive` packages, where the `airflow-mysql-hive` package
depends on the two
I've created AIP-8:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827303
To follow-up from the discussion about splitting hooks/operators out of the
core Airflow package at
http://mail-archives.apache.org/mod_mbox/airflow-dev/201809.mbox/%3c308670db-bd2a-4738-81b1-3f6fb312c..
Hi Fokko,
Currently there are still a few test files which are not pretended with
“test_”, including tests/core.py, tests/models.py, and
tests/plugins_manager.py.
The tests cases in the tests/core.py & tests/models.py are actually working
because they are imported explicitly in tests/__init__.py.
Yes, let's sort out our CI with collective effort. We need to make sure our
CI is green + all the tests are being checked.
Regards,
Kaxil
On Mon, Jan 7, 2019, 09:20 Driesprong, Fokko Hi Björn,
>
> This is an issue indeed. Only picking up the files with the test_ prepended
> is the default behavi
Hi Björn,
This is an issue indeed. Only picking up the files with the test_ prepended
is the default behavior. We've had this issue before:
https://lists.apache.org/thread.html/fe868e1cad2d1c645670f38e92d14fc8e096c6552b72edf3d9c2f94c@%3Cdev.airflow.apache.org%3E
Deng Xiaodong fixed a couple of th
