The issue may related to differences between test and run commands as they
produce different result. In case of any long running processes, test command
should also fail, but it consistently running with result within seconds.
Regards, Daniel Kipnis.
On Feb 5, 2020, at 6:42 PM, Ash Berlin-Taylo
Hi,
Created new DAG with 3 tasks, successfully verified all tasks by using “airflow
test”, but still failed from scheduler or run without any log details.
1. airflow test five9_load_dag f9_ext_s3 2020-01-01 generated
proper result and messages
2. airflow run five9_load_dag
The "Process timed out" means that something is taking a long time and getting
killed by a timeout before it can do anything.
Is the machine or DB where this is running under a lot of load?
On Feb 5 2020, at 11:40 pm, Daniel Kipnis wrote:
>
> Hi,
>
>
> Created new DAG with 3 tasks, successfully
+1 (binding). Tested using sqlite + python 3.6.
J.
On Mon, Feb 3, 2020 at 10:46 PM Kaxil Naik wrote:
> Hello Airflow Community,
>
> This is a call for the vote to release Apache Airflow version 1.10.8.
>
> The release candidate:
> https://dist.apache.org/repos/dist/dev/airflow/1.10.8rc1/
>
> *a
Wrote this small proposal
https://docs.google.com/document/d/1ammZ2iGmGuoXNCtbpLdnj2D23nEYrCaLMbETjim3C0Q/edit?usp=sharing
I can add it to Confluence as well if pointed to where should it live (sorry
not really familiar enough w Airflow Confluence).
Working to add some more context on existing
Hello everyone,
We are hosting an Airflow Meetup in San Francisco on April 8! This meetup
is scheduled around Google Next, so if you are travelling for Google Next,
please come and join us for the meetup as well at Google office in San
Francisco.
We have our first speaker, Mike Clarke, who will s
Ok let's try it, don't know if we're violating some Apache process here but
I guess we'll find out :).
On Wed, Feb 5, 2020 at 4:22 PM Tomasz Urbaszek
wrote:
> Google docs is good to work out final version that can be published on
> confluence. But that’s only my opinion.
>
> T.
>
> On Wed, 5 Feb
Google docs is good to work out final version that can be published on
confluence. But that’s only my opinion.
T.
On Wed, 5 Feb 2020 at 22:12, Dan Davydov
wrote:
> Traditionally we've done this in confluence within the AIP although I think
> I would prefer google docs at some point in the futur
Traditionally we've done this in confluence within the AIP although I think
I would prefer google docs at some point in the future maybe :). I would
use confluence though for this.
On Wed, Feb 5, 2020 at 3:52 PM Gerard Casas Saez
wrote:
> Happy to drive this. What would be a good place to put th
Happy to drive this. What would be a good place to put this design doc?
Guessing confluence, not sure under what directory though.
Gerard Casas Saez
Twitter | Cortex | @casassaez
On Feb 4, 2020, 1:18 PM -0700, Jarek Potiuk , wrote:
> +1 short design doc would be cool.
>
> wt., 4 lut 2020, 21:16 u
Hi everyone,
There is a early proposal for optimizing airflow sensor tasks in AIP-17
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-17+Airflow+sensor+optimization
The smart sensor mentioned in this document was deployed internally in our
company and have largely reduced sensor workers. A
Yep. Learning from the bigger move I will see how small is the "smaller"
one I would love to see and can make informed decision.
On Wed, Feb 5, 2020 at 1:35 PM Ash Berlin-Taylor wrote:
> Ah right okay, not an impossible task then.
>
> Still, I am very strongly in favour of the smallest change fo
Ah right okay, not an impossible task then.
Still, I am very strongly in favour of the smallest change for the goal (which
is to allow back-ported providers to be installed on 1.10.x) - both cos then
most things are more normal, including code under a provider, and then that way
we don't have t
>
> If we make every namespace under airflow.* implict, then we have to ensure
> that our modules do not clash with _any and every_ possible top level
> package that exists in PyPi, because we cannot know what might be installed
> by our users. That seems like an impossible feat. Have I mis-underst
On Feb 5 2020, at 11:59 am, Jarek Potiuk wrote:
> So I would like that we - as a community - make an informed decision about
> it without over-simplifications and biases. I have a kind request to those
> interested in AIP-21 - can you please take a look and comment? I would not
> like to be held b
I don't think such quick assessment on a "global level" with the size of
change is complete or even correct - the alternative is more than "1 file"
change.
I think we should all try to understand what is changing, the reasoning and
consequences, and not only to look at the total numbers. Particula
There's two things I don't like: 1 the name, and that two it's more work than
we are needed for (to what I can see) no gain. Making everything an implicit
package touches 776 files, when 1 is all that is needed.
To make airflow.providers an implicit namepsace package we need to remove a
single
Hey Ash,
What do you think is the downside of changing all the packages to implicit
Except the -6000 or so useless comments in empty __init__.py ?
J.
On Wed, Feb 5, 2020 at 11:34 AM Ash Berlin-Taylor wrote:
> And to be clear: I'd rather we didn't make everything an implicit
> namespace package
On Wed, Feb 5, 2020 at 11:31 AM Ash Berlin-Taylor wrote:
> Are you talking about/test with making _ALL_ packages implicit namespaces?
>
> I would think that we would only make airflow.providers an implicit
> package, but leave all the sub packages as explicit packages: i.e. this:
>
Nope.
Pretty
And to be clear: I'd rather we didn't make everything an implicit namespace
package, but only the "official" extension points of airflow.providers. I cant
immediately think of a reason to make anything else a namespace package, and
limiting this to a single place makes the change smaller (tiny e
Are you talking about/test with making _ALL_ packages implicit namespaces?
I would think that we would only make airflow.providers an implicit package,
but leave all the sub packages as explicit packages: i.e. this:
airflow/__init__.py
airflow/utils/__init__.py
airflow/providers/ (no __init__.py)
*TL;DR; I am asking for opinions on some of the changes required to
introduce implicit native package support. This is the easiest way to make
backports of master providers packages to 1.10.x. *
NOTE!: @Ash Berlin-Taylor @Fokko Driesprong
@Felix Uellendall @Kamil Breguła
@Kaxil Naik
@basharen
Hi,
I'm running a custom operator with proper logging. However, upon running my
DAGs it shows a failed status but upon checking the logs it shows no error
logs and neither the logs prompts I had set inside my operator. Also, upon
manually checking if the task ran successfully with the output being
23 matches
Mail list logo
