Hello,
I have a quite important DAG (~20 tasks) using multiple Operators. I
dynamically create DAG using a dag = XXXDagFactory.create(**kwargs). I
would like to test this dag factory. For now I have a big method create. It
is something like
with Dag() as dag:
t1 = PythonOperator(**kwargs)
t2 = BashOperator(**kwargs)
t3 = ECSOperator(**kwargs)
...
t1 >> t2 >> t3
...
return dag
With 300 lines of code :)
What do you recommend to write unit tests to assess that Operators are
called with right arguments and so on.
I don't want to have a unique test_create() method.
Should I create "business" method Operator wrapping technical Operator. For
instance :
class DoSomethindParticularOperator(PythonOperator):
def __init__(**specific_args):
Or rather write just method returning task and test them.
def scoringTask(a, b, c):
return ECSOperator(...)
Do you have some tips ?
Thanks!
--
*Vincent Poulain*
Senior Software Engineer
Office +33 1 75 50 67 26 <+33%201%2075%2050%2067%2026> | Mobile +33 6 21 82
87 62 | [email protected] <[email protected]>
Tinyclues | 51 rue Étienne Marcel, 75001 Paris
www.tinyclues.com <http://bit.ly/2hNL4Fs> | @tinyclues
<https://twitter.com/Tinyclues>
