[
https://issues.apache.org/jira/browse/AIRFLOW-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Herrera updated AIRFLOW-1252:
------------------------------------
Flags: Important
Affects Version/s: 2.0.0
1.10.0
1.9.0
Description:
When someones calls to the endpoint _POST
<host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
that request if the body of that contains _conf_.
This occurs due to a mismatch between types when calling function
_trigger_dag()_, which is also used by *CLI*. That function perform a
_json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
other side, from *experimental API*, that type is _dict_ (because _Json_ is
processed before to get all data, such as execution_date).
There are two possibilities:
1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
from outside the function.
2. In the *experimental API*, put the conf in a string (with _json.dumps()_)
to allow _trigger_dag()_ transform into _dict_.
I have implemented the second option, so I can make a PR with that if you want.
Thank you a lot
EDIT: Also, there are no tests which uses conf in the Json passed through
request currently.
Examples:
- Before fix:
{{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
{{Content-Type: application/json}}
{{{}}
{{ "conf": {}}
{{ \"k1\": \"v1\",}}
{{ \"k2\": \"v2\",}}{{ }}
{{ \"k3\": {{[\"av1\", \"av2\", \"av3\"]}},}}{{ }}
{{ \"k4\": }}{{{ }}
{{ \"sk1\": \"sv1\",}}
{{ \"sk2\": \"sv2\" }}
{{ }}}
{{ }}}
{{ }}}
- After fix:
{{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
{{Content-Type: application/json}}{{ }}
{{{}}{{ }}
{{ "conf": {}}
{{ "k1": "v1",}}
{{ "k2": "v2",}}{{ }}
{{ "k3": {{["av1", "av2", "av3"]}},}}{{ }}
{{ "k4": }}{{{ }}
{{ "sk1": "sv1", }}
{{ "sk2": "sv2" }}
{{ }}}
{{ }}}
{{}}}
was:
When someones calls to the endpoint _POST
<host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
that request if the body of that contains _conf_.
This occurs due to a mismatch between types when calling function
_trigger_dag()_, which is also used by *CLI*. That function perform a
_json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
other side, from *experimental API*, that type is _dict_ (because _Json_ is
processed before to get all data, such as execution_date).
There are two possibilities:
1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
from outside the function.
2. In the *experimental API*, put the conf in a string (with _json.dumps()_) to
allow _trigger_dag()_ transform into _dict_.
I have implemented the second option, so I can make a PR with that if you want.
Thank you a lot
EDIT: Also, there are no tests which uses conf in the Json passed through
request currently.
> Experimental API - exception when conf is present in JSON body
> --------------------------------------------------------------
>
> Key: AIRFLOW-1252
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1252
> Project: Apache Airflow
> Issue Type: Bug
> Components: api
> Affects Versions: Airflow 1.8, 1.8.1, 1.9.0, 1.10.0, 2.0.0
> Reporter: Sergio Herrera
> Assignee: Sergio Herrera
> Priority: Major
> Labels: api
>
> When someones calls to the endpoint _POST
> <host>:<port>/api/experimental/dags/<dag_id>/dag_runs {}_, Airflow never run
> that request if the body of that contains _conf_.
> This occurs due to a mismatch between types when calling function
> _trigger_dag()_, which is also used by *CLI*. That function perform a
> _json.loads(conf)_ because from CLI the type of conf is _string_, but, in the
> other side, from *experimental API*, that type is _dict_ (because _Json_ is
> processed before to get all data, such as execution_date).
> There are two possibilities:
> 1. Look for every use of _trigger_dag()_ function and put _Json_ formatting
> from outside the function.
> 2. In the *experimental API*, put the conf in a string (with _json.dumps()_)
> to allow _trigger_dag()_ transform into _dict_.
> I have implemented the second option, so I can make a PR with that if you
> want.
> Thank you a lot
> EDIT: Also, there are no tests which uses conf in the Json passed through
> request currently.
> Examples:
> - Before fix:
> {{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
> {{Content-Type: application/json}}
> {{{}}
> {{ "conf": {}}
> {{ \"k1\": \"v1\",}}
> {{ \"k2\": \"v2\",}}{{ }}
> {{ \"k3\": {{[\"av1\", \"av2\", \"av3\"]}},}}{{ }}
> {{ \"k4\": }}{{{ }}
> {{ \"sk1\": \"sv1\",}}
> {{ \"sk2\": \"sv2\" }}
> {{ }}}
> {{ }}}
> {{ }}}
> - After fix:
> {{POST /api/experimental/dags/test_conf/dag_runs HTTP/1.1}}
> {{Content-Type: application/json}}{{ }}
> {{{}}{{ }}
> {{ "conf": {}}
> {{ "k1": "v1",}}
> {{ "k2": "v2",}}{{ }}
> {{ "k3": {{["av1", "av2", "av3"]}},}}{{ }}
> {{ "k4": }}{{{ }}
> {{ "sk1": "sv1", }}
> {{ "sk2": "sv2" }}
> {{ }}}
> {{ }}}
> {{}}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
