[GitHub] [airflow] glingle05 commented on issue #14417: DruidOperator failing to submit ingestion tasks : Getting 500 error code from Druid
glingle05 commented on issue #14417:
URL: https://github.com/apache/airflow/issues/14417#issuecomment-900799207
This fix is a workaround to a bug in the DruidHook.submit_indexing_job()
itself.
The requests.post line in submit_indexing_job is the real problem:
`
req_index = requests.post(url, data=json_index_spec,
headers=self.header, auth=self.get_auth())
`
should be
`
req_index = requests.post(url, json=json_index_spec,
headers=self.header, auth=self.get_auth())
`
Notice that passing the json_index_spec Dict using the data argument causes
the dict to be converted
using simple string conversion. This results in the string keys and values
to be quoted with single-tick (')
which results in the payload Not being json-compliant. If the
json_index_spec is passed in using the json param
it is converted to string using json lib and the payload is json with string
keys and values being quoted with
double-tick (") and request will succeed.
HTH, Glenn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [airflow] glingle05 commented on issue #14417: DruidOperator failing to submit ingestion tasks : Getting 500 error code from Druid
glingle05 commented on issue #14417:
URL: https://github.com/apache/airflow/issues/14417#issuecomment-900799207
This fix is a workaround to a bug in the DruidHook.submit_indexing_job()
itself.
The requests.post line in submit_indexing_job is the real problem:
`
req_index = requests.post(url, data=json_index_spec,
headers=self.header, auth=self.get_auth())
`
should be
`
req_index = requests.post(url, json=json_index_spec,
headers=self.header, auth=self.get_auth())
`
Notice that passing the json_index_spec Dict using the data argument causes
the dict to be converted
using simple string conversion. This results in the string keys and values
to be quoted with single-tick (')
which results in the payload Not being json-compliant. If the
json_index_spec is passed in using the json param
it is converted to string using json lib and the payload is json with string
keys and values being quoted with
double-tick (") and request will succeed.
HTH, Glenn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
