[ https://issues.apache.org/jira/browse/AIRFLOW-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982926#comment-16982926 ]
Kamil Bregula commented on AIRFLOW-5544: ---------------------------------------- [~coolcute] Is it still valid? My team has made improvements in this integration. > GCP datastore operation pull should have some retry to avoid false alarm > ------------------------------------------------------------------------ > > Key: AIRFLOW-5544 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5544 > Project: Apache Airflow > Issue Type: Bug > Components: gcp > Affects Versions: 1.10.5 > Reporter: Yang Hu > Priority: Minor > > Hi Airflow, > We noticed that sometime, google status url will return us 500 and airflow > will mark the whole task failed, but is actually runs fine. > > I think we need some retry before failing the task, at least at this 500 case > Sample request: > > [2019-09-18 21:45:19,543] \{base_task_runner.py:98} INFO - Subtask: > googleapiclient.errors.HttpError: <HttpError 500 when requesting <a > href="https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json" > > target="_blank">https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json</a> > returned "Unknown Error."> > Ref code: > [https://github.com/apache/airflow/blob/7e4330cce0b4f333b1658cdc315b06505cf9dd76/airflow/gcp/hooks/datastore.py#L269] -- This message was sent by Atlassian Jira (v8.3.4#803005)