[jira] [Commented] (AIRFLOW-5544) GCP datastore operation pull should have some retry to avoid false alarm

[Yang Hu (Jira)]

    [ 
https://issues.apache.org/jira/browse/AIRFLOW-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982972#comment-16982972
 ] 

Yang Hu commented on AIRFLOW-5544:
----------------------------------

Glad there are fixes there, we are on GCP Cloud Composer so that we have to 
wait Google pick up the version which has this update, but thanks for informing 
us!

> GCP datastore operation pull should have some retry to avoid false alarm
> ------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5544
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5544
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: gcp
>    Affects Versions: 1.10.5
>            Reporter: Yang Hu
>            Priority: Minor
>
> Hi Airflow,
> We noticed that sometime, google status url will return us 500 and airflow 
> will mark the whole task failed, but is actually runs fine.
>  
> I think we need some retry before failing the task, at least at this 500 case
> Sample request:
> > [2019-09-18 21:45:19,543] \{base_task_runner.py:98} INFO - Subtask: 
> googleapiclient.errors.HttpError: &lt;HttpError 500 when requesting <a 
> href="https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json";
>  
> target="_blank">https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json</a>
>  returned &quot;Unknown Error.&quot;&gt;
> Ref code:
> [https://github.com/apache/airflow/blob/7e4330cce0b4f333b1658cdc315b06505cf9dd76/airflow/gcp/hooks/datastore.py#L269]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)