tgmof opened a new issue, #72385:
URL: https://github.com/apache/airflow/issues/72385
### Under which category would you file this issue?
Providers
### Apache Airflow version
2.11.1
### What happened and how to reproduce it?
- Running a bunch of CloudDataFusionStartPipelineOperator in parallel
- Observing a third to half of them failing with error:
```
File
"/opt/python3.11/lib/python3.11/site-packages/airflow/providers/google/cloud/hooks/datafusion.py",
line 185, in _check_response_status_and_data
raise HTTPError(message)
requests.exceptions.HTTPError: Retrieving a pipeline state failed with code
404
```
### What you think should happen instead?
- It should not fail because this introduces a regression of a fix delivered
6 years ago: https://github.com/apache/airflow/pull/10031
- Clearly the PR makes no sense because the "new exceptions chosen" are not
in sync: One function raises HTTPError while the one calling that function has
an except KeyError... I guess AI has at least this advantage that it would
catch such simple error as it's 100% a logic bug and 100% contained in a single
PR... Maybe activate CodeRabbit or whatever competitor that is free for OSS?
### Operating System
_No response_
### Deployment
Google Managed Service for Apache Airflow
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
apache-airflow-providers-google==22.2.2
https://docs.cloud.google.com/composer/docs/versions-packages#composer-2-17-9-airflow-2-11-1
### Official Helm Chart version
Not Applicable
### Kubernetes Version
Not Applicable
### Helm Chart configuration
Not Applicable
### Docker Image customizations
Not Applicable
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
