set92 opened a new issue, #55804: URL: https://github.com/apache/airflow/issues/55804
### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.10.3 ### What happened? I tried using the deferrable parameter = True. I got an error of lack of permissions in AWS (`botocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the FilterLogEvents operation: ...`), and thought that since I got an error, the job in Glue was cancelled. So, after fixing the permissions, I relaunched the task and duplicated the data. So, basically when it was launched the first time it launched the Glue job, tried to pass it deferrable mode, it couldn't, so it threw an error, but didn't cancel the glue job. ### What you think should happen instead? I think the normal behavior should be that if an error is shown, any work that is being done should be stopped. ### How to reproduce In my case, the glue jobs are doing queries in redshift and saving them to s3 in a specific bucket. When it runs several times in a row it doesn't overwrite the contents, it only exports the data again to the same folder. ### Operating System MWAA AWS ### Versions of Apache Airflow Providers ``` [apache-airflow-providers-amazon](https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.0.0) 9.0.0 Amazon integration (including [Amazon Web Services (AWS)](https://aws.amazon.com/)). [apache-airflow-providers-celery](https://airflow.apache.org/docs/apache-airflow-providers-celery/3.8.3) 3.8.3 [Celery](https://docs.celeryq.dev/en/stable/) [apache-airflow-providers-common-compat](https://airflow.apache.org/docs/apache-airflow-providers-common-compat/1.7.3) 1.7.3 Common Compatibility Provider - providing compatibility code for previous Airflow versions [apache-airflow-providers-common-io](https://airflow.apache.org/docs/apache-airflow-providers-common-io/1.4.2) 1.4.2 ``Common IO Provider`` [apache-airflow-providers-common-sql](https://airflow.apache.org/docs/apache-airflow-providers-common-sql/1.28.0) 1.28.0 [Common SQL Provider](https://en.wikipedia.org/wiki/SQL) [apache-airflow-providers-fab](https://airflow.apache.org/docs/apache-airflow-providers-fab/1.5.0) 1.5.0 [Flask App Builder](https://flask-appbuilder.readthedocs.io/) [apache-airflow-providers-ftp](https://airflow.apache.org/docs/apache-airflow-providers-ftp/3.11.1) 3.11.1 [File Transfer Protocol (FTP)](https://tools.ietf.org/html/rfc114) [apache-airflow-providers-http](https://airflow.apache.org/docs/apache-airflow-providers-http/4.13.2) 4.13.2 [Hypertext Transfer Protocol (HTTP)](https://www.w3.org/Protocols/) [apache-airflow-providers-imap](https://airflow.apache.org/docs/apache-airflow-providers-imap/3.7.0) 3.7.0 [Internet Message Access Protocol (IMAP)](https://tools.ietf.org/html/rfc3501) [apache-airflow-providers-microsoft-mssql](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-mssql/3.9.1) 3.9.1 [Microsoft SQL Server (MSSQL)](https://www.microsoft.com/sql-server/) [apache-airflow-providers-pagerduty](https://airflow.apache.org/docs/apache-airflow-providers-pagerduty/5.0.2) 5.0.2 [Pagerduty](https://www.pagerduty.com/) [apache-airflow-providers-postgres](https://airflow.apache.org/docs/apache-airflow-providers-postgres/5.13.1) 5.13.1 [PostgreSQL](https://www.postgresql.org/) [apache-airflow-providers-slack](https://airflow.apache.org/docs/apache-airflow-providers-slack/9.2.0) 9.2.0 [Slack](https://slack.com/) services integration including: - [Slack API](https://api.slack.com/) - [Slack Incoming Webhook](https://api.slack.com/messaging/webhooks) [apache-airflow-providers-smtp](https://airflow.apache.org/docs/apache-airflow-providers-smtp/1.8.0) 1.8.0 [Simple Mail Transfer Protocol (SMTP)](https://tools.ietf.org/html/rfc5321) [apache-airflow-providers-snowflake](https://airflow.apache.org/docs/apache-airflow-providers-snowflake/6.5.3) 6.5.3 [Snowflake](https://www.snowflake.com/) [apache-airflow-providers-sqlite](https://airflow.apache.org/docs/apache-airflow-providers-sqlite/3.9.0) 3.9.0 [SQLite](https://www.sqlite.org/) ``` ### Deployment Amazon (AWS) MWAA ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
