abhijeets25012-tech opened a new pull request, #76740: URL: https://github.com/apache/airflow/pull/76740
A Power BI dataset can only run one refresh at a time. If a refresh is already running, for example one started manually from the Power BI portal, the refresh this operator starts fails. The only way to handle that today is to retry until no refresh is in progress. This adds `wait_for_current_refresh` to `PowerBIDatasetRefreshOperator`. When enabled, the operator checks the dataset refresh history before starting its own refresh. If a refresh is in progress, the task defers and waits for that refresh to reach a terminal state, then starts its own. The wait reuses the existing `PowerBITrigger` with `cancel_on_error=False`: the waiting task did not start that refresh and must not cancel it. A previous refresh that ends in `Failed`, `Cancelled` or `Disabled` does not fail the task, since it only needs the dataset to be free. Only a timeout fails the task, as the dataset may still be busy. The check is made in `submit_job()` and in the non-durable `execute()` path, so it applies to both execution modes and to a durable retry that resubmits after a terminal failure, while a retry that reconnects to its own active refresh is unaffected. Note: the issue proposed a separate `PowerBIDatasetRefreshWithWaitOperator` built on `defer(trigger=PowerBITrigger(...))`. That snapshot predates the current `ResumableJobMixin`/`durable` execution model, where `execute()` delegates to `execute_resumable()`. An option on the existing operator avoids duplicating the lifecycle logic and keeps the reconnect behaviour. The template has also been on `BaseOperator` rather than a standalone class. closes: #76711 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Qoder following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. --- ##### Testing - `uv run --project providers/microsoft/azure pytest providers/microsoft/azure/tests/unit/microsoft/azure/operators/test_powerbi.py -q` → 34 passed - `uv run --project providers/microsoft/azure ruff check ...` and `ruff format --check ...` on both changed Python files → clean - Full `pre-commit` suite on the commit → passed New tests cover: deferring when a refresh is in progress, starting immediately when none is, not querying the history by default, the non-durable path, resuming after the waited refresh succeeds, resuming after it fails, and failing on timeout or a missing event. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
