Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-12 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2346033661 cool, will create a new pr for the same. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-11 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2345335723 if that's the case, it makes sense to have one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-10 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2342581145 ya but not always, sometime root `state_message` = `Task t1, t2... failed with message: Workload failed, see run output for details. This caused all downstream tasks to get skipped.`

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-10 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2340551364 @Lee-W any suggestions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-03 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2327834405 > @Lee-W currently we are checking in root state_message only, should we also check `error` key in every fail task array as well ? Could you provide an example to illustrate this? T

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-09-03 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2325875691 @Lee-W currently we are checking in root state_message only, should we also check `error` key in every fail task array as well ? -- This is an automated message from the Apache Gi

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-30 Thread via GitHub
Lee-W merged PR #41412: URL: https://github.com/apache/airflow/pull/41412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-30 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2320755725 LGTM 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-28 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2314990423 @Lee-W changes are done, can you please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-28 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2314726476 thanks @Lee-W , making the changes, Yes, I also agree with same default behaviour. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-28 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2314696702 > @Lee-W I'm thinking for an option not to trigger repair if reason matches nothing in the dict `databricks_repair_reason_new_settings` , what's your thought on this? I don't think

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-27 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2313980116 @Lee-W I'm thinking for an option not to trigger repair if reason matches nothing in the dict `databricks_repair_reason_new_settings` , what's your thought on this? -- This is an

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-27 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2312598134 > > > @Lee-W should we have one more configurable like user want to repair only in case if `databricks_repair_reason_new_settings` dict key contains the reason, otherwise don't repair at

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2308679006 > > @Lee-W should we have one more configurable like user want to repair only in case if `databricks_repair_reason_new_settings` dict key contains the reason, otherwise don't repair

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
Lee-W commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2308675722 > @Lee-W should we have one more configurable like user want to repair only in case if `databricks_repair_reason_new_settings` dict key contains the reason, otherwise don't repair at all(

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2308671800 @Lee-W should we have one more configurable like user want to repair only in case if `databricks_repair_reason_new_settings` dict key contains the reason, otherwise don't repair at

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2308661220 @Lee-W require changes done, also added assert_called_with check for `update_job` method as well. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
gaurav7261 commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1730205681 ## airflow/providers/databricks/operators/databricks.py: ## @@ -123,6 +124,30 @@ def _handle_databricks_operator_execution(operator, hook, log, context) -> None:

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
gaurav7261 commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1730205636 ## airflow/providers/databricks/operators/databricks.py: ## @@ -123,6 +124,30 @@ def _handle_databricks_operator_execution(operator, hook, log, context) -> None:

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-24 Thread via GitHub
Lee-W commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1730204759 ## airflow/providers/databricks/operators/databricks.py: ## @@ -123,6 +124,30 @@ def _handle_databricks_operator_execution(operator, hook, log, context) -> None:

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-23 Thread via GitHub
gaurav7261 commented on PR #41412: URL: https://github.com/apache/airflow/pull/41412#issuecomment-2307989361 @Lee-W @shahar1 can you please review now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-20 Thread via GitHub
gaurav7261 commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1723654239 ## airflow/providers/databricks/operators/databricks.py: ## @@ -872,7 +896,23 @@ def execute_complete(self, context: Context, event: dict[str, Any] | None = None

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-15 Thread via GitHub
Lee-W commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1718111953 ## airflow/providers/databricks/operators/databricks.py: ## @@ -107,6 +107,22 @@ def _handle_databricks_operator_execution(operator, hook, log, context) -> None:

Re: [PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-14 Thread via GitHub
shahar1 commented on code in PR #41412: URL: https://github.com/apache/airflow/pull/41412#discussion_r1717304462 ## airflow/providers/databricks/operators/databricks.py: ## @@ -107,6 +107,22 @@ def _handle_databricks_operator_execution(operator, hook, log, context) -> None:

[PR] [FEAT] databricks repair run with reason match and appropriate new settings [airflow]

2024-08-12 Thread via GitHub
gaurav7261 opened a new pull request, #41412: URL: https://github.com/apache/airflow/pull/41412 [FEAT] databricks repair run with reason match and appropriate new settings in `DatabricksRunNowOperator` This is useful in scenario like where we get databr