[airflow-site] 01/01: 2.6.0 blog post and announcement

2023-04-29 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch 26_blog_post in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit f91f90d92dcd33c2db71f4e5c418d7c653c238bf Author: Jed Cunningham AuthorDate: Sat Apr 29 23:34:31 2023

[airflow-site] branch 26_blog_post created (now f91f90d92d)

2023-04-29 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch 26_blog_post in repository https://gitbox.apache.org/repos/asf/airflow-site.git at f91f90d92d 2.6.0 blog post and announcement This branch includes the following new commits:

[airflow] branch main updated: Upgrade PIP to 23.1.2 (#30964)

2023-04-29 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8b13a8bd85 Upgrade PIP to 23.1.2 (#30964) 8b

[GitHub] [airflow] jedcunningham merged pull request #30964: Upgrade PIP to 23.1.2

2023-04-29 Thread via GitHub
jedcunningham merged PR #30964: URL: https://github.com/apache/airflow/pull/30964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[airflow] branch main updated (17d1f3a7bb -> 21ee151cbb)

2023-04-29 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 17d1f3a7bb Fix backfill KeyError when try_number out of sync (#30653) add 21ee151cbb Upgrade the dev versio

[GitHub] [airflow] jedcunningham merged pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
jedcunningham merged PR #30958: URL: https://github.com/apache/airflow/pull/30958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] parth166 commented on issue #30840: RunID has changes

2023-04-29 Thread via GitHub
parth166 commented on issue #30840: URL: https://github.com/apache/airflow/issues/30840#issuecomment-1528932176 > Another error that seems to be a duplicate in #30898 - @parth166 if you are still working on it, that might give you more clues on reproducibility Hey @potiuk - Yess, I wa

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30965: fix issue: DAG's on_failure_callback is not invoked when task failed during testing dag.

2023-04-29 Thread via GitHub
boring-cyborg[bot] commented on PR #30965: URL: https://github.com/apache/airflow/pull/30965#issuecomment-1528909811 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] mis98zb opened a new pull request, #30965: fix issue: DAG's on_failure_callback is not invoked when task failed during testing dag.

2023-04-29 Thread via GitHub
mis98zb opened a new pull request, #30965: URL: https://github.com/apache/airflow/pull/30965 The DAG.test() code is as below: ```python # Instead of starting a scheduler, we run the minimal loop possible to check # for task readiness and dependency management. This i

[GitHub] [airflow] github-actions[bot] commented on pull request #28970: Fix support for macros with dots in DataProcJobBuilder

2023-04-29 Thread via GitHub
github-actions[bot] commented on PR #28970: URL: https://github.com/apache/airflow/pull/28970#issuecomment-1528902449 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] github-actions[bot] commented on pull request #30093: [WIP]: Add deferrable mode to `RedshiftDataOperator`

2023-04-29 Thread via GitHub
github-actions[bot] commented on PR #30093: URL: https://github.com/apache/airflow/pull/30093#issuecomment-1528902436 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] cjames23 commented on pull request #28187: Add IAM authentication to Amazon Redshift Connection by AWS Connection

2023-04-29 Thread via GitHub
cjames23 commented on PR #28187: URL: https://github.com/apache/airflow/pull/28187#issuecomment-1528897547 @o-nikolas @shubham22 @eladkal so far testing against a redshift cluster with other Redshift connections not using IAM I am not seeing anything break connections wise. It is just setti

[GitHub] [airflow] potiuk opened a new pull request, #30964: Upgrade PIP to 23.1.2

2023-04-29 Thread via GitHub
potiuk opened a new pull request, #30964: URL: https://github.com/apache/airflow/pull/30964 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow] potiuk commented on pull request #29602: Change config max tis per query

2023-04-29 Thread via GitHub
potiuk commented on PR #29602: URL: https://github.com/apache/airflow/pull/29602#issuecomment-1528891181 There are still static check problems. I recommend installing pre-commits and running them. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [airflow] potiuk commented on pull request #30704: Save scheduler execution time by caching dags

2023-04-29 Thread via GitHub
potiuk commented on PR #30704: URL: https://github.com/apache/airflow/pull/30704#issuecomment-1528890874 Second pair of eyes needed here , as this is core part. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[airflow] branch main updated (b0a40bb812 -> 17d1f3a7bb)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from b0a40bb812 Optimize deferred execution mode (#30946) add 17d1f3a7bb Fix backfill KeyError when try_number out of s

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30653: Fix backfill KeyError when try_number out of sync

2023-04-29 Thread via GitHub
boring-cyborg[bot] commented on PR #30653: URL: https://github.com/apache/airflow/pull/30653#issuecomment-1528890456 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[GitHub] [airflow] potiuk merged pull request #30653: Fix backfill KeyError when try_number out of sync

2023-04-29 Thread via GitHub
potiuk merged PR #30653: URL: https://github.com/apache/airflow/pull/30653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
potiuk commented on PR #30658: URL: https://github.com/apache/airflow/pull/30658#issuecomment-1528890251 > @potiuk Actually when I posted my comment, CI was all green and I thought for a moment I had solved the issue. As I can see it takes time so that all pipelines finish. Complain

[GitHub] [airflow] potiuk commented on a diff in pull request #30946: Optimize deferrable mode execution in bigquery sensors

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30946: URL: https://github.com/apache/airflow/pull/30946#discussion_r1181136391 ## airflow/providers/google/cloud/sensors/bigquery.py: ## @@ -110,20 +110,24 @@ def poke(self, context: Context) -> bool: def execute(self, context: Context) ->

[airflow] branch main updated (f3e82b2011 -> b0a40bb812)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f3e82b2011 Update faq.rst about missing task logs (#30717) add b0a40bb812 Optimize deferred execution mode (#30946

[GitHub] [airflow] potiuk merged pull request #30946: Optimize deferrable mode execution in bigquery sensors

2023-04-29 Thread via GitHub
potiuk merged PR #30946: URL: https://github.com/apache/airflow/pull/30946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #30726: Adding loggroomer sidecar to dag processor

2023-04-29 Thread via GitHub
potiuk commented on PR #30726: URL: https://github.com/apache/airflow/pull/30726#issuecomment-1528887188 The output is pretty clear: https://user-images.githubusercontent.com/595491/235326703-6e5de7c0-3c6b-49ab-beae-3f8b37685d57.png";> The formatting is wrong. * Did you r

[GitHub] [airflow] potiuk commented on pull request #29602: Change config max tis per query

2023-04-29 Thread via GitHub
potiuk commented on PR #29602: URL: https://github.com/apache/airflow/pull/29602#issuecomment-1528885674 > The errors has been addressed. Let's see. Approved the run now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] potiuk commented on pull request #29732: Loop through paginated response to check for cluster id

2023-04-29 Thread via GitHub
potiuk commented on PR #29732: URL: https://github.com/apache/airflow/pull/29732#issuecomment-1528885560 unexpected ident does not signal anything good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] potiuk commented on pull request #30923: Update return types of `get_key` methods on `S3Hook`

2023-04-29 Thread via GitHub
potiuk commented on PR #30923: URL: https://github.com/apache/airflow/pull/30923#issuecomment-1528885419 Ah no ... pre-commit install/run and docs build are recommende/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[airflow] branch main updated (a7eb32a5b2 -> f3e82b2011)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from a7eb32a5b2 Bump minimum Airflow version in providers (#30917) add f3e82b2011 Update faq.rst about missing task log

[GitHub] [airflow] potiuk merged pull request #30717: Update faq.rst about missing task logs

2023-04-29 Thread via GitHub
potiuk merged PR #30717: URL: https://github.com/apache/airflow/pull/30717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] lyso commented on pull request #29602: Change config max tis per query

2023-04-29 Thread via GitHub
lyso commented on PR #29602: URL: https://github.com/apache/airflow/pull/29602#issuecomment-1528885117 > Commenting on this one to ensure it doesn't get aurtoclosed. It has a very nice performance enhancement! > > @lyso What is the current status of this PR? Looks like there are some

[GitHub] [airflow] lyso commented on pull request #29602: Change config max tis per query

2023-04-29 Thread via GitHub
lyso commented on PR #29602: URL: https://github.com/apache/airflow/pull/29602#issuecomment-1528885057 > Commenting on this one to ensure it doesn't get aurtoclosed. It has a very nice performance enhancement! > > @lyso What is the current status of this PR? Looks like there are some

[GitHub] [airflow] potiuk commented on pull request #30501: DynamoDBToS3Operator - Add feature to export table to a point in time

2023-04-29 Thread via GitHub
potiuk commented on PR #30501: URL: https://github.com/apache/airflow/pull/30501#issuecomment-1528884643 CI is not happy. @utkarsharma2 -> you need to fix those . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[airflow] branch main updated (4ee2de1e38 -> a7eb32a5b2)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 4ee2de1e38 Use the Task Group explicitly passed to 'partial' if any (#30933) add a7eb32a5b2 Bump minimum Airflow

[GitHub] [airflow] potiuk merged pull request #30917: Bump minimum Airflow version in providers

2023-04-29 Thread via GitHub
potiuk merged PR #30917: URL: https://github.com/apache/airflow/pull/30917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on issue #27399: CronTriggerTimetable lost one task occasionally

2023-04-29 Thread via GitHub
potiuk commented on issue #27399: URL: https://github.com/apache/airflow/issues/27399#issuecomment-1528883872 Hard to say. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk commented on pull request #30742: Fix ExternalTaskSensor to work correctly with task groups

2023-04-29 Thread via GitHub
potiuk commented on PR #30742: URL: https://github.com/apache/airflow/pull/30742#issuecomment-1528883599 LGTM. @uranusjr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[airflow] branch remove-python-3-7 updated (4c8e5b8499 -> 2bb7bf51b7)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch remove-python-3-7 in repository https://gitbox.apache.org/repos/asf/airflow.git discard 4c8e5b8499 Remove Python 3.7 support add 2bb7bf51b7 Remove Python 3.7 support This update added new

[airflow] branch main updated: Use the Task Group explicitly passed to 'partial' if any (#30933)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4ee2de1e38 Use the Task Group explicitly passed to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30933: Use the Task Group explicitly passed to 'partial' if any

2023-04-29 Thread via GitHub
boring-cyborg[bot] commented on PR #30933: URL: https://github.com/apache/airflow/pull/30933#issuecomment-1528876339 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[GitHub] [airflow] potiuk closed issue #30932: Tasks created using "dynamic task mapping" ignore the Task Group passed as argument

2023-04-29 Thread via GitHub
potiuk closed issue #30932: Tasks created using "dynamic task mapping" ignore the Task Group passed as argument URL: https://github.com/apache/airflow/issues/30932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [airflow] potiuk merged pull request #30933: Use the Task Group explicitly passed to 'partial' if any

2023-04-29 Thread via GitHub
potiuk merged PR #30933: URL: https://github.com/apache/airflow/pull/30933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch remove-python-3-7 updated (17e408ef90 -> 4c8e5b8499)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch remove-python-3-7 in repository https://gitbox.apache.org/repos/asf/airflow.git omit 17e408ef90 Remove Python 3.7 support add 4c8e5b8499 Remove Python 3.7 support This update added new

[airflow] branch remove-python-3-7 updated (3de334936e -> 17e408ef90)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch remove-python-3-7 in repository https://gitbox.apache.org/repos/asf/airflow.git omit 3de334936e Remove Python 3.7 support add 17e408ef90 Remove Python 3.7 support This update added new

[airflow] branch remove-python-3-7 updated (f898d3eaec -> 3de334936e)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch remove-python-3-7 in repository https://gitbox.apache.org/repos/asf/airflow.git discard f898d3eaec Remove Python 3.7 support add 3de334936e Remove Python 3.7 support This update added new

[airflow] branch remove-python-3-7 created (now f898d3eaec)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch remove-python-3-7 in repository https://gitbox.apache.org/repos/asf/airflow.git at f898d3eaec Remove Python 3.7 support No new revisions were added by this update.

[GitHub] [airflow] potiuk opened a new pull request, #30963: Remove Python 3.7 support

2023-04-29 Thread via GitHub
potiuk opened a new pull request, #30963: URL: https://github.com/apache/airflow/pull/30963 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow] potiuk closed pull request #30962: Remove Python 3.7 support

2023-04-29 Thread via GitHub
potiuk closed pull request #30962: Remove Python 3.7 support URL: https://github.com/apache/airflow/pull/30962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[GitHub] [airflow] potiuk commented on a diff in pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30958: URL: https://github.com/apache/airflow/pull/30958#discussion_r1181117814 ## airflow/config_templates/default_airflow.cfg: ## @@ -250,6 +250,13 @@ daemon_umask = 0o077 # Example: dataset_manager_kwargs = {{"some_param": "some_value"}} # da

[GitHub] [airflow] potiuk commented on a diff in pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30958: URL: https://github.com/apache/airflow/pull/30958#discussion_r1181117814 ## airflow/config_templates/default_airflow.cfg: ## @@ -250,6 +250,13 @@ daemon_umask = 0o077 # Example: dataset_manager_kwargs = {{"some_param": "some_value"}} # da

[GitHub] [airflow] potiuk opened a new pull request, #30962: Remove Python 3.7 support

2023-04-29 Thread via GitHub
potiuk opened a new pull request, #30962: URL: https://github.com/apache/airflow/pull/30962 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30961: implement gcs_schema_object for BigQueryCreateExternalTableOperator

2023-04-29 Thread via GitHub
boring-cyborg[bot] commented on PR #30961: URL: https://github.com/apache/airflow/pull/30961#issuecomment-1528847519 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] attilaszombati opened a new pull request, #30961: implement gcs_schema_object for BigQueryCreateExternalTableOperator

2023-04-29 Thread via GitHub
attilaszombati opened a new pull request, #30961: URL: https://github.com/apache/airflow/pull/30961 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for m

[GitHub] [airflow] pierrejeambrun commented on a diff in pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
pierrejeambrun commented on code in PR #30958: URL: https://github.com/apache/airflow/pull/30958#discussion_r1181110567 ## airflow/config_templates/default_airflow.cfg: ## @@ -250,6 +250,13 @@ daemon_umask = 0o077 # Example: dataset_manager_kwargs = {{"some_param": "some_value"

[GitHub] [airflow] potiuk opened a new pull request, #30960: Add description on how we handle security issues in Airflow

2023-04-29 Thread via GitHub
potiuk opened a new pull request, #30960: URL: https://github.com/apache/airflow/pull/30960 This is a proposed description of the security policy we have in Apache Airflow. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://githu

[airflow] branch constraints-main updated: Updating constraints. Github run id:4839637167

2023-04-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 2727bd4506 Updating con

[GitHub] [airflow] potiuk commented on pull request #30727: Kubernetes Executor Load Time Optimizations

2023-04-29 Thread via GitHub
potiuk commented on PR #30727: URL: https://github.com/apache/airflow/pull/30727#issuecomment-1528823712 > Which is especially frustrating when reviews are hard to come by and I must constanty manage the conflicts from main. I know - been there, done that, but let's say now since I as

[GitHub] [airflow] o-nikolas commented on pull request #30727: Kubernetes Executor Load Time Optimizations

2023-04-29 Thread via GitHub
o-nikolas commented on PR #30727: URL: https://github.com/apache/airflow/pull/30727#issuecomment-1528821699 Hey @potiuk, thanks for the review! 😃 > Just a thought. @o-nikolas . MAYBE there is a way to split that one into two (at least) prs -> one extracting/renaming code/methods, may

[airflow] branch main updated (36fe6d0377 -> 1d4b1410b0)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 36fe6d0377 Remove print from dag trigger command (#30921) add 1d4b1410b0 Fix `order_by` request in list DAG rest a

[GitHub] [airflow] potiuk closed issue #30900: REST API, order_by parameter in dags list is not taken into account

2023-04-29 Thread via GitHub
potiuk closed issue #30900: REST API, order_by parameter in dags list is not taken into account URL: https://github.com/apache/airflow/issues/30900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [airflow] potiuk merged pull request #30926: Fix `order_by` request in list DAG rest api

2023-04-29 Thread via GitHub
potiuk merged PR #30926: URL: https://github.com/apache/airflow/pull/30926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #30727: Kubernetes Executor Load Time Optimizations

2023-04-29 Thread via GitHub
potiuk commented on PR #30727: URL: https://github.com/apache/airflow/pull/30727#issuecomment-1528818513 Just a thought. @o-nikolas . MAYBE there is a way to split that one into two prs -> one extracting/renaming code/methods, maybe extracting the common k8S executor types, and another mov

[GitHub] [airflow] potiuk commented on pull request #30727: Kubernetes Executor Load Time Optimizations

2023-04-29 Thread via GitHub
potiuk commented on PR #30727: URL: https://github.com/apache/airflow/pull/30727#issuecomment-1528817230 Not an easy one to review, even if it is just a refactor :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk commented on pull request #30669: [WIP] Fix try_number calculation and add a new column poke_number

2023-04-29 Thread via GitHub
potiuk commented on PR #30669: URL: https://github.com/apache/airflow/pull/30669#issuecomment-1528816474 > Hi all, I'm the author of #30653 and just wanted to share results of monkey patching our large airflow instance containing thousands of DAGs with that PR: we've had no more issues and

[GitHub] [airflow] potiuk commented on a diff in pull request #30864: Change Deferrable implementation for RedshiftResumeClusterOperator to follow standard

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30864: URL: https://github.com/apache/airflow/pull/30864#discussion_r1181094511 ## airflow/providers/amazon/aws/triggers/redshift_cluster.py: ## @@ -137,3 +137,52 @@ async def run(self): }, ) yield TriggerEve

[GitHub] [airflow] potiuk commented on a diff in pull request #30864: Change Deferrable implementation for RedshiftResumeClusterOperator to follow standard

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30864: URL: https://github.com/apache/airflow/pull/30864#discussion_r1181094511 ## airflow/providers/amazon/aws/triggers/redshift_cluster.py: ## @@ -137,3 +137,52 @@ async def run(self): }, ) yield TriggerEve

[GitHub] [airflow] potiuk commented on a diff in pull request #30864: Change Deferrable implementation for RedshiftResumeClusterOperator to follow standard

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30864: URL: https://github.com/apache/airflow/pull/30864#discussion_r1181094274 ## airflow/providers/amazon/aws/triggers/redshift_cluster.py: ## @@ -137,3 +137,52 @@ async def run(self): }, ) yield TriggerEve

[GitHub] [airflow] potiuk commented on pull request #30706: Save scheduler execution time by checking if DAG has interval or timetable

2023-04-29 Thread via GitHub
potiuk commented on PR #30706: URL: https://github.com/apache/airflow/pull/30706#issuecomment-1528814621 LGTM. @uranusjr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [airflow] potiuk merged pull request #30921: Remove print from dag trigger command

2023-04-29 Thread via GitHub
potiuk merged PR #30921: URL: https://github.com/apache/airflow/pull/30921 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated (d327642d04 -> 36fe6d0377)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d327642d04 Remove DAGs.can_create on access control doc, adjust test fixture (#30862) add 36fe6d0377 Remove print

[GitHub] [airflow] potiuk commented on pull request #28187: Add IAM authentication to Amazon Redshift Connection by AWS Connection

2023-04-29 Thread via GitHub
potiuk commented on PR #28187: URL: https://github.com/apache/airflow/pull/28187#issuecomment-1528813989 I rebased this one, it looks cool, but maybe someone from teh AWS team can comment on it ? -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [airflow] potiuk commented on issue #11134: Not able to delete the DAG from UI -

2023-04-29 Thread via GitHub
potiuk commented on issue #11134: URL: https://github.com/apache/airflow/issues/11134#issuecomment-1528813675 @gurudevkumar51 then either UPGRADE to fix the problem (you can look at the history and see linked PRs and see when they were fixed) or UPGRADE to latest version and if it sill ther

[GitHub] [airflow] potiuk commented on pull request #30914: Added support for custom KEDA scaling

2023-04-29 Thread via GitHub
potiuk commented on PR #30914: URL: https://github.com/apache/airflow/pull/30914#issuecomment-1528812710 This is an interesting one and will need a bit more thorough look from the maintainers (I also added few people who were involved in the original KEDA integration. But no matter w

[GitHub] [airflow] potiuk closed issue #30863: ValueError: Unable to configure handler 'task'

2023-04-29 Thread via GitHub
potiuk closed issue #30863: ValueError: Unable to configure handler 'task' URL: https://github.com/apache/airflow/issues/30863 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [airflow] potiuk commented on a diff in pull request #30911: Save DAG parsing time if dag is not scheduleable

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30911: URL: https://github.com/apache/airflow/pull/30911#discussion_r1181091669 ## airflow/models/dag.py: ## @@ -2755,25 +2755,28 @@ def bulk_write_to_db( session.add(orm_dag) orm_dags.append(orm_dag) -# Get the

[GitHub] [airflow] potiuk commented on pull request #30429: Add running status tab for DAGs on the UI

2023-04-29 Thread via GitHub
potiuk commented on PR #30429: URL: https://github.com/apache/airflow/pull/30429#issuecomment-1528809257 > Put it anywhere you like, this feature is very much needed, especially when you have hundreds of DAGs in your Airflow... I do not mind even if it goes to the menu somewhere, as long as

[GitHub] [airflow] shahar1 commented on issue #29115: GCSToGCSOperator delimiter bug

2023-04-29 Thread via GitHub
shahar1 commented on issue #29115: URL: https://github.com/apache/airflow/issues/29115#issuecomment-1528809010 > Due to name of argument as `delimiter`, description and code of Operator and [google-cloud-storage](https://cloud.google.com/python/docs/reference/storage/latest/buckets#listblob

[GitHub] [airflow] shahar1 commented on issue #29115: GCSToGCSOperator delimiter bug

2023-04-29 Thread via GitHub
shahar1 commented on issue #29115: URL: https://github.com/apache/airflow/issues/29115#issuecomment-1528808991 > Due to name of argument as `delimiter`, description and code of Operator and [google-cloud-storage](https://cloud.google.com/python/docs/reference/storage/latest/buckets#listblob

[GitHub] [airflow] potiuk commented on pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
potiuk commented on PR #30958: URL: https://github.com/apache/airflow/pull/30958#issuecomment-1528807559 Not sure which part of the release process it is in, but it looks like it shoudl be done slightly earlier (i.e. when we cut-off the 2.6.0 branch). -- This is an automated message from

[GitHub] [airflow] potiuk commented on a diff in pull request #30827: Save scheduler execution time by adding new Index idea for dag_run

2023-04-29 Thread via GitHub
potiuk commented on code in PR #30827: URL: https://github.com/apache/airflow/pull/30827#discussion_r1181089601 ## airflow/migrations/versions/0125_2_6_0_add_index_on_last_scheduling_decision_.py: ## @@ -0,0 +1,57 @@ +# +# Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [airflow] potiuk commented on pull request #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
potiuk commented on PR #30958: URL: https://github.com/apache/airflow/pull/30958#issuecomment-1528806834 We already have a few PRs that have wrongly generated version (https://github.com/apache/airflow/pull/30827 for example) so likely it's a good time to merge 2.6.0.dev0 -> 2.7.0.dev0 :)

[GitHub] [airflow] potiuk opened a new pull request, #30958: Upgrade the dev version of Airflow to 2.7.0

2023-04-29 Thread via GitHub
potiuk opened a new pull request, #30958: URL: https://github.com/apache/airflow/pull/30958 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow] potiuk commented on issue #30840: RunID has changes

2023-04-29 Thread via GitHub
potiuk commented on issue #30840: URL: https://github.com/apache/airflow/issues/30840#issuecomment-1528799363 Another error that seems to be a duplicate in #30898 - @parth166 if you are still working on it, that might give you more clues on reproducibility -- This is an automated message

[GitHub] [airflow] potiuk closed issue #30898: Error viewing logs for a task: AttributeError: 'NoneType' object has no attribute 'execution_date'

2023-04-29 Thread via GitHub
potiuk closed issue #30898: Error viewing logs for a task: AttributeError: 'NoneType' object has no attribute 'execution_date' URL: https://github.com/apache/airflow/issues/30898 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [airflow] potiuk commented on issue #30898: Error viewing logs for a task: AttributeError: 'NoneType' object has no attribute 'execution_date'

2023-04-29 Thread via GitHub
potiuk commented on issue #30898: URL: https://github.com/apache/airflow/issues/30898#issuecomment-1528799147 This is duplicate of https://github.com/apache/airflow/issues/30840 hopefully we will solve it soon - there is a person who works on it, you can join the effort. -- This is an au

[GitHub] [airflow] shahar1 commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
shahar1 commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1181081895 ## airflow/providers/google/cloud/transfers/bigquery_to_sql.py: ## @@ -0,0 +1,126 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[GitHub] [airflow] potiuk merged pull request #30862: Improve access control documentation on DAG level permission

2023-04-29 Thread via GitHub
potiuk merged PR #30862: URL: https://github.com/apache/airflow/pull/30862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #30862: Improve access control documentation on DAG level permission

2023-04-29 Thread via GitHub
potiuk commented on PR #30862: URL: https://github.com/apache/airflow/pull/30862#issuecomment-1528795616 Nice catch! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[airflow] branch main updated (344799984c -> d327642d04)

2023-04-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 344799984c Add unit test for static path location for flask blueprints (#30955) add d327642d04 Remove DAGs.can_cr

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #30862: Improve access control documentation on DAG level permission

2023-04-29 Thread via GitHub
boring-cyborg[bot] commented on PR #30862: URL: https://github.com/apache/airflow/pull/30862#issuecomment-1528795660 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

[GitHub] [airflow] shahar1 commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
shahar1 commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1181081895 ## airflow/providers/google/cloud/transfers/bigquery_to_sql.py: ## @@ -0,0 +1,126 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[GitHub] [airflow] shahar1 commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
shahar1 commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1181080201 ## airflow/providers/google/cloud/transfers/bigquery_to_sql.py: ## @@ -0,0 +1,126 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[GitHub] [airflow] jbbqqf commented on pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
jbbqqf commented on PR #30658: URL: https://github.com/apache/airflow/pull/30658#issuecomment-1528789111 > To start - maybe get all the tests succeed :D (they are failing currently)? Touché :wink: @potiuk Actually when I posted my comment, CI was all green and I thought for a mom

[GitHub] [airflow] shahar1 commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
shahar1 commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1181077707 ## airflow/providers/google/cloud/transfers/bigquery_to_mssql.py: ## @@ -39,86 +37,39 @@ class BigQueryToMsSqlOperator(BaseOperator): For more information on

[GitHub] [airflow] jbbqqf commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
jbbqqf commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1181077440 ## tests/system/providers/google/cloud/bigquery/example_bigquery_to_mysql.py: ## @@ -0,0 +1,99 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] jbbqqf commented on a diff in pull request #30658: Add BigQueryToPostgresOperator

2023-04-29 Thread via GitHub
jbbqqf commented on code in PR #30658: URL: https://github.com/apache/airflow/pull/30658#discussion_r1174545448 ## tests/system/providers/google/cloud/bigquery/example_bigquery_to_mysql.py: ## @@ -0,0 +1,99 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# o

[GitHub] [airflow] pankajastro commented on a diff in pull request #30946: Optimize deferrable mode execution in bigquery sensors

2023-04-29 Thread via GitHub
pankajastro commented on code in PR #30946: URL: https://github.com/apache/airflow/pull/30946#discussion_r1181062264 ## airflow/providers/google/cloud/sensors/bigquery.py: ## @@ -110,20 +110,24 @@ def poke(self, context: Context) -> bool: def execute(self, context: Contex

[GitHub] [airflow] pankajastro commented on a diff in pull request #30948: Add deferrable mode to AWS glue operators (Job & Crawl)

2023-04-29 Thread via GitHub
pankajastro commented on code in PR #30948: URL: https://github.com/apache/airflow/pull/30948#discussion_r1181060265 ## airflow/providers/amazon/aws/triggers/glue_crawler.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] pankajastro commented on a diff in pull request #30948: Add deferrable mode to AWS glue operators (Job & Crawl)

2023-04-29 Thread via GitHub
pankajastro commented on code in PR #30948: URL: https://github.com/apache/airflow/pull/30948#discussion_r1181055003 ## airflow/providers/amazon/aws/triggers/glue_crawler.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] pankajastro commented on a diff in pull request #30948: Add deferrable mode to AWS glue operators (Job & Crawl)

2023-04-29 Thread via GitHub
pankajastro commented on code in PR #30948: URL: https://github.com/apache/airflow/pull/30948#discussion_r1181055003 ## airflow/providers/amazon/aws/triggers/glue_crawler.py: ## @@ -0,0 +1,62 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[airflow] branch constraints-main updated: Updating constraints. Github run id:4838212791

2023-04-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new ae0afb61e9 Updating con

svn commit: r61543 - /dev/airflow/2.6.0rc4/

2023-04-29 Thread ephraimanierobi
Author: ephraimanierobi Date: Sat Apr 29 10:50:34 2023 New Revision: 61543 Log: Remove old release: 2.6.0rc4 Removed: dev/airflow/2.6.0rc4/

  1   2   >