Re: [PR] Add pyspark decorator [airflow]

2023-10-28 Thread via GitHub
eladkal commented on code in PR #35247: URL: https://github.com/apache/airflow/pull/35247#discussion_r1375379337 ## airflow/example_dags/example_pyspark.py: ## Review Comment: We use system tests for example dags see https://github.com/apache/airflow/blob/bfcae349b88

Re: [PR] Use ExitStack to manage mutation of secrets_backend_list in dag.test [airflow]

2023-10-28 Thread via GitHub
eladkal commented on PR #34620: URL: https://github.com/apache/airflow/pull/34620#issuecomment-1784014051 @dstandish needs rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [I] Relax mandatory requirement for `start_date` when `schedule=None` [airflow]

2023-10-28 Thread via GitHub
profwacko commented on issue #35199: URL: https://github.com/apache/airflow/issues/35199#issuecomment-1784012828 I'll take a look at this as well! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
jtmiclat commented on PR #35249: URL: https://github.com/apache/airflow/pull/35249#issuecomment-1784011487 Closed this in favor of https://github.com/apache/airflow/pull/35250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[PR] Allow additional columns in cliu dag list [airflow]

2023-10-28 Thread via GitHub
jtmiclat opened a new pull request, #35250: URL: https://github.com/apache/airflow/pull/35250 This closes #34765 This adds an additional optional parameter to the cli command `airflow dag list` ```bash root@956035f65680:/opt/airflow# airflow dags list --additional-columns=l

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
jtmiclat closed pull request #35249: Add last_parsed_time to cli command, airflow dags list URL: https://github.com/apache/airflow/pull/35249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Status of testing Providers that were prepared on October 28, 2023 [airflow]

2023-10-28 Thread via GitHub
eladkal commented on issue #35240: URL: https://github.com/apache/airflow/issues/35240#issuecomment-1784009171 > I have tested, S3 particularly, for #34729. However, this only becomes relevant for Airflow 2.8 - integrations until that time won't be active / work. Yes this is expected

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
jtmiclat commented on code in PR #35249: URL: https://github.com/apache/airflow/pull/35249#discussion_r1375373310 ## airflow/models/dag.py: ## @@ -1375,6 +1375,11 @@ def get_is_paused(self, session=NEW_SESSION) -> None: """Return a boolean indicating whether this DAG is

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
uranusjr commented on code in PR #35249: URL: https://github.com/apache/airflow/pull/35249#discussion_r1375372071 ## airflow/models/dag.py: ## @@ -1375,6 +1375,11 @@ def get_is_paused(self, session=NEW_SESSION) -> None: """Return a boolean indicating whether this DAG is

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
uranusjr commented on code in PR #35249: URL: https://github.com/apache/airflow/pull/35249#discussion_r1375372071 ## airflow/models/dag.py: ## @@ -1375,6 +1375,11 @@ def get_is_paused(self, session=NEW_SESSION) -> None: """Return a boolean indicating whether this DAG is

Re: [PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
boring-cyborg[bot] commented on PR #35249: URL: https://github.com/apache/airflow/pull/35249#issuecomment-1784000443 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Add last_parsed_time to cli command, airflow dags list [airflow]

2023-10-28 Thread via GitHub
jtmiclat opened a new pull request, #35249: URL: https://github.com/apache/airflow/pull/35249 This closes #34765 This adds the last_parsed_time to the cli command `airflow dag list` Before ``` root@956035f65680:/opt/airflow# airflow dags list dag_id | filepath | ow

Re: [PR] Suppress false warning when TI state is QUEUED and TI doesn't have a start_date [airflow]

2023-10-28 Thread via GitHub
kzosabe commented on PR #34771: URL: https://github.com/apache/airflow/pull/34771#issuecomment-1783995424 Yes, but I would like some advice regarding the points I mentioned above. @eladkal @uranusjr -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk closed pull request #35248: Include 'use_dill' in venv hash caching for PythonVirtualenvOperator URL: https://github.com/apache/airflow/pull/35248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35248: URL: https://github.com/apache/airflow/pull/35248#issuecomment-1783956146 False alarm. It was a dependency issue :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35248: URL: https://github.com/apache/airflow/pull/35248#issuecomment-1783952986 Hmmm.. Actually... Not such easy. I will have to take a closer look at tests failing because _requirements_list **should** already include dill.. Need a bit closer look. -- This is an

Re: [PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35248: URL: https://github.com/apache/airflow/pull/35248#issuecomment-1783952456 cc: @jens-scheffler-bosch :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35248: URL: https://github.com/apache/airflow/pull/35248#issuecomment-1783952416 Found that while testing #35160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[PR] Include 'use_dill' in venv hash caching for PythonVirtualenvOperator [airflow]

2023-10-28 Thread via GitHub
potiuk opened a new pull request, #35248: URL: https://github.com/apache/airflow/pull/35248 The #33355 added caching possibility to Python Virtualenv Operator in order to improve speed of task execution. However cache calculation did not include "use_dill" parameter - use_dill modifies the

Re: [PR] full screen mode for logs [airflow]

2023-10-28 Thread via GitHub
github-actions[bot] commented on PR #34233: URL: https://github.com/apache/airflow/pull/34233#issuecomment-1783948834 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Fix passing dags.gitSync.sshKeySecret with dags.persistence.enabled=true [airflow]

2023-10-28 Thread via GitHub
github-actions[bot] commented on PR #34331: URL: https://github.com/apache/airflow/pull/34331#issuecomment-1783948819 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] added default_run_id parameter [airflow]

2023-10-28 Thread via GitHub
github-actions[bot] commented on PR #34234: URL: https://github.com/apache/airflow/pull/34234#issuecomment-1783948826 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [I] Status of testing Providers that were prepared on October 28, 2023 [airflow]

2023-10-28 Thread via GitHub
bolkedebruin commented on issue #35240: URL: https://github.com/apache/airflow/issues/35240#issuecomment-1783940406 I have tested, S3 particularly, for #34729. However, this only becomes relevant for Airflow 2.8 - integrations until that time won't be active / work. -- This is an automate

Re: [PR] Don't get DAG out of DagBag when we already have it [airflow]

2023-10-28 Thread via GitHub
jens-scheffler-bosch commented on PR #35243: URL: https://github.com/apache/airflow/pull/35243#issuecomment-1783937825 Yes, in deep I was also scratching my head. Obviously there is a kind of basic caching but also with expiry check. The main driver for the `lru_cache` was the use in `_get_

[PR] Add pyspark decorator [airflow]

2023-10-28 Thread via GitHub
bolkedebruin opened a new pull request, #35247: URL: https://github.com/apache/airflow/pull/35247 This add the pyspark decorator so that spark can be run inline so that results, like dataframes, can be shared. The spark context gets injected into the python callable.

Re: [PR] Add a public interface for custom weight_rule implementation [airflow]

2023-10-28 Thread via GitHub
hussein-awala commented on PR #35210: URL: https://github.com/apache/airflow/pull/35210#issuecomment-1783932883 > I am just not sure whether the pickling of DAGs after parsing is sufficient such that a custom weight rule gets to Scheduler. Have you tested with separated DAG parser and Sched

Re: [PR] Add description on what goes into the next release [airflow]

2023-10-28 Thread via GitHub
aritra24 commented on code in PR #35245: URL: https://github.com/apache/airflow/pull/35245#discussion_r1375327877 ## README.md: ## @@ -446,6 +450,42 @@ If you would like to become a maintainer, please review the Apache Airflow [committer requirements](https://github.com/apach

(airflow) branch main updated: Add Duckdb as devel dependency (#35246)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new af0cdb586b Add Duckdb as devel dependency (#35246)

Re: [PR] Add Duckdb as devel dependency [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35246: URL: https://github.com/apache/airflow/pull/35246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch constraints-main updated: Updating constraints. Github run id:6678811043

2023-10-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 8f01156f62 Updating con

Re: [PR] Add Duckdb as devel dependency [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35246: URL: https://github.com/apache/airflow/pull/35246#issuecomment-1783922445 Most importantly it will avoids this messsage on initdb in Breeze :) ``` [2023-10-28T21:14:16.458+] {db.py:1620} INFO - Creating tables INFO [alembic.runtime.migration] C

Re: [PR] Add Duckdb as devel dependency [airflow]

2023-10-28 Thread via GitHub
potiuk commented on code in PR #35246: URL: https://github.com/apache/airflow/pull/35246#discussion_r1375322453 ## setup.py: ## @@ -443,6 +443,10 @@ def write_version(filename: str = str(AIRFLOW_SOURCES_ROOT / "airflow" / "git_ve "wheel", ] +_devel_only_duckdb = [ +

Re: [I] Make `_run_parsing_loop` in the dag-processor component execute only once. [airflow]

2023-10-28 Thread via GitHub
boushphong closed issue #35231: Make `_run_parsing_loop` in the dag-processor component execute only once. URL: https://github.com/apache/airflow/issues/35231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Make `_run_parsing_loop` in the dag-processor component execute only once. [airflow]

2023-10-28 Thread via GitHub
boushphong commented on issue #35231: URL: https://github.com/apache/airflow/issues/35231#issuecomment-1783922029 Closing this! There is a `num_run` option in the dag-processor that I was unaware of 🙇 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] OAUTH2/OKTA Authentication not working with Airflow2.7.1 [airflow]

2023-10-28 Thread via GitHub
codeprasan commented on issue #35239: URL: https://github.com/apache/airflow/issues/35239#issuecomment-1783921120 > Did this ever worked in any Airflow version or is it something you try for the first time? > > This feels more like troubleshooting question to [FAB](https://github.com

[PR] Add Duckdb as devel dependency [airflow]

2023-10-28 Thread via GitHub
potiuk opened a new pull request, #35246: URL: https://github.com/apache/airflow/pull/35246 Duckdb becomes very popular and we already have some examples that are using duckdb in examples. This makes those examples fail in the breeze CI environment. Having duckdb in the environment would be

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-28 Thread via GitHub
hussein-awala closed pull request #35225: Fix K8S executor override config using pod_override_object URL: https://github.com/apache/airflow/pull/35225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1783916472 FYI. I removed the "full tests needed" and rebased to check if there is indeed an issue that "K8S System tests" are not triggered for such PR - I think they should be - looking at select

Re: [I] Airflow progressive slowness [airflow]

2023-10-28 Thread via GitHub
potiuk commented on issue #32928: URL: https://github.com/apache/airflow/issues/32928#issuecomment-1783915309 I prepared pr #35245 and started proposal on devlist to add description on the process/approach we are using. See https://lists.apache.org/thread/05njmmqvwl0gn20f2go9d420xhzptrw2 -

Re: [I] Status of testing Providers that were prepared on October 28, 2023 [airflow]

2023-10-28 Thread via GitHub
potiuk commented on issue #35240: URL: https://github.com/apache/airflow/issues/35240#issuecomment-1783913977 Min version for presto is nicely bumped :) #35132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] remove K8S 1.24 support [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35214: URL: https://github.com/apache/airflow/pull/35214#issuecomment-1783913253 > > > I think we should consider [Alibaba Cloud](https://www.alibabacloud.com/help/en/ack/product-overview/support-for-kubernetes-versions) as a major, no? Otherwise, we should replace "

Re: [PR] enhance druid connection parameters [airflow]

2023-10-28 Thread via GitHub
eladkal commented on code in PR #35244: URL: https://github.com/apache/airflow/pull/35244#discussion_r1375316519 ## docs/apache-airflow-providers-apache-druid/connections/druid.rst: ## @@ -0,0 +1,48 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] enhance druid connection parameters [airflow]

2023-10-28 Thread via GitHub
eladkal commented on code in PR #35244: URL: https://github.com/apache/airflow/pull/35244#discussion_r1375316519 ## docs/apache-airflow-providers-apache-druid/connections/druid.rst: ## @@ -0,0 +1,48 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] Add description on what goes into the next release [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35245: URL: https://github.com/apache/airflow/pull/35245#issuecomment-1783910830 This is a follow-up after discussion and question from the user https://github.com/apache/airflow/issues/32928#issuecomment-1783709930 on what can expect to go to the next release. We've

[PR] Add description on what goes into the next release [airflow]

2023-10-28 Thread via GitHub
potiuk opened a new pull request, #35245: URL: https://github.com/apache/airflow/pull/35245 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)*

Re: [I] Relax mandatory requirement for `start_date` when `schedule=None` [airflow]

2023-10-28 Thread via GitHub
vishnucoder1 commented on issue #35199: URL: https://github.com/apache/airflow/issues/35199#issuecomment-1783909865 I would like to give a try. Please assign. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[PR] enhance druid connection parameters [airflow]

2023-10-28 Thread via GitHub
blcksrx opened a new pull request, #35244: URL: https://github.com/apache/airflow/pull/35244 closes: #35012 --- Enhancement on this PR: - Support all the other connection parameters that exists on the ``pydruid.connect`` in the `get_conn` method. - Connect

Re: [PR] Don't get DAG out of DagBag when we already have it [airflow]

2023-10-28 Thread via GitHub
ashb commented on PR #35243: URL: https://github.com/apache/airflow/pull/35243#issuecomment-1783904570 >> Because the time on our slow DB to query the dag took between 50ms and 250ms and if you execute this only once or 60 times during one scheduler loop run this makes a big change.

Re: [PR] Don't get DAG out of DagBag when we already have it [airflow]

2023-10-28 Thread via GitHub
ashb commented on code in PR #35243: URL: https://github.com/apache/airflow/pull/35243#discussion_r1375310439 ## airflow/jobs/scheduler_job_runner.py: ## @@ -1064,12 +1063,8 @@ def _do_scheduling(self, session: Session) -> int: callback_tuples = self._schedule_all_d

Re: [PR] Allow optional defaults in required fields with manual triggered dags [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #31301: URL: https://github.com/apache/airflow/pull/31301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Allow optional defaults in required fields with manual triggered dags (#31301)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3d7c8f7d05 Allow optional defaults in required fiel

Re: [I] Params Definition is Required to be JSON Schema Valid also if Manually Triggered [airflow]

2023-10-28 Thread via GitHub
potiuk closed issue #31299: Params Definition is Required to be JSON Schema Valid also if Manually Triggered URL: https://github.com/apache/airflow/issues/31299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

(airflow) branch main updated: Add connexion<3.0 upper bound (#35218)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f592b2c40b Add connexion<3.0 upper bound (#35218) f

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-28 Thread via GitHub
boring-cyborg[bot] commented on PR #35218: URL: https://github.com/apache/airflow/pull/35218#issuecomment-1783902992 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [I] 2.7.X should add dep constraint: "connexion<3" [airflow]

2023-10-28 Thread via GitHub
potiuk closed issue #35217: 2.7.X should add dep constraint: "connexion<3" URL: https://github.com/apache/airflow/issues/35217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35218: URL: https://github.com/apache/airflow/pull/35218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] OAUTH2/OKTA Authentication not working with Airflow2.7.1 [airflow]

2023-10-28 Thread via GitHub
eladkal commented on issue #35239: URL: https://github.com/apache/airflow/issues/35239#issuecomment-1783902412 Did this ever worked in any Airflow version or is it something you try for the first time? This feels more like troubleshooting question to [FAB](https://github.com/dpgaspar

(airflow) branch main updated: Add common.io to boring-cyborg.yml (#35242)

2023-10-28 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 44d494bae8 Add common.io to boring-cyborg.yml (#35

Re: [PR] Add common.io to boring-cyborg.yml [airflow]

2023-10-28 Thread via GitHub
eladkal merged PR #35242: URL: https://github.com/apache/airflow/pull/35242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783897920 > @potiuk Thanks for the heads-up, I'll address all those issues in the LLM ops providers before merging. :) You will have no other choice now 😁 ... Your PRs will not get green :)

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
utkarsharma2 commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783897696 @potiuk Thanks for the heads-up, I'll address all those issues in the LLM ops providers before merging. :) -- This is an automated message from the Apache Git Service. To respond

Re: [I] Status of testing Providers that were prepared on October 28, 2023 [airflow]

2023-10-28 Thread via GitHub
simonprydden commented on issue #35240: URL: https://github.com/apache/airflow/issues/35240#issuecomment-1783892712 #35137 HttpToS3Operator tested and working as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

(airflow) branch main updated: Rename the "system" tests where we run Airflow in K8S (#35238)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new a9a77c2cb3 Rename the "system" tests where we run A

Re: [PR] Rename the "system" tests where we run Airflow in K8S [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35238: URL: https://github.com/apache/airflow/pull/35238 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch dont-get-dag-we-already-have-it created (now 2352f35bb3)

2023-10-28 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch dont-get-dag-we-already-have-it in repository https://gitbox.apache.org/repos/asf/airflow.git at 2352f35bb3 Don't get DAG out of DagBag when we already have it No new revisions were added by

[PR] Don't get DAG out of DagBag when we already have it [airflow]

2023-10-28 Thread via GitHub
ashb opened a new pull request, #35243: URL: https://github.com/apache/airflow/pull/35243 Two things here: 1. By the ponit we are looking at the "callbacks" `dagrun.dag` will already be set, (the `or dagbag.get_dag` is a safety precaution. It might not be required or worth it) 2.

[PR] Add common.io to boring-cyborg.yml [airflow]

2023-10-28 Thread via GitHub
eladkal opened a new pull request, #35242: URL: https://github.com/apache/airflow/pull/35242 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] fix changelog of common-io [airflow]

2023-10-28 Thread via GitHub
eladkal opened a new pull request, #35241: URL: https://github.com/apache/airflow/pull/35241 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [I] OAUTH2/OKTA Authentication not working with Airflow2.7.1 [airflow]

2023-10-28 Thread via GitHub
codeprasan commented on issue #35239: URL: https://github.com/apache/airflow/issues/35239#issuecomment-1783883089 Here is the screenshot of the issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[I] Status of testing Providers that were prepared on October 28, 2023 [airflow]

2023-10-28 Thread via GitHub
eladkal opened a new issue, #35240: URL: https://github.com/apache/airflow/issues/35240 ### Body I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers? The guidelines on h

(airflow-site) branch add-documentation-2023-10-28 created (now 562df6712d)

2023-10-28 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2023-10-28 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 562df6712d Add documentation for packages - 2023-10-28 This branch includes the followin

[I] OAUTH2/OKTA Authentication not working with Airflow2.7.1 [airflow]

2023-10-28 Thread via GitHub
codeprasan opened a new issue, #35239: URL: https://github.com/apache/airflow/issues/35239 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened The OKTA authentication is identifying and verifying the users in OKTA console. Logs are

[PR] Add documentation for packages - 2023-10-28 [airflow-site]

2023-10-28 Thread via GitHub
eladkal opened a new pull request, #885: URL: https://github.com/apache/airflow-site/pull/885 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

svn commit: r64894 - /dev/airflow/providers/

2023-10-28 Thread eladkal
Author: eladkal Date: Sat Oct 28 17:30:17 2023 New Revision: 64894 Log: Add artifacts for Airflow Providers 2023-10-28 Added: dev/airflow/providers/apache-airflow-providers-amazon-8.10.0.tar.gz (with props) dev/airflow/providers/apache-airflow-providers-amazon-8.10.0.tar.gz.asc dev

Re: [PR] Save scheduler execution time by caching dags [airflow]

2023-10-28 Thread via GitHub
ashb commented on PR #30704: URL: https://github.com/apache/airflow/pull/30704#issuecomment-1783877399 @uranusjr @jens-scheffler-bosch @potiuk This PR makes no sense btw -- DagBag.get_dag _already_ has a cache: https://github.com/apache/airflow/blob/d1c58d86de1267d9268a1efe0a0c102633

(airflow) branch main updated: Prepare docs 3rd wave of Providers October 2023 - FIX (#35233)

2023-10-28 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d1c58d86de Prepare docs 3rd wave of Providers Octo

Re: [PR] Prepare docs 3rd wave of Providers October 2023 - FIX [airflow]

2023-10-28 Thread via GitHub
eladkal merged PR #35233: URL: https://github.com/apache/airflow/pull/35233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[PR] Rename the "system" tests where we run Airflow in K8S [airflow]

2023-10-28 Thread via GitHub
potiuk opened a new pull request, #35238: URL: https://github.com/apache/airflow/pull/35238 Not to confuse the tests with Helm tests (where we just process Helm chart and verify if rendered templates look good) the Kubernetes test suite actually create K8S clusters (with Kind), build custom

(airflow) branch main updated: Fix errors in common.io release documentation generation with prevention (#35236)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e637f08188 Fix errors in common.io release document

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35236: URL: https://github.com/apache/airflow/pull/35236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: feat: make `cc` and `bcc` templated fields in EmailOperator (#35235)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7857ca2d20 feat: make `cc` and `bcc` templated fiel

Re: [PR] Make `cc` and `bcc` templated fields in EmailOperator [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35235: URL: https://github.com/apache/airflow/pull/35235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Remove leftover mesos reference in generic hooks (#35232)

2023-10-28 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8ea094f1f5 Remove leftover mesos reference in gener

Re: [PR] Remove leftover mesos reference in generic hooks [airflow]

2023-10-28 Thread via GitHub
potiuk merged PR #35232: URL: https://github.com/apache/airflow/pull/35232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-28 Thread via GitHub
potiuk commented on code in PR #35218: URL: https://github.com/apache/airflow/pull/35218#discussion_r1375270160 ## setup.cfg: ## @@ -79,7 +79,13 @@ install_requires = # Update CustomTTYColoredFormatter to remove colorlog>=4.0.2, <5.0 configupdater>=3.1.1 -conn

[PR] Make ODBCHook result JSON-serializable [airflow]

2023-10-28 Thread via GitHub
Joffreybvn opened a new pull request, #32319: URL: https://github.com/apache/airflow/pull/32319 The ODBCHook returns pyodbc.Row objects when used with SQLExecuteQueryOperator to do SELECT queries, which cause serialization errors in the XCom backend. This PR follows the [discuss

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783840939 Actually seems that linting found quite a few problems across multiple dosc of ours :). Those are minor issues (but worth fixing I think I will disable it tempoarily to allow the

[PR] Update webserver.rst to include information of supported OAuth2 providers [airflow]

2023-10-28 Thread via GitHub
rqi14 opened a new pull request, #35237: URL: https://github.com/apache/airflow/pull/35237 To include information of supported OAuth2 providers Related to issue #35131 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](htt

Re: [PR] Update webserver.rst to include information of supported OAuth2 providers [airflow]

2023-10-28 Thread via GitHub
boring-cyborg[bot] commented on PR #35237: URL: https://github.com/apache/airflow/pull/35237#issuecomment-1783839399 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783830793 Also cc: @bolkedebruin -> re, our discussion - this is why we keep on adding more and more checks like that and it's "difficutl" to add a new provider. But if we do not do that, it

Re: [PR] Fix UI DAG counts including deleted DAGs [airflow]

2023-10-28 Thread via GitHub
Taragolis commented on PR #33778: URL: https://github.com/apache/airflow/pull/33778#issuecomment-1783829634 I think in Airflow 2.7.2 was added some other bug - https://github.com/apache/airflow/pull/34944 - https://github.com/apache/airflow/issues/34955 So this is a good chan

Re: [PR] Add connexion<3.0 upper bound [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35218: URL: https://github.com/apache/airflow/pull/35218#issuecomment-1783829100 Committed the link and rebased to build again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Fix UI DAG counts including deleted DAGs [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #33778: URL: https://github.com/apache/airflow/pull/33778#issuecomment-1783828642 Ah cool. Seems to be already marked for 2.7.3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783827561 Related #35233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk commented on PR #35236: URL: https://github.com/apache/airflow/pull/35236#issuecomment-1783825435 Might also prove useful when we will be merging the 5 new ML providers as it catches a lot more potential "publishing" errors - also improves management of exceptional "min-airflow" ver

[PR] Fix errors in common.io release documentation generation with prevention [airflow]

2023-10-28 Thread via GitHub
potiuk opened a new pull request, #35236: URL: https://github.com/apache/airflow/pull/35236 The common.io provider release caused a few problems when preparing the new wave of the providers: * it missed link to security.rst (which is generated automatically anticipating SBOM generati

Re: [PR] Fix UI DAG counts including deleted DAGs [airflow]

2023-10-28 Thread via GitHub
boushphong commented on PR #33778: URL: https://github.com/apache/airflow/pull/33778#issuecomment-1783815885 Yes. confirmed that 2.7.2 has this issue. The change however was added back in the main branch after 2.7.2 at: https://github.com/apache/airflow/commit/54e7e0824d271900fde35

[PR] Make `cc` and `bcc` templated fields in EmailOperator [airflow]

2023-10-28 Thread via GitHub
Joffreybvn opened a new pull request, #35235: URL: https://github.com/apache/airflow/pull/35235 Hello, This is a small PR to enable templating on `cc` and `bcc` parameters in the EmailOperator. *Use case*: I'm sending emails to some people based on a SQL quer

(airflow-site) branch gh-pages updated (a1c7db26f7 -> 0317796b2b)

2023-10-28 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard a1c7db26f7 Rewritten history to remove past gh-pages deployments new 0317796b2b Rewritten history to

Re: [PR] Fix UI DAG counts including deleted DAGs [airflow]

2023-10-28 Thread via GitHub
boushphong commented on PR #33778: URL: https://github.com/apache/airflow/pull/33778#issuecomment-1783800155 > This change is present in 2.7.2 https://github.com/apache/airflow/blob/2.7.2/airflow/www/views.py#L842C1-L844C24 I don't think this change was in the link you provided.

  1   2   >