[GitHub] [airflow] uranusjr commented on a diff in pull request #34026: AIP-44 Migrate Job to Internal API

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34026: URL: https://github.com/apache/airflow/pull/34026#discussion_r1316823578 ## airflow/jobs/job.py: ## @@ -17,6 +17,7 @@ # under the License. from __future__ import annotations +from datetime import datetime Review Comment: I think w

[GitHub] [airflow] uranusjr commented on a diff in pull request #34026: AIP-44 Migrate Job to Internal API

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34026: URL: https://github.com/apache/airflow/pull/34026#discussion_r1316822714 ## airflow/jobs/base_job_runner.py: ## @@ -29,20 +29,19 @@ J = TypeVar("J", "Job", "JobPydantic", "Job | JobPydantic") - -class BaseJobRunner(Generic[J]): +clas

[GitHub] [airflow] uranusjr commented on pull request #34111: Refactor: Consolidate "import numpy as np"

2023-09-05 Thread via GitHub
uranusjr commented on PR #34111: URL: https://github.com/apache/airflow/pull/34111#issuecomment-1707766582 I don’t really have issues either way, but why do we consolidate to `datetime` instead of `dt` but `np` instead of `numpy`? It seems slightly inconsistent. -- This is an automated m

[airflow] branch main updated: Refactor: Consolidate import datetime (#34110)

2023-09-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7731255975 Refactor: Consolidate import datetime (#

[GitHub] [airflow] potiuk merged pull request #34110: Refactor: Consolidate import datetime

2023-09-05 Thread via GitHub
potiuk merged PR #34110: URL: https://github.com/apache/airflow/pull/34110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] uranusjr commented on a diff in pull request #34108: Refactor: Consolidate import and usage of random

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34108: URL: https://github.com/apache/airflow/pull/34108#discussion_r1316796894 ## tests/models/test_cleartasks.py: ## @@ -580,25 +581,23 @@ def test_dags_clear(self): assert tis[i].max_tries == 1 # test only_failed -

[GitHub] [airflow] uranusjr commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1316791815 ## airflow/auth/managers/base_auth_manager.py: ## @@ -102,3 +154,8 @@ def security_manager(self, security_manager: AirflowSecurityManager): :param security

[GitHub] [airflow] Lee-W commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Lee-W commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316789161 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -404,6 +479,40 @@ def start_cluster(self, json: dict) -> None: """ self._do_api_call(START_

[GitHub] [airflow] uranusjr commented on a diff in pull request #34120: Fix unfound ab_user table in the CLI session

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34120: URL: https://github.com/apache/airflow/pull/34120#discussion_r1316776989 ## airflow/models/dagrun.py: ## @@ -1411,7 +1411,12 @@ class DagRunNote(Base): __tablename__ = "dag_run_note" user_id = Column(Integer, nullable=True) -

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316775212 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -404,6 +479,40 @@ def start_cluster(self, json: dict) -> None: """ self._do_api_call(S

[GitHub] [airflow] uranusjr commented on a diff in pull request #34121: Refactor: Remove trailing whitespace from strings

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34121: URL: https://github.com/apache/airflow/pull/34121#discussion_r1316774976 ## airflow/secrets/local_filesystem.py: ## @@ -238,7 +238,7 @@ def load_variables(file_path: str) -> dict[str, str]: if invalid_keys: raise AirflowExce

[GitHub] [airflow] uranusjr commented on a diff in pull request #34121: Refactor: Remove trailing whitespace from strings

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34121: URL: https://github.com/apache/airflow/pull/34121#discussion_r1316774161 ## airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py: ## @@ -149,7 +149,7 @@ def execute(self, context: Context): ) raise Air

[GitHub] [airflow] uranusjr commented on a diff in pull request #34121: Refactor: Remove trailing whitespace from strings

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34121: URL: https://github.com/apache/airflow/pull/34121#discussion_r1316773471 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -819,7 +819,7 @@ def process_pod_deletion(self, pod: k8s.V1Pod, *, reraise=True):

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316772936 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -388,6 +437,32 @@ def repair_run(self, json: dict) -> None: """ self._do_api_call(REPA

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316772594 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -388,6 +437,32 @@ def repair_run(self, json: dict) -> None: """ self._do_api_call(REPA

[GitHub] [airflow] Lee-W commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
Lee-W commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707637926 Also tested #33424 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [airflow] Lee-W commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
Lee-W commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707636169 Tested https://github.com/apache/airflow/pull/33403, https://github.com/apache/airflow/pull/33401, https://github.com/apache/airflow/pull/33196 -- This is an automated message fro

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #34127: Remove duplicated logs by reusing PodLogsConsumer

2023-09-05 Thread via GitHub
boring-cyborg[bot] commented on PR #34127: URL: https://github.com/apache/airflow/pull/34127#issuecomment-1707628845 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] dkim010 opened a new pull request, #34127: Remove duplicated logs by reusing PodLogsConsumer

2023-09-05 Thread via GitHub
dkim010 opened a new pull request, #34127: URL: https://github.com/apache/airflow/pull/34127 * reuse the kubernetes connection not to re-consume the logs * If a failure occurs while consuming logs through `PodLogsConsumer`, a new `PodLogsConsumer` is created. * But, at this time th

[GitHub] [airflow] uranusjr commented on issue #32311: Pydantic 2.0.0 support for Airflow

2023-09-05 Thread via GitHub
uranusjr commented on issue #32311: URL: https://github.com/apache/airflow/issues/32311#issuecomment-1707600347 I think 2.7 still has some incompatibilities in serialisation, but it should work adequately. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] ldacey commented on issue #32311: Pydantic 2.0.0 support for Airflow

2023-09-05 Thread via GitHub
ldacey commented on issue #32311: URL: https://github.com/apache/airflow/issues/32311#issuecomment-1707571739 Would upgrading to pydantic 2 break core Airflow functionality? I have sometimes updated other packages (pandas, pyarrow) beyond the constraints Airflow sets. The latest release of

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316633410 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -388,6 +437,32 @@ def repair_run(self, json: dict) -> None: """ self._do_api_call(REPA

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316618087 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -114,6 +116,53 @@ def from_json(cls, data: str) -> RunState: return RunState(**json.loads(data)

[GitHub] [airflow] Seokyun-Ha commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
Seokyun-Ha commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316613974 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -114,6 +116,53 @@ def from_json(cls, data: str) -> RunState: return RunState(**json.loads(data)

[GitHub] [airflow] uranusjr commented on a diff in pull request #34118: Refactor: Simplify string generation

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34118: URL: https://github.com/apache/airflow/pull/34118#discussion_r1316560874 ## airflow/providers/microsoft/azure/hooks/wasb.py: ## @@ -173,8 +173,8 @@ def get_conn(self) -> BlobServiceClient: # provided the DNS name without

[GitHub] [airflow] github-actions[bot] closed pull request #31008: Add startup_check_interval_seconds to PodManager's await_pod_start

2023-09-05 Thread via GitHub
github-actions[bot] closed pull request #31008: Add startup_check_interval_seconds to PodManager's await_pod_start URL: https://github.com/apache/airflow/pull/31008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] uranusjr commented on a diff in pull request #34118: Refactor: Simplify string generation

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34118: URL: https://github.com/apache/airflow/pull/34118#discussion_r1316528170 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -166,9 +166,8 @@ def hook(self): def path_to_s3_dataset(path) -> Dataset: from openlineag

[airflow] branch main updated: Fix the required permissions to clear a TI from the UI (#34123)

2023-09-05 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2f5777c082 Fix the required permissions to clear

[GitHub] [airflow] uranusjr closed issue #34092: Can Delete on Task Instances permission is required to clear task

2023-09-05 Thread via GitHub
uranusjr closed issue #34092: Can Delete on Task Instances permission is required to clear task URL: https://github.com/apache/airflow/issues/34092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [airflow] uranusjr merged pull request #34123: Fix the required permissions to clear a TI from the UI

2023-09-05 Thread via GitHub
uranusjr merged PR #34123: URL: https://github.com/apache/airflow/pull/34123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] uranusjr commented on a diff in pull request #34112: Switch to internal JSON SA Field Types

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #34112: URL: https://github.com/apache/airflow/pull/34112#discussion_r1316516583 ## airflow/utils/sqlalchemy.py: ## @@ -105,21 +105,59 @@ def load_dialect_impl(self, dialect): return super().load_dialect_impl(dialect) -class Extended

[GitHub] [airflow] uranusjr commented on a diff in pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
uranusjr commented on code in PR #33901: URL: https://github.com/apache/airflow/pull/33901#discussion_r1316512464 ## airflow/auth/managers/base_auth_manager.py: ## @@ -36,8 +38,9 @@ class BaseAuthManager(LoggingMixin): Auth managers are responsible for any user management r

[GitHub] [airflow] syun64 opened a new pull request, #34126: New 'cleared' boolean attribute for dag_run

2023-09-05 Thread via GitHub
syun64 opened a new pull request, #34126: URL: https://github.com/apache/airflow/pull/34126 related: https://github.com/apache/airflow/discussions/33677 Currently, Airflow supports clearing Dag runs in one of three ways: REST API endpoint [clear](https://airflow

[GitHub] [airflow] patgarz commented on a diff in pull request #34051: AIP-56: Change FAB username identifier

2023-09-05 Thread via GitHub
patgarz commented on code in PR #34051: URL: https://github.com/apache/airflow/pull/34051#discussion_r1316503648 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -61,17 +61,27 @@ def get_cli_commands() -> list[CLICommand]: SYNC_PERM_COMMAND, # not in a comma

[airflow] branch constraints-main updated: Updating constraints. Github run id:6089932435

2023-09-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new d15abbeaad Updating con

[GitHub] [airflow] patgarz commented on a diff in pull request #34051: AIP-56: Change FAB username identifier

2023-09-05 Thread via GitHub
patgarz commented on code in PR #34051: URL: https://github.com/apache/airflow/pull/34051#discussion_r1316499155 ## airflow/models/log.py: ## @@ -37,6 +37,7 @@ class Log(Base): event = Column(String(30)) execution_date = Column(UtcDateTime) owner = Column(String(5

[GitHub] [airflow] patgarz commented on a diff in pull request #34051: AIP-56: Change FAB username identifier

2023-09-05 Thread via GitHub
patgarz commented on code in PR #34051: URL: https://github.com/apache/airflow/pull/34051#discussion_r1316493843 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -61,17 +61,27 @@ def get_cli_commands() -> list[CLICommand]: SYNC_PERM_COMMAND, # not in a comma

[GitHub] [airflow] hussein-awala commented on pull request #34110: Refactor: Consolidate import datetime

2023-09-05 Thread via GitHub
hussein-awala commented on PR #34110: URL: https://github.com/apache/airflow/pull/34110#issuecomment-1707338998 merge main to fix static checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] hussein-awala commented on a diff in pull request #34110: Refactor: Consolidate import datetime

2023-09-05 Thread via GitHub
hussein-awala commented on code in PR #34110: URL: https://github.com/apache/airflow/pull/34110#discussion_r1316431193 ## airflow/providers/cncf/kubernetes/pod_launcher_deprecated.py: ## Review Comment: This module is deprecated. @eladkal is it ok to update it? -- Th

[GitHub] [airflow] potiuk commented on pull request #34123: Fix the required permissions to clear a TI from the UI

2023-09-05 Thread via GitHub
potiuk commented on PR #34123: URL: https://github.com/apache/airflow/pull/34123#issuecomment-1707332249 > I agree, bug. The [API also requires edit](https://github.com/apache/airflow/blob/1bd321d07885d659ce91d7c3ab08764362da0093/airflow/api_connexion/endpoints/task_instance_endpoint.py#L454

[airflow] branch main updated: Fix RAT check for added DOAP file (#34124)

2023-09-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ba524ce4b2 Fix RAT check for added DOAP file (#3412

[GitHub] [airflow] potiuk merged pull request #34124: Fix RAT check for added DOAP file

2023-09-05 Thread via GitHub
potiuk merged PR #34124: URL: https://github.com/apache/airflow/pull/34124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] Taragolis commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316410423 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +58,18 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +VALID_ES_CONFIG_KEYS

[GitHub] [airflow] potiuk opened a new pull request, #34124: Fix RAT check for added DOAP file

2023-09-05 Thread via GitHub
potiuk opened a new pull request, #34124: URL: https://github.com/apache/airflow/pull/34124 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)*

[airflow] 01/01: Fix RAT check for added DOAP file

2023-09-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch fix-rat-check-licence in repository https://gitbox.apache.org/repos/asf/airflow.git commit a07ea4221d7a522e40bdc8f821f443d05a886bd9 Author: Jarek Potiuk AuthorDate: Tue Sep 5 23:04:40 2023 +0200

[airflow] branch fix-rat-check-licence created (now a07ea4221d)

2023-09-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch fix-rat-check-licence in repository https://gitbox.apache.org/repos/asf/airflow.git at a07ea4221d Fix RAT check for added DOAP file This branch includes the following new commits: ne

[GitHub] [airflow] Taragolis commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316405846 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +58,18 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +VALID_ES_CONFIG_KEYS

[GitHub] [airflow] Taragolis commented on pull request #34116: Purge `[elasticsearch_configs] use_ssl` config param in `ElasticsearchTaskHandler`

2023-09-05 Thread via GitHub
Taragolis commented on PR #34116: URL: https://github.com/apache/airflow/pull/34116#issuecomment-1707307943 closed in favour of https://github.com/apache/airflow/pull/34119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [airflow] Taragolis closed pull request #34116: Purge `[elasticsearch_configs] use_ssl` config param in `ElasticsearchTaskHandler`

2023-09-05 Thread via GitHub
Taragolis closed pull request #34116: Purge `[elasticsearch_configs] use_ssl` config param in `ElasticsearchTaskHandler` URL: https://github.com/apache/airflow/pull/34116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [airflow] Taragolis commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316403711 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +58,18 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +VALID_ES_CONFIG_KEYS

[GitHub] [airflow] Taragolis commented on a diff in pull request #34122: openlineage: fix import in `get_custom_facets`

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34122: URL: https://github.com/apache/airflow/pull/34122#discussion_r1316396564 ## airflow/providers/openlineage/plugins/facets.py: ## @@ -33,7 +33,7 @@ class AirflowMappedTaskRunFacet(BaseFacet): @classmethod def from_task_instance(c

[GitHub] [airflow] potiuk commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
potiuk commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316395950 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +57,23 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +# These are the only ke

[airflow] branch main updated: Added more example code for XCom push and pull (#34016)

2023-09-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1bd321d078 Added more example code for XCom push

[GitHub] [airflow] Taragolis merged pull request #34016: Added more example code for XCom push and pull

2023-09-05 Thread via GitHub
Taragolis merged PR #34016: URL: https://github.com/apache/airflow/pull/34016 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] JDarDagran commented on a diff in pull request #34122: openlineage: fix import in `get_custom_facets`

2023-09-05 Thread via GitHub
JDarDagran commented on code in PR #34122: URL: https://github.com/apache/airflow/pull/34122#discussion_r1316377686 ## airflow/providers/openlineage/plugins/facets.py: ## @@ -33,7 +33,7 @@ class AirflowMappedTaskRunFacet(BaseFacet): @classmethod def from_task_instance(

[GitHub] [airflow] hussein-awala opened a new pull request, #34123: Fix the required permissions to clear a TI from the UI

2023-09-05 Thread via GitHub
hussein-awala opened a new pull request, #34123: URL: https://github.com/apache/airflow/pull/34123 closes: #34092 To clear a TI, we don't delete the row, but we just update some fields like the state, job_id, task_retries, ... so technically we edit it. Currently th

[GitHub] [airflow] Taragolis commented on a diff in pull request #34122: openlineage: fix import in `get_custom_facets`

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34122: URL: https://github.com/apache/airflow/pull/34122#discussion_r1316367583 ## airflow/providers/openlineage/plugins/facets.py: ## @@ -33,7 +33,7 @@ class AirflowMappedTaskRunFacet(BaseFacet): @classmethod def from_task_instance(c

[GitHub] [airflow] Taragolis commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316364199 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +57,23 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +# These are the only

[GitHub] [airflow] JDarDagran opened a new pull request, #34122: openlineage: fix import in `get_custom_facets`

2023-09-05 Thread via GitHub
JDarDagran opened a new pull request, #34122: URL: https://github.com/apache/airflow/pull/34122 This PR fixes broken import in `get_custom_facets` and adds test for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [airflow] Taragolis commented on a diff in pull request #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
Taragolis commented on code in PR #34119: URL: https://github.com/apache/airflow/pull/34119#discussion_r1316345822 ## airflow/providers/elasticsearch/log/es_task_handler.py: ## @@ -56,6 +57,23 @@ USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template") +# These are the only

[GitHub] [airflow] eumiro opened a new pull request, #34121: Refactor: Remove trailing whitespace from strings

2023-09-05 Thread via GitHub
eumiro opened a new pull request, #34121: URL: https://github.com/apache/airflow/pull/34121 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [airflow] darkag commented on a diff in pull request #34041: Custom fetch all handler for vertica to not miss errors

2023-09-05 Thread via GitHub
darkag commented on code in PR #34041: URL: https://github.com/apache/airflow/pull/34041#discussion_r1316341182 ## airflow/providers/vertica/hooks/vertica.py: ## @@ -17,9 +17,24 @@ # under the License. from __future__ import annotations +from typing import Any, Callable, Ite

[GitHub] [airflow] hussein-awala opened a new pull request, #34120: Fix unfound ab_user table in the CLI session

2023-09-05 Thread via GitHub
hussein-awala opened a new pull request, #34120: URL: https://github.com/apache/airflow/pull/34120 closes: #34109 For some reason, the sqlalchemy sessions created in the CLI do not find the table `ab_user`, I tested different solutions such as providing the session to the

[GitHub] [airflow] potiuk opened a new pull request, #34119: Make sure that only valid elasticsearch keys are passed to handler

2023-09-05 Thread via GitHub
potiuk opened a new pull request, #34119: URL: https://github.com/apache/airflow/pull/34119 The elasticsearch handler got all configuraiton parameters from the "elasticsearch_config" section but it means that in airflow versions pre 2.7 it could get old config keys which renders the new pro

[GitHub] [airflow] darkag commented on pull request #34041: Custom fetch all handler for vertica to not miss errors

2023-09-05 Thread via GitHub
darkag commented on PR #34041: URL: https://github.com/apache/airflow/pull/34041#issuecomment-1707224938 Added docstring (hope this is understandable enough) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [airflow] ashokballolli commented on issue #34117: BigQueryInsertJobOperator doesn't log the query its executed

2023-09-05 Thread via GitHub
ashokballolli commented on issue #34117: URL: https://github.com/apache/airflow/issues/34117#issuecomment-1707214049 Its not a blocker as I am able to see the query executed as part of the `Rendered Template` tab in the **Airflow UI**. But it would be great to log it so that if someone is o

[GitHub] [airflow] ldacey closed issue #34022: Unable to read or write logs on Azure Blob after 2.7.0 update

2023-09-05 Thread via GitHub
ldacey closed issue #34022: Unable to read or write logs on Azure Blob after 2.7.0 update URL: https://github.com/apache/airflow/issues/34022 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [airflow] ldacey commented on issue #34022: Unable to read or write logs on Azure Blob after 2.7.0 update

2023-09-05 Thread via GitHub
ldacey commented on issue #34022: URL: https://github.com/apache/airflow/issues/34022#issuecomment-1707209690 Hm, I have been using the same conn_id since 2018 for Azure Blob (HTTP conn type): ```yaml host: https://account_name.blob.core.windows.net/etl01?restype=container&comp=list

[GitHub] [airflow] ashokballolli opened a new issue, #34117: BigQueryInsertJobOperator doesn't log the query its executed

2023-09-05 Thread via GitHub
ashokballolli opened a new issue, #34117: URL: https://github.com/apache/airflow/issues/34117 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Version: 2.2.3+composer The BigQueryInsertJobOperator from airflow.providers.g

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34117: BigQueryInsertJobOperator doesn't log the query its executed

2023-09-05 Thread via GitHub
boring-cyborg[bot] commented on issue #34117: URL: https://github.com/apache/airflow/issues/34117#issuecomment-1707203429 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] Taragolis opened a new pull request, #34116: Purge `[elasticsearch_configs] use_ssl` config param in `ElasticsearchTaskHandler`

2023-09-05 Thread via GitHub
Taragolis opened a new pull request, #34116: URL: https://github.com/apache/airflow/pull/34116 closes: #34099 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-req

[GitHub] [airflow] vincbeck commented on a diff in pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
vincbeck commented on code in PR #33901: URL: https://github.com/apache/airflow/pull/33901#discussion_r1316316171 ## airflow/auth/managers/base_auth_manager.py: ## @@ -24,9 +24,11 @@ from airflow.utils.log.logging_mixin import LoggingMixin if TYPE_CHECKING: +from flask i

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
vandonr-amz commented on code in PR #33901: URL: https://github.com/apache/airflow/pull/33901#discussion_r1316314556 ## airflow/auth/managers/base_auth_manager.py: ## @@ -24,9 +24,11 @@ from airflow.utils.log.logging_mixin import LoggingMixin if TYPE_CHECKING: +from flas

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
vandonr-amz commented on code in PR #33901: URL: https://github.com/apache/airflow/pull/33901#discussion_r1316311143 ## airflow/auth/managers/base_auth_manager.py: ## @@ -36,8 +38,9 @@ class BaseAuthManager(LoggingMixin): Auth managers are responsible for any user managemen

[GitHub] [airflow] vandonr-amz commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
vandonr-amz commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707192432 good for me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [airflow] SamWheating opened a new pull request, #34114: (WIP) Allow users to set empty access control on DAGs

2023-09-05 Thread via GitHub
SamWheating opened a new pull request, #34114: URL: https://github.com/apache/airflow/pull/34114 Follow-up to https://github.com/apache/airflow/pull/33632, which only tested the relevant code in isolation. While verifying this behaviour in order to test the noticed two things -

[GitHub] [airflow] eumiro opened a new pull request, #34113: Refactor: Consolidate 'import textwrap'

2023-09-05 Thread via GitHub
eumiro opened a new pull request, #34113: URL: https://github.com/apache/airflow/pull/34113 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [airflow] hussein-awala commented on pull request #34016: Added more example code for XCom push and pull

2023-09-05 Thread via GitHub
hussein-awala commented on PR #34016: URL: https://github.com/apache/airflow/pull/34016#issuecomment-1707143247 @Taragolis could you take a second look before merging? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [airflow] hussein-awala commented on a diff in pull request #34071: Databricks ClusteState & Clsuter get, activate

2023-09-05 Thread via GitHub
hussein-awala commented on code in PR #34071: URL: https://github.com/apache/airflow/pull/34071#discussion_r1316248117 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -114,6 +116,53 @@ def from_json(cls, data: str) -> RunState: return RunState(**json.loads(da

[GitHub] [airflow] Taragolis opened a new pull request, #34112: Switch to internal JSON SA Field Types

2023-09-05 Thread via GitHub
Taragolis opened a new pull request, #34112: URL: https://github.com/apache/airflow/pull/34112 Main motivation to get rid of `sqlalchemy_jsonfield.JSONField` it is the fact, that this third party custom type doesn't define `cache_ok` as result if we try to access to single f

[GitHub] [airflow] eumiro opened a new pull request, #34111: Refactor: Consolidate "import numpy as np"

2023-09-05 Thread via GitHub
eumiro opened a new pull request, #34111: URL: https://github.com/apache/airflow/pull/34111 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [airflow] eumiro opened a new pull request, #34110: Refactor: Consolidate import datetime

2023-09-05 Thread via GitHub
eumiro opened a new pull request, #34110: URL: https://github.com/apache/airflow/pull/34110 There are several files with `from airflow.utils.timezone import datetime` that at the same time `import datetime as dt`, making the two variables confusing. How about changing that `airflow.utils.ti

[GitHub] [airflow] SamWheating commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
SamWheating commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707074982 I have pulled the 2.7.1rc2 commit and verified the following fixes in Breeze: https://github.com/apache/airflow/pull/33622 https://github.com/apache/airflow/pull/33931

[GitHub] [airflow] alexsanderp commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
alexsanderp commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707062654 The `airflow db check` command seems to be broken in this version: ``` ERROR! Maximum number of retries (20) reached. Last check result: $

[GitHub] [airflow] eumiro commented on a diff in pull request #33837: Refactor unneeded 'continue' jumps in models

2023-09-05 Thread via GitHub
eumiro commented on code in PR #33837: URL: https://github.com/apache/airflow/pull/33837#discussion_r1316222356 ## airflow/models/mappedoperator.py: ## @@ -97,15 +97,11 @@ def validate_mapping_kwargs(op: type[BaseOperator], func: ValidationSource, valu continue

[GitHub] [airflow] boring-cyborg[bot] commented on issue #34109: sqlalchemy error when running CLI command `airflow tasks test`

2023-09-05 Thread via GitHub
boring-cyborg[bot] commented on issue #34109: URL: https://github.com/apache/airflow/issues/34109#issuecomment-1707061891 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[GitHub] [airflow] mbarugelCA opened a new issue, #34109: sqlalchemy error when running CLI command `airflow tasks test`

2023-09-05 Thread via GitHub
mbarugelCA opened a new issue, #34109: URL: https://github.com/apache/airflow/issues/34109 ### Apache Airflow version 2.7.0 ### What happened Calling `airflow tasks test ` runs the task correctly, but yields the following error after it's done (regardless of failure/suc

[GitHub] [airflow] phanikumv commented on pull request #33720: Optimise and migrate to SA2-compatible syntax for TaskReschedule

2023-09-05 Thread via GitHub
phanikumv commented on PR #33720: URL: https://github.com/apache/airflow/pull/33720#issuecomment-1707053777 We need to run full tests on the changes related to SqlAlchemy 2.0 , hence applied the label -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] khmelevskiy commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
khmelevskiy commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1707051901 i checked the version of airflow:2.7.1rc2-python3.10. run dag (test) -> let it work -> rename dag (add _v2, eg = test_v2) result - bug, there are 2 dagas test an

[GitHub] [airflow] potiuk commented on issue #34099: Elasticsearch.__init__() got an unexpected keyword argument 'use_ssl'

2023-09-05 Thread via GitHub
potiuk commented on issue #34099: URL: https://github.com/apache/airflow/issues/34099#issuecomment-1707044956 I am quite positive that this should be fixed in 2.7.1 via https://github.com/apache/airflow/pull/33135 There was a bit of problem with 2.7.0 that elasticsearch configuration

[GitHub] [airflow] potiuk closed issue #34099: Elasticsearch.__init__() got an unexpected keyword argument 'use_ssl'

2023-09-05 Thread via GitHub
potiuk closed issue #34099: Elasticsearch.__init__() got an unexpected keyword argument 'use_ssl' URL: https://github.com/apache/airflow/issues/34099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on issue #34093: The allowed_deserialization_classes regex is broken

2023-09-05 Thread via GitHub
potiuk commented on issue #34093: URL: https://github.com/apache/airflow/issues/34093#issuecomment-1707020916 Absolutely - feel free to open a new PR (even now before getting feedback from the others). We can always iterate over the PR and get to the final solution there. And we hav

[GitHub] [airflow] eumiro opened a new pull request, #34108: Refactor: Consolidate import and usage of random

2023-09-05 Thread via GitHub
eumiro opened a new pull request, #34108: URL: https://github.com/apache/airflow/pull/34108 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [airflow] vincbeck commented on pull request #33901: Fix inheritance chain in security manager

2023-09-05 Thread via GitHub
vincbeck commented on PR #33901: URL: https://github.com/apache/airflow/pull/33901#issuecomment-1707005235 We should double check it resolves: #34107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] vincbeck opened a new issue, #34107: OpenShift OAuth not working when upgrading from 2.6.3 -> 2.7.0

2023-09-05 Thread via GitHub
vincbeck opened a new issue, #34107: URL: https://github.com/apache/airflow/issues/34107 ### Discussed in https://github.com/apache/airflow/discussions/33913 Originally posted by **allekai** August 30, 2023 Hi everyone, we have Airflow deployed via the official helm c

[GitHub] [airflow] jedcunningham commented on a diff in pull request #33620: Deferrable Operators Docs Edits

2023-09-05 Thread via GitHub
jedcunningham commented on code in PR #33620: URL: https://github.com/apache/airflow/pull/33620#discussion_r1316139157 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -18,45 +18,48 @@ Deferrable Operators & Triggers === -Stand

[GitHub] [airflow] argibbs commented on issue #33410: SlaMiss Records Never Created for Packaged DAGs

2023-09-05 Thread via GitHub
argibbs commented on issue #33410: URL: https://github.com/apache/airflow/issues/33410#issuecomment-1706964333 Ok, Have managed to get to a real computer with a keyboard and everything and have a proper look. First up, my apologies for breaking something, that was obviously not

[GitHub] [airflow] ohaibbq commented on issue #34065: Status of testing of Apache Airflow 2.7.1rc2

2023-09-05 Thread via GitHub
ohaibbq commented on issue #34065: URL: https://github.com/apache/airflow/issues/34065#issuecomment-1706958990 Confirmed #32731 is fixed in RC1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch main updated (a3da008250 -> 5e6e107f73)

2023-09-05 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from a3da008250 Improve documentation for AzureDataExplorerHook DefaultAzureCredential support (#34095) add 5e6

[GitHub] [airflow] jedcunningham merged pull request #34106: Use `cached_property` for hook in SparkKubernetesSensor

2023-09-05 Thread via GitHub
jedcunningham merged PR #34106: URL: https://github.com/apache/airflow/pull/34106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

  1   2   >