Re: [PR] Secret masker ignores passwords with special chars [airflow]

2024-01-10 Thread via GitHub
aritra24 commented on PR #36692: URL: https://github.com/apache/airflow/pull/36692#issuecomment-1884372024 @hussein-awala I see the test failing on main ``` === short test summary info FAILED tests/always/test_connection.py::Test

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36705: URL: https://github.com/apache/airflow/pull/36705#discussion_r1447022943 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,25 @@ def variables_get(args): @providers_configuration_loaded def variables_set(args): """Create n

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36705: URL: https://github.com/apache/airflow/pull/36705#discussion_r1447022943 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,25 @@ def variables_get(args): @providers_configuration_loaded def variables_set(args): """Create n

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36705: URL: https://github.com/apache/airflow/pull/36705#discussion_r1447022943 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,25 @@ def variables_get(args): @providers_configuration_loaded def variables_set(args): """Create n

Re: [PR] Support encryption for triggers parameters [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36492: URL: https://github.com/apache/airflow/pull/36492#issuecomment-1884389895 > Hi, just saw this. Just curious, why not just always encrypt, instead of forcing each implementer to mess with the kwargs naming? My take: encrupting all of it makes trigger data

Re: [PR] Secret masker ignores passwords with special chars [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36692: URL: https://github.com/apache/airflow/pull/36692#discussion_r1447045190 ## airflow/models/connection.py: ## @@ -140,6 +140,7 @@ def __init__( if self.password: mask_secret(self.password) +mask_secret(quo

Re: [PR] Secret masker ignores passwords with special chars [airflow]

2024-01-10 Thread via GitHub
aritra24 commented on code in PR #36692: URL: https://github.com/apache/airflow/pull/36692#discussion_r1447056664 ## airflow/models/connection.py: ## @@ -140,6 +140,7 @@ def __init__( if self.password: mask_secret(self.password) +mask_secret(q

Re: [PR] Add SparkKubernetesOperator crd implementation [airflow]

2024-01-10 Thread via GitHub
bolkedebruin commented on PR #22253: URL: https://github.com/apache/airflow/pull/22253#issuecomment-1884432760 Almost there @hamedhsn ! ``` airflow/providers/cncf/kubernetes/operators/custom_object_launcher.py:106: error: Incompatible default for argument "driver" (default has t

Re: [PR] [WIP] Update test_file_not_new_enough in test_sftp.py with logical_date example [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36681: URL: https://github.com/apache/airflow/pull/36681#discussion_r1447080614 ## tests/providers/sftp/sensors/test_sftp.py: ## @@ -86,16 +86,15 @@ def test_file_new_enough(self, sftp_hook_mock): @patch("airflow.providers.sftp.sensors.sftp

Re: [PR] [WIP] Update test_file_not_new_enough in test_sftp.py with logical_date example [airflow]

2024-01-10 Thread via GitHub
dirrao commented on code in PR #36681: URL: https://github.com/apache/airflow/pull/36681#discussion_r1447080614 ## tests/providers/sftp/sensors/test_sftp.py: ## @@ -86,16 +86,15 @@ def test_file_new_enough(self, sftp_hook_mock): @patch("airflow.providers.sftp.sensors.sftp

Re: [I] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
aritra24 commented on issue #36694: URL: https://github.com/apache/airflow/issues/36694#issuecomment-1884498099 Is this not achieved with [this](https://github.com/apache/airflow/blob/9cb2052810a8a4b191e77d804fc79927f046c8bb/airflow/cli/commands/variable_command.py#L79)? -- This is an aut

[PR] Fix ODBC Hook sql select return empty table [airflow]

2024-01-10 Thread via GitHub
roel-w opened a new pull request, #36709: URL: https://github.com/apache/airflow/pull/36709 When querying using a select statement with the odbc hook and the return is an empty table (sqlalchemy []). ```python File "/usr/local/lib/python3.9/site-packages/airflow/prov

Re: [PR] Fix ODBC Hook sql select return empty table [airflow]

2024-01-10 Thread via GitHub
boring-cyborg[bot] commented on PR #36709: URL: https://github.com/apache/airflow/pull/36709#issuecomment-1884508050 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] Displaying "actual" try number in TaskInstance view [airflow]

2024-01-10 Thread via GitHub
eladkal commented on PR #34635: URL: https://github.com/apache/airflow/pull/34635#issuecomment-1884560300 @bbovenzi can we merge it then? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Add log lookup exception for empty op subtypes [airflow]

2024-01-10 Thread via GitHub
eladkal commented on PR #35536: URL: https://github.com/apache/airflow/pull/35536#issuecomment-1884562897 @vchiapaikeo are you still working on this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fix ODBC Hook sql select return empty table [airflow]

2024-01-10 Thread via GitHub
Lee-W commented on code in PR #36709: URL: https://github.com/apache/airflow/pull/36709#discussion_r1447188723 ## tests/providers/odbc/hooks/test_odbc.py: ## @@ -329,6 +329,25 @@ def mock_handler(*_): result = hook.run("SQL", handler=mock_handler) assert ho

Re: [PR] Fix ODBC Hook sql select return empty table [airflow]

2024-01-10 Thread via GitHub
Lee-W commented on code in PR #36709: URL: https://github.com/apache/airflow/pull/36709#discussion_r1447188723 ## tests/providers/odbc/hooks/test_odbc.py: ## @@ -329,6 +329,25 @@ def mock_handler(*_): result = hook.run("SQL", handler=mock_handler) assert ho

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
richa-sawant commented on code in PR #36705: URL: https://github.com/apache/airflow/pull/36705#discussion_r1447199915 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,25 @@ def variables_get(args): @providers_configuration_loaded def variables_set(args): """Cr

Re: [PR] Provide the logger_name param in providers hooks in order to override the logger name [airflow]

2024-01-10 Thread via GitHub
eladkal commented on PR #36675: URL: https://github.com/apache/airflow/pull/36675#issuecomment-1884614830 > @eladkal is it ok to merge this PR, or do we need to create a PR per provider? No need. 1 PR is fine :) -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Provide the logger_name param in providers hooks in order to override the logger name [airflow]

2024-01-10 Thread via GitHub
eladkal merged PR #36675: URL: https://github.com/apache/airflow/pull/36675 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow) branch main updated (8fb55f2127 -> 6bd450da1e)

2024-01-10 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8fb55f2127 Support encryption for triggers parameters (#36492) add 6bd450da1e Provide the logger_name param in pr

Re: [PR] Support encryption for triggers parameters [airflow]

2024-01-10 Thread via GitHub
jscheffl commented on PR #36492: URL: https://github.com/apache/airflow/pull/36492#issuecomment-1884625637 > > Hi, just saw this. Just curious, why not just always encrypt, instead of forcing each implementer to mess with the kwargs naming? > > My take: encrypting all of it makes trig

Re: [PR] Support encryption for triggers parameters [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36492: URL: https://github.com/apache/airflow/pull/36492#issuecomment-1884639602 > Consindering that the Triggering information is very volatile (I am not sure but was expecting after a task is completed it is clened-up and does not pile-up?) it would be really reaso

Re: [PR] Support encryption for triggers parameters [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36492: URL: https://github.com/apache/airflow/pull/36492#issuecomment-1884640857 https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-json/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [I] apache-airflow-providers-cncf-kubernetes requires latest kubernetes version [airflow]

2024-01-10 Thread via GitHub
gbonazzoli commented on issue #36678: URL: https://github.com/apache/airflow/issues/36678#issuecomment-1884644808 @paramjeet01 Well said !!! I also add that working with apache-airflow-providers-cncf-kubernetes on a cluster kubernetes 1.28.x it seems to have logging capture problems d

Re: [I] Support writing custom DAG tags in metrics [airflow]

2024-01-10 Thread via GitHub
dirrao commented on issue #36704: URL: https://github.com/apache/airflow/issues/36704#issuecomment-1884728953 @cbuffett Yes, this is a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(airflow) branch switch-to-hatch updated (1f8a5c4a6b -> 8633e533b1)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 1f8a5c4a6b Standardize airflow build process and switch to Hatchling build backend add 98b9e491e5 Upda

[PR] Introduce DatasetTimetable [airflow]

2024-01-10 Thread via GitHub
uranusjr opened a new pull request, #36710: URL: https://github.com/apache/airflow/pull/36710 This special timetable allows a DAG to be run against a time-based schedule and dataset events at the same time. The logic is nothing special---scheduled runs are created based on a time-based time

Re: [I] Support writing custom DAG tags in metrics [airflow]

2024-01-10 Thread via GitHub
dirrao commented on issue #36704: URL: https://github.com/apache/airflow/issues/36704#issuecomment-1884750795 @cbuffett It is a big feature request and it requires a discussion. So, converting it to a discussion. -- This is an automated message from the Apache Git Service. To respond

Re: [I] Support writing custom DAG tags in metrics [airflow]

2024-01-10 Thread via GitHub
dirrao closed issue #36704: Support writing custom DAG tags in metrics URL: https://github.com/apache/airflow/issues/36704 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] Airflow Scheduler with Kubernetes Executor has errors in logs and stuck slots with no running tasks [airflow]

2024-01-10 Thread via GitHub
dirrao commented on issue #36478: URL: https://github.com/apache/airflow/issues/36478#issuecomment-1884755897 > @dirrao Yes, thank you! > > I rollback our cluster to 1 Scheduler and 64 parallelism to have the same capacity :) About restart - I found that it could be made in the Airflo

Re: [PR] check job_status before BatchOperator execute in deferrable mode [airflow]

2024-01-10 Thread via GitHub
phanikumv commented on PR #36523: URL: https://github.com/apache/airflow/pull/36523#issuecomment-1884774292 > I also think this is a bit redundant. I like the time savings, but it complicates the code a bit to have result checking/hanlding in multiple places. I think if we move forward with

Re: [PR] check job_status before BatchOperator execute in deferrable mode [airflow]

2024-01-10 Thread via GitHub
phanikumv merged PR #36523: URL: https://github.com/apache/airflow/pull/36523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: check job_status before BatchOperator execute in deferrable mode (#36523)

2024-01-10 Thread phanikumv
This is an automated email from the ASF dual-hosted git repository. phanikumv pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 88c9596f4a check job_status before BatchOperator

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1884781241 cc: @jscheffl and others - @ephraimbuddy but also @uranusjr - we've been discussing about security of the build scripts/plugins/extensions. Also related to the https://medium.com/apach

(airflow) branch switch-to-hatch updated (8633e533b1 -> c63a6772b7)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 8633e533b1 Standardize airflow build process and switch to Hatchling build backend new c63a6772b7 Stan

Re: [I] apache-airflow-providers-cncf-kubernetes requires latest kubernetes version [airflow]

2024-01-10 Thread via GitHub
raphaelauv commented on issue #36678: URL: https://github.com/apache/airflow/issues/36678#issuecomment-1884787127 @gbonazzoli ``` Otherall, the XComs examples in python do not work at all ``` it works on my side , could you please open a new issue with a reproducible example , t

Re: [PR] Add log lookup exception for empty op subtypes [airflow]

2024-01-10 Thread via GitHub
vchiapaikeo commented on PR #35536: URL: https://github.com/apache/airflow/pull/35536#issuecomment-1884789286 Whoops, sorry about that. Rebased and ready for review here. I retested and it still works as expected - https://github.com/apache/airflow/assets/9200263/fff927fe-b51f-4dc3-a9

Re: [PR] add use_regex argument for allowing S3KeySensor to check s3 keys with regular expression [airflow]

2024-01-10 Thread via GitHub
phanikumv commented on code in PR #36578: URL: https://github.com/apache/airflow/pull/36578#discussion_r1447344954 ## tests/system/providers/amazon/aws/example_s3.py: ## @@ -200,7 +202,18 @@ def check_fn(files: list) -> bool: check_fn=check_fn, deferrable=True,

Re: [PR] check ProcessingJobStatus status before deferring SageMakerProcessingOperator [airflow]

2024-01-10 Thread via GitHub
phanikumv merged PR #36658: URL: https://github.com/apache/airflow/pull/36658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: check sagemaker processing job status before deferring (#36658)

2024-01-10 Thread phanikumv
This is an automated email from the ASF dual-hosted git repository. phanikumv pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5a6f959bd5 check sagemaker processing job status

Re: [PR] Implement deferrable mode for CreateHyperparameterTuningJobOperator [airflow]

2024-01-10 Thread via GitHub
VladaZakharova commented on PR #36594: URL: https://github.com/apache/airflow/pull/36594#issuecomment-1884854770 Hi @potiuk @eladkal @Lee-W ! Could you please check changes in this PR? Thank you! -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] Add data_interval_start and data_interval_end in dagrun create API endpoint [airflow]

2024-01-10 Thread via GitHub
karakanb commented on code in PR #36630: URL: https://github.com/apache/airflow/pull/36630#discussion_r1447391125 ## airflow/api_connexion/endpoints/dag_run_endpoint.py: ## @@ -319,11 +320,34 @@ def post_dag_run(*, dag_id: str, session: Session = NEW_SESSION) -> APIResponse:

[PR] Remove dot value [airflow]

2024-01-10 Thread via GitHub
fuatcakici opened a new pull request, #36712: URL: https://github.com/apache/airflow/pull/36712 The update_state function in the DagRun class fails as ever since an update was made to Airflow, the state of the DagRun is returned as a string instead of an actual DagRunState. However, should

Re: [PR] Remove dot value [airflow]

2024-01-10 Thread via GitHub
boring-cyborg[bot] commented on PR #36712: URL: https://github.com/apache/airflow/pull/36712#issuecomment-1884884746 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] Fix Callback exception when a removed task is the last one in the task instance list [airflow]

2024-01-10 Thread via GitHub
joaopamaral commented on code in PR #36693: URL: https://github.com/apache/airflow/pull/36693#discussion_r1447442412 ## airflow/models/dag.py: ## @@ -1466,6 +1466,9 @@ def fetch_callback( # context for the callback. if dag.partial: tis

Re: [PR] Add support of Pendulum 3 [airflow]

2024-01-10 Thread via GitHub
Taragolis commented on PR #36281: URL: https://github.com/apache/airflow/pull/36281#issuecomment-1884967587 Just wondering shall we merge it or better to wait for someone else review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] Does not support this configuration : redis sentinel [airflow]

2024-01-10 Thread via GitHub
TicyYang commented on issue #35619: URL: https://github.com/apache/airflow/issues/35619#issuecomment-1884983750 Hi, I have also tried many times and found that the following configuration approach connects successfully: ``` [celery] broker_url = sentinel://:26379/0;sentinel://:

Re: [PR] Attempt to update Connexion library to version 3 [airflow]

2024-01-10 Thread via GitHub
MaksYermak commented on PR #36052: URL: https://github.com/apache/airflow/pull/36052#issuecomment-1884982252 Hello Team, I see several issues which we need to fix for finishing this PR. 1. Fix unit tests which use `app`. After connexion update `create_app` function returns `conn

Re: [PR] Add log lookup exception for empty op subtypes [airflow]

2024-01-10 Thread via GitHub
eladkal commented on code in PR #35536: URL: https://github.com/apache/airflow/pull/35536#discussion_r1447517335 ## airflow/utils/log/file_task_handler.py: ## @@ -357,7 +367,9 @@ def _read( worker_log_full_path = Path(self.local_base, worker_log_rel_path)

Re: [PR] Doc: fix version to 2.8.0 [airflow]

2024-01-10 Thread via GitHub
boring-cyborg[bot] commented on PR #36714: URL: https://github.com/apache/airflow/pull/36714#issuecomment-1885104673 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Doc: fix version to 2.8.0 [airflow]

2024-01-10 Thread via GitHub
turbotimon opened a new pull request, #36714: URL: https://github.com/apache/airflow/pull/36714 --- Change version from 2.6.x to 2.8.0 so every version on the page is the same and latest -- This is an automated message from the Apache Git Service. To res

Re: [PR] Remove tenacity on KPO logs inner func consume_logs [airflow]

2024-01-10 Thread via GitHub
vincbeck commented on PR #35504: URL: https://github.com/apache/airflow/pull/35504#issuecomment-1885120967 > Hmm I see, looking at the traceback, that the error is raised from `read_pod_logs`, which is already wrapped with tenacity. Perhaps instead of adding "more tenacity" at higher level,

Re: [PR] Fix security manager inheritance in fab provider [airflow]

2024-01-10 Thread via GitHub
vincbeck commented on PR #36538: URL: https://github.com/apache/airflow/pull/36538#issuecomment-1885128112 My bad, can you try by inheriting from `AirflowSecurityManager` instead of `FabAirflowSecurityManagerOverride `. Basically `class AzureCustomSecurity(AirflowSecurityManager):` instead

Re: [PR] Attempt to update Connexion library to version 3 [airflow]

2024-01-10 Thread via GitHub
vincbeck commented on PR #36052: URL: https://github.com/apache/airflow/pull/36052#issuecomment-1885139430 > Update init_api_auth_provider function. In get_api_endpoints function we use connexion_app.add_api and expect FlaskApi as a return object, but add_api function always return None.

Re: [PR] add use_regex argument for allowing S3KeySensor to check s3 keys with regular expression [airflow]

2024-01-10 Thread via GitHub
pankajkoti commented on code in PR #36578: URL: https://github.com/apache/airflow/pull/36578#discussion_r1447604749 ## tests/system/providers/amazon/aws/example_s3.py: ## @@ -200,7 +202,18 @@ def check_fn(files: list) -> bool: check_fn=check_fn, deferrable=True

Re: [PR] Do not let EventsTimetable schedule past events if catchup=False [airflow]

2024-01-10 Thread via GitHub
eladkal merged PR #36134: URL: https://github.com/apache/airflow/pull/36134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [I] EventsTimetable schedules past events even if catchup=False [airflow]

2024-01-10 Thread via GitHub
eladkal closed issue #33948: EventsTimetable schedules past events even if catchup=False URL: https://github.com/apache/airflow/issues/33948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

(airflow) branch main updated: Do not let EventsTimetable schedule past events if catchup=False (#36134)

2024-01-10 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c01daf8119 Do not let EventsTimetable schedule pas

Re: [PR] Fix assignment of template field in `__init__` in `papermill.py` [airflow]

2024-01-10 Thread via GitHub
eladkal merged PR #36530: URL: https://github.com/apache/airflow/pull/36530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow) branch main updated (c01daf8119 -> 7a2a4a98e0)

2024-01-10 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c01daf8119 Do not let EventsTimetable schedule past events if catchup=False (#36134) add 7a2a4a98e0 Fix assignme

Re: [PR] [FEAT] adds repair run functionality for databricks [airflow]

2024-01-10 Thread via GitHub
gaurav7261 commented on PR #36601: URL: https://github.com/apache/airflow/pull/36601#issuecomment-1885225508 Hi @dirrao , fixed static checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [I] Docker libmariadb.so.3 is missing from official Docker image [airflow]

2024-01-10 Thread via GitHub
boring-cyborg[bot] commented on issue #36715: URL: https://github.com/apache/airflow/issues/36715#issuecomment-1885233465 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Docker libmariadb.so.3 is missing from official Docker image [airflow]

2024-01-10 Thread via GitHub
ant-bl opened a new issue, #36715: URL: https://github.com/apache/airflow/issues/36715 ### Apache Airflow version 2.8.0 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? The libmariadb.so.3 is missing from the airflow Doc

Re: [I] Pause DAG such that all tasks complete [airflow]

2024-01-10 Thread via GitHub
aritra24 commented on issue #36539: URL: https://github.com/apache/airflow/issues/36539#issuecomment-1885241496 Hey @potiuk /@eladkal just wanted to bring this onto your radar, would you have any suggestions on the direction of implementation on this? -- This is an automated message from

(airflow) branch constraints-main updated: Updating constraints. Github run id:7477896229

2024-01-10 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 4858703340 Updating con

(airflow) branch switch-to-hatch updated (c63a6772b7 -> 9ad0ab39c9)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit c63a6772b7 Standardize airflow build process and switch to Hatchling build backend new 9ad0ab39c9 Stan

Re: [I] Docker libmariadb.so.3 is missing from official Docker image [airflow]

2024-01-10 Thread via GitHub
Taragolis commented on issue #36715: URL: https://github.com/apache/airflow/issues/36715#issuecomment-1885307402 Yep, seems like for `mysqlclient` it built by use MariaDB libraries, however in prod image it install MySQL libraries instead ```console ❯ docker run -it --rm apache/ai

(airflow) branch switch-to-hatch updated (9ad0ab39c9 -> 8fb613979e)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git discard 9ad0ab39c9 Standardize airflow build process and switch to Hatchling build backend add 88c9596f4a chec

(airflow) branch fix-default-mysql-client-for-prod created (now a986862627)

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git at a986862627 Use mariadb by default when build final prod image This branch includes the

Re: [PR] Allow custom columns in cli dags list [airflow]

2024-01-10 Thread via GitHub
ephraimbuddy commented on code in PR #35250: URL: https://github.com/apache/airflow/pull/35250#discussion_r1447723745 ## airflow/cli/commands/dag_command.py: ## @@ -362,15 +374,16 @@ def dag_list_dags(args) -> None: "For details, run `airflow dags list-import-errors

(airflow) 01/01: Use mariadb by default when build final prod image

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git commit a986862627f9a43a1f64f2afd265be9dd5577e77 Author: Andrey Anshin AuthorDate: Wed Jan 10 21:

(airflow) branch fix-default-mysql-client-for-prod updated (a986862627 -> 6e183fc76b)

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git discard a986862627 Use mariadb by default when build final prod image new 6e183fc76b Use `

(airflow) 01/01: Use `mariadb` by default when build final prod image

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6e183fc76b63df5a53cf556c2c6a31fd6ed75012 Author: Andrey Anshin AuthorDate: Wed Jan 10 21:

Re: [I] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
aritra24 commented on issue #36694: URL: https://github.com/apache/airflow/issues/36694#issuecomment-1885331442 Closing this since it already exists -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
aritra24 closed issue #36694: Bulk Variable Updates to allow transactional rollbacks URL: https://github.com/apache/airflow/issues/36694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Use `mariadb` by default when build final prod image [airflow]

2024-01-10 Thread via GitHub
Taragolis opened a new pull request, #36716: URL: https://github.com/apache/airflow/pull/36716 We forget to change default value for `INSTALL_MYSQL_CLIENT_TYPE` argument to `mariadb`. So we build packages by MariaDB libraries, however finally install mysql. This only af

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1885343909 All right I verified the contents of generated files and added a few modifications. Generally speaking - we have few more files added: * README files * openapi.yml * .pyi files

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36537: URL: https://github.com/apache/airflow/pull/36537#issuecomment-1885344869 Comparision for sdist packages looks similar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] add use_regex argument for allowing S3KeySensor to check s3 keys with regular expression [airflow]

2024-01-10 Thread via GitHub
vincbeck commented on code in PR #36578: URL: https://github.com/apache/airflow/pull/36578#discussion_r1447736675 ## tests/system/providers/amazon/aws/example_s3.py: ## @@ -200,7 +202,18 @@ def check_fn(files: list) -> bool: check_fn=check_fn, deferrable=True,

Re: [PR] add use_regex argument for allowing S3KeySensor to check s3 keys with regular expression [airflow]

2024-01-10 Thread via GitHub
vincbeck merged PR #36578: URL: https://github.com/apache/airflow/pull/36578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated (7a2a4a98e0 -> 9f04716179)

2024-01-10 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7a2a4a98e0 Fix assignment of template field in `__init__` in `papermill.py` (#36530) add 9f04716179 Add use_reg

Re: [PR] Use `mariadb` by default when build final prod image [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36716: URL: https://github.com/apache/airflow/pull/36716#issuecomment-1885351034 I guess we should also mention that in the docker-stack release notes (changing 2.8.0 -> 2.8.1) ? -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-10 Thread via GitHub
vincbeck commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r1447742983 ## airflow/providers/fab/provider.yaml: ## @@ -36,6 +36,11 @@ versions: dependencies: - apache-airflow>=2.9.0 - flask>=2.2,<2.3 + # We are tightly coupled wi

Re: [PR] Use `mariadb` by default when build final prod image [airflow]

2024-01-10 Thread via GitHub
Taragolis commented on PR #36716: URL: https://github.com/apache/airflow/pull/36716#issuecomment-1885360700 Yeah I think also need to mention that if someone build custom image from 2.8.0 branch then should specify `INSTALL_MYSQL_CLIENT_TYPE=mariadb` Let me find better place for that

(airflow) branch switch-to-hatch updated (8fb613979e -> 28977c4070)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 8fb613979e Standardize airflow build process and switch to Hatchling build backend add 28977c4070 Stan

Re: [PR] Standardize airflow build process and switch to Hatchling build backend [airflow]

2024-01-10 Thread via GitHub
potiuk commented on code in PR #36537: URL: https://github.com/apache/airflow/pull/36537#discussion_r1447746091 ## airflow/providers/fab/provider.yaml: ## @@ -36,6 +36,11 @@ versions: dependencies: - apache-airflow>=2.9.0 - flask>=2.2,<2.3 + # We are tightly coupled with

(airflow) branch fix-default-mysql-client-for-prod updated (6e183fc76b -> e07ab89dac)

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git from 6e183fc76b Use `mariadb` by default when build final prod image add e07ab89dac Add

Re: [I] default_webserver_config.py isn't compatible with the latest changes [airflow]

2024-01-10 Thread via GitHub
potiuk commented on issue #36702: URL: https://github.com/apache/airflow/issues/36702#issuecomment-1885402039 cc: @vincbeck -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] airflow workers failed to connect redis. pod log of workers: [airflow]

2024-01-10 Thread via GitHub
potiuk closed issue #36707: airflow workers failed to connect redis. pod log of workers: URL: https://github.com/apache/airflow/issues/36707 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] Status of testing Providers that were prepared on January 07, 2024 [airflow]

2024-01-10 Thread via GitHub
shohamy7 commented on issue #36644: URL: https://github.com/apache/airflow/issues/36644#issuecomment-1885406782 I've checked my changes, all looks good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
potiuk commented on code in PR #36705: URL: https://github.com/apache/airflow/pull/36705#discussion_r1447781472 ## airflow/cli/commands/variable_command.py: ## @@ -61,8 +61,25 @@ def variables_get(args): @providers_configuration_loaded def variables_set(args): """Create n

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
potiuk closed pull request #36705: Bulk Variable Updates to allow transactional rollbacks URL: https://github.com/apache/airflow/pull/36705 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Bulk Variable Updates to allow transactional rollbacks [airflow]

2024-01-10 Thread via GitHub
potiuk commented on PR #36705: URL: https://github.com/apache/airflow/pull/36705#issuecomment-1885413586 Closing it - this change is just proxying existing command and makes little sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

svn commit: r66544 - /dev/airflow/providers/ /release/airflow/providers/

2024-01-10 Thread eladkal
Author: eladkal Date: Wed Jan 10 18:45:51 2024 New Revision: 66544 Log: Release Airflow Providers on 2024-01-10 Added: release/airflow/providers/apache_airflow_providers_amazon-8.16.0-py3-none-any.whl - copied unchanged from r66543, dev/airflow/providers/apache_airflow_providers_amazo

Re: [PR] Add documentation for packages - 2024-01-07 [airflow-site]

2024-01-10 Thread via GitHub
eladkal merged PR #931: URL: https://github.com/apache/airflow-site/pull/931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow-site) branch add-documentation-2024-01-07 deleted (was 52580d73d0)

2024-01-10 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-01-07 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 52580d73d0 Add documentation for packages - 2024-01-07 The revisions that were on this b

(airflow) branch fix-default-mysql-client-for-prod updated: Validate import MySQLdb (mysqlclient) in prod image test

2024-01-10 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch fix-default-mysql-client-for-prod in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/fix-default-mysql-client-for-prod by this push

(airflow) branch switch-to-hatch updated (28977c4070 -> 1c8184e0eb)

2024-01-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch switch-to-hatch in repository https://gitbox.apache.org/repos/asf/airflow.git omit 28977c4070 Standardize airflow build process and switch to Hatchling build backend add 9f04716179 Add

Re: [PR] Use `mariadb` by default when build final prod image [airflow]

2024-01-10 Thread via GitHub
Taragolis commented on code in PR #36716: URL: https://github.com/apache/airflow/pull/36716#discussion_r1447796040 ## docker_tests/test_prod_image.py: ## @@ -158,6 +158,7 @@ def test_pip_dependencies_conflict(self): "grpc": ["grpc", "google.auth", "google_auth_httplib2"

  1   2   3   >