Re: [PR] change spark connection form and add spark connections docs [airflow]

2024-03-25 Thread via GitHub
YvesMentens commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-2018787802 But how do I pass the extra parameters then? If I cant add them in the UI? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] change spark connection form and add spark connections docs [airflow]

2024-03-25 Thread via GitHub
eladkal commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-2018756477 > How should we pass the extra params that have been removed in this PR now? It's not removed. This is just friendlier UI form. -- This is an automated message from the Apach

Re: [PR] change spark connection form and add spark connections docs [airflow]

2024-03-25 Thread via GitHub
YvesMentens commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-2018731687 How should we pass the extra params that have been removed in this PR now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-29 Thread via GitHub
eladkal merged PR #36419: URL: https://github.com/apache/airflow/pull/36419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1437270084 ## docs/apache-airflow-providers-apache-spark/connections/spark.rst: ## @@ -19,15 +19,15 @@ .. _howto/connection:spark: Review Comment: renamed :) -- This

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
shohamy7 commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-1870649740 @josh-fell Good point! I didn't know about this one. I've checked and saw that it is causing issues with other connections. But I saw that when I add default values, the probl

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
josh-fell commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-1870393776 @shohamy7 Have you tried to create a connection that _isn't_ one of the connections being updated here? Behind the scenes with connections, _all_ of the connection form data is submit

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
josh-fell commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1437086157 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls)

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
bolkedebruin commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436918236 ## docs/apache-airflow-providers-apache-spark/connections/spark.rst: ## @@ -19,15 +19,15 @@ .. _howto/connection:spark: Review Comment: Lets rename the fi

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
Taragolis commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436880841 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls)

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436848168 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls) -

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-27 Thread via GitHub
Taragolis commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436826808 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls)

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436635037 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls) -

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436621647 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls) -

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436621647 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls) -

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
josh-fell commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436608468 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -98,10 +98,42 @@ class SparkSubmitHook(BaseHook, LoggingMixin): def get_ui_field_behaviour(cls)

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
shohamy7 commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436359527 ## docs/apache-airflow-providers-apache-spark/connections/spark-sql.rst: ## @@ -0,0 +1,48 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-26 Thread via GitHub
Taragolis commented on code in PR #36419: URL: https://github.com/apache/airflow/pull/36419#discussion_r1436345176 ## docs/apache-airflow-providers-apache-spark/connections/spark-sql.rst: ## @@ -0,0 +1,48 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

Re: [PR] change spark connection form and add spark connections docs [airflow]

2023-12-25 Thread via GitHub
shohamy7 commented on PR #36419: URL: https://github.com/apache/airflow/pull/36419#issuecomment-1869107822 @Taragolis Please review, would like to here what you are think on this PR :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[PR] change spark connection form and add spark connections docs [airflow]

2023-12-25 Thread via GitHub
shohamy7 opened a new pull request, #36419: URL: https://github.com/apache/airflow/pull/36419 --- Related: #28790 This PR aims to add docs about each Spark connection in a separate page. In addition, adding a more user-friendly connection form for Spark an