Re: [PR] k8s spark operator compatibility [airflow]

2024-03-26 Thread via GitHub
github-actions[bot] closed pull request #36268: k8s spark operator compatibility URL: https://github.com/apache/airflow/pull/36268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] k8s spark operator compatibility [airflow]

2024-03-20 Thread via GitHub
github-actions[bot] commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-2010947712 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] k8s spark operator compatibility [airflow]

2024-02-04 Thread via GitHub
blcksrx commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1925874694 This part is simple,the operator it self got completely changed, check it from the main branch -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] k8s spark operator compatibility [airflow]

2024-02-04 Thread via GitHub
romsharon98 commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1925874173 > A major change happened in the operator actually, that makes it hard to do it! The new change generates a new job name(at least what i understood from the code) and it needs colla

Re: [PR] k8s spark operator compatibility [airflow]

2024-02-04 Thread via GitHub
blcksrx commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1925867495 A major change happened in the operator actually, that makes it hard to do it! The new change generates a new job name(at least what i understood from the code) and it needs collabor

Re: [PR] k8s spark operator compatibility [airflow]

2024-02-04 Thread via GitHub
romsharon98 commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1925866476 I think it will be nice to add this `override_existing` as templated field. If it's not to much can u add it? -- This is an automated message from the Apache Git Service. To res

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-25 Thread via GitHub
blcksrx commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1910942786 @potiuk I see a lot of changes, It gonna check it during the weekened -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-24 Thread via GitHub
potiuk commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1907882516 Can you rebase it please @blcksrx ? There are some conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-24 Thread via GitHub
ruslanguns commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1907626900 This PR is very important for us, we are start working with Airflow and Spark and this additional means a lot. I really encourage the reviewers to review this. Thanks in advanced:

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-23 Thread via GitHub
blcksrx commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1906582870 @assaf-xm no review yet :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-23 Thread via GitHub
assaf-xm commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1906020596 @blcksrx is there any progress with the review of this PR? (this is a small PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] k8s spark operator compatibility [airflow]

2023-12-17 Thread via GitHub
blcksrx commented on code in PR #36268: URL: https://github.com/apache/airflow/pull/36268#discussion_r1429261594 ## airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py: ## @@ -59,7 +59,7 @@ class SparkKubernetesOperator(BaseOperator): def __init__( self

Re: [PR] k8s spark operator compatibility [airflow]

2023-12-17 Thread via GitHub
blcksrx commented on code in PR #36268: URL: https://github.com/apache/airflow/pull/36268#discussion_r1429261594 ## airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py: ## @@ -59,7 +59,7 @@ class SparkKubernetesOperator(BaseOperator): def __init__( self

Re: [PR] k8s spark operator compatibility [airflow]

2023-12-17 Thread via GitHub
josh-fell commented on code in PR #36268: URL: https://github.com/apache/airflow/pull/36268#discussion_r1429239828 ## airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py: ## @@ -59,7 +59,7 @@ class SparkKubernetesOperator(BaseOperator): def __init__( se

[PR] k8s spark operator compatibility [airflow]

2023-12-17 Thread via GitHub
blcksrx opened a new pull request, #36268: URL: https://github.com/apache/airflow/pull/36268 Changes in this PR: - Added `override_existing` argument with default to the operator. - Uses the K8S application `uid` instead of naming --- **^ Add meaningful