Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864270367
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
I will resolve it, as I think it's done, we can always update it later :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk merged PR #44492: URL: https://github.com/apache/airflow/pull/44492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864250920
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
> Is it worth adding that in the message -- "that you don't need to rebuild
everytime -- but this is useful when deps change" etc?
Welll. I though I did :). Is the last line not enough :) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864251058
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
(Or maybe it could be yellow or something @kaxil .?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864249341
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
Yeah. I will think of a better way of doing it later -> this part is very
"slow" to iterate on and there are more gotchas and inefficiences of the `K8S
tests` experience - which is far from ideal. It's been experienced in the past
by @ephraimbuddy and yesterday by @ashb . While having the tests is super
useful to test the whole stack of ours (`airflow` + `image` + `chart`) - and as
the `LocalExceutor` failure the day before have shown it's great to have them
to catch it early, iterating on them is a pain.
I created an issue in our "CI/CD/DEVENV" project
https://github.com/apache/airflow/issues/44508 describing all the defficiencies
and potential ways we can improve them.
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
Yeah. I will think of a better way of doing it later -> this part is very
"slow" to iterate on and there are more gotchas and inefficiences of the `K8S
tests` experience - which is far from ideal. It's been experienced in the past
by @ephraimbuddy and yesterday by @ashb .
While having the tests is super useful to test the whole stack of ours
(`airflow` + `image` + `chart`) - and as the `LocalExceutor` failure the day
before have shown it's great to have them to catch it early, iterating on them
is a pain.
I created an issue in our "CI/CD/DEVENV" project
https://github.com/apache/airflow/issues/44508 describing all the defficiencies
and potential ways we can improve them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
kaxil commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864032528
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
Is it worth adding that in the message -- "that you don't need to rebuild
everytime -- but this is useful when deps change" etc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864009483
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
We need - in case dependencies changed (this was likely the reason why it
failed for @ashb today) - and for sure failed for me when I followed it. The
problem is if you build the image in the past, and use it as a base, airflow
might not start. The problem for me was that `asyncpg` was missing in the image
- because we added it last week and latest sources of airflow required it.
I have no good idea for now how to make sure that the prod image is only
rebuilt when needed (it takes time) - so next best thing is to ask the user to
force it when they go through the normal k8s testing process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
kaxil commented on code in PR #44492:
URL: https://github.com/apache/airflow/pull/44492#discussion_r1864005605
##
dev/breeze/src/airflow_breeze/commands/kubernetes_commands.py:
##
@@ -974,8 +974,15 @@ def configure_cluster(
output=None,
)
if return_code == 0:
-get_console().print("\n[warning]NEXT STEP:[/][info] You might now
build your k8s image by:\n")
-get_console().print("\nbreeze k8s build-k8s-image\n")
+get_console().print(
+"\n[warning]NEXT STEP:[/][info] You might now build your k8s
image "
+"with all latest dependencies:\n"
+)
+get_console().print("\n breeze k8s build-k8s-image
--rebuild-base-image\n")
Review Comment:
Do we need `--rebuild-base-image` for building image? From command itself
thought: `breeze k8s build-k8s-image` should work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Better description on how to build image for k8s [airflow]
potiuk commented on PR #44492: URL: https://github.com/apache/airflow/pull/44492#issuecomment-2508733157 https://github.com/user-attachments/assets/7c969345-f0b4-4ef0-81a5-59a2136a250a";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
