Javier162380 opened a new issue #7904: Airflow Kubernetes Pod Operator Debugger
URL: https://github.com/apache/airflow/issues/7904
 
 
    so<!--
   
   Welcome to Apache Airflow!  For a smooth issue process, try to answer the 
following questions.
   Don't worry if they're not all applicable; just try to include what you can 
:-)
   
   If you need to include code snippets or logs, please put them in fenced code
   blocks.  If they're super-long, please use the details tag like
   <details><summary>super-long log</summary> lots of stuff </details>
   
   Please delete these comment blocks before submitting the issue.
   
   -->
   
   **Description**
   
   Airflow and Kubernetes integration are great. Although one of the hardest 
part comes when we need to debug.
   
   I found some time problematic when I create or change some configuration in 
a  dag using the KubernetesPodOperator to know if the task is going to render 
all their attributes as I want. For instance, some times on my job I found 
inconsistencies in the way arguments or entry points are parsing, or Kubernetes 
resources are created or referenced.
   
   Moreover having a tool for generating k8s pod templates from Airflow dag, 
can be also really convenient for recover historical data if you are using an 
incremental approach for some dags or trigger some tasks with as slightly 
different configuration for one-time demand. In this way, we can interact 
directly with the Kubernetes API without needing to change airflow dags 
configuration, so Airflow will give us more speed and flexibility
   
   **Use case / motivation**
   
   <!-- What do you want to happen?
   
   For this solution, I will like to have a feature in Airflow that given a dag 
or path containing multiple dags Airflow can generate one or multiple k8s pod 
templates in a path or giving an output in a terminal for debugging, and maybe 
in a most ambitious way for deploying into prod using a Kubernetes command.
    Something similar to the helm installs --debug dry-run command 
(https://helm.sh/docs/chart_template_guide/debugging/). 
   
   This feature could be added directly to the CLI so it can be easily used for 
end-users. In the same way
   
   I have created a library implementing this new functionality
   
   -->
   
   **Related Issues**
   
   <!-- Is there currently another issue associated with this? -->

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to