Re: Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Jarek Potiuk
Yeah. This seems like a really good feature (I saw that you added it to the Awesome Apache Airflow :). And I will also be happy to help with getting it in the CLI. It's rather simple task (especially after some of the refactorings we've done in

Re: Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Javier Llorente Mañas
Hi, Thank you! For the detail explanation. I will read everything and create a ticket with all the expectations. Cheers, Javier El jue., 26 mar. 2020 a las 23:46, Kamil Breguła () escribió: > Hello, > > Can you create a ticket in Github and describe your expectations? This > will allow you to

Re: Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Kamil Breguła
Hello, Can you create a ticket in Github and describe your expectations? This will allow you to discuss this topic more easily. https://github.com/apache/airflow/issues/new/choose First of all, you should read the contribution guide, which contains a very detailed description of how to propose ch

Re: Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Javier Llorente Mañas
Hi, Yes, I missed this feature in the Airflow CLI. For me, the ideal scenario is to have something similar to the helm install --debug --dry-run command ( https://helm.sh/docs/chart_template_guide/debugging/). So the same that for a helm chart you generate multiple k8s templates, for an airflow da

Re: Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Kamil Breguła
Hello, The idea is fantastic. I like it very much and it will facilitate the work with Kubernethes. I'm just afraid that this tool will not be available when I need it. Did you think to add this tool to CLI of Airflow? This is the best place to share useful tools. When I missed the DAG preview in

Airflow Kubernetes pod Operator debug library

2020-03-26 Thread Javier Llorente Mañas
Hi all! it's Javier. I am a Data Engineer, I have been working with Airflow and it's Kubernetes integration for almost one year and a half. It's great and it has helped a lot to me and the data engineering team I have been working with. Although I and my colleagues find a recurring issue sometime