Hi all! it's Javier. I am a Data Engineer, I have been working with Airflow and it's Kubernetes integration for almost one year and a half. It's great and it has helped a lot to me and the data engineering team I have been working with.
Although I and my colleagues find a recurring issue sometimes whenever we were creating new dags using the KubernetesPodOperator as sometimes the pod created was not as we expected. As an example arguments of the container were not parsing as we expected or Secrets or Kubernetes resources were not referenced as we want. I just created this library https://github.com/Javier162380/AirflowKuberentesDebugger. The idea is that it can be a simple interface that can generate k8s pod YAML files before deploying dags into a production environment so we can test if the dag is going to generate all the k8s pods as we want or something is wrong. The idea is to have a kind of helm debugger for Airflow. Also, it can be really useful to recover historical data for recurrent dags. just changing the pod resources and the container arguments or entry points. All your feedback is appreciated. Cheers, Javier
