dondaum opened a new issue #17604: URL: https://github.com/apache/airflow/issues/17604
Hi core Airflow dev teams, I would like to say a big 'thank you' first for this great plattform and for all of your efforts to constantly improve and mantain it. It is just a joy to work with.. Our company has recently switched to the Airflow 2 and we love the overall improvements, such as scheduler speed, UI but also all your efforts in knowledge sharing and community building. **Description** The KubernetesPodOperator does not have an option to either disable showing environment variables information or masking variables in the task logs if a failure or an error happen on a created pod. We use the KubernetesPodOperator and k8s.V1EnvVar to pass sensitive information to a pod, such as target connection, user name or passwords. If an error or failure happens the task logs show all pod details including the environment variables. An option, such as adding a new param to the KubernetesPodOperator that allows to either remove env variables completely or mask env variables would prevent stop sharing those senstive informations in the logs. **Use case / motivation** When running the KubernetesPodOperator with environment variables it would be great to have the option to remove environmental variables or to mask environment variables information in the task logs if there are failures or errors. **Are you willing to submit a PR?** Yes I would. Currently if the final_state not equals State.SUCCESS, we are handling this by checking if we have env variables set and if so we start to masking them with **** before raising the exception that then also logs the complete remote_pod object. I guess this is rather a quick fix that we have applied. So it would be good to think about other use cases or pod objects that could benefit as well. The relevant section is ` launcher = self.create_pod_launcher() if len(pod_list.items) == 1: try_numbers_match = self._try_numbers_match(context, pod_list.items[0]) final_state, remote_pod, result = self.handle_pod_overlap( labels, try_numbers_match, launcher, pod_list.items[0] ) else: self.log.info("creating pod with labels %s and launcher %s", labels, launcher) final_state, remote_pod, result = self.create_new_pod_for_operator(labels, launcher) if final_state != State.SUCCESS: raise AirflowException(f'Pod {self.pod.metadata.name} returned a failure: {remote_pod}') ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org