[ https://issues.apache.org/jira/browse/AIRFLOW-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik updated AIRFLOW-6350: -------------------------------- Issue Type: New Feature (was: Bug) > security - spark submit operator logging+exceptions should mask passwords > -------------------------------------------------------------------------- > > Key: AIRFLOW-6350 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6350 > Project: Apache Airflow > Issue Type: New Feature > Components: hooks, operators > Affects Versions: 1.10.3 > Reporter: t oo > Assignee: t oo > Priority: Major > Fix For: 1.10.8 > > > contrib/hooks/spark_submit_hook.py > Mask passwords in spark submit cmd AND error stacktrace > > *add* > def _mask_cmd(self, connection_cmd): > # Mask any password related fields in application args with key value pair > where key contains password (case insensitive), e.g. HivePassword='abc' > connection_cmd_masked = > re.sub(r"(\S*?(?:secret|password)\S*?\s*=\s*')[^']*(?=')", r'\1******', ' > '.join(connection_cmd), flags=re.I) > return connection_cmd_masked > > *BEFORE* > self.log.info("Spark-Submit cmd: %s", connection_cmd) > > *AFTER* > self.log.info("Spark-Submit cmd: %s", self._mask_cmd(connection_cmd)) > > *BEFORE* > if returncode or (self._is_kubernetes and self._spark_exit_code != 0): > raise AirflowException( > "Cannot execute: {}. Error code is: {}.".format( > spark_submit_cmd, returncode > ) > ) > > *AFTER* > if returncode or (self._is_kubernetes and self._spark_exit_code != 0): > raise AirflowException( > "Cannot execute: {}. Error code is: {}.".format( > self._mask_cmd(spark_submit_cmd), returncode > ) > ) -- This message was sent by Atlassian Jira (v8.3.4#803005)