Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
github-actions[bot] closed issue #54981: BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred URL: https://github.com/apache/airflow/issues/54981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
github-actions[bot] commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-4455854305 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
github-actions[bot] commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-440192 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
JiwaniZakir commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-4071677976 Looking into this, will submit a fix if I can confirm the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
galafis commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-4068960955 Hey, I took a look at this and I think I understand what's going on. When the operator runs in non-deferred mode, Airflow's `SecretsMasker` catches sensitive values in the logs because they go through the standard logging pipeline. But in deferred mode, the trigger gets serialized via `serialize()` which dumps the raw `variables` dict (including passwords, API tokens, etc.) into the trigger's serialized state. When the triggerer process picks it up and logs the trigger info, those values show up unmasked. I traced it through `BeamPythonPipelineTrigger.serialize()` in `providers/apache/beam/src/airflow/providers/apache/beam/triggers/beam.py` - the `variables` dict is returned as-is. The fix should ensure that sensitive keys in the variables are properly masked when the trigger state is logged. I'd like to work on this if it's available. Will have a PR up shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
jroachgolf84 commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-3227961754 Can you include a snippet of the non-deferred logs and the deferred logs with sensitive values removed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] BeamPythonPipelineTrigger logs sensitive pipeline options when pipeline is in deferred state but not when its not deferred [airflow]
boring-cyborg[bot] commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-3227595045 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
