CurtHagenlocher commented on PR #4027: URL: https://github.com/apache/arrow-adbc/pull/4027#issuecomment-4277643050
> Let me know if you have some time to review this PR. I'm sorry for having taken so long to get to this, but I knew I'd need to be able to spend some time understanding the bigger picture. If I understand the goal of this change correctly, it's that it would let unredacted traces in a service environment be written to the file system on-demand for individual troubleshooting purposes, while using `System.Diagnostics.ActivityListener` to integrate with the telemetry system in our proprietary application in a way that allows conditional redaction of sensitive values. Is that right? The main concern I have with the approach in the PR is that it only works for drivers implemented in .NET that haven't been AOT-compiled and are running in same CLR instance as the `ActivityListener`. Do you have any ideas on how this could be extended to e.g. Snowflake or FlightSQL or future drivers implemented in Go or Rust? The other thing which feels a little unsettling is that it basically projects elements of our own internal privacy model onto the shared specification for (.NET-based) ADBC drivers, and in a way that limits the ability to (for instance) use an OTel-compatible exporter to get data that we consider sensitive but which another consumer might not. Unfortunately, the Open Telemetry guidance around "sensitive values" seems to be to "don't log them at all", which isn't very helpful in an environment where the regulatory landscape varies so much between jurisdictions and over time. The only other approach that immediately comes to mind is to encode the semantics of the value in its key name, but I'm still thinking about this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
