On 7/15/20 3:14 PM, Alex Rousskov wrote: > I propose to add a new tls_key_log directive to record TLS > pre-master secret (and related encryption details) for to- and > from-Squid TLS connections. This very useful triage feature is common > for browsers and some networking tools. Wireshark supports it[1]. You > might know it as SSLKEYLOGFILE. It has been requested by several Squid > admins. A draft documentation of the proposed directive is at the end of > this email. > > [1] https://wiki.wireshark.org/TLS#Using_the_.28Pre.29-Master-Secret > > If you have any feature scope adjustments, implementation wishes, or > objections to this feature going in, please let me know!
FYI: Factory is starting to implement this feature. Alex. > FTR, we have considered providing similar support by adding new > logformat %code(s) to the existing access_log. While doing so would > reduce and simplify code changes, we ultimately rejected that design > because the combination of the following factors renders it inferior and > insufficient on several levels: > > 1. Some TLS connections are reflected in access logs a long time > after they are established (e.g., when the connection serves > a long CONNECT tunnel). During triage, admins may need the > ability to decipher a live connection much sooner. > > 2. Some TLS connections are never reflected in access logs at all > (e.g., when Squid opens but does not use an outgoing TLS connection > or crashes when parsing the first request on an incoming one). Info > gaps often create triage suspicions: Did we drop something else? > > 3. A single access_log record may correspond to many from-Squid > connections, especially when Squid retries peer failures. Logging > keys for all these connections would require accumulating the keys in > the master transaction and then dumping them as a part of a new > %code. Adding (and dumping) repeated ALE fields is awkward. > > 4. Manually creating ACLs to limit access log records to the first > transaction on a connection would be a must for most deployments > using this feature. Doing so is far from trivial and related > configuration errors are difficult to triage. We could add a new > ACL type for this purpose, but even that is tricky because a > single master transaction may have many associated connections. > And logging secrets for every transaction creates too much noise. > > 5. Configuration flexibility offered by logformat is likely to > remain largely unused by the new feature because tools like > Wireshark _automatically_ find relevant records when deciphering > captured traffic. Augmenting these logs with other transaction > details (typical for access log uses) would be mostly useless. > > 6. New %codes would be awkward to use in a regular access log because > they may expand into a variable number of lines, going against the > traditional line-oriented, "fixed" format access log use. > > While some of the above items have workarounds, a few do not, and the > whole combination looks rather grim/unfriendly. We should attack this > problem from the other end -- a new simple configuration dedicated to > this useful feature. > > We propose to structure this new directive so that it is easy to add > advanced access_log-like features later if needed (while reusing the > corresponding access_log code). For example, if users find that they > want to maintain multiple TLS key logs or augment log records with > connection details, we can add that support by borrowing access_log > options and code without backward compatibility concerns. The new > required "if" keyword in front of the ACL list allows for seamless > addition of new directive options in the future. > > > Cheers, > > Alex. > ---------- draft squid.conf directive documentation ------------ > > tls_key_log > > Configures whether and where Squid records pre-master secret and > related encryption details for TLS connections accepted or established > by Squid. These connections include connections accepted at > https_port, TLS connections opened to origin servers/cache_peers/ICAP > services, and TLS tunnels bumped by Squid using the SslBump feature. > This log (a.k.a. SSLKEYLOGFILE) is meant for triage with traffic > inspection tools like Wireshark. > > tls_key_log <filename> if <acl>... > > WARNING: This log allows anybody to decrypt the corresponding > encrypted TLS connections, both in-flight and postmortem. > > At most one log file is supported at this time. Repeated tls_key_log > directives are treated as fatal configuration errors. By default, no > log is created or updated. > > If the log file does not exist, Squid creates it. Otherwise, Squid > appends an existing log file. > > The directive is consulted whenever a TLS connection is accepted or > established by Squid. TLS connections that fail the handshake may be > logged if Squid got enough information to form a log record. A record > is logged only if all of the configured ACLs match. > > Squid does not buffer these log records -- the worker blocks until > each record is written. File system buffering may speed things up, but > consider placing this triage log in a memory-based partition. > > This log is rotated based on the logfile_rotate settings. > > Logging errors are reported to cache.log but are otherwise ignored. > > While transport-related ACLs like src and dst should work, Squid may > not have access to higher-level information. For example, when logging > accepted https_port connections, Squid does not yet have access to the > expected HTTPS request. Similarly, an HTTPS response is not available > when logging most TLS connections established by Squid. > > The log record format is meant to be compatible with TLS deciphering > features of Wireshark which include support for TLS v1.3 fields such > as CLIENT_EARLY_TRAFFIC_SECRET and SERVER_TRAFFIC_SECRET_0. A single > log record usually spans multiple lines. Technical documentation for > that format is maintained inside the Wireshark code (e.g., see > tls_keylog_process_lines() comments as of Wireshark commit > e3d44136f0f0026c5e893fa249f458073f3b7328). > > This clause only supports fast acl types. > See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details. > _______________________________________________ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev