Sudheer Vinukonda created TS-4291:
-------------------------------------
Summary: Custom log field {{pqhl}} should not count internal
headers.
Key: TS-4291
URL: https://issues.apache.org/jira/browse/TS-4291
Project: Traffic Server
Issue Type: Bug
Components: Logging
Reporter: Sudheer Vinukonda
[~zwoop] suggested to add a simple API (not necessarily exposed to plugins at
the moment), to count the length of @-custom headers and deduct that when
reporting {{pqhl}}
IRC discussion below.
{code}
oag
7:55 If anyone is around who is familiar with the @-headers trick, I have a
question:
7:55 I have observed that @custom-header in an outgoing request changes the
value of the pqhl log field, even though the header is never actually sent.
7:55 This is true in 5.3.1. Does anyone know if this as been changed in 6.x.x ?
7:55 Would like it if pqhl was an accurate reflection of data to the origin
server.
7:55 (pqhl + pqbl that is.)
zwoop
7:55 oag I'm pretty sure that is intended, if you don't want it there, why not
add it to another header ?
7:55 the way it works is that it "filters" out the @ header before sending, and
returning, headers
7:55 (afaik)
7:55 entirely possible I'm wrong too, but that was my impression from before.
***
7:55 Playback Complete.
7:57 esproul [~ad...@static-108-48-124-82.washdc.fios.verizon.net] entered the
room.
sudheerv
8:07 zwoop: oag: umm..interesting, like zwoop said while it is intentional that
@-custom headers are meant never to be leaked outside, the pqhl adding the
@-headers seems like not something desirable
8:08 i'd be okay if we agree to not add it to pqhl
8:08 pqhl
The proxy request header length; the header length in Traffic Server’s request
to the origin server.
8:09 given that pqhl is defined as the header length in the outbound request
8:09 adding @-headers to it seems inconsistent/inaccurate
8:10 as a matter of fact, i use @headers pretty "heavily" in our products..but,
don't enable pqhl
8:10 (at the moment anyway)
zwoop
8:15 sudheerv but that sets a poor precedence. You want to exclude @ headers
from pqhl, but not the other 3 internal header log tags that we have ?
zwoop
8:16 One very important feature of the @-headers is to be able to log internal
(plugin generated) information in our custom logs
sudheerv
8:16 zwoop: but, pqhl is just the length of the outgoing headers?
zwoop
8:16 ah
sudheerv
8:16 i haven't looked at in detail, but, excluding @ headers ( and the other
intenral headers)
zwoop
8:16 in bytes ?
sudheerv
8:16 from pqhl
8:16 will it mess up the log fields?
8:16 yeah
zwoop
8:16 I thought it was the request header object.
8:16 Yeah, I don't know
sudheerv
8:17 yeah, worth double checking - when the wise man complains :)
zwoop
8:17 maybe you'd have to keep two lengths then, one with, and one without the @
headers ?
sudheerv
8:17 but, yeah, i'd not want to exclude the headers from log fields
8:17 just the length
8:17 sure, yeah unless it's already computing them in one place
zwoop
8:17 m_proxy_request->length_get()
sudheerv
8:17 where it'd be easier to exclude them - need ot check that part of the code
8:17 ah, ok
8:19 so, would you be okay to modify pqhl (lenght in bytes) to exclude the
internal headers?
zwoop
8:19 http://pastebin.com/svwPmaa0
8:19 looks like it counts the size of the "blocks" used, and not individual
header. It'd likely get much, much more expensive if you have to walk through
every header value.
8:21 actually, maybe it does count it individually, cause it calls
mime_field_length_get(MIMEField *field)
sudheerv
8:21 yeah
zwoop
8:22 But regardless, I'd be concerned that this might break other behavior
internally, so at a minimum, you'd want this exclusion only for a log tag. And
even so, since it's incompatible, etiher for v7.0.0, or as a separate log tag
sudheerv
8:22 and it does it each time lenght_get() is called?
zwoop
8:22 looks like it
sudheerv
8:22 umm..ok - yeah, may be, we just live with it then :)
8:22 we can update the docs though
8:23 oag: may be, you could help with that :)
zwoop
8:23 this is probably a huge can of worm :)
sudheerv
8:23 agree
zwoop
8:23 i'm not touching it :)
sudheerv
8:24 lol, although, i wasn't going to change the mime_hdr_lenght_get() for
general API usage
8:24 only the pqhl part
8:24 for e.g. add another version of mime_hdr_lenght_get()
8:24 and use that for loggin
8:24 but, i agree - may be it's not worth touching this can
zwoop
8:25 maybe another option would be to have a new tag showing the (various)
sizes of @-headers only?
8:25 Then you can do the subtraction
sudheerv
8:25 yeah that sounds reasonable
8:25 actually, even cleaner
8:25 someone can just get the lenght of @headers if he wants
zwoop
8:26 right
sudheerv
8:26 and the logging part can show the delta
zwoop
8:26 yeah, logs_xml.config supports "math" I think
8:26 psp [~Adium@207.38.47.126] entered the room.
sudheerv
8:26 yeah, or we can even fix the pqhl if it's acceptable
8:26 as it stands right now, it seems to not match its' definition
zwoop
8:26 although, with jpeach's work on the Lua metrics, maybe we can get Lua
"logs configs" for 7.x :)
sudheerv
8:26
http://trafficserver.readthedocs.org/en/latest/admin-guide/monitoring/logging/log-formats.en.html
8:26 qhl
The proxy request header length; the header length in Traffic Server’s request
to the origin server.
8:27 cool
zwoop
8:28 Oh, looks like the "subtraction" feature is only on the milestone tag ?
8:28 {Milestone field name1-Milestone field name2}msdms
8:28 that's unfortunate, it'd be nice to generalize that
sudheerv
8:28 huh, how's that?
8:28 yeah, would have thought it's generic
8:28 we do have some aggregate operators?
8:28 like avg, count etc
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
