Willy, Am 02.05.2018 um 11:47 schrieb Willy Tarreau: > Nice one, though I'd argue that sites which do this know that they > are manipulating large contents (it's visible in the config file and > sometimes they are the ones asking to relax the config parsing rules). > So they're also aware of the need to increase maxrewrite.
Clearly I was not aware that I needed to increase maxrewrite, despite adding a ton of headers to my responses. Yes, I could have found out if I read the complete configuration manual. But the tunables are defined as "Performance Tuning" in the headline. I don't have performance issues, I handle less than 15 r/s in peak situations. It did not even occur to me that there could possibly be a limit to header rewriting. In fact I found about this whole issue by accident. >> I'd start of with *logging* when the call fails for the short term. > > Adding extra logs not related to traffic actually makes debugging much > worse because these logs don't end up in the correct file/storage/whatever. > It's much easier to filter on 400/502 and find all the request elements to > understand what happens than seeing a random error out of any context. > Part of the difficulty here is to properly indicate what was attempted > and failed, in complex configs where it's hard to even enumerate rulesets. To me a message like: "Unable to add-header Content-Security-Policy to response. Possibly the amount of headers exceeds tune.maxrewrite." would have been more helpful than random 502 without any further information. Especially since the issue happens randomly: Sometimes the additional headers fit by chance. Sometimes they don't. I would start by investigating the connection to the backend services, not investigating some random tunables (See my paragraph above.). Actually I'm not sure whether a 502 would even be the correct response. The issue is not with the backend, but with the proxy. I'd expect a 500 here: > The 502 (Bad Gateway) status code indicates that the server, while > acting as a gateway or proxy, received an *invalid response from an > inbound server* it accessed while attempting to fulfill the request. (highlighting mine) After digging into it I might be able to deduce that the addition of the new `http-response add-header` line caused the issues. But still I would be non the wiser. I would have to stumble upon the tunable by accident. Or ask on the list, like I did. > Maybe one solution could be to tag transactions that experienced such a > failure and to put a warning on them, that would be reported in the logs > with a "W" flag in the termination codes. It would not indicate what > failed, obviously, but would very quickly raise awareness of the fact > that there is a problem. Usually thanks to the second flag indicating > the transaction state and to the rest of the log, it's easier to follow > the processing and to figure what went wrong. Later once we add the > ability to reject again such failures, we could add an "F" (for "failure" > to process). > > What do you think ? It clearly would be better than silently failing, but it still would have wasted a few hours I suspect (again: see above paragraphs). I want to note at this point that I'm not running haproxy at scale or with serious monitoring. The haproxy instance I'm experiencing this issue with is my personal server, not some company or business one. It runs my mail and some side / hobby projects. My needs or expectations might be different. Best regards Tim Düsterhus
