Hello everyone, I'd like to not put this item on the agenda for next Monday so I'll briefly present it here since nobody at Varnish Software thinks it will be controversial. Attendees outside of Varnish Software, please ack that this should have a de-facto "send patches" green light or we can discuss this further next Monday.
It comes in 3 simple steps. 1) Multi-line queries When someone wishes to capture logs for multiple reasons, they need to OR them, possibly with brackets. Instead the proposal is to support one query per line, ignore empty lines, and treat tokens starting with '#' as comments. Example: varnishlog -q '(my-first-query) or (my-second-query)' varnishlog -q ' my-first-query my-second-query ... ' 2) Read queries from a file Similar to reading varnishncsa's format from a file: cat >bad-things.txt <<EOF # internal errors RespStatus >= 500 # backend problems ... # internal incident 4253 ... # varnish issue 1799 ... EOF varnishlog -Q bad-things.txt 3) Query scoping The current scope is always the transaction group, but sometimes we may want conditions to apply to a subset of the `-g` option (for example two fields of the same VSL record). Proposed solution: inside brackets have the possibility to reduce the scope with the very first token. The scope cannot outlive the parent scope, and the query keeps its default @group transaction. Example: # beresp cached more than 2m before v_b_r{} @tag TTL[1] eq RFC and TTL[2] >= 120 The groups would be: - @group (default when omitted, matches -g option) - @request (same as -g request) - @vxid (same as -g vxid) - @tag (single record) Things that shouldn't work: varnishlog -q '@request ...' varnishlog -q '@tag ... and (@group ...)' varnishlog -g vxid -q '@request ...' varnishlog -q '@tag SomeRecord and DifferentRecord' My opinion was that the current VSL query architecture should accommodate such changes nicely, and Martin is also not too worried about it. 4) Bonus topic I also wanted to put a small performance topic on the agenda: mmap'ing regular files when reading offline VSL. The current code treats files (-r option) the same way live SHMLOG VSL is and allocates copies. The idea is to mmap regular files and avoid needless copies, and keep the current `-r file` handling for the stdin case. I hope to speed up processing when it comes to dumps in the GBs. Someone asked about binary VSL future plans. All of this should be orthogonal to changes in the underlying VSL format. Nothing should require an soname major bump in libvarnishapi (minor bump for the -Q option). Dridi _______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev