Hashar has uploaded a new change for review. https://gerrit.wikimedia.org/r/312504
Change subject: logstash: parse runJobs messages ...................................................................... logstash: parse runJobs messages Extract informations from messages in mediawiki.runJobs, namely: * the job arguments and parameters as reported by Job::toString() * the job type/class coming from the RPC call (which is guaranteed to be correct unlike the message which a job can mangle however it wants) * job duration (in miliseconds) * the job error string if any I have forged the rules via https://grokdebug.herokuapp.com/ based on Grok doc: https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html Bug: T146469 Change-Id: I5dfa80364aa455725c51afd0e6a37dd4774b94b6 --- M files/logstash/filter-mediawiki.conf 1 file changed, 23 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/04/312504/1 diff --git a/files/logstash/filter-mediawiki.conf b/files/logstash/filter-mediawiki.conf index 3373774..fe8e614 100644 --- a/files/logstash/filter-mediawiki.conf +++ b/files/logstash/filter-mediawiki.conf @@ -95,6 +95,29 @@ } } # end [channel] == "api-feature-usage" + if [channel] == "runJobs" { + grok { + # The syslog message is forged by Job->toString() which is not + # guaranteed to contain the job type (the method is not final and can + # be overriden by child classes. + # The job type should be the first word, but it is not guaranteed, so + # capture everything under job_payload. + match => [ + "message", + "^%{DATA:job_payload} t=%{NUMBER:job_duration} error=%{DATA:job_error}$" + ] + # Extract the job type from the jobrunner RPC call passed as the 'type' + # URI parameter. We take in account the parameter can be passed + # anywhere, eg as last parameter. + match => [ + "url", + "/rpc/RunJobs.php\?.*?type=%{DATA:job_type}(?:&.*|$)" + ] + named_captures_only => true + } + + } # end [channel] == "runJobs" + if [channel] == "xff" { # Copy XFF addresses from message grok { -- To view, visit https://gerrit.wikimedia.org/r/312504 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5dfa80364aa455725c51afd0e6a37dd4774b94b6 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Hashar <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
