Re: [rsyslog] suspend messages (relp)

2018-04-11 Thread deoren
On 4/11/2018 8:24 AM, Plof Jacht via rsyslog wrote: Hello, Hoping this is not too RTFM , i see messages like this and dont know where to start solving: action 'hostname' resumed (module 'omrelp') [v8.32.0 try http://www.rsyslog.com/e/2359 ] action 'hostname' suspended (module 'omrelp'),

[rsyslog] Does liblognorm match against tabs?

2018-03-29 Thread deoren
111.222.333.444 mouse123456 rule=:%datestamp:date-iso%%-:whitespace%%timestamp:time-24hr%%-:whitespace%%event:word%%-:whitespace%%ip:ipv4%%-:whitespace%%auth:word% I tried using both v1 and v2 rules, all with no luck; evidently I'm doing something wrong. As noted, I had to first strip out tab before

[rsyslog] liblognorm error: rulebase file test-log-patterns.rb[55]: invalid record type detected: ']%'

2018-03-27 Thread deoren
I started with the example here: https://github.com/rsyslog/liblognorm-rulebases/blob/master/rules/v2/apache_common.rb and attempted to modify it like so: rule=login_intruder_ip:%[ {"type": "date-iso", "name": "datestamp"}, {"type": "whitespace"}, {"type": "time-24hr", "name":

Re: [rsyslog] Getting Empty Log entry

2018-03-19 Thread deoren
On 3/19/2018 11:49 AM, Carsten Lange via rsyslog wrote: Dear all, currently I am facing an issue with empty LOG entries. I have setup a rsyslog server with TLS receiving events via the internet from a cloud provider. The rsyslog server is behind a load balancer which is doing some NAT. I do

[rsyslog] What is the best way to have a receiver feed a copy of local messages back into itself?

2018-03-13 Thread deoren
Setup: * Latest stable rsyslog from Ubuntu PPA * 50-60 clients, sending to central receiver via omrelp (JSON payloads) * About 5 clients, sending to central receiver via omfwd/tcp (standard syslog) I use a standard "client" configuration for all nodes, including a central receiver that feeds

Re: [rsyslog] Can I specify a single level for a log?

2018-03-13 Thread deoren
On 3/13/2018 12:34 PM, LuKreme wrote: On Mar 13, 2018, at 11:15, deoren <rsyslog-users-lists.adiscon@whyaskwhy.org> wrote: Here is an untested solution using the "advanced" format: Oh my, that is fascinating,. I hadn't come across the advanced format yet, but that doe

Re: [rsyslog] Can I specify a single level for a log?

2018-03-13 Thread deoren
4 then { action(type="omfile" file="/var/log/ftp-error.log") } # Drop all 'ftp' facility messages. By this point those messages # should have already been logged in one of the previously # specified files. stop } Link to file on GitHub: https:

[rsyslog] Can the new option.jsonf="on" template parameter be used to generate numeric JSON field entries?

2018-03-12 Thread deoren
http://www.rsyslog.com/doc/v8-stable/configuration/templates.html#generating-json Example from the docs: template(name="outfmt" type="list" option.jsonf="on") { property(outname="@timestamp" name="timereported" dateFormat="rfc3339" format="jsonf") property(outname="host"

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-11 Thread deoren
On 3/9/2018 1:53 AM, deoren wrote: On 3/9/2018 1:48 AM, Rainer Gerhards wrote: 2018-03-09 8:35 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 3/9/2018 1:24 AM, Rainer Gerhards wrote: 2018-03-09 8:21 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whya

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-11 Thread deoren
On 3/10/2018 5:53 PM, David Lang wrote: On Fri, 9 Mar 2018, Rainer Gerhards wrote: 1) switch forum to read-only 2) mention github as issue tracker (email integration is great) 3) try this e.g. for 3 month If it works out - great. If not, move on to something else. New users will probably not

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-09 Thread deoren
On 3/9/2018 3:10 AM, Rainer Gerhards wrote: So how about this? - do not mention forum any longer - say "github is for questions" - say "SE is experimentally for questions, but less likely to draw answers from team" - keep "mailing list is great to ask questions" update this in doc, site etc.

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-09 Thread deoren
On 3/9/2018 1:53 AM, deoren wrote: On 3/9/2018 1:48 AM, Rainer Gerhards wrote: IMO its not the right thing to send it to the list. Whoever is interested in those questions can subscribe. Actually I am for quite a while. Ah OK, I'm glad to hear I'm wrong. I'll look into subscribing

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-08 Thread deoren
On 3/9/2018 1:48 AM, Rainer Gerhards wrote: 2018-03-09 8:35 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 3/9/2018 1:24 AM, Rainer Gerhards wrote: 2018-03-09 8:21 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 3/9/2018 1:17 AM, Rai

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-08 Thread deoren
On 3/9/2018 1:24 AM, Rainer Gerhards wrote: 2018-03-09 8:21 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 3/9/2018 1:17 AM, Rainer Gerhards wrote: 2018-03-09 4:06 GMT+01:00 David Lang <da...@lang.hm>: I would like to replace the existing forum wi

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-08 Thread deoren
On 3/9/2018 1:17 AM, Rainer Gerhards wrote: 2018-03-09 4:06 GMT+01:00 David Lang : I would like to replace the existing forum with something that ties in to the mailing lists (and/or retire the forum entirely if setting something up that can integrate is too much work) But I'm

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-08 Thread deoren
On 3/8/2018 12:58 AM, Rainer Gerhards wrote: 2018-03-08 7:24 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 2/6/2018 1:56 PM, David Lang wrote: Resurrecting this thread. Any further thoughts regarding retiring the forums? Are we at a point where the rsyslog team

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-03-07 Thread deoren
On 2/6/2018 1:56 PM, David Lang wrote: On Tue, 6 Feb 2018, Simon Lundström wrote: My only negative experience of the rsyslog unpaid community support is github issues where I've submitted and they weren't answered and/or the responses stopped coming after a while. I know that the Github

Re: [rsyslog] optimizer error: we see a NOP, how come?

2018-03-07 Thread deoren
On 2/13/2018 1:04 PM, deoren wrote: On 2/13/2018 1:13 AM, Rainer Gerhards wrote: 2018-02-13 5:12 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: Setup: * Ubuntu 16.04 * ppa:adiscon/v8-devel PPA I was applying patches to an Ubuntu 16.04 test box (which uses the ) an

Re: [rsyslog] What is the syntax for comparing a global variable against a local variable?

2018-03-06 Thread deoren
On 3/6/2018 2:53 AM, Rainer Gerhards wrote: 2018-03-06 9:04 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 3/6/2018 1:43 AM, Rainer Gerhards wrote: 2018-03-05 19:17 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: Hi, When refactor

Re: [rsyslog] What is the syntax for comparing a global variable against a local variable?

2018-03-06 Thread deoren
On 3/6/2018 1:43 AM, Rainer Gerhards wrote: 2018-03-05 19:17 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: Hi, When refactoring an older configuration I figured I would give global variables a try. I'm attempting to create a generic email notification ruleset th

Re: [rsyslog] What is the syntax for comparing a global variable against a local variable?

2018-03-05 Thread deoren
On 3/5/2018 7:48 PM, David Lang wrote: On Mon, 5 Mar 2018, deoren wrote: if $.email-notification!sender == '' then {    $.email-notification!sender = $/default-email-notification!sender; This needs to be set $.email-notification!sender = $/default-email-notification!sender; (you forgot

[rsyslog] What is the syntax for comparing a global variable against a local variable?

2018-03-05 Thread deoren
Hi, When refactoring an older configuration I figured I would give global variables a try. I'm attempting to create a generic email notification ruleset that can be called after setting values in a subtree of the $.email-notification local variable. Example (email templates and associated

Re: [rsyslog] rsyslogd0: action 'action 11' resumed (module 'builtin:omfwd') [try http://www.rsyslog.com/e/0 ]

2018-03-05 Thread deoren
On 3/5/2018 8:09 AM, sophie.loewenthal--- via rsyslog wrote: Dear all, This was a Monday morning moment. Syslog works. However a restart of the daemon earlier caused IP to name resolution is change name so the log files were different. This threw me initially. Can you elaborate on that? I

Re: [rsyslog] Forward Raw TCP?

2018-02-27 Thread deoren
On 2/27/2018 2:40 PM, Naftuli Kay via rsyslog wrote: I am emitting JSON lines using a custom template and attempting to forward them to TCP logs-logstash port 515: https://gist.github.com/naftulikay/47e5f7708cd422f29d97747de0e82869 If I simply "cat blob.json | nc logs-logstash 515", I can get

Re: [rsyslog] Embedding JSON Dictionaries in Templates

2018-02-27 Thread deoren
On 2/27/2018 11:59 AM, Naftuli Kay via rsyslog wrote: These don't seem to address what I am trying to do, but thanks for looking in the right direction :) Welcome. My regular expressions work. I just need to find a way to decode a map into a valid JSON map. Gotcha. I'm afraid that is

Re: [rsyslog] Seeking Help with variables

2018-02-27 Thread deoren
On 2/27/2018 5:39 AM, putcha narayana via rsyslog wrote: Hi I am set the variables and using them in filters. But the following are not working as expected or not working at all. Appreciate if you can share a working example or suggest what i am doing wrong. Not Working: set

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-26 Thread deoren
On 2/27/2018 12:36 AM, David Lang wrote: On Mon, 26 Feb 2018, deoren wrote: you are better using mmnormalize, with your example you would have a rule rule=: %ip:ipv4% - %host:word% [%timestamp:char-to:]%]%-:rest% this would create $!ip, $!host and $!timestamp (note I did this from memory

Re: [rsyslog] Embedding JSON Dictionaries in Templates

2018-02-26 Thread deoren
On 2/26/2018 9:16 PM, Naftuli Kay via rsyslog wrote: I am trying the following: # RSYSLOG IS NOT PCRE COMPLIANT!!! According to this site: http://www.rsyslog.com/regex/ rsyslog uses POSIX ERE (and optionally BRE). If dealing just with re_extract, I found I had to escape the backslash

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-26 Thread deoren
On 2/25/2018 5:37 PM, David Lang wrote: On Fri, 23 Feb 2018, deoren wrote: liblognorm is so fast you really have to use it to believe it. At $lastjob I had a 1400 line ruleset handling >100K logs/sec without the liblognorm effort being noticable Wow, that's pretty impressive. I may

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-26 Thread deoren
On 2/20/2018 6:58 PM, David Lang wrote: On Tue, 20 Feb 2018, deoren wrote: On 2/20/2018 6:39 PM, deoren wrote: I've been attempting to use the re_extract() function quite a bit lately to write some simple "filters" for notification purposes. I struggled with the syntax for a whi

Re: [rsyslog] regex when used in conf files is giving hard-time

2018-02-23 Thread deoren
On 2/23/2018 3:19 AM, putcha narayana via rsyslog wrote: Hello Experts, Any help on my query about regular expression in rsyslog. The code below is not stripping off the .cpp from syslog. if ( $syslogseverity-text != 'debug') then { if re_match($msg, "[a-zA-Z0-9]+\\.cpp:[0-9]+")

Re: [rsyslog] imklog ratelimit on Linux

2018-02-23 Thread deoren
On 2/23/2018 8:35 AM, Berend De Schouwer via rsyslog wrote: Hi, I've recently come across some machines that flooded rsyslog via /proc/kmsg on Linux. This means that printk_ratelimit doesn't apply to all kernel messages. This resulted in >100 GB log in 24 hours, so I added ratelimit to

Re: [rsyslog] Question regarding imfile Input Parameters

2018-02-23 Thread deoren
On 2/22/2018 4:11 AM, putcha narayana wrote: Hi, The changes to the description are clear, no ambiguity now. It is also inline with the text provided for Facility and Severity. I vote for it. Warm Regards Lak. Thanks for your feedback. The changes have been merged. They'll show up on in

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-23 Thread deoren
On 2/21/2018 11:23 PM, matthew.gaetano wrote: Liblognorm is love, Liblognorm is life To Echo Dave, $currentjob uses REK to provided services to various $client at anywhere from 60-80k mps in realtime, plus spikes upwards of over 100k mps. For redundancy (load balancing - waste not want not) we

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-23 Thread deoren
On 2/21/2018 7:02 PM, David Lang wrote: On Wed, 21 Feb 2018, deoren wrote: On 2/20/2018 6:58 PM, David Lang wrote: On 2/20/2018 6:39 PM, deoren wrote: In this case, my specific goal is to look for log messages containing "SPECIFIC_PATTERN_HERE" (as shown in sample l

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-21 Thread deoren
On 2/20/2018 10:28 PM, Andrew Griffin via rsyslog wrote: I’ll second David and say that mmnormalize is your better option. Though whenever I get in a discussion about troubleshooting regex I always make a point to recommend the Regex Rx app (if you’re a Mac user):

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-21 Thread deoren
On 2/20/2018 6:58 PM, David Lang wrote: On 2/20/2018 6:39 PM, deoren wrote: >> I've read that mmnormalize is recommended over regexes for performance reasons, but I have little experience with liblognorm (other than knowing it exists). Am I better off writing a few regex matches lik

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-21 Thread deoren
On 2/20/2018 6:50 PM, David Lang wrote: you really should look at using mmnormalize to extract fields from the logs, it's FAR faster. Will do. I was looking over the liblognorm doc last night and it makes a little sense. The v2 options look to have expanded the support quite a bit, at the

Re: [rsyslog] Question regarding imfile Input Parameters

2018-02-21 Thread deoren
What do you think of these potential changes to the description? https://github.com/rsyslog/rsyslog-doc/pull/584/files Does that make the coverage any clearer, or worse? On 2/21/2018 3:20 AM, putcha narayana via rsyslog wrote: Thank you David Lang for a quick response. Appreciate it. Lak.

Re: [rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-20 Thread deoren
On 2/20/2018 6:39 PM, deoren wrote: I've been attempting to use the re_extract() function quite a bit lately to write some simple "filters" for notification purposes. I struggled with the syntax for a while until I realized tha the  and have been struggling quite a bit with the reg

[rsyslog] Anyone have any good guides for the specific regex format/syntax required for re_extract() ?

2018-02-20 Thread deoren
I've been attempting to use the re_extract() function quite a bit lately to write some simple "filters" for notification purposes. I struggled with the syntax for a while until I realized tha the and have been struggling quite a bit with the regex support for the re_extract() function.

Re: [rsyslog] Forward messages from rsyslog server to JSON elasticSeach connector

2018-02-19 Thread deoren
On 2/19/2018 10:17 AM, sophie.loewenthal--- via rsyslog wrote: Thank you Deoren for your thoughts. Welcome. Hopefully others will chime in with more details. I've seen some junk hostnames already appear in the logging directory. Thanks for your explanation. I can create an IP to Hostname

Re: [rsyslog] Rsyslog: how do I override the hostname when forwarding log messages?

2018-02-19 Thread deoren
On 2/19/2018 9:26 AM, deoren wrote: On 2/19/2018 8:52 AM, Graham Leggett via rsyslog wrote: Hi all, I have a number of java services that include support for logging to syslog, but unfortunately they can only log by sending udp packets to port 514. This is not in itself a problem, however

Re: [rsyslog] Forward messages from rsyslog server to JSON elasticSeach connector

2018-02-19 Thread deoren
On 2/19/2018 9:29 AM, sophie.loewenthal--- via rsyslog wrote: Hi, Does this configuration look ok begore I let this configuration rip in production? A server running rsyslog 8.7.4 on Solaris 11 that receives TCP and UDP messages from a mixture of syslog and rsyslog clients . Each client

Re: [rsyslog] central syslog and cisco device hostnames

2018-02-19 Thread deoren
On 2/16/2018 3:56 PM, John Ratliff wrote: When my rsyslog server receives packets from our cisco switches, instead of logging it with the hostname, it logs it with the IP address. How can I get rsyslog to use the hostname instead? See the "how do I override the hostname when forwarding log

Re: [rsyslog] Rsyslog: how do I override the hostname when forwarding log messages?

2018-02-19 Thread deoren
On 2/19/2018 8:52 AM, Graham Leggett via rsyslog wrote: Hi all, I have a number of java services that include support for logging to syslog, but unfortunately they can only log by sending udp packets to port 514. This is not in itself a problem, however these services have no stable

Re: [rsyslog] Fighting with re_extract, not going well

2018-02-16 Thread deoren
On 2/16/2018 1:15 PM, deoren wrote: Hi all, Can someone familiar with re_extract point out what I'm doing wrong? I have this message: Server bk_postfix/relay5 is UP/READY (leaving forced maintenance). that I'm attempting to match on like so: set $.relayserver = re_extract($msg,     "S

[rsyslog] Fighting with re_extract, not going well

2018-02-16 Thread deoren
Hi all, Can someone familiar with re_extract point out what I'm doing wrong? I have this message: Server bk_postfix/relay5 is UP/READY (leaving forced maintenance). that I'm attempting to match on like so: set $.relayserver = re_extract($msg, "Server bk_postfix\\/([0-9A-Za-z]+)", 0,

Re: [rsyslog] optimizer error: we see a NOP, how come?

2018-02-13 Thread deoren
On 2/13/2018 1:13 AM, Rainer Gerhards wrote: 2018-02-13 5:12 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: Setup: * Ubuntu 16.04 * ppa:adiscon/v8-devel PPA I was applying patches to an Ubuntu 16.04 test box (which uses the ) and just happened to spot check the /v

[rsyslog] optimizer error: we see a NOP, how come?

2018-02-12 Thread deoren
Setup: * Ubuntu 16.04 * ppa:adiscon/v8-devel PPA I was applying patches to an Ubuntu 16.04 test box (which uses the ) and just happened to spot check the /var/log/rsyslog.log when I saw that error. When I run a validation check I get the following: root@sawmill3:/var/log# rsyslogd -N2

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-02-11 Thread deoren
On 2/5/2018 4:46 AM, Simon Lundström wrote: That's great! I was trying to make a point, but failed apparently, that the docs that the original poster thinks are unclear which creates frequently asked questions should be updated, not a FAQ article. Though some questions have no natural place

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-02-11 Thread deoren
On 2/4/2018 10:46 PM, David Lang wrote: * Forums are shut down and visitors are directed to Stack Exchange/Overflow/whatever instead. It would appear there is already solid participation there for questions tagged with rsyslog: https://stackoverflow.com/questions/tagged/rsyslog This is

Re: [rsyslog] Help select a new logo

2018-02-02 Thread deoren
@Rainer +1 for logo 1 out of the provided options (also voted using the provided poll) On 2/2/2018 1:27 AM, Ciprian Hacman wrote: > Nice. Logo 1 from me also (voted). Seems the cleanest one. > > Ciprian > > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch

Re: [rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-02-02 Thread deoren
On 2/2/2018 5:41 AM, Simon Lundström wrote: Thank you for your feedback! > I like mailinglists and IRC but the most important for me is that the > questions are answered, be it by employees or the community I completely agree with this. Seeing how spread out the current community is between

Re: [rsyslog] rsyslog+elasticsearch (and some Kafka and a few others) eBook

2018-02-01 Thread deoren
On 2/1/2018 6:27 AM, Radu Gheorghe wrote: Hi, Today we just published what I hope to be a quite complete eBook about centralizing logs with rsyslog. The destination I had in mind was Elasticsearch, but I think it should apply to many other use-cases. Here's the blog post with more details on

[rsyslog] POLL (informal): What are your thoughts regarding current and potential rsyslog support channels?

2018-01-31 Thread deoren
Hi, ## Forum support requests ## I'd like to kick start some discussion around ways that we may better support users seeking help, not those who are reporting bugs (perceived or genuine). In particular, I have noticed the level of inactivity on the

Re: [rsyslog] Can a single logfile be part of multiple imfile configs?

2018-01-26 Thread deoren
[mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of deoren Sent: Friday, January 26, 2018 4:36 PM To: rsyslog@lists.adiscon.com Subject: Re: [rsyslog] Can a single logfile be part of multiple imfile configs? On 1/26/2018 3:29 PM, Scot Kreienkamp wrote: Hi everyone, My basic question: Can the same logfile

Re: [rsyslog] Can a single logfile be part of multiple imfile configs?

2018-01-26 Thread deoren
On 1/26/2018 3:29 PM, Scot Kreienkamp wrote: Hi everyone, My basic question: Can the same logfile be used in two imfile inputs? There may be other ways of doing this, but this comes to mind: 1. A single input object which specifies the file you want to monitor. 2. Attach a single ruleset to

Re: [rsyslog] Is there an advanced/current/RainerScript version of the obsolete legacy ActionFileDefaultTemplate directive?

2018-01-16 Thread deoren
On 1/12/2018 2:43 PM, deoren wrote: I'm looking through the docs and I haven't spotted it. I think I answered my own question: module(load="builtin:omfile" template="RSYSLOG_FileFormat") or just: module(load="builtin:omfile") if I am fine with the default

[rsyslog] Is there an advanced/current/RainerScript version of the obsolete legacy ActionFileDefaultTemplate directive?

2018-01-12 Thread deoren
I'm looking through the docs and I haven't spotted it. Thanks. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE

Re: [rsyslog] Request: Feedback on parameter formats for rsyslog module docs

2018-01-08 Thread deoren
On 1/8/2018 9:22 AM, deoren wrote: On 1/8/2018 9:11 AM, Andrew Griffin via rsyslog wrote: This looks great, I love it! Can someone refresh my memory on the process for contributing to the documentation?  I’d like to chip in Andrew Griffin Thanks for the feedback. Do I understand your

Re: [rsyslog] Request: Feedback on parameter formats for rsyslog module docs

2018-01-08 Thread deoren
On 1/8/2018 3:30 AM, Simon Lundström wrote: On Sun, 2018-01-07 at 23:56:27 -0600, deoren wrote: Hi all, I'd like to get your feedback on some proposed formatting changes to the imuxsock module doc. I mention my preference below, but the end goal is to standardize the formatting and make

Re: [rsyslog] Request: Feedback on parameter formats for rsyslog module docs

2018-01-08 Thread deoren
s in place. Everything from typos, to clarifications of content to adding missing coverage. I'm still learning the ins/outs of everything (including Git), but I'd be happy to answer any questions that I'm able to. Just mention me (e.g., "@deoren") on whatever GitHub issue/PR you work o

[rsyslog] Request: Feedback on parameter formats for rsyslog module docs

2018-01-07 Thread deoren
Hi all, I'd like to get your feedback on some proposed formatting changes to the imuxsock module doc. I mention my preference below, but the end goal is to standardize the formatting and make the docs easier to work with, so my personal preferences can take a backseat in the scheme of things. ;)

Re: [rsyslog] Error with omkafka

2017-12-23 Thread deoren
On 12/21/2017 1:10 PM, Andrew Akins via rsyslog wrote: > I was wondering if anyone had seen this. I’m running rsyslog on Alpine Linux, > deployed as a container. Rsyslog was built from source, and is version 8.31.0 > > Basically, initialization of a omkafka message is failing: > >

Re: [rsyslog] Imkafka/omkafka tuning

2017-12-23 Thread deoren
On 12/22/2017 9:52 AM, Luigi Tagliamonte via rsyslog wrote: > Hi there! > What are the tunable parameters for this module, like: > - an option to increase the number of threads for kafka processing > - number of messages to process per req. > - etc.. > Regards > L. Module docs: *

Re: [rsyslog] klogLocalIPIF

2017-12-21 Thread deoren
On 12/20/2017 4:24 AM, deoren wrote: > On 12/14/2017 8:19 PM, Rory Toma wrote: >> I have put in >> $ModLoad imklog >> $klogLocalIPIF eth0 >> >> in my rsyslog.conf file (8.30.0) However, >> rsyslogd: invalid or yet-unknown config file command 'klogLoca

Re: [rsyslog] klogLocalIPIF

2017-12-20 Thread deoren
On 12/14/2017 8:19 PM, Rory Toma wrote: I have put in $ModLoad imklog $klogLocalIPIF eth0 in my rsyslog.conf file (8.30.0) However, rsyslogd: invalid or yet-unknown config file command 'klogLocalIPIF' - have you forgotten to load a module? [v8.30.0 try http://www.rsyslog.com/e/3003 ]

[rsyslog] Who manages the Rsyslog documentation on Read the Docs?

2017-12-06 Thread deoren
Is this something that the Rsyslog team manages or is it someone at the Read the Docs team that manages the content? I ask because it appears that the version of the docs (stable, latest) available there are outdated and I wanted to make sure to report the issue to the correct place.

Re: [rsyslog] imuxsock module documentation: What exactly does the 'SysDock.Name' parameter do?

2017-11-30 Thread deoren
On 11/30/2017 5:28 PM, deoren wrote: Is that parameter used to specify the replacement for /dev/log or is that parameter used to specify another socket that is in addition to /dev/log as an input source? I assume that SysSock.Use defaults to /dev/log, but if SysSock.Name is specified, does

[rsyslog] imuxsock module documentation: What exactly does the 'SysDock.Name' parameter do?

2017-11-30 Thread deoren
Is that parameter used to specify the replacement for /dev/log or is that parameter used to specify another socket that is in addition to /dev/log as an input source? I assume that SysSock.Use defaults to /dev/log, but if SysSock.Name is specified, does SysSock.Use now refer to using that

Re: [rsyslog] Are configuration parameters case sensitive?

2017-11-17 Thread deoren
On 11/17/2017 11:05 AM, Rainer Gerhards wrote: 2017-11-17 18:04 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: I noticed this commit focused on fixing the case in the source code for comparison purposes: https://github.com/rgerhards/rsyslog/

[rsyslog] Are configuration parameters case sensitive?

2017-11-17 Thread deoren
I noticed this commit focused on fixing the case in the source code for comparison purposes: https://github.com/rgerhards/rsyslog/commit/b9cda4602b26a4778fdfec4990a62b6faf2bc86b which leads me to ask: Are configuration parameters case sensitive? For example, are these all equivalent? global

Re: [rsyslog] rsyslog status ABRT or SEGV

2017-11-14 Thread deoren
On November 14, 2017 10:49:06 PM CST, "Войнович Андрей Александрович via rsyslog" wrote: >Thank you, David > >We have upgraded our linux box to the latest available (Debian 9) and >now rsyslog version is 8.24 (the newest from deb repo), but we still >experience the

Re: [rsyslog] Ubuntu 16 rsyslogd not creating log files

2017-11-10 Thread deoren
On 11/10/2017 9:33 AM, dchappelle via rsyslog wrote: Thanks for all of the info deoren. I do have the file you speak of installed on my system: dchappelle@L164:~$ cat /usr/lib/tmpfiles.d/00rsyslog.conf # Override systemd's default tmpfiles.d/var.conf to make /var/log writable

Re: [rsyslog] Ubuntu 16 rsyslogd not creating log files

2017-11-09 Thread deoren
On November 9, 2017 10:21:04 PM CST, dchappelle via rsyslog wrote: >Apologies for not including the config. Here is >/etc/rsyslog.d/10-example.conf: > >dchappelle@L164:/etc/rsyslog.d$ cat 10-example.conf >local0.* /var/log/test.log

Re: [rsyslog] Ubuntu 16 rsyslogd not creating log files

2017-11-09 Thread deoren
On November 9, 2017 6:47:11 PM CST, dchappelle via rsyslog wrote: >I am running a vanilla install of Ubuntu 16 and my rsyslogd is not >creating >new log files for me. I added a new filter rule and restarted rsyslogd. >After doing so and generating log messages for

Re: [rsyslog] preserving metadata on message split

2017-11-09 Thread deoren
which actually have set it to a couple of MB (and occasionally use it). Rainer 2017-11-09 17:25 GMT+01:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: On 11/9/2017 10:24 AM, Scot Kreienkamp wrote: I have it set at 128k now... I thought I read in the list archives that was

Re: [rsyslog] preserving metadata on message split

2017-11-09 Thread deoren
On 11/9/2017 10:24 AM, Scot Kreienkamp wrote: I have it set at 128k now... I thought I read in the list archives that was the maximium value? https://github.com/rsyslog/rsyslog/issues/1741 Looks like it (for now). ___ rsyslog mailing list

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-11-09 Thread deoren
On 11/9/2017 4:08 AM, Thomas Deutschmann via rsyslog wrote: Hi, no distribution will probably _require_ network for rsyslog per default because in the default configuration distributions are shipping, no network is required. Due to the fact that most init systems nowadays support parallel

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-11-07 Thread deoren
On 11/7/2017 12:25 PM, deoren wrote: On 11/7/2017 10:31 AM, matthew.gaetano wrote: With the exception of the relation to storage, yes, for the most part. We encountered the issue on a physical server using SCSI/SATA drives. Our secondary tester were in vmware. I initially emphasized the boot

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-11-07 Thread deoren
On 11/7/2017 10:31 AM, matthew.gaetano wrote: With the exception of the relation to storage, yes, for the most part. We encountered the issue on a physical server using SCSI/SATA drives. Our secondary tester were in vmware. I initially emphasized the boot speed from running the Ubuntu 16.04 VM

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-11-07 Thread deoren
https://github.com/rsyslog/rsyslog/issues/1656 See if that matches what you are fighting with. On November 7, 2017 9:51:51 AM CST, "matthew.gaetano" wrote: >Queue's aside, regardless of the order rsyslog loads (before or after >network) its retry function should not

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-10-31 Thread deoren
On 10/31/2017 4:05 PM, matthew.gaetano wrote: Seems like your on the right track. We changed the dns names in the conf to the destination IPs and this somewhat resolved the issue. Rsyslog would still suspend the two destination actions however once the system settled the actions were resumed.

Re: [rsyslog] Legacy FWD Failes on startup (v8)

2017-10-31 Thread deoren
On 10/31/2017 12:42 PM, matthew.gaetano wrote: Hello, I'm not sure this is an issue considering Legacy format shouldn't really be used in version 8, however it seems that when using legacy forwarding (as described in the default rsyslog.conf file) rsyslog suspends the actions and never retries.

Re: [rsyslog] How is the imrelp MaxDataSize parameter related to the global() maxMessageSize parameter?

2017-10-29 Thread deoren
parameter messages that were previously "stuck", flow once more. On 10/29/2017 1:22 PM, Rainer Gerhards wrote: quick answer: I guess you ran into this https://github.com/rsyslog/rsyslog/issues/1741 Let me know if more info is needed. Rainer 2017-10-29 19:15 GMT+01:00 deoren <

[rsyslog] How is the imrelp MaxDataSize parameter related to the global() maxMessageSize parameter?

2017-10-29 Thread deoren
I originally sent this as part of another thread, but I think this got buried and lost among the noise the rest of my notes generated. Posting a cleaner version here in case others know the answer. I'm trying to avoid using legacy configuration options where I can, but just in case the order

Re: [rsyslog] Qualys scan against rsyslog causes it to segfault

2017-10-27 Thread deoren
On 10/23/2017 7:55 PM, deoren wrote: On 10/23/2017 7:51 PM, deoren wrote: On 10/23/2017 7:38 PM, deoren wrote: On 10/23/2017 7:11 PM, David Lang wrote: do you have a tcpdump or info from Qualys saying what it sends as part of the scan? David Lang Thankfully (for troubleshooting purposes

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

2017-10-27 Thread deoren
On 10/27/2017 5:19 PM, Naftuli Kay via rsyslog wrote: Can anyone shed any light on how to set global variables? Environment variables won't change over the lifetime of the process so it would make sense to not have to allocate for every log message. Thanks, - Naftuli Kay I've not used them

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

2017-10-25 Thread deoren
On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote: So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then reference this variable? I'm still trying to learn more about rsyslog variables and how to use them in templates. I'm still learning myself, so I completely

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

2017-10-25 Thread deoren
On 10/25/2017 3:48 PM, Naftuli Kay via rsyslog wrote: I have a few environment variables that I'd like to include in my log messages that I'm formatting in JSON format. I have a service that runs on boot which generates /etc/sysconfig/ec2 which contains variables like EC2_INSTANCE_ID,

Re: [rsyslog] If messages are stuck in a queue, do you have any option other than nuking the queue file(s)?

2017-10-24 Thread deoren
On 10/19/2017 6:58 PM, deoren wrote: On 10/19/2017 3:12 PM, Rainer Gerhards wrote: Am 19.10.2017 21:55 schrieb "David Lang" <da...@lang.hm>: RELP has it's place, but most of the time I'm willing to loose some logs under rare failure conditions and so haven't bothered to

Re: [rsyslog] Qualys scan against rsyslog causes it to segfault

2017-10-23 Thread deoren
On 10/23/2017 7:51 PM, deoren wrote: On 10/23/2017 7:38 PM, deoren wrote: On 10/23/2017 7:11 PM, David Lang wrote: do you have a tcpdump or info from Qualys saying what it sends as part of the scan? David Lang Thankfully (for troubleshooting purposes), the problem isn't specific

Re: [rsyslog] Qualys scan against rsyslog causes it to segfault

2017-10-23 Thread deoren
On 10/23/2017 7:38 PM, deoren wrote: On 10/23/2017 7:11 PM, David Lang wrote: do you have a tcpdump or info from Qualys saying what it sends as part of the scan? David Lang Thankfully (for troubleshooting purposes), the problem isn't specific to the Qualys scan. I later learned

Re: [rsyslog] Qualys scan against rsyslog causes it to segfault

2017-10-23 Thread deoren
On 10/23/2017 7:11 PM, David Lang wrote: do you have a tcpdump or info from Qualys saying what it sends as part of the scan? David Lang Thankfully (for troubleshooting purposes), the problem isn't specific to the Qualys scan. I later learned that messages coming from our ESXi hosts

Re: [rsyslog] Qualys scan against rsyslog causes it to segfault

2017-10-23 Thread deoren
On 10/7/2017 10:44 AM, deoren wrote: On 10/7/2017 5:25 AM, Rainer Gerhards wrote: 2017-10-07 7:57 GMT+02:00 deoren <rsyslog-users-lists.adiscon@whyaskwhy.org>: As I dig more into this, I'm beginning to think the only thing the Qualys scan did was aggravate an existing problem and

[rsyslog] What is the expected behavior when checking non-existent variable in the $! object?

2017-10-22 Thread deoren
This is a tangent of another issue I was dealing with a few weeks back, but it appears that problem was related to checking whether a non-existent $!variable was empty. Is the expected behavior for that check to fail? This is with v8.29.0 and I have not tested with 8.30.0 yet, but I am more

Re: [rsyslog] If messages are stuck in a queue, do you have any option other than nuking the queue file(s)?

2017-10-19 Thread deoren
On 10/19/2017 3:12 PM, Rainer Gerhards wrote: Am 19.10.2017 21:55 schrieb "David Lang" : RELP has it's place, but most of the time I'm willing to loose some logs under rare failure conditions and so haven't bothered to use it. large maxmessagesize leads to wasted memory in

Re: [rsyslog] If messages are stuck in a queue, do you have any option other than nuking the queue file(s)?

2017-10-19 Thread deoren
On 10/18/2017 8:10 PM, David Lang wrote: On Wed, 18 Oct 2017, deoren wrote: On 10/18/2017 3:15 PM, David Lang wrote: On Wed, 18 Oct 2017, deoren wrote: On 10/18/2017 1:36 PM, David Lang wrote: On Wed, 18 Oct 2017, deoren wrote: Since the sender and receiver in this are both the latest

Re: [rsyslog] If messages are stuck in a queue, do you have any option other than nuking the queue file(s)?

2017-10-18 Thread deoren
On 10/18/2017 3:15 PM, David Lang wrote: On Wed, 18 Oct 2017, deoren wrote: On 10/18/2017 1:36 PM, David Lang wrote: On Wed, 18 Oct 2017, deoren wrote: Since the sender and receiver in this are both the latest versions of rsyslog (with the plan for the setup to remain that way), can I scale

  1   2   >