Hi Michael, and David, @david: thanks for your great and timely responses, as always. I really appreciate them, even though I haven't been very vocal about it since the pandemics...
@michael: thanks for your questions and welcome. We highly appreciate your interest. David already pointed you in the right direction. I will now answer in two steps: 1. immediately for the most pressing things 2. a bit later after I added some developer docs #2 is because I think you asked questions that probably many potential contributors have, so I would like to have the answer better accessible. Also make sure they are sufficiently well. I would appreciate feedback on all material and answers. Note that I am so deeply involved for many years with the project that I may overlook the obvious. Just tell me than! Immediate needs: Prometheus: there has already work done already, and native prometheus format is supported by impstats. imhttp now also provides an endpoint for publishing these values. A sidecar container w/ external process is also coming up for native cloud integration. See this PR: https://github.com/rsyslog/rsyslog/pull/5983 Note that we do NOT use JSON arrays - I simple was not aware this also works with Prometheus. --- Workflow: As David said: fork repo and keep it in sync. This is best practice. Right now the development is rapid and the base you work on may otherwise become outdated. Create Pull requests, this will also automatically trigger AI review (usually very useful) and ~40 CI test runs executing each one executing 1000+ tests. Good at catching regression, sometime there are flakes (test the fail albeit there is not real issue). I'll usually baby-sit such flakes. With PR, no harm can be done, I or co-maintainer will always rdo manual review before final merge (so the responsibility ultimately is mine if something goes wrong). --- Likelihood of acceptance: We are highly interested in contributions of all types, be it small or large, code, doc or otherwise. We are also interested to learn use cases and recently in missing feature for cloud-native. Good Feature requests can in a number of cases be addressed by AI workflow for rapid implementation (does not work for large features but increasingly good for small and smallest). Often small things have big effect, so do not be shy. --- Seprate Plugin: definitely not, you took the right approach. Beside prometheus, I think your change makes a lot of sense. Jsut be sure to make it configurable. We insist on backwards compatibility. --- Note: besides the general "rsyslog assitant" AI advisor at https://rsyslog.ai I also work on a developer-focussed AI, experimentally available at https://chatgpt.com/g/g-68ab335c662c8191853b11fe69877f45-ryslog-dev-assistant?model=gpt-5 But be warned: it is in its infancy, the quality is not yet very good. As it looks, I dug a bit deeper than I originally intended, so the part 2 is probably no longer that much ;-) Any questions pls let us know. I would strongly suggest to open either a PR or Feature issue on github - this platform is better suited to to development work,e.g. because of interlinking capabilities. Once again: many thanks for your interest, and this for sure will become an important addition! Rainer El lun, 15 sept 2025 a las 3:58, Redbourne,Michael via rsyslog (<[email protected]>) escribió: > > Hey Folks, > > My advance apologies for the rather beginner questions and a wall of text. > I'm not accustomed to sharing code with enterprise projects that large scale > integrations. I'm also not a C-developer (or really a developer at all beyond > bash scripting). I'm also still testing this to make sure it'll integrate > correctly with my systems (and hopefully others). Once I'm done verifying > this works for both rsyslog (stability, performance, etc.) and my systems, > I'll submit this if it's of interest. > > High Level Overview > We use the impstats module to monitor the performance of rsyslog in our SAM > system. One of the predominant issues we have on two of our servers is our > pipelines are periodically updated to incorporate new pipelines. This is > notable for applications such as Kafka (our use case), where each topic we > connect to has it's own configuration (SAS Key, topic, broker, etc.) The long > standing issue between rsyslog and our SAM system is rsyslog's implementation > of the JSON format. Pre-processed, the implementation of rsyslog's JSON isn't > technically incorrect, but it lacks compatibility with dynamic discovery in > our SAM system. This is because rsyslog outputs a new line for each > statistic, rather than concat'ing the statistics into a single JSON object. > > The way I explained this to GPT-5 was this: > > Mon Sep 15 10:32:45 2025: action-0-builtin:omfile: origin=core.action > processed=4444 failed=0 suspended=0 suspended.duration=0 resumed=0 > [...] > Mon Sep 15 10:32:45 2025: action-3-builtin:omfile: origin=core.action > processed=1111 failed=0 suspended=0 suspended.duration=0 resumed=0 > > should be turned into (these logs aren't identical): > > {"timedate": "Mon Sep 15 10:52:54 2025", "stats": [{ "name": > "action-0-builtin:omfile", "origin": "core.action", "processed": 2641, > "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 },{ > "name": "imtcp(514)", "origin": "imtcp", "submitted": 0 },{ "name": > "w1\/imtcp", "origin": "imtcp", "runs": 0, "read": 0, "emptyread": 0, > "starvation_protect": 0, "accept": 0 }]} > > The difference lies in the stats: [] JSON array. This can be iterated through > in programs like Zabbix to dynamically discover (create) or delete items > automatically, rather than having to manually create new items each time we > update the Kafka pipelines to incorporate new environments. > > To do this, we took the original impstats.c plugin code and added a new > format type called "Prometheus". (Prometheus is a common monitoring > application used in conjunction with something like Grafana, for those who > don't use it.) This keeps the legacy formats intact (legacy, cee, json, > json-es, etc.) allowing backwards compatibility with current installations, > while enabling the new format for individuals seeking this functionality. The > new impstats.c file modifies implements some new functions such as > generatePrometheusStats() > > Again, I still have some testing to do, code to remove (some debug print > statements), etc. But the modified impstats.c file can be seen here > (https://github.com/MRedbourne-BPSI/rsyslog/blob/main/plugins/impstats/impstats.c) > > Note: I know rsyslog is AI-friendly. The prometheus section was generated in > part by GPT-5, with a lot of prompting and describing issues that were > arising (specifically: UAF bugs with elastic strings.) It's been the better > part of a decade since I've done any C-development, and nothing with mutexs, > threads, or anything remotely complex. > > Which brings me to my final questions: > > 1. > Assuming this is stable and performant, what is the procedure to submit this > to rsyslog? I'm assuming I just fork the repository and create a pull request > for impstats.c? > 2. > (Again, assume this is stable/performant): What is the likelihood that this > type of contribution is accepted? It's not a particularly novel approach, and > I haven't seen much discussion around this in the community. > 3. > Does rsyslog have a testing pipeline (unit files, etc.) that might help me > test functionality? I compiled this in the available stable tarball > (8.2508.0) via Rocky EL9. > 4. > Would this be better contributed as an individual plugin (eg: > impstats-prometheus) or as a drop-in replacement for the current impstats > module? > > ________________________________________ > This e-mail communication (including any or all attachments) is intended only > for the use of the person or entity to which it is addressed and may contain > confidential and/or privileged material. If you are not the intended > recipient of this e-mail, any use, review, retransmission, distribution, > dissemination, copying, printing, or other use of, or taking of any action in > reliance upon this e-mail, is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete the original and any > copy of this e-mail and any printout thereof, immediately. If you have any > questions or concerns, please contact our Customer Service Desk at > 1-877-274-2349. Your co-operation is appreciated. > > Le présent courriel (y compris toute pièce jointe) s'adresse uniquement à son > destinataire, qu'il soit une personne ou un organisme, et pourrait comporter > des renseignements privilégiés ou confidentiels. Si vous n'êtes pas le > destinataire du courriel, il est interdit d'utiliser, de revoir, de > retransmettre, de distribuer, de disséminer, de copier ou d'imprimer ce > courriel, d'agir en vous y fiant ou de vous en servir de toute autre façon. > Si vous avez reçu le présent courriel par erreur, prière de communiquer avec > l'expéditeur et d'éliminer l'original du courriel, ainsi que toute copie > électronique ou imprimée de celui-ci, immédiatement. Si vous avez des > questions ou des préoccupations, veuillez contacter notre centre de service à > la clientèle au 1-877-274-2349. Nous sommes reconnaissants de votre > collaboration. > ________________________________________ > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
