Re: [asterisk-dev] Journald support for Asterisk
I've two remarks about that: 1. Even if a journald support is one day included in Asterisk, it will be for Asterisk 14 or 15 at least. The time you have that as stable version, it will be 2016 or 2017. Even if we already have some few CentOS on production because it's mandatory for some products, we stopped to install new CentOS 6 a long time ago. At least for us, when we want to have a new version of Asterisk, we install first a new VM with the latest stable version, in this case, CentOS 7, because it's very easier to setup a new VM, migrate data, and finally switch DNS records, instead of to cut the service. 2. Moreover, the support of journald doesn't mean you must drop syslog logging: I don't know how the standard logging mechanism works in C, but in Python: a. You log has usual your message with logging.info(), and you add a parameter extra with the metadata/dict of the log. b. You have a list of handlers already included in Python to log in syslog, console, file...: https://docs.python.org/3.4/library/logging.handlers.html#module-logging.handlers Most handlers don't support extra parameter, its drop extra data. For handlers that support that like journald, it's included. Ludovic Gasc (GMLudo) http://www.gmludo.eu/ On 12 May 2015 07:25, Corey Farrell g...@cfware.com wrote: CentOS/RHEL 6 are under production support until Nov 2020 [1] and journald is not available. Even if journald could improve Asterisk logging, that's not an acceptable reason to force Asterisk users to upgrade from a supported OS. If there was a module that allowed Asterisk logging to talk to journald I wouldn't care, but it should not replace the logging system or be required to run Asterisk. [1] http://wiki.centos.org/FAQ/General#head-fe8a0be91ee3e7dea812e8694491e1dde5b75e6d On Sun, May 10, 2015 at 5:22 PM, Ludovic Gasc gml...@gmail.com wrote: 2015-05-10 20:46 GMT+02:00 Bruce Ferrell bferr...@baywinds.org: On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from
Re: [asterisk-dev] Journald support for Asterisk
2015-05-12 13:48 GMT+02:00 Tzafrir Cohen tzafrir.co...@xorcom.com:
On Mon, May 11, 2015 at 11:26:31PM +0200, Ludovic Gasc wrote:
The idea behind structured log is to retrieve easily a context with a log
to reduce the effort of categorization after, because you don't need to
parse and recognize patterns in log message.
Technically, instead of to have a String, you have a Dict of Strings.
That's nice in theory. Could you please give an example of the required
output? An example of how it would help?
For now, this is the situation to log for most applications:
1. In your source code, where you want to log, you aggregate data from
several variables to generate a log message. Example:
logger.info('This is a call from %s with the uniqueid %s and
callerid %s to call %s extension in %s context', endpoint, uniqueid,
callerid, extension, context)
2. This message is stored in the text format in /var/log/asterisk/debug.log
3. For example, to retrieve all logs about a specific extension, the
sysadmin needs to use: grep -i \%s\
extension /var/log/asterisk/debug.log or write regex for a log analysis
tool, with a risk of false positive or miss something.
At the beginning, in your source code, you have structured data, you encode
as string for log.
After, to get metadata in logs, you need to decode your logs with regex to
retrieve structured data to query inside.
The final situation should be:
logger.info('This is a call from %(endpoint)s with the uniqueid
%(uniqueid)s and callerid %(callerid)s to call %(extension)s
extension in %(context)s context', {'endpoint': endpoint, 'uniqueid':
uniqueid, 'callerid': callerid, 'extension': extension, 'context': context})
And after, depends on the handler:
a. For syslog, you use the dict only to replace values in string
b. For journald, you push also the dict with the log message. With
that, you don't need to encode/decode, you retrieve directly structured
data included in message log.
And yes, ideally, it should be useful if we can generate log messages with
the structured data directly in the dialplan.
BTW, you just give me an interesting idea to have this feature easily: I
can make a FastAGI daemon that generate my journald message, based on data
pushed as parameter with a JSON dict structure.
Not really funny to write the dialplan source code, however, it will work.
If it isn't enough concrete for you, I can give you a concrete example with
a click2call HTTP endpoint, where the order starts from WebBrowser, to
finish in asterisk.
--
Tzafrir Cohen
icq#16849755 jabber:tzafrir.co...@xorcom.com
+972-50-7952406 mailto:tzafrir.co...@xorcom.com
http://www.xorcom.com
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-dev
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
On Mon, May 11, 2015 at 11:26:31PM +0200, Ludovic Gasc wrote: The idea behind structured log is to retrieve easily a context with a log to reduce the effort of categorization after, because you don't need to parse and recognize patterns in log message. Technically, instead of to have a String, you have a Dict of Strings. That's nice in theory. Could you please give an example of the required output? An example of how it would help? -- Tzafrir Cohen icq#16849755 jabber:tzafrir.co...@xorcom.com +972-50-7952406 mailto:tzafrir.co...@xorcom.com http://www.xorcom.com -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
On Sat, May 9, 2015 at 9:15 AM, Bruce Ferrell bferr...@baywinds.org wrote: systemd and journald SUCK to high heaven. I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. You're welcome to your opinion, but I find systemd/journald very nice. I'd welcome having Asterisk log directly to journald as an option. I wouldn't support dropping other logging methods because there are systems that don't have journald, or their admins would prefer some other logging method. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. The Apache httpd mod_journald is very new so it's not surprising that there are bugs. It also breaks fail2ban site security. As a point of fact, journald support was added to fail2ban over a year ago, and works quite well. -- Jeff Ollie -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
2015-05-11 0:17 GMT+02:00 Bruce Ferrell bferr...@baywinds.org: On 05/10/2015 02:22 PM, Ludovic Gasc wrote: 2015-05-10 20:46 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto: bferr...@baywinds.org: On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org mailto:bferr...@baywinds.org mailto: bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org http://wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from the apache build. Correct operation was now observed Interesting, it seems like a regression bug. The question now is the patchs from opensuse has broken this module, or the bug is in Apache itself. The bug seems to be too obvious to be present in Apache source code, but who knows ? The response to the bug report was Use Nginex To be really honest with you, I've thought that when you explain your problem, I've dropped Apache on our production a long time ago for efficiency issues. Please do NOT do this Why ? If it works for me, what's the problem for you ? Systemd integration has been insidious. Should you wish to discuss further, please off this list. It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 1.) I don't know of any asterisk systems that use any form of system logging. It can, and the fact that you keep bringing syslog logging
Re: [asterisk-dev] Journald support for Asterisk
CentOS/RHEL 6 are under production support until Nov 2020 [1] and journald is not available. Even if journald could improve Asterisk logging, that's not an acceptable reason to force Asterisk users to upgrade from a supported OS. If there was a module that allowed Asterisk logging to talk to journald I wouldn't care, but it should not replace the logging system or be required to run Asterisk. [1] http://wiki.centos.org/FAQ/General#head-fe8a0be91ee3e7dea812e8694491e1dde5b75e6d On Sun, May 10, 2015 at 5:22 PM, Ludovic Gasc gml...@gmail.com wrote: 2015-05-10 20:46 GMT+02:00 Bruce Ferrell bferr...@baywinds.org: On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from the apache build. Correct operation was now observed Interesting, it seems like a regression bug. The question now is the patchs from opensuse has broken this module, or the bug is in Apache itself. The bug seems to be too obvious to be present in Apache source code, but who knows ? The response to the bug report was Use Nginex To be really honest with you, I've thought that when you explain your problem, I've dropped Apache on our production a long time ago for efficiency issues. Please do NOT do this Why ? If it works for me, what's the problem for you ? Systemd integration has been insidious. Should you wish to discuss further, please off this list. It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 1.) I don't know of any asterisk systems that use any form of system logging. It can, and the fact that you keep bringing syslog logging into this indicates a lack of understanding of asterisk usage
Re: [asterisk-dev] Journald support for Asterisk
2015-05-10 20:46 GMT+02:00 Bruce Ferrell bferr...@baywinds.org: On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto: bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from the apache build. Correct operation was now observed Interesting, it seems like a regression bug. The question now is the patchs from opensuse has broken this module, or the bug is in Apache itself. The bug seems to be too obvious to be present in Apache source code, but who knows ? The response to the bug report was Use Nginex To be really honest with you, I've thought that when you explain your problem, I've dropped Apache on our production a long time ago for efficiency issues. Please do NOT do this Why ? If it works for me, what's the problem for you ? Systemd integration has been insidious. Should you wish to discuss further, please off this list. It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 1.) I don't know of any asterisk systems that use any form of system logging. It can, and the fact that you keep bringing syslog logging into this indicates a lack of understanding of asterisk usage on your part. 2.) Not released code and still under test. This is disingenuous, at best. 3.) Just because everyone else is jumping off of bridges isn't a good enough reason to do it. It's NOT broken and this is NOT an improvement. Concretely, we have an infrastructure that for now handles more than 3 millions of SIP calls by month with a dedicated telco team for that, and we're continuing to grow. For now, even with voipmonitor and homer, it's a nightmare for our telco team to debug that. One part of the problem is because we have an heterogeneous
Re: [asterisk-dev] Journald support for Asterisk
On 05/10/2015 02:22 PM, Ludovic Gasc wrote: 2015-05-10 20:46 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org: On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org mailto:bferr...@baywinds.org mailto:bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org http://wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from the apache build. Correct operation was now observed Interesting, it seems like a regression bug. The question now is the patchs from opensuse has broken this module, or the bug is in Apache itself. The bug seems to be too obvious to be present in Apache source code, but who knows ? The response to the bug report was Use Nginex To be really honest with you, I've thought that when you explain your problem, I've dropped Apache on our production a long time ago for efficiency issues. Please do NOT do this Why ? If it works for me, what's the problem for you ? Systemd integration has been insidious. Should you wish to discuss further, please off this list. It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 1.) I don't know of any asterisk systems that use any form of system logging. It can, and the fact that you keep bringing syslog logging into this indicates a lack of understanding of asterisk usage on your part. 2.) Not released code and still under
Re: [asterisk-dev] Journald support for Asterisk
On 05/09/2015 01:53 PM, Ludovic Gasc wrote: 2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org mailto:bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? I would suggest you try it for yourself. Procedure to reproduce: Perform fresh install of OpenSUSE 13.2 assure apache/php/MySQL support are installed install wordpress from wordpress.org Observe CSS is NOT correctly served. I did this several times to assure my findings. I rebuilt the opensuse rpms from the source rpms to omit systemd/journald from the apache build. Correct operation was now observed The response to the bug report was Use Nginex Please do NOT do this Why ? If it works for me, what's the problem for you ? Systemd integration has been insidious. Should you wish to discuss further, please off this list. It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 1.) I don't know of any asterisk systems that use any form of system logging. It can, and the fact that you keep bringing syslog logging into this indicates a lack of understanding of asterisk usage on your part. 2.) Not released code and still under test. This is disingenuous, at best. 3.) Just because everyone else is jumping off of bridges isn't a good enough reason to do it. It's NOT broken and this is NOT an improvement. Asterisk works well now. It integrates easily. For this point, I'm agree with you, it's very important to continue to be integrated easily, I use that a lot. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
2015-05-09 16:52 GMT+02:00 Matthew Jordan mjor...@digium.com:
On Sat, May 9, 2015 at 9:15 AM, Bruce Ferrell bferr...@baywinds.org
wrote:
On 05/09/2015 01:17 AM, Ludovic Gasc wrote:
Hi,
Systemd and Journald is now by default on Debian Jessie and Ubuntu
15.04, as on RHEL/CentOS.
Journald supports syslog format, nevertheless, at least for us, the
structured log system provided with journald helps us to debug the
production.
The idea behind that is to attach metadata with a log line to
facilitate the search with journalctl, you can write queries to find the
errors.
For example, with Apache:
http://httpd.apache.org/docs/trunk/mod/mod_journald.html
For our Python daemons, for each log line, we store account_id,
request_id, endpoint and method used to be easy to retrieve quickly
interesting logs.
Moreover, not yet used for us, but you can generate statistics about
your source code real usage based
on:
http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE=
For Asterisk, for example with dialplan logging, you should attach the
context, the extension and the channel.
You can simulate this journald feature with a specific format message
for your logs, nevertheless, journalctl is more user-friendly to retrieve
pertinent logs compare to the
classical grep usage.
ave no idea if i
I'm permitted to raise the question about an eventual journald support
in Asterisk because I don't find anything about that in issues tracker nor
mailing list.
Moreover, I understand that even if you add the journald support, it's
certainly necessary to change logging everywhere in Asterisk, however, I
should help to do that.
Regards
--
Ludovic Gasc (GMLudo)
http://www.gmludo.eu/
systemd and journald SUCK to high heaven. I have no idea if the issues
I've had with them (OpenSUSE) are distro related or inherent.
I have seen that the implementation of apache with a static
systemd/journald module will no longer correctly serve content.
Please do NOT do this. It also breaks fail2ban site security. Asterisk
works well now. It integrates easily.
Being very honest, my knowledge of systemd and journald is not
extensive. I have no real position on systemd/journald one way or the
other. I will say that I do not view it as an inevitable conclusion,
nor a necessary requirement for Asterisk. As such, I'm viewing this
only as a proposal for we'd like to change the vast majority of log
statements in Asterisk.
First, as Bruno pointed out, a lot of external systems rely on parsing
log messages. Asterisk does not exist in a vacuum, and is often a
component in a much larger system. As such, any modification to
Asterisk's logging would have to be configurable.
The idea is to add metadata to logs, not to change logs mechanism, like
with Python logging where you have an extra parameter to give more
details.
If your handler like syslog doesn't handle metadata, it isn't used, example
with Python:
http://www.freedesktop.org/software/systemd/python-systemd/journal.html#journalhandler-class
I can have a journalhandler, consolehandler and sysloghandler,
only journalhandler will handle extra parameter without disturb others
handlers.
Even if you wish drop syslog and files support in asterisk, with journald,
it's possible to repush logs in a rsyslog, to continue to use log parsers.
Second, the logging system in Asterisk has a measurable impact on
performance. Unfortunately, adding a configuration option that is
evaluated on every log statement will further degrade performance when
logging verbosity is increased. A compile time option is not a
pleasant alternative either. Note that we experienced a similar
situation in Asterisk 11 with the remote console independent verbosity
setting, and spent a lot of time over many point releases getting the
performance back in line.
Yes, it's also my fair, I need to do a macro-benchmark to measure
globally the real impact to provide more data in logs.
It should appear catastrophic on a micro-benchmark, but on a
macro-benchmark with all stack, it should be a detail compare to the rest
of processing.
It is also telling that Apache's mod_journald states:
{quote}
Performance warning
Currently, systemd-journald is not designed for high-throughput
logging and logging access_log to systemd-journald could decrease the
performance a lot.
{quote}
Very interesting, I'll benchmark that to have a number rate behind not
designed for high-throughput
logging sentence.
Third, journald seems to be providing structured log data. While some
logging statements in Asterisk may benefit from such a system, many
may not. Asterisk can be *extremely* chatty.
I'm a little bit aware of that: with my colleagues, we're managing between
400 and 500 Asterisks (no, it isn't a joke) for our infrastructure and at
our clients ;-)
I'm not sure that every
log statement in Asterisk should be converted
Re: [asterisk-dev] Journald support for Asterisk
2015-05-09 16:15 GMT+02:00 Bruce Ferrell bferr...@baywinds.org: On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I've no problems to believe you, nevertheless, could you be more precise ? What are the facts that you arrive to this conclusion ? I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. It's issues on your production with several servers ? Or on your laptop ? Could give us links with unresolved bugs you have ? For now, to my experience, with 6 VMs on Debian Jessie on production with small clients and Asterisk 13, I've no issues with systemd nor journald. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Could you give me a link about that ? Please do NOT do this. Why ? If it works for me, what's the problem for you ? It also breaks fail2ban site security. I don't understand the issue here, I see at least two solutions: 1. You can continue to use rsyslog even if you have journald, BTW, it's the default setup of Debian Jessie, where journald forwards everything to rsyslog to avoid to perturb sysadmins with log files. 2. Apparently, fail2ban supports journald: https://github.com/fail2ban/fail2ban/pull/82 Asterisk works well now. It integrates easily. For this point, I'm agree with you, it's very important to continue to be integrated easily, I use that a lot. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
On 05/09/2015 01:17 AM, Ludovic Gasc wrote: Hi, Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as on RHEL/CentOS. Journald supports syslog format, nevertheless, at least for us, the structured log system provided with journald helps us to debug the production. The idea behind that is to attach metadata with a log line to facilitate the search with journalctl, you can write queries to find the errors. For example, with Apache: http://httpd.apache.org/docs/trunk/mod/mod_journald.html For our Python daemons, for each log line, we store account_id, request_id, endpoint and method used to be easy to retrieve quickly interesting logs. Moreover, not yet used for us, but you can generate statistics about your source code real usage based on: http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE= For Asterisk, for example with dialplan logging, you should attach the context, the extension and the channel. You can simulate this journald feature with a specific format message for your logs, nevertheless, journalctl is more user-friendly to retrieve pertinent logs compare to the classical grep usage. ave no idea if i I'm permitted to raise the question about an eventual journald support in Asterisk because I don't find anything about that in issues tracker nor mailing list. Moreover, I understand that even if you add the journald support, it's certainly necessary to change logging everywhere in Asterisk, however, I should help to do that. Regards -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ systemd and journald SUCK to high heaven. I have no idea if the issues I've had with them (OpenSUSE) are distro related or inherent. I have seen that the implementation of apache with a static systemd/journald module will no longer correctly serve content. Please do NOT do this. It also breaks fail2ban site security. Asterisk works well now. It integrates easily. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Journald support for Asterisk
On Sat, May 9, 2015 at 9:15 AM, Bruce Ferrell bferr...@baywinds.org wrote:
On 05/09/2015 01:17 AM, Ludovic Gasc wrote:
Hi,
Systemd and Journald is now by default on Debian Jessie and Ubuntu 15.04, as
on RHEL/CentOS.
Journald supports syslog format, nevertheless, at least for us, the
structured log system provided with journald helps us to debug the
production.
The idea behind that is to attach metadata with a log line to facilitate the
search with journalctl, you can write queries to find the errors.
For example, with Apache:
http://httpd.apache.org/docs/trunk/mod/mod_journald.html
For our Python daemons, for each log line, we store account_id, request_id,
endpoint and method used to be easy to retrieve quickly interesting logs.
Moreover, not yet used for us, but you can generate statistics about your
source code real usage based
on:
http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html#CODE_FILE=
For Asterisk, for example with dialplan logging, you should attach the
context, the extension and the channel.
You can simulate this journald feature with a specific format message for
your logs, nevertheless, journalctl is more user-friendly to retrieve
pertinent logs compare to the
classical grep usage.
ave no idea if i
I'm permitted to raise the question about an eventual journald support in
Asterisk because I don't find anything about that in issues tracker nor
mailing list.
Moreover, I understand that even if you add the journald support, it's
certainly necessary to change logging everywhere in Asterisk, however, I
should help to do that.
Regards
--
Ludovic Gasc (GMLudo)
http://www.gmludo.eu/
systemd and journald SUCK to high heaven. I have no idea if the issues I've
had with them (OpenSUSE) are distro related or inherent.
I have seen that the implementation of apache with a static systemd/journald
module will no longer correctly serve content.
Please do NOT do this. It also breaks fail2ban site security. Asterisk
works well now. It integrates easily.
Being very honest, my knowledge of systemd and journald is not
extensive. I have no real position on systemd/journald one way or the
other. I will say that I do not view it as an inevitable conclusion,
nor a necessary requirement for Asterisk. As such, I'm viewing this
only as a proposal for we'd like to change the vast majority of log
statements in Asterisk.
First, as Bruno pointed out, a lot of external systems rely on parsing
log messages. Asterisk does not exist in a vacuum, and is often a
component in a much larger system. As such, any modification to
Asterisk's logging would have to be configurable.
Second, the logging system in Asterisk has a measurable impact on
performance. Unfortunately, adding a configuration option that is
evaluated on every log statement will further degrade performance when
logging verbosity is increased. A compile time option is not a
pleasant alternative either. Note that we experienced a similar
situation in Asterisk 11 with the remote console independent verbosity
setting, and spent a lot of time over many point releases getting the
performance back in line.
It is also telling that Apache's mod_journald states:
{quote}
Performance warning
Currently, systemd-journald is not designed for high-throughput
logging and logging access_log to systemd-journald could decrease the
performance a lot.
{quote}
Third, journald seems to be providing structured log data. While some
logging statements in Asterisk may benefit from such a system, many
may not. Asterisk can be *extremely* chatty. I'm not sure that every
log statement in Asterisk should be converted or support such a
system. A more concrete proposal over what messages are appropriate
for journald would be nice.
I'd be curious if another option would be to treat journald as we
currently treat syslog - not as a formatting option, but as a
destination. If that were possible, support could be added for it
without necessarily impacting the existing log statements. Asterisk
supports the concept of having dynamic log channels, and support for
journald could then also be added as an external module without
impacting the core - or impacting users who don't care for it.
--
Matthew Jordan
Digium, Inc. | Director of Technology
445 Jan Davis Drive NW - Huntsville, AL 35806 - USA
Check us out at: http://digium.com http://asterisk.org
--
_
-- Bandwidth and Colocation Provided by http://www.api-digital.com --
asterisk-dev mailing list
To UNSUBSCRIBE or update options visit:
http://lists.digium.com/mailman/listinfo/asterisk-dev
