Re: [Spacewalk-devel] Audit Logging
Hey Bo, what was the original use/business case for the audit logging add-on? How do you plan to present the audit information? Regards, Tomas -- Tomas Lestach RHN Satellite Engineering, Red Hat On 08/23/2011 06:26 PM, Bo Maryniuk wrote: Hi, all! Here at SUSE we are working on an audit logging. As we all perfectly know, the Audit Logging is not just a logging in the sense as a syslog for like debugging or alerting etc. We developing an add-on for the Spacewalk. It should collect events from various components and then bring them all to a single point, thus send later to a various destinations (XDAS, databases, indexers etc). Hence I would like shortly introduce what this is all about. Architecture The main core of the logger is a separate daemon that serves XML-RPC listener to a localhost only. Then each software component in the Spacewalk is going to send an logging event message through it. Daemon is designed in a queue fashion, where request gets immediate response, message is accepted inside an embedded database and then processed accordingly. This way it won't impact software components performance. The API for XML-RPC are intentionally generic: they accept a login ID, a string message and a key/value map. While first two parameters are pretty obvious how to use, the key/value map is specific to the scope of audited software. The keys are validated according to a defined structure format, which can be totally different for another components or software group. The validator is a plug-in, which implements specific structure validation. In this way we have a generic auditing collector that can be reused elsewhere or integrated with just anything. There are plugins for: 1. Delivering messages. 2. Validating key/value mapping to prevent posting inconsistent nonsense. Delivering message plugins are: 1. RDBMS (PostgreSQL and Oracle). Stores data in two tables. 2. XML file. Appends data to an XML file. 3. Syslog (TCP, UDP and local). Has advantage to allow reconcile multi-line posts, once messages are too big to fit in one line. 4. STDERR and STDOUT :-) Validator plugins: 1. Spacewalk schema validator. Plugins can be reused in an unlimited combinations and instances: you can feed several databases, syslogs, XML files etc, if needed. Impact on a Spacewalk Entire Spacewalk components are going to be altered, where each audit-able action will perform a call. We already developed an log4j appender and now processing all Java part and XML-RPC backend and a frontend. This will help community to reconnect audit to something totally else, if they would really like to do so. Perl/Python-based and other components will always talk to the audit logger over XML-RPC, thus never link directly. The feature will be turned off by a default and will be able to turn it on in a Spacewalk conf, like: audit = on. You might ask... Q: Why not AMQP or other messaging? A: We considered that. But concluded that we rather want it small, straightforward and simple, available everywhere: http://goo.gl/usti8 :) Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) Q: A daemon? A: Yes. A daemon. Because if this would be a servlet on a Tomcat, so once Tomcat feels sour during various reasons, like a star wars satellite accidentally blew up WAN or endothermal recalibration caused failure converting big to little endian, for example :-), then the rest of the stack won't properly functioning. That should not happen. Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. Q: Does it takes care of being tamper-proof? A: No. The software component is responsible only to collect various wild components and comb the output to a single standard that will be delivered to the enterprise archive, like EMC Centera, for example. Contribution plans We are going to open it under Apache Licence 2.0 and contribute back to the community along with a mega-patch for Spacewalk 1.2 base. If you have any feedback to this and ideas what else could be good, we welcome you to share your thoughts. Have a lot of fun! -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] spacewalk with oracle 11.2 has problems with the monitoring code
On 08/24/2011 01:05 PM, Michael Calmer wrote: Some monitoring code expect special columns with id 1 which fail if id 1 column does not exist. E.g: java/code/src/com/redhat/rhn/domain/monitoring/satcluster/SatClusterFactory.java private static final PhysicalLocation PHYSICAL_LOCATION = lookupPhysicalLocation(new Long(1)); A quick fix is, to disable this feature in oracle with: alter system set deferred_segment_creation=FALSE; but a real fix would be better. Requiring presence of row with id=1 is definitely bug. I do not like that Oracle workaround. That code which lookup id=1 should be replaced. Patches are welcome. :) -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/23/2011 06:26 PM, Bo Maryniuk wrote: Entire Spacewalk components are going to be altered, where each audit-able action will perform a call. We already developed an log4j appender and now processing all Java part and XML-RPC backend and a frontend. This will help community to reconnect audit to something So you already have something. Can you send what you have, so we can review the implementation? It would be unfortunate if you send mega patch and we will do not like several items and reject it. It does not need to be polished or apply to current HEAD. And it does not need to be complete. Just to see the code behind your ideas. The feature will be turned off by a default and will be able to turn it on in a Spacewalk conf, like: audit = on. Definitelly. Most users will not need this one, so it should be disabled by default. Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) This is where I disagree. But I would like to see your code and maybe I will change my mind. Q: A daemon? A: Yes. A daemon. Because if this would be a servlet on a Tomcat, so once Tomcat feels sour during various reasons, like a star wars satellite accidentally blew up WAN or endothermal recalibration caused failure converting big to little endian, for example :-), then the rest of the stack won't properly functioning. That should not happen. *nod* Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. That is quite slow IMO for logger. We are going to open it under Apache Licence 2.0 and contribute back to the community along with a mega-patch for Spacewalk 1.2 base. Please no mega-patch. And definitelly no unless you send few iteration of what-you-have patch, so we can review and discuss whether it is good approach or not. This is thing which can be done good and be great benefit, but with only few things screwed can slow down whole Spacewalk even in disabled state. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/24/2011 01:20 PM, Tomas Lestach wrote: what was the original use/business case for the audit logging add-on? To know who did what, when and from where. How do you plan to present the audit information? I plan to send it where it is needed in the way the destination is asking. So it depends. -- Bo ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] spacewalk with oracle 11.2 has problems with the monitoring code
Hi, Am Mittwoch, 24. August 2011, 13:29:30 schrieb Miroslav Suchý: On 08/24/2011 01:05 PM, Michael Calmer wrote: Some monitoring code expect special columns with id 1 which fail if id 1 column does not exist. E.g: java/code/src/com/redhat/rhn/domain/monitoring/satcluster/SatClusterFactory.java private static final PhysicalLocation PHYSICAL_LOCATION = lookupPhysicalLocation(new Long(1)); A quick fix is, to disable this feature in oracle with: alter system set deferred_segment_creation=FALSE; but a real fix would be better. Requiring presence of row with id=1 is definitely bug. 100% agree I do not like that Oracle workaround. That code which lookup id=1 should be replaced. Patches are welcome. :) I agree. The problem is, that there are a lot more places where this happens. To find out all these is a lot of work :-) -- Regards Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) signature.asc Description: This is a digitally signed message part. ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/24/2011 01:41 PM, Miroslav Suchý wrote: Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) This is where I disagree. But I would like to see your code and maybe I will change my mind. Probably I had not the best explanation here. Actually, there *are* log appenders at the Spacewalk part, so we just continue to use the same log4j the same fashion in the Java part, Python logger etc. But the appenders are actually an XML-RPC clients to the daemon. Then since you have various destinations, like a slow and remote RDBMS, like other solutions, the daemon is *separately* delivers them, after an event already been accepted. Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. That is quite slow IMO for logger. Well, if your admin can do more than 2000 changes-per-second, then just let me know... :) -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
