On 02/06/2013 11:59 PM, David Lang wrote: > On Wed, 6 Feb 2013, Risto Vaarandi wrote: > >> hi all, >> >> as you know, currently there are number of outputs which are not >> directly supported by sec. For example, with 'pipe' action one can feed >> only one event to an external program (with most programs exiting after >> the 'pipe' action closes the pipe). Also, the 'write' action opens a >> given file before each access and closes it afterwards. While this makes >> output file rotation worry-free, it is not very efficient. Finally, sec >> does not directly support various sockets as output (for example, it is >> not possible to connect directly to a syslog server on another host and >> issue syslog events without any intermediaries). >> >> Therefore, would there be any interest to an output action which would >> leave the output open/running/connected, so that the next access of the >> same output would not have to do reopen/restart/connect again? This >> action would have to support a number of output types, e.g., >> >> write2 udp myserver:514 This is my event >> write2 tcp myserver:514 Another event >> write2 file /var/log/myevents Third event >> write2 program /bin/logger -p user.notice Fourth event >> >> It is probably easy to see that with sockets and files it is not that >> hard to distinguish them from each other, since the filename or server >> name plus port number can serve as unique identifiers. With command >> lines, however, it is somewhat more difficult, since an extra space or >> different ordering of options makes two command lines different, even if >> their effect is exactly the same. Also, since sec uses positional >> parameters, there is an issue with parsing, although this could be >> solved with the use of (). In addition, one workaround for both issues >> would be storing command line in an action list variable. >> The action outlines above would also need an output rotation mechanism, >> but this could be implemented with adding this to SIGUSR2 handler >> (closing all currently open outputs). >> >> Would the action above be of interest, and are there any other ideas for >> handling these situations? > > I think this sort of thing would be useful, a lot of 'action scripts' end up > being trivial wrappers to do these sorts of things, and opening/closing files > and starting programs can have a surprisingly large effect on performance (I > suspect that SEC has other bottlenecks, but even if it's not a bottleneck, > speeding it up will leave more time for the bottleneck code :-) > > I would not try to match up command lines, as you note it's hard to tell when > you really have two identical ones, and it makes it hard to have multiple > connectons open to the same destination (important for network connections) > > This crys out for a filehandle approach, have something define the output and > then write2 can use it. You can have calendar events close and re-open the > files > for log rotation
The use of filehandles introduces another problem. Suppose the connection to other host is closed for some reason. If connections should be explicitly created by the end user with 'open' action which also creates a handle, it would also be the responsibility of the end user to check the connection status and re-create it if necessary. On the other hand, if 'write2' action is reopening connections when necessary, the end user would be free of defining any extra connection management rules. > > now, as to specific output options. My first thing is make this extendable, > people will have lots of strange outputs they want to talk to. Make it so they > can define a perl subroutine and use that. > > you need pipe to command line (think |gzip -c ->/var/log/alerts.gz) > I am actually also thinking of reusing the already existing 'spawn' action for arbitrary command lines. One very common approach of using 'spawn' has been to start a command line with it which is reading its input through a named pipe. In other parts of the ruleset, the 'write' action can then be used to write into this particular named pipe. This will drop the need for distinguishing command lines from each other and having handles for them, since the filename of the named pipe provides a natural unique identifier. In the current version, there has been one issue related to using 'spawn' in the above way -- since 'write' opens and closes the named pipe at each write operation, a process listening to the pipe will see EOF after each event and could terminate (this will happen to many common tools like 'logger' or 'cat'). However, if the 'write' action would keep the pipe constantly open, many unix command line tools would not need such frequent restarting. > you need send to syslog, but this may not be just send the raw string. Syslog > message formatting is common across the different transport types, so it would > be more 'open syslog destination' than 'open tcp destination' > > In formatting the syslog message, options that jump out at me include: > > facility/priority > > syslog tag (program name) > > source hostname (it's useful to be able to generate alert messages with the > name of the relavent server, not just the SEC box) > > syslog format (historic syslog rfc3164, new syslog rfc5424, lumberjack > JSON, > CEE, CEF, ....) > > how to handle multiline (report) data (send as multiple log messages, send > as > one message with newlines, send as one message with a different separator > character,...) > > for these, there should be a default set at open time, but it can make sense > to > override these for particular messages. > > Much of this formatting stuff is useful even if just writing to files, so > it > may be that the right thing is to define the transport at open time and then > have a function call to format the message at log time. > > > thoughts? > > David Lang > I'd leave the message formatting completely open to the end user. First, I think it would be quite hard to figure out defaults for many cases (e.g. how to handle scenarios where a newline is inside the message text). Also, the number of potential formats can be very large. Fortunately, for syslog it is very easy to manually craft an rfc3164 and rfc5424 message, and I've often used telnet and nc for connecting remote syslog servers and interactively typing in valid messages. So it is probably not very hard to craft a proper syslog message from the sec rule... kind regards, risto ------------------------------------------------------------------------------ Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
