Hello, world. I discussed this in private with Rainer, and he suggested me to bring the discussion here.
I'm already developing an output module for feeding an Oracle database with rsyslog input. Rainer already committed some patches to the "oracle" branch, in git. Let me remember that this is highly experimental, and I'm sending a big semantic change today. But, in principle, the module does what you'd expect from it: it connects to a DB, receives a SQL statement via doAction, prepares that statement, runs it, commits. It works, but it's way too slow for my needs. As I said when I started this project, I need to be very fast, to prepare the statement at connection time, run it many times, and definitely want batch operations. Say, I want to insert 1000 entries with a single call to the Oracle interface, then commit. With what I know now of rsyslog, I can do it more or less like this: $OmoracleStatementTemplate,"insert into foo(field1, field2, field3) values(:val1, :val2, :val3)" which is the statement to prepare by Oracle. This way, I can prepare the statement at createInstance() time. Then, I can specify the batch size with something like $OmoracleBatchSize 1000 With this, also at createInstance() time I can specify that doAction is called only if there are 1000 entries pending for this selector, like this: CODE_STD_STRING_REQUESTparseSelectorAct(batch_size); The bad part is that rsyslog will deliver to the output module a single string per entry. So, I'd have to split each entry into its fields as part of the doAction() code. I'd need some funny separator for each field, to avoid problems. So far, it can be done. But the configuration would look like this: $OmoracleDB logdb $OmoracleDBUser dbuser $OmoracleDBPassword dbpassword $OmoracleStatement "insert into foo(col1, col2) values (:fied1, :field2)" $OmoracleBatchSize 1000 $OmoracleFieldSeparator **** *.* :omoracle:;"%field1%****%field2%" and make doAction split the fields appropriately. I bet it works. But it's probably too ugly for users. Cleaner ways may need deeper changes into rsyslog's API so that the module gets direct access to each field. That's probably a lot of work and I can't wait for that. So, my questions (at last!): Are there any other alternatives? Is this "ugly" way of working good for other users? Should I keep it for internal use? Thanks a lot. -- Luis Fernando Muñoz Mejías [email protected] _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
