Re: [RADIATOR] MongoDB \ Accounting
On 7/28/13 11:21 AM, Joe Hughes wrote: Hello Joe, > I am considering the best ways of getting the accounting data from our > RADIUS servers \ SQL databases into MongoDB. > > Looking for some feedback\comments. > > Some options; > > a) Write a accounting hook to break apart the accounting message, > construct a JSON request and send it off to a remote application server. > * Downside is the risk of blocking\disrupting the main process. > > b) Spool the messages to disk, have an out-of-process script parse the > files, construct a JSON (or MongoDB request) , send it to a remote > server and delete the file. Downside is some disk\write IO, nothing too > taxing. * Out of process = good. > > c) At the DB level, clone the accounting messages into another table. > Script reads the rows, processes as above, then deletes the rows. * Some > extra DB load. > > d) Possibly silently forwarding (or replicating) the accounting message > to another server and doing one of the above > > Anything I have missed. I am leaning towards b) or c) > > Is anybody else using NoSQL for this type of application? Any feedback? You may have missed that you can actually utilise to push the whole accounting request to external script to be stored in MongoDB database. I was also curious about utilising NoSQL to store RADIUS accounting information as well as SysLog messages so I did a quick proof-of-concept for one postgraduate seminar. The full presentation is available here: http://www.slideshare.net/khuhtanen/using-nosql-databases-to-store-syslog-and-radius but the really short example of the handler needed is this: # # send all RADIUS accounting requests to external script # Command %D/acct2mongo.py AcctLogFileName %L/acct-acct2mongodb-%Y-%M.log The script (included in the presentation) was rather simple as well as Radiator already expands RADIUS accounting message to Attribute -- Value pairs. For the seminar I just tested if this was possible and it was, but I did not yet consider the performance issues or measure the performance of this kind of solution. As we run a separate external command for each accounting request this may probably be inefficient and better way to do this might be to code a Radiator module with Perl for this. I used Python for script, because it was faster for me to write. Of course doing what Hugh suggested, running separate instances for accounting, also helps. Hope this helps and please share you experiences if you plan to experiment with Radiator and MongoDB further based on this. // kh -- Karri Huhtanen Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc. ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator
Re: [RADIATOR] MongoDB \ Accounting
Hello Joe - I would be inclined to use method d) so you get a copy of the accounting requests in a separate process where you can do whatever you need to without impacting your main process. You would do something like this (assuming you are using Handlers): ….. AuthByPolicy ContinueAlways # forward a copy to a separate process …… IgnoreAccountingResponse # do normal accounting ….. Its also a good idea to have separate Radiator processes for authentication and accounting in any case. regards Hugh On 28 Jul 2013, at 18:21, Joe Hughes wrote: > Hi > > Simple question really. > > I want to introduce MongoDB as a "test" server for storing accounting and > session data. > > We currently use MSSQL, it works well, but the large amount of data (and > related joins into other data islands) can become unwieldy over time - > especially for historic reporting. I have done some work with MongoDB and > other systems (with relatively straight forward schemas), and storing > accounting\session seems well suited for this. Don't get me wrong, its not > that MSSQL\MySQL aren't up to the task, I just think this is well suited for > NoSQL and I am keen to satisfy my technical curiosity.. > > I am considering the best ways of getting the accounting data from our RADIUS > servers \ SQL databases into MongoDB. > > Looking for some feedback\comments. > > Some options; > > a) Write a accounting hook to break apart the accounting message, construct a > JSON request and send it off to a remote application server. * Downside is > the risk of blocking\disrupting the main process. > > b) Spool the messages to disk, have an out-of-process script parse the files, > construct a JSON (or MongoDB request) , send it to a remote server and delete > the file. Downside is some disk\write IO, nothing too taxing. * Out of > process = good. > > c) At the DB level, clone the accounting messages into another table. Script > reads the rows, processes as above, then deletes the rows. * Some extra DB > load. > > d) Possibly silently forwarding (or replicating) the accounting message to > another server and doing one of the above > > Anything I have missed. I am leaning towards b) or c) > > Is anybody else using NoSQL for this type of application? Any feedback? > > Regards > > Joe > > > > > > > ___ > radiator mailing list > radiator@open.com.au > http://www.open.com.au/mailman/listinfo/radiator -- Hugh Irvine h...@open.com.au Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, Active Directory, EAP, TLS, TTLS, PEAP, TNC, WiMAX, RSA, Vasco, Yubikey, MOTP, HOTP, TOTP, DIAMETER etc. Full source on Unix, Windows, MacOSX, Solaris, VMS, NetWare etc. ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator
[RADIATOR] MongoDB \ Accounting
Hi Simple question really. I want to introduce MongoDB as a "test" server for storing accounting and session data. We currently use MSSQL, it works well, but the large amount of data (and related joins into other data islands) can become unwieldy over time - especially for historic reporting. I have done some work with MongoDB and other systems (with relatively straight forward schemas), and storing accounting\session seems well suited for this. Don't get me wrong, its not that MSSQL\MySQL aren't up to the task, I just think this is well suited for NoSQL and I am keen to satisfy my technical curiosity.. I am considering the best ways of getting the accounting data from our RADIUS servers \ SQL databases into MongoDB. Looking for some feedback\comments. Some options; a) Write a accounting hook to break apart the accounting message, construct a JSON request and send it off to a remote application server. * Downside is the risk of blocking\disrupting the main process. b) Spool the messages to disk, have an out-of-process script parse the files, construct a JSON (or MongoDB request) , send it to a remote server and delete the file. Downside is some disk\write IO, nothing too taxing. * Out of process = good. c) At the DB level, clone the accounting messages into another table. Script reads the rows, processes as above, then deletes the rows. * Some extra DB load. d) Possibly silently forwarding (or replicating) the accounting message to another server and doing one of the above Anything I have missed. I am leaning towards b) or c) Is anybody else using NoSQL for this type of application? Any feedback? Regards Joe ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator
