Hi,
I have been in the background working on a companion to log4j that
I'm initially calling 'pinpoint'. It is designed as a log context
tool to facilitate production queries.
Logging events to a DB for querying is fine, except SQL queries,
IHMO, on that are time consuming and unsatisfying, and miss some
important features such as full text indexing. I find for my own use
I tend to rely on grep/fgrep to analyse logs files because the regexp
is handy, but that often trims off important stack traces. Usually
you have to dice the file several times to work with it. It seemed
to make sense to index these log files and be able to analyse them
using Lucene.
pinpoint is designed to create a Context which is a container of
indexed LoggingEvents. The Context contains an Lucene index, and a
serialized version of the events. searches against the index then
return hits that contain offset+length pointers to the serialization
file. That way the full LoggingEvent can be brought back for display
in the results, formatted anyway you'd like.
Using a CLI client, one can open that index and do arbitrary queries.
An example is worth a thousand words:
$ sh pinpoint-cli/target/appassembler/bin/pinpoint -h /temp/pinpoint-
home/ -c testlog -s
pinpoint>search level:INFO factory -english
+message:factory -message:english
[2007-06-08 12:13:00,122 INFO ][eNotificationsAction][Maintenance
Tasks Scheduler_Worker-9] Executing
OverdueCorrespondenceNotificationsAction: exclusionDate=2006-06-08,
maxEmailSendRate=240, maxNumEmails=2147483647, sendEmailEnabled=TRUE
[2007-06-08 12:13:00,553 ERROR][urityInfoRetrieveCmd][Maintenance
Tasks Scheduler_Worker-9][][] Could not find class for command
factory. Class com.aconex.security.command.SecuredAssetListCmd
.......
pinpoint>
Only the basics are there right now, with a simple importer utility
that uses LogFilePatternReceiver to soak events from text-based log
files, but obviously eventually any Receiver might be usable to soak
events into a context. The CLI module has handy command-line
completion support, but it could definitely do with some extensions.
Pinpoint is made up of:
* pinpoint-core
* pinpoint-importer
* pinpoint-search
* pinpoint-cli
and lives in a maven multi-project module setup. It relies on the
log4j companion 1.0-SNAPSHOTs and log4j 1.2.15-SNAPSHOT.
I'd like to eventually add a pinpoint-service module allowing
standard socket/JMS-based receivers to be configured out of the box
to allow easy connection to it as a remote service. Right now
indexing the logs is likely to be slower than the rate of incoming
events, so we'd need to build in some asynch behaviour to allow
pinpoint to buffer some logs (say via an internal JMS implementation
using ActiveMQ) as it processes them. This allows the producing
application to not be slowed down by the indexing.
I'd also like to create pinpoint-webapp to create a nice and easy
deployable webapp than can browse the pinpoint contexts and easily
search logs based on MDC variables etc. (Imagine browsing all logs
generated by a User when using, say, MDC), and once found an
interesting logging event quickly see more context lines from around
that time.
I'd be happy to have this added as a log4j module if other people are
interested. Otherwise I'm just as happy to tinker with it and use it
for my own use. Perhaps I'm the only one that wants to use logging
this way?
cheers,
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
