On 03/18/2011 10:10 AM, Vincent Massol wrote: > > On Mar 18, 2011, at 10:01 AM, Thomas Mortagne wrote: > >> On Fri, Mar 18, 2011 at 09:46, Vincent Massol<[email protected]> wrote: >>> I'm stupid, there's no need to relate it to the AS. The AS can simply >>> ignore those events and we can have a separate listener to receive them.... >> >> Yes can be a special tool for that. >> >>> >>> -Vincent >>> >>> On Mar 18, 2011, at 9:39 AM, Vincent Massol wrote: >>> >>>> Hi, >>>> >>>> Right now logs go to a file on the filesystem. However this is not right >>>> since most logs are application logs and should be visible to wiki >>>> developers. For ex, if I use a deprecated API, I need to see it. It >>>> shouldn't go to admins only and shouldn't "pollute" the system logs.
I agree and don't agree at the same time. * Certain logs are indeed useful only if visible somewhere. If I, as a user, am doing something wrong, I should be notified about it, but I sure can't read catalina.out... * Most logs generated on myxwiki.org don't concern user at all, so I don't agree that we should move all the log data in the wiki. By the way, myxwiki.org generates gigabytes of data, are you sure you want to move that much information in the database? * Stacktraces can't be stored in the activity stream, since it has a limit of 2000 characters for the event body. So, what I agree with is that *short* messages should be loggable both from applications and from Java code. I'm not sure how to display them, though. I can see three different usecases. A. As a user, if I do something wrong in a request, I want to be notified of it immediately. If I have to visit some page, then it's not useful at all, I probably won't do it. I shouldn't see that the visited page uses a deprecated API; this would mean that I get to be warned about the programmer's errors, and in general error notifications tend to scare users. B. As an application writer, I want to be able to see all the unexpected errors raised when people use my application. I shouldn't see expected errors, like "X tried to change his password but provided the wrong verification password". C. As a site admin, I should be able to see all the logs generated by the wiki. These three usecases are completely different and require different approaches. A. should be fixed as a generic error reporting tool in the UI. Currently there are some messy approaches to this, for example expecting an $errors variable to be set in the context, or returning an error string from APIs. B. means that we should indeed make it easy to do commons-logging-like error reporting visible in the wiki. Is this the main use case you were thinking of when writing this proposal? There's also http://extensions.xwiki.org/xwiki/bin/Extension/Log+Application which tries to do something similar. C. should instead make the current logs visible to administrators from the wiki. I don't think that a rethinking of the whole logging mechanism is needed for this; on the contrary, the current logs written to catalina.out are very good. They can be accessed using the Show Log snippet from http://extensions.xwiki.org/xwiki/bin/Extension/Show+Log >>>> Hence I believe we need a Log Console available somewhere (we could make >>>> it avail in the Admin UI FTM). >>>> >>>> I'd like to discuss an implementation idea I've had this morning: >>>> >>>> * Send application logs as Observation Events and make the available in >>>> the Activity Stream (AS) Or send them as Events in the new EventStream (we still have to debate about this some time). Personally I find it weird that logs are sent out as observable events that anyone can listen to, and someone specific *should* listen to in order to persist them. For me logs have a precise configurable path, X tells Y to log something. Your approach suggests that X yells something, and various Y that might be present note what they find interesting. Sure, it's a lot more flexible, but is all that flexibility really needed? Aren't you over-engineering something just for the sake of engineering and reusability? >>>> Pros: >>>> * Infrastructure already in place >>>> * Fits the AS goal: temporary information and is purged regularly >>>> * (Of course the Activity gadget would not display them) >>>> * They can be sent remotely as remote events in the future; this allows >>>> implementing a remote console to monitor an XE or XEM from a distance >>>> >>>> Cons: >>>> * We need to assess the performance risk and more generally we need to >>>> make the AS scalable (I don't think it is now). >>>> * 2 ideas for scaling up the Observation/AS: >>>> 1) Have the Observation Manager save events to be notified into a Queue >>>> and have one or several separate threads take those events and send them >>>> to listeners. Right now if one listener takes time in its onEvent() method >>>> it slows down the whole chain since they are called serially. Note that if >>>> we want even better scalability, the Queue could be stored externally to >>>> XWiki (a JMS queue for ex) and scalability can be achieved by app server >>>> instances listening to this queue to process it. >> >> IMO if a listener takes time in its onEvent() it can always do that >> message queue itself, that's what Lucene plugin do for example. > > Yes i know but it's nice from an architecture POV to do it at this level > rather than *hope* that all listeners (including those not coded by us) will > be good citizens. Once you have the need for synchronous messages, it's hard to break that need for purely architectural reasons. How would you implement the ScriptExecution filter that prevents nested scripts without synchronous events? We could split events into two categories, sync and async, but then we'd have two architectures instead of one. >> We >> dono't absolutely need to implement that in Observation Manager. Also >> you can't do that for all events since some of them are >> question/answer events or event responsible for setting and unsetting >> the contexte before and after a task for example. >> >>>> 2) Have a way to tell the AS what storage to use for specific Event Types. >>>> For example the AS could use an in-memory storage for Log Events while >>>> using a DB storage for other events. This would be useful since I don't >>>> think we really need to store logs in the DB. Note that the cache could be >>>> indexed on the message so only one instance of each log message is >>>> preserved (no need for dups), possibly with a counter to mention how many >>>> of them there were (that's an optimization). Well, how many logs do you expect to have? MyXWiki generates hundreds of megabytes of log data in a day, do you expect all that to fit in memory? The memory has the advantage that it's faster so it can quickly return control to the log caller, but it has the disadvantage that it's very limited in size. >>>> I believe 2) might be enough for performances in a first implementation of >>>> the log console. >>>> >>>> WDYT? >> >> Make the default org.xwiki.component.logging.Logger take care of >> producing theses events so that it support already existing log and >> makes it easier for a component than having to do that using >> Observation Manager. > > Yes although I think I'd prefer to slowly move them to send events instead of > logging and have the logging be done in the listener (ie turn the logic > upside down). > > I'm not 100% sure though, needs some more thoughts. Or we could switch to the slf4j API and write an implementation that creates events. >> +1 for the general idea > > Thanks > -Vincent -- Sergiu Dumitriu http://purl.org/net/sergiu/ _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
