On Sun, 26 Sep 2004, Ceki G�lc� wrote: | At 01:53 AM 9/23/2004, Elias Ross wrote: | | >Single lock strategy: (current one) | > | >Thread 1 --> Appender --> Database.insert | >Thread 2 --> (wait) | > | >This tends to create bottlenecks if two threads are logging at the same | >time. | > | > | >Concurrent strategy: | > | >Thread 1 --> Appender --> Database.insert | >Thread 2 --> Appender --> Database.insert | > | >This has an advantage on multiprocessor machines or when the database | >can handle inserts simultaneously. | | How can you write events concurrently from two distinct threads to the | same file and expect coherent output?
One must assume that the database knows how to handle syncronization, which a database definately must! Judging from the thread (-but I haven't read the code-), I think Elias have a point: if the current implementation locks the Appender (or an object "on the same level") while outputting, then there might be room for improvement: Lets construct an example: you have an dual-CPU box, running an application using log4j extensively, and the log-receiving DB is on another box, also dual-CPU. Then you could conceivably log two different messages at the exact same time, using two different connections to the database machine. On the database side, the two different processors read from the two different connections, then close the connection, and the log4j log-method will return, and everything will have been "totally concurrent". Note that on the thread's way into the logging system (deciding whether to log or not), the methods obviously will have to synchronize on different elements, but this would just have to be to check different ints and booleans - it might even suffice to use transient variables here. ( How the database side handles the actual concurrency, and the writing/updating to disk, is really out of the scope of the discussion. However, here's an idea: each connection will cache the INSERT message to ram (the two CPU's have different ram-sections, obviously), then do dependency-analysis on other tables. The database will realize that it is a simple insert, with no dependecies, and the caller requests "just do it when you can", so it can just acknowledge the update, and release the connection right away (if you do MySQL "low priority" inserts, this is actually what can happen). It'll further stick the update's in a queue, this step is obviously synchronized, and another thread will read from this queue, also syncrhonized, and write to the transaction log first, then yet another thread will later commit the changes to the actual table-file (or whatever). However, the log4j-part of this scenario won't be affected by the database's struggles with getting this to disk! ) Another more easily conceivable appender that would benefit from full not-synchronzied appending-logic, is an UDP-appender (or multicast-appender?), that just sends it's log-messages "out in the dark", hoping that there are some receiver that picks them up. This could be done without using synchronizatin on actual appender-part on the log4j side. Things might (often) be received out-of-order, but this is no problem: each thread will be in-order, and you'll also send the timestamp (from the server) and the thread-name, in this UDP packet. One might also envision a counter that ensures received order, which could either be an unsynchronized ThreadLocal, or a appender-global, synchronized counter. Another thing I think I understood from the mail-thread, is that the -rendering- of the messages are also done within the syncronization block. Again, I don't know whether this is true, as I haven't read the code there yet. But if this is true, I can't see the point: the rendering of the message can be done simultaneously on the two different CPUs in any regard, even if you'd like the actual appending to be synchronized. And this would increase logging throughput. For the file-appender-case, the situation is slightly more complicated, of course: this particular appender must either write its entire log-line in one go (one write-method invocation), but it can be unsyncrhonized (as the write method already is syncrhonzied), -or- one must syncrhonzie on the "file itself" manually, while doing the multiple writes that the one log-method invocation requires. But this is a particular specificity of the file-appender, and not of appending in general. Sorry if I've missed all the targets royally here. Endre --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
