Re: Multithreaded Appender
Ceki Gülcü wrote: I think I understand. Can you point me to a sample implementation? I have a sample implementation of an optimized Category and AppenderAttachableImpl. How can I give it to you? Ingo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multithreaded Appender
Ceki Gülcü wrote: I think I understand. Can you point me to a sample implementation? I use the implementation from http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html which provides a SyncCollection (a wrapper around collections) which can be initialized with a WriterPreferenceReadWriteLock. WriterPreferenceReadWriteLock implements the needed strategy. The only problem I see is the repository.fireAddAppenderEvent(this, newAppender); in Category.addAppender. This could lead to deadlocks, if not used right. Fortunately I don't see many uses of this event handling inside log4j except the MBean. I can try to implement the desired feature in Category - if you want. Ingo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multithreaded Appender
I think I understand. Can you point me to a sample implementation? At 07:58 PM 3/27/2003 +0100, you wrote: Ceki, A number of internal components rely on the fact that the Appender.doAppend() method is synchronized. It is certainly possible to write appenders which do not require synchronization. However, this will require a substantial investment. There is no simple and safe workaround except rewriting the appenders and layouts you would like to use. The problem is not writing the appenders. I know that I have to write a special appender for my purposes. The problem seems to be, that all accesses to this appender will be synchronized by the Category if they have a common Category in the hierarchy - what they usually do. The problem is implemented in method Category.callAppenders which synchronizes calls to all parent categories, especially those with appenders. Please see my reply to my mail. Do you think it's possible to make the categories more "concurrency friendly"? Otherwise all the efforts on the appender side won't work. "Concurrency friendly" means: Calls to Category.callApenders should be possible by many threads concurrently (readers) except when one threads wants to call addAppender, removeAppender or something similar (writer). Then all readers should have to wait for the writer's completion - a very short time. I think a pattern like this is important for large scale concurrent applications that want to use log4j not only to log to console or file but also to databases and archives. Ingo. -- Ceki - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multithreaded Appender
Ceki, A number of internal components rely on the fact that the Appender.doAppend() method is synchronized. It is certainly possible to write appenders which do not require synchronization. However, this will require a substantial investment. There is no simple and safe workaround except rewriting the appenders and layouts you would like to use. The problem is not writing the appenders. I know that I have to write a special appender for my purposes. The problem seems to be, that all accesses to this appender will be synchronized by the Category if they have a common Category in the hierarchy - what they usually do. The problem is implemented in method Category.callAppenders which synchronizes calls to all parent categories, especially those with appenders. Please see my reply to my mail. Do you think it's possible to make the categories more "concurrency friendly"? Otherwise all the efforts on the appender side won't work. "Concurrency friendly" means: Calls to Category.callApenders should be possible by many threads concurrently (readers) except when one threads wants to call addAppender, removeAppender or something similar (writer). Then all readers should have to wait for the writer's completion - a very short time. I think a pattern like this is important for large scale concurrent applications that want to use log4j not only to log to console or file but also to databases and archives. Ingo. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multithreaded Appender
At 09:37 PM 3/26/2003 +0100, you wrote: Hi, I want to write a multithreaded appender. Threads that use this appender shouldn't be synchronized because writing can take some time and their back end takes care of synchronization (database or MQ-Series). On the other hand the appender must notify their caller if something goes wrong, for instance with RuntimeExceptions. So using the AsyncAppender isn't an option. My problem is that the "way" from Logger.info() to Appender.append() is synchronized on two locations. Therefore every (potentially parallel) call to the Appender is synchronized. 1.) AppenderSkeleton.doAppend is synchronized on the appender object: workaround - write a new "UnsynchronizedAppenderSkeleton" and derive from this class 2.) Category.callApenders is synchronized on the category object: workaround - create categories with the thread's name as part of their name Unfortunately 2.) doesn't work when threads with new names are created and destroyed at a high rate like in IBM WebSphere and categories are stored in the log4j-repository forever. At some time all the memory will be used by categories. Questions: 1.) Are my observations and conclusions right? 2.) Is there a workaround? A number of internal components rely on the fact that the Appender.doAppend() method is synchronized. It is certainly possible to write appenders which do not require synchronization. However, this will require a substantial investment. There is no simple and safe workaround except rewriting the appenders and layouts you would like to use. Regards, Ingo -- Ceki - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multithreaded Appender
I do not reply to myself usually.
2.) Category.callApenders is synchronized on the category object:
workaround - create categories with the thread's name as part of their
name
Unfortunately 2.) doesn't work when threads with new names are created
and destroyed at a high rate like in IBM WebSphere and categories are
stored in the log4j-repository forever. At some time all the memory
will be used by categories.
The workaround doesn't work at all. Category.callAppenders is also
synchronized on the parent categories. One of them is configured to call
the appender. So all the concurrent threads with their categories
(loggers) have to wait anyway.
I know that the reason for synchronization is this code in
appendLoopOnAppenders which is called from callAppenders:
size = appenderList.size();
for (int i = 0; i < size; i++) {
appender = (Appender) appenderList.elementAt(i);
appender.doAppend(event);
}
But why have the readers (every call to log.debug, etc.) of this
appenderList to by synchronized, when there are very rare occurences of
writers (addAppender, removeAppender).
Regards,
Ingo
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Multithreaded Appender
Hi, I want to write a multithreaded appender. Threads that use this appender shouldn't be synchronized because writing can take some time and their back end takes care of synchronization (database or MQ-Series). On the other hand the appender must notify their caller if something goes wrong, for instance with RuntimeExceptions. So using the AsyncAppender isn't an option. My problem is that the "way" from Logger.info() to Appender.append() is synchronized on two locations. Therefore every (potentially parallel) call to the Appender is synchronized. 1.) AppenderSkeleton.doAppend is synchronized on the appender object: workaround - write a new "UnsynchronizedAppenderSkeleton" and derive from this class 2.) Category.callApenders is synchronized on the category object: workaround - create categories with the thread's name as part of their name Unfortunately 2.) doesn't work when threads with new names are created and destroyed at a high rate like in IBM WebSphere and categories are stored in the log4j-repository forever. At some time all the memory will be used by categories. Questions: 1.) Are my observations and conclusions right? 2.) Is there a workaround? Regards, Ingo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
