Hi Eric, just downloading it..
About the tread safety. I don't think so. I'm still wonder if the exception was only thrown because of the OOM or if its the cause of the OOM. Hopefully I will have a better view of the whole story once I looked at the heap dump... Bye, Norman 2010/4/24 Eric Charles <[email protected]>: > Hi Norman, > > You can download it from > http://apache.u-mangate.com/james/oom/java_pid23898.hprof.zip > > > "This error seems to get thrown when you write to a closed stream.." > Could it be a thread-safety problem ? (on thread using the session/state of > another one ?) > > I will also monitor with lsof -p <pid> | grep deleted | wc -l > > As config change, I activated a few days ago the ValidRcptHandler fastfail > handler. > > Tks, > > Eric > > > On 04/24/2010 03:13 PM, Norman Maurer wrote: >> >> Could you upload the heap dump too ? >> >> >> 2010/4/24, Norman Maurer<[email protected]>: >> >>> >>> Hi Eric, >>> >>> thx for the report. Comments are inline... >>> >>> >>> 2010/4/24 Eric Charles<[email protected]>: >>> >>>> >>>> Hi All, >>>> >>>> I just got oom with netty impl. >>>> See http://apache.u-mangate.com/james/oom/20090424.png >>>> (http://apache.u-mangate.com/james/oom/index.html for history). >>>> >>>> Everything was fine with the trunk of last week that was really stable. >>>> I redeployed 3 days ago a new trunk version after a stop/start that >>>> showed >>>> ClassCastException of ImapMessage (ClassCastException was solved with >>>> the >>>> redeploy) >>>> >>> >>> The ClassCastException should be fixed since yesterday. This was >>> caused by the EntityManager got confused by entities.. >>> >>> >>> >>>> >>>> Short infos about the today oom: >>>> - The logs had IOException "Bad file descriptor" (see copy of the >>>> exception >>>> after this mail) >>>> >>> >>> This error seems to get thrown when you write to a closed stream... >>> >>> >>>> >>>> - The begining time of these exceptions in the log is 00:10, and there >>>> are >>>> numerous ones during 1 minute. >>>> - The heap dump automatically generated via >>>> -XX:+HeapDumpOnOutOfMemoryError >>>> -XX:HeapDumpPath=/ occured on 00:10, exactly on the same time the >>>> IOException "Bad file descriptor" began. >>>> >>> >>> Ok.. >>> >>> >>>> >>>> - I forgot to lsof to see the open files... but when if the error came >>>> from >>>> too many open files, the received exception in the log would have been >>>> "java.io.IOException: too many open files". >>>> >>> >>> right.. >>> >>> >>>> >>>> - Seems like an infinite loop as Norman anticipated, but now stacktraces >>>> from logs seem to confirm this. >>>> - Looking at line 150 of >>>> >>>> http://svn.apache.org/viewvc/james/server/trunk/smtpserver/src/main/java/org/apache/james/smtpserver/DataLineJamesMessageHookHandler.java?view=markup >>>> , out.write(line) seem to be the place that triggers the exception, out >>>> variable being = (OutputStream) >>>> session.getState().get(SMTPConstants.DATA_MIMEMESSAGE_OUTPUTSTREAM); >>>> >>> >>> Good catch will see how this could happen.. >>> >>> >>>> >>>> - The server filesystem seems sane (new hardware, running with enough >>>> privilege, other applications running on the same machine without any >>>> problem). I don't see why I have a "Bad file descriptor" >>>> >>>> >>> >>> See above.. >>> >>> >>>> >>>> There are file in tmp with creation date that span 1 week (so not >>>> produced >>>> on 00:10): >>>> - A few messagestream*.ms >>>> - More rewindable*.tmp >>>> >>> >>> Will investigate .. >>> >>> >>>> >>>> I restarted james. >>>> It still showed some issues (one account via pop3 gave errors in my >>>> client). >>>> oouch... >>>> >>>> I updated from trunk, redeployed, copied the var directory to migrate >>>> the >>>> account/domain/mail database, and restarted. >>>> Hopefully, now everything is working as before. >>>> >>>> So, we've got now a useful stacktrace that clearly shows where the oom >>>> "trigger" could be. >>>> I'm pretty sure that jconsole would have shown the same behaviour as >>>> http://apache.u-mangate.com/james/oom/oom.png >>>> >>>> Possible cause on the SMTP channel: >>>> - A weird behaviour of the smtp clients (hacking or respectfulness >>>> protocols >>>> clients) >>>> - A design in james causing infinite loop: thread clash (a thread >>>> accessing >>>> the same file as another thread?,...) >>>> - Additional James config (spamassassin or whatever) that leads to such >>>> behaviour >>>> - Hardware problem >>>> >>>> Currently, I run with default config + jdbcdomainlist. >>>> I except to receive another oom one of these days, unless the oom comes >>>> from >>>> an unactivated config (spamassassin or whatever). >>>> >>>> Tks, >>>> >>>> Eric >>>> >>>> ERROR 00:10:54,772 | james.smtpserver | Unknown error occurred while >>>> processing DATA. >>>> java.io.IOException: Bad file descriptor >>>> at java.io.FileOutputStream.writeBytes(Native Method) >>>> at java.io.FileOutputStream.write(FileOutputStream.java:247) >>>> at >>>> >>>> org.apache.james.smtpserver.DataLineJamesMessageHookHandler.onLine(DataLineJamesMessageHookHandler.java:150) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.ReceivedDataLineFilter.onLine(ReceivedDataLineFilter.java:54) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.esmtp.MailSizeEsmtpExtension.onLine(MailSizeEsmtpExtension.java:151) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.socket.netty.LineHandlerUpstreamHandler.messageReceived(LineHandlerUpstreamHandler.java:62) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.timeout.IdleStateHandler.messageReceived(IdleStateHandler.java:237) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:117) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:331) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:226) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:191) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:191) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) >>>> at >>>> >>>> org.jboss.netty.channel.Channels.fireChannelDisconnected(Channels.java:502) >>>> at >>>> org.jboss.netty.channel.socket.nio.NioWorker.close(NioWorker.java:583) >>>> at >>>> >>>> org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:119) >>>> at >>>> >>>> org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:76) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) >>>> at >>>> >>>> org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:583) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:789) >>>> at >>>> >>>> org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:90) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:583) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:578) >>>> at org.jboss.netty.channel.Channels.close(Channels.java:1065) >>>> at >>>> org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:178) >>>> at >>>> >>>> org.apache.james.smtpserver.netty.SMTPChannelUpstreamHandler.exceptionCaught(SMTPChannelUpstreamHandler.java:85) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:117) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) >>>> at >>>> org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:619) >>>> at >>>> >>>> org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:637) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:562) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:134) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:134) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) >>>> at >>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345) >>>> at >>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332) >>>> at >>>> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323) >>>> at >>>> >>>> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275) >>>> at >>>> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196) >>>> at >>>> >>>> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) >>>> at >>>> >>>> org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) >>>> at >>>> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>>> at >>>> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>>> at java.lang.Thread.run(Thread.java:619) >>>> ERROR 00:10:54,772 | james.smtpserver | Unknown error occurred while >>>> processing DATA. >>>> java.io.IOException: Bad file descriptor >>>> at java.io.FileOutputStream.writeBytes(Native Method) >>>> at java.io.FileOutputStream.write(FileOutputStream.java:247) >>>> at >>>> >>>> org.apache.james.smtpserver.DataLineJamesMessageHookHandler.onLine(DataLineJamesMessageHookHandler.java:150) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.ReceivedDataLineFilter.onLine(ReceivedDataLineFilter.java:54) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.esmtp.MailSizeEsmtpExtension.onLine(MailSizeEsmtpExtension.java:151) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75) >>>> at >>>> >>>> org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:60) >>>> at >>>> >>>> org.apache.james.socket.netty.LineHandlerUpstreamHandler.messageReceived(LineHandlerUpstreamHandler.java:62) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.timeout.IdleStateHandler.messageReceived(IdleStateHandler.java:237) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:117) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:331) >>>> at >>>> >>>> org.jboss.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:226) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:191) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) >>>> at >>>> >>>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) >>>> at >>>> >>>> org.jboss.netty.channel.SimpleChannelUpstreamHandler.channelDisconnected(SimpleChannelUpstreamHandler.java:191) >>>> ... >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>>> >>> >>> Bye, >>> Norman >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
