sessionClosed is not called sometimes in heavy stress test contions.
Dear Mina developers.
I'm trying to use mina 1.1.5 to develop the very big stress server.
I test my server with jmeter like some echo test now.
test environment
* jmeter :
echo message size : 1024 bytes
jmeter threads : 100
loop count : 1000
sampler : TCP Sampler
reuse connection : false
* server : HP-UX 2 cores
java HotSpot(TM) Server VM (build 1.5.0.07 jinteg:03.20.07-11:05
IA64, mixed mode)
my server(mina)
* client : pc (dual core).
jmeter
when I tested the stress, I found sessionCreated called counts are a little
bigger than sessionClosed called counts.
so, I tried to find out why that mismatch occured during several days.
and I found the followings :
org.apache.mina.common.support.IoServiceListenerSupport
{
public void fireSessionCreated(IoSession session) {
SocketAddress serviceAddress = session.getServiceAddress();
// Get the session set.
SetIoSession s = new IdentityHashSetIoSession();
SetIoSession sessions =
managedSessions.putIfAbsent(serviceAddress,
Collections
.synchronizedSet(s));
boolean firstSession;
if (null == sessions) {
sessions = s;
firstSession = true;
} else {
firstSession = false;
}
// If already registered, ignore.
if (!sessions.add(session)) {
return;
}
// If the first connector session, fire a virtual service
activation
// event.
if (session.getService() instanceof IoConnector
firstSession) {
fireServiceActivated(session.getService(),
session.getServiceAddress(),
session
.getHandler(),
session.getServiceConfig());
}
// Fire session events.
session.getFilterChain().fireSessionCreated(session); --
called
correctly
session.getFilterChain().fireSessionOpened(session); --
called correctly
// Fire listener events.
for (IoServiceListener listener : listeners) {
listener.sessionCreated(session); -- called correctly
}
}
public void fireSessionDestroyed(IoSession session) {
boolean lastSession = false;
SocketAddress serviceAddress = session.getServiceAddress();
// Get the session set.
SetIoSession sessions = managedSessions.get(serviceAddress);
// Ignore if unknown.
if (sessions == null) {
// logger.error(fireSessionDestroyed() : sessions ==
null +
session);
return; -- occured sometimes !!
}
sessions.remove(session);
// Try to remove the remaining empty session set after removal.
if (sessions.isEmpty()) {
lastSession = managedSessions.remove(serviceAddress,
sessions);
}
// Fire session events.
session.getFilterChain().fireSessionClosed(session); --
skipped sometimes
// Fire listener events.
try {
for (IoServiceListener listener : listeners) {
listener.sessionDestroyed(session); -- skipped
sometimes
}
} finally {
// Fire a virtual service deactivation event for the
last session of
// the connector.
// TODO double-check that this is *STILL* the last
session. May not
// be the case
if (session.getService() instanceof IoConnector
lastSession) {
fireServiceDeactivated(session.getService(),
session.getServiceAddress(), session
.getHandler(),
session.getServiceConfig()); -- skipped sometimes
}
}
}
the above fireSessionDestroyed method called as the same number as
fireSessionCreated method.
but the mysterious event is that the sessions is not gotten from
managedSessions in fireSessionDestroyed,
i.e. SocketAddress key is not found in managedSessions.
then the remaining logic was skipped.
so fireSessionClosed, sessionDestroyed ... are not called sometimes.
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null
(SOCKET, R: /172.26.73.107:15059, L: /10.88.50.205:3, S:
0.0.0.0/0.0.0.0:3)
[IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null
(SOCKET, R: /172.26.73.107:15961, L: /10.88.50.205:3, S:
0.0.0.0/0.0.0.0:3)
[IoServiceListenerSupport] - fireSessionDestroyed()
use isPrimitive() instead of comparing the string to a list of primitives
Hi
This is not really a problem, but I noticed that on line 1612 of
AbstractIOBuffer in the current trunk determining whether a class is
primitive could be done nicer and faster with this:
Replace:
String className = desc.getName();
if (primitiveTypeNames.contains(className)) {
with:
if (desc.forClass().isPrimitive()) {
isPrimitive is a native Function. Also if a new primitive Type were to be
added, the code would still work.
It should do the same, I looked it up in the code.
Keep it up!
Stivo
--
Stefan Ackermann,
[EMAIL PROTECTED]
[OT] Asynchronous database drivers
This past weekend I made a release of the asynchronous database driver project I've been working on for the past few months. I've been working on this project as part of my master's thesis. I call the project Asynchronous Database Connectivity in Java (ADBCJ), http://adbcj.org/. I've implemented asynchronous MySQL and Postgresql drivers using MINA. I've also implemented a driver that wraps JDBC and achieves asynchronism using a thread pool. The API and drivers are still a bit immature and incomplete but I thought that making this project more public would help with getting some feedback before I get too far. I have a simple tutorial for using ADBCJ at http://code.google.com/p/adbcj/wiki/Tutorial. Once I'm close to finishing my thesis, I would like to move ADBCJ over to Apache and get these drivers production ready but that's a ways down the road. -Mike
Re: [AsyncHttpClient] On bringing the code bases and communities together
Rick McGuire wrote: I had some time this morning, and decided to take a look at this. It was fairly straightforward merging the changes back in to the 2.0 sandbox branch. There's a bit of a problem going on here with the jsps used for the tests. In the 1.1.5-based version, there was no eol-style property set for the jsps. This caused the strings that were returned by the tests to use \n for line terminators, which the unit tests expected to find in the message responses. In the 2.0 version, the eol-style is set to native, which causes the unit tests to fail when run on a Windows system. I was able to hack these up so they're now running cleanly, but I'm not terribly confident these won't end up breaking again in the future. I suspect a less encoding-specific approach is going to be needed for validating the responses should be used. Rick Rick, Does http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ contain all the bug fixes and latest features of AsyncHttpClient? Can we use this to merge the changes over to the client under MINA? -Mike snip
Re: [AsyncHttpClient] On bringing the code bases and communities together
Yeah I believe so (as far as I can tell). It was more or less a straightforward porting, so there might be interesting and subtle behavior changes we may need to look at. But it should be a good solid baseline. Thanks, Sangjin On Feb 4, 2008 2:50 PM, Mike Heath [EMAIL PROTECTED] wrote: Rick McGuire wrote: I had some time this morning, and decided to take a look at this. It was fairly straightforward merging the changes back in to the 2.0 sandbox branch. There's a bit of a problem going on here with the jsps used for the tests. In the 1.1.5-based version, there was no eol-style property set for the jsps. This caused the strings that were returned by the tests to use \n for line terminators, which the unit tests expected to find in the message responses. In the 2.0 version, the eol-style is set to native, which causes the unit tests to fail when run on a Windows system. I was able to hack these up so they're now running cleanly, but I'm not terribly confident these won't end up breaking again in the future. I suspect a less encoding-specific approach is going to be needed for validating the responses should be used. Rick Rick, Does http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ contain all the bug fixes and latest features of AsyncHttpClient? Can we use this to merge the changes over to the client under MINA? -Mike snip
Re: [AsyncHttpClient] On bringing the code bases and communities together
I've been looking into merging http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ into http://svn.apache.org/repos/asf/mina/asyncweb/trunk/client/ and I'm trying to figure out the best way to proceed. Here are some of the pain points I see: - The only redeeming quality about the AsyncWeb client is that it uses the AsyncWeb codec. - There are a few things in AHC that duplicate functionality already available in MINA (timeouts for example) - The AHC codec and the AsycnWeb codec have diverged quite a bit. To resolve these pain points, here is how I would like to proceed. I would like to copy the AHC async-http-client-mina2 code as is into the AsyncWeb client repository. Because AHC and AsyncWeb client use different package names, there shouldn't be any conflicts doing this. Once that is done, we can look at removing the AHC code that reimplements functionality available in MINA. As long as all the tests are passing, we should be fine and if there are gaps in testing, we should fill those. Once that is done, we can look at refactoring the AHC code base to use the AsyncWeb codec while at the same time, make sure that the AsyncWeb codec isn't missing any functionality that is implemented in AHC. I think at this point, we can rename the AHC packages and move it over top the current AsyncWeb client. Am I missing anything in AsyncWeb client that needs to merged into AHC? It's only four classes and I didn't see anything in there that isn't available in AHC. Before going ahead and doing this, I wanted to know what everyone else thinks of this plan. Does anyone else have a different idea as to how best to undertake this merge? Am I missing anything? -Mike Sangjin Lee wrote: Yeah I believe so (as far as I can tell). It was more or less a straightforward porting, so there might be interesting and subtle behavior changes we may need to look at. But it should be a good solid baseline. Thanks, Sangjin On Feb 4, 2008 2:50 PM, Mike Heath [EMAIL PROTECTED] wrote: Rick McGuire wrote: I had some time this morning, and decided to take a look at this. It was fairly straightforward merging the changes back in to the 2.0 sandbox branch. There's a bit of a problem going on here with the jsps used for the tests. In the 1.1.5-based version, there was no eol-style property set for the jsps. This caused the strings that were returned by the tests to use \n for line terminators, which the unit tests expected to find in the message responses. In the 2.0 version, the eol-style is set to native, which causes the unit tests to fail when run on a Windows system. I was able to hack these up so they're now running cleanly, but I'm not terribly confident these won't end up breaking again in the future. I suspect a less encoding-specific approach is going to be needed for validating the responses should be used. Rick Rick, Does http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ contain all the bug fixes and latest features of AsyncHttpClient? Can we use this to merge the changes over to the client under MINA? -Mike snip
Re: [AsyncHttpClient] On bringing the code bases and communities together
Sounds good to me. The codec has evolved quite a bit, so yeah, we'd need to pay attention to the codec to make sure all the features are properly migrated... Please let me know how I can participate in this effort. We'll open new JIRA issues as we identify individual tasks, no? Thanks, Sangjin On Feb 4, 2008 4:45 PM, Mike Heath [EMAIL PROTECTED] wrote: I've been looking into merging http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ into http://svn.apache.org/repos/asf/mina/asyncweb/trunk/client/ and I'm trying to figure out the best way to proceed. Here are some of the pain points I see: - The only redeeming quality about the AsyncWeb client is that it uses the AsyncWeb codec. - There are a few things in AHC that duplicate functionality already available in MINA (timeouts for example) - The AHC codec and the AsycnWeb codec have diverged quite a bit. To resolve these pain points, here is how I would like to proceed. I would like to copy the AHC async-http-client-mina2 code as is into the AsyncWeb client repository. Because AHC and AsyncWeb client use different package names, there shouldn't be any conflicts doing this. Once that is done, we can look at removing the AHC code that reimplements functionality available in MINA. As long as all the tests are passing, we should be fine and if there are gaps in testing, we should fill those. Once that is done, we can look at refactoring the AHC code base to use the AsyncWeb codec while at the same time, make sure that the AsyncWeb codec isn't missing any functionality that is implemented in AHC. I think at this point, we can rename the AHC packages and move it over top the current AsyncWeb client. Am I missing anything in AsyncWeb client that needs to merged into AHC? It's only four classes and I didn't see anything in there that isn't available in AHC. Before going ahead and doing this, I wanted to know what everyone else thinks of this plan. Does anyone else have a different idea as to how best to undertake this merge? Am I missing anything? -Mike Sangjin Lee wrote: Yeah I believe so (as far as I can tell). It was more or less a straightforward porting, so there might be interesting and subtle behavior changes we may need to look at. But it should be a good solid baseline. Thanks, Sangjin On Feb 4, 2008 2:50 PM, Mike Heath [EMAIL PROTECTED] wrote: Rick McGuire wrote: I had some time this morning, and decided to take a look at this. It was fairly straightforward merging the changes back in to the 2.0 sandbox branch. There's a bit of a problem going on here with the jsps used for the tests. In the 1.1.5-based version, there was no eol-style property set for the jsps. This caused the strings that were returned by the tests to use \n for line terminators, which the unit tests expected to find in the message responses. In the 2.0 version, the eol-style is set to native, which causes the unit tests to fail when run on a Windows system. I was able to hack these up so they're now running cleanly, but I'm not terribly confident these won't end up breaking again in the future. I suspect a less encoding-specific approach is going to be needed for validating the responses should be used. Rick Rick, Does http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ contain all the bug fixes and latest features of AsyncHttpClient? Can we use this to merge the changes over to the client under MINA? -Mike snip
connect timeout
I had a quick question on the connect timeout... The connect timeout supplied to connectors is in the unit of seconds, and it appears the minimum value you can use is 1 second ( AbstractIoConnector.setConnectTimeout() in the case of the trunk). Is this by design? I can see cases where one needs to have a shorter connect timeout, but it seems it is not possible today. One solution might be to use ConnectFuture.join() with a timeout, but that works only if you want to block until it times out... It also seems that this minimum timeout value is somewhat tied to the timeout value used in the select() loop in the connector, which is hard coded to be 1 second. Would it be a good idea to support connect timeout values in milliseconds, and make it shorter than 1 second? Thanks, Sangjin
Re: connect timeout
great question..Im interested in this as well. Jeff Sangjin Lee wrote: I had a quick question on the connect timeout... The connect timeout supplied to connectors is in the unit of seconds, and it appears the minimum value you can use is 1 second ( AbstractIoConnector.setConnectTimeout() in the case of the trunk). Is this by design? I can see cases where one needs to have a shorter connect timeout, but it seems it is not possible today. One solution might be to use ConnectFuture.join() with a timeout, but that works only if you want to block until it times out... It also seems that this minimum timeout value is somewhat tied to the timeout value used in the select() loop in the connector, which is hard coded to be 1 second. Would it be a good idea to support connect timeout values in milliseconds, and make it shorter than 1 second? Thanks, Sangjin
Re: [AsyncHttpClient] On bringing the code bases and communities together
Mike Heath wrote: I've been looking into merging http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ into http://svn.apache.org/repos/asf/mina/asyncweb/trunk/client/ and I'm trying to figure out the best way to proceed. Here are some of the pain points I see: - The only redeeming quality about the AsyncWeb client is that it uses the AsyncWeb codec. - There are a few things in AHC that duplicate functionality already available in MINA (timeouts for example) I think this timeout is a different timeout in implementation...that is unless this was added to 2.0, then duplication may be correct. Its a response timeout, not socket timeout...I think thats different. - The AHC codec and the AsycnWeb codec have diverged quite a bit. To resolve these pain points, here is how I would like to proceed. I would like to copy the AHC async-http-client-mina2 code as is into the AsyncWeb client repository. Because AHC and AsyncWeb client use different package names, there shouldn't be any conflicts doing this. Once that is done, we can look at removing the AHC code that reimplements functionality available in MINA. As long as all the tests are passing, we should be fine and if there are gaps in testing, we should fill those. Once that is done, we can look at refactoring the AHC code base to use the AsyncWeb codec while at the same time, make sure that the AsyncWeb codec isn't missing any functionality that is implemented in AHC. I think at this point, we can rename the AHC packages and move it over top the current AsyncWeb client. Am I missing anything in AsyncWeb client that needs to merged into AHC? It's only four classes and I didn't see anything in there that isn't available in AHC. Before going ahead and doing this, I wanted to know what everyone else thinks of this plan. Does anyone else have a different idea as to how best to undertake this merge? Am I missing anything? I like the plan...seems like a good way to go. -Mike Sangjin Lee wrote: Yeah I believe so (as far as I can tell). It was more or less a straightforward porting, so there might be interesting and subtle behavior changes we may need to look at. But it should be a good solid baseline. Thanks, Sangjin On Feb 4, 2008 2:50 PM, Mike Heath [EMAIL PROTECTED] wrote: Rick McGuire wrote: I had some time this morning, and decided to take a look at this. It was fairly straightforward merging the changes back in to the 2.0 sandbox branch. There's a bit of a problem going on here with the jsps used for the tests. In the 1.1.5-based version, there was no eol-style property set for the jsps. This caused the strings that were returned by the tests to use \n for line terminators, which the unit tests expected to find in the message responses. In the 2.0 version, the eol-style is set to native, which causes the unit tests to fail when run on a Windows system. I was able to hack these up so they're now running cleanly, but I'm not terribly confident these won't end up breaking again in the future. I suspect a less encoding-specific approach is going to be needed for validating the responses should be used. Rick Rick, Does http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2/ contain all the bug fixes and latest features of AsyncHttpClient? Can we use this to merge the changes over to the client under MINA? -Mike snip
MINA 2.0 IPv6 NioDatagramAcceptor
Using MINA 2.0, I cannot seem to create and bind to the local IPv6 wildcard address: org.apache.mina.transport.socket.nio.NioDatagramAcceptor acceptor = new org.apache.mina.transport.socket.nio.NioDatagramAcceptor(); acceptor.setDefaultLocalAddress(new java.net.InetSocketAddress(::0, 555)); acceptor.setHandler(new org.apache.mina.common.IoHandlerAdapter()); acceptor.bind(); java.net.SocketException: Address family not supported by protocol family: bind at sun.nio.ch.Net.bind(Native Method) at sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:495) at sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:79) at org.apache.mina.transport.socket.nio.NioDatagramAcceptor.open(NioDatagramAcceptor.java:117) at org.apache.mina.transport.socket.nio.NioDatagramAcceptor.open(NioDatagramAcceptor.java:46) at org.apache.mina.common.AbstractPollingConnectionlessIoAcceptor.registerHandles(AbstractPollingConnectionlessIoAcceptor.java:505) at org.apache.mina.common.AbstractPollingConnectionlessIoAcceptor.access$500(AbstractPollingConnectionlessIoAcceptor.java:48) at org.apache.mina.common.AbstractPollingConnectionlessIoAcceptor$Worker.run(AbstractPollingConnectionlessIoAcceptor.java:334) at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595) Is this something that is supposed to work, or is this not available in MINA? Regards, Greg -- View this message in context: http://www.nabble.com/MINA-2.0-IPv6-NioDatagramAcceptor-tp15282319s16868p15282319.html Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.
Re: [AsyncHttpClient] On bringing the code bases and communities together
Jeff Genender wrote: snip I think this timeout is a different timeout in implementation...that is unless this was added to 2.0, then duplication may be correct. Its a response timeout, not socket timeout...I think thats different. MINA has support for triggering events for read idle, write idle, or both. See IoSession.setIdleTime(IdleStatus, int) in MINA 1.x and IoSession.getConfig().setIdleTime(IdleStatus,int) in MINA 2. The idle events get handled by IoHandler.sessionIdle(IoSession, IdleStatus) in both versions. snip I like the plan...seems like a good way to go. Thanks for the feedback. I'll get started on this tomorrow. -Mike
Re: [AsyncHttpClient] On bringing the code bases and communities together
Jeff Genender wrote: Mike Heath wrote: Jeff Genender wrote: snip I think this timeout is a different timeout in implementation...that is unless this was added to 2.0, then duplication may be correct. Its a response timeout, not socket timeout...I think thats different. MINA has support for triggering events for read idle, write idle, or both. See IoSession.setIdleTime(IdleStatus, int) in MINA 1.x and IoSession.getConfig().setIdleTime(IdleStatus,int) in MINA 2. The idle events get handled by IoHandler.sessionIdle(IoSession, IdleStatus) in both versions. Will idle work here? The Mina FAQ states that sessionIdle cannot be used in a request-response timeout: http://tinyurl.com/25gnja Also, with pipelining, can idle time even be used? What the FAQ is saying is that the idle event wont tell you which request message was issued when the timeout occured. So using an idle read to detect an unresponsive connection is perfectly legitimate. I don't see why using idle wouldn't work for pipelining. If the session times out, it gets closed which would cause any pending requests to fail anyway. I may just be presumptuous. Am I missing something? -Mike
Re: [AsyncHttpClient] On bringing the code bases and communities together
Mike Heath wrote: What the FAQ is saying is that the idle event wont tell you which request message was issued when the timeout occured. So using an idle read to detect an unresponsive connection is perfectly legitimate. I don't see why using idle wouldn't work for pipelining. If the session times out, it gets closed which would cause any pending requests to fail anyway. I may just be presumptuous. Am I missing something? This is pipelining...using a single connection for reuse for multiple requests, which tells me that a sessionIdle would probably not work. But if you are saying its possible, then great! Jeff -Mike
Re: [AsyncHttpClient] On bringing the code bases and communities together
Mike Heath wrote: To clarify, my understanding of pipelining is that it is sending multiple requests and then waiting for the corresponding responses. This is more than HTTP keep alive which would also allow using a single connection for multiple requests but doesn't necessarily involve the batching of pipelining. I understand it slightly different..that you can send multiple requests through the same socket. The difference between what you said and what I said is I think pipelining does not have stacked waiting. But if you are saying its possible, then great! What I'm saying is that in the AHC code at http://svn.apache.org/repos/asf/geronimo/sandbox/async-http-client-mina2, the HttpIoHandler.TimeoutTask is unaware of which request has been sent. It only knows which IoSession is being used (that's the only field in the TimeoutTask). So essentially the timeout logic as it is implemented in AHC is no different than using the MINA idle logic. Does the AHC client currently support pipelining? If I issue two pipelined requests, and the first one times out causing the session to close. How does the future of the second request get notified of the timeout? I only see code for working with CURRENT_REQUEST (from the IoSession attributes). I don't see any code for dealing with pending requests. You are rightno AHC does not support pipelining but we were coding it to go in that direction so we could. We knew that it would be a lot more complex to implement, but wanted to sow the seeds of the code to handle it when we did. OTOH, I must admit I implemented the ScheduledExecutor based on the FAQ and a few IRC discussions ;-) but if sessionIdle will work, then that would be the way to go. -Mike BTW - Thanks for the discussion, Jeff. It's really helped me to go back and look deeper in the AHC code. Thank you...and thanks for taking an interest...this is great ;-) -Mike
