[
https://issues.apache.org/jira/browse/HTTPCORE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657317#action_12657317
]
Marc Beyerle commented on HTTPCORE-155:
---------------------------------------
Hi folks,
I received feedback from our customer: They tested my patch under Linux on
System z and it works. However, they also performed some tests under Windows,
where they discovered some - what they call - "dropouts". Therefore, I also
conducted some more tests and found that the Sun JVM produces intermittent
connection time-outs (with my patch enabled) under very heavy load. Funny
enough, I also saw a connection time-out when I tested the Sun JVM with the
original, un-modified code, but couldn't reproduce it reliably.
I am beginning to agree with our Java VM L3 support, that we are probably
facing a thread timing issue here. Why else would synchronization have an
influence on all this? Anyway, I completely re-wrote my patch (and had to go
through the entire IBM approval process again...) in such a way that it uses a
queueing mechanism, as proposed by Oleg. With this, I now achieve "natural"
synchronization: The I/O operations are carried out one after the other, in
order to avoid the blocking behavior, but now without Java monitor involvement.
I conducted quite extensive testing with the latest / greatest 1.5 JVM versions
now, and all scenarios worked just fine without any exceptions. I tested 50
concurrent threads hammering at Synapse ("Stock Quote"), with 400 requests each
(in addition, I performed each test several times):
- Intel Linux & Sun JVM
- Intel Linux & Sun JVM & patch
- Intel Linux & IBM JVM & patch
- Windows & Sun JVM
- Windows & Sun JVM & patch
- Windows & IBM JVM & patch
- Linux on System z & IBM JVM & patch
As a nice side-effect of the queueing implementation, the patch is ways faster
now :-) And it "future-proofs" HttpComponents, in case Sun decides to change
NIO behavior in a later release of the JDK.
@Asankha: I would really like to have your opinion - could you please perform
your tests also with the new patch?
Cheers,
Marc
> Performance issues with IBM JRE 6.0
> -----------------------------------
>
> Key: HTTPCORE-155
> URL: https://issues.apache.org/jira/browse/HTTPCORE-155
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore NIO
> Affects Versions: 4.0-beta1
> Environment: Windows 2003 SP2 - IBM J2RE 1.6.0 build 2.4 - HTTPCore
> Beta1 - Dual Core CPU 3.0Ghz - 1Gbps networking
> Reporter: Tom McSorley
> Fix For: 4.1
>
> Attachments: AbstractIOReactor.diff, AbstractIOReactor.java,
> IOSessionImpl.diff, IOSessionImpl.java,
> javacore.20081203.153723.32300.0001.txt, patch.08-12-17.tar.gz
>
>
> I'm issuing a second HTTP Request on a connection that has very recently
> returned a null for the submitRequest() call... this 2nd request is being
> issued approximately 500ms after the submitRequest() null is returned... so
> the connection has just been established, an HTTP Request/Response-200 cycle
> has completed just prior to this 2nd request being issued. I'm seeing
> unusually long delays in the requestOutput() call (verified by surrounding
> timing prints)... that can range anywhere from a few milliseconds on up to 60
> seconds... It eventually unwinds, and then the submitRequest() is called...
> this 2nd request is dispatched and works fine... but, it is delayed
> considerably... Is this a known issue and is there a possible work-around?
> Here's the JVM related thread information:
> The thread being delayed and stuck in the requestOutput() call for a long
> time (mostly longer than 5 seconds):
> 3XMTHREADINFO "pool-2-thread-5" TID:0x2AEECE00, j9thread_t:0x2A7189A8,
> state:B, prio=5
> 3XMTHREADINFO1 (native thread ID:0x1B44, native priority:0x5,
> native policy:UNKNOWN)
> 4XESTACKTRACE at
> sun/nio/ch/SelectionKeyImpl.interestOps(SelectionKeyImpl.java:60)
> 4XESTACKTRACE at
> org/apache/http/impl/nio/reactor/IOSessionImpl.setEvent(IOSessionImpl.java:113)
> 4XESTACKTRACE at
> org/apache/http/impl/nio/NHttpConnectionBase.requestOutput(NHttpConnectionBase.java:158)
> .... (non important stack information removed)
> 4XESTACKTRACE at
> java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
> 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735)
> Here's the monitor that this thread is blocked and waiting on:
> 2LKMONINUSE sys_mon_t:0x2A708AF8 infl_mon_t: 0x2A708B30:
> 3LKMONOBJECT sun/nio/ch/uti...@00b09208/00B09214: Flat locked by "I/O
> dispatcher 7" (0x2A208E00), entry count 1
> 3LKWAITERQ Waiting to enter:
> 3LKWAITER "pool-2-thread-5" (0x2AEECE00)
> And here's the thread that currently has this monitor locked:
> 3XMTHREADINFO "I/O dispatcher 7" TID:0x2A208E00, j9thread_t:0x2A6EC73C,
> state:R, prio=5
> 3XMTHREADINFO1 (native thread ID:0x830, native priority:0x5,
> native policy:UNKNOWN)
> 4XESTACKTRACE at
> sun/nio/ch/WindowsSelectorImpl$SubSelector.poll0(Native Method)
> 4XESTACKTRACE at
> sun/nio/ch/WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:308(Compiled
> Code))
> 4XESTACKTRACE at
> sun/nio/ch/WindowsSelectorImpl$SubSelector.access$500(WindowsSelectorImpl.java(Compiled
> Code))
> 4XESTACKTRACE at
> sun/nio/ch/WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:162(Compiled
> Code))
> 4XESTACKTRACE at
> sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:69(Compiled Code))
> 4XESTACKTRACE at
> sun/nio/ch/SelectorImpl.select(SelectorImpl.java:80(Compiled Code))
> 4XESTACKTRACE at
> org/apache/http/impl/nio/reactor/AbstractIOReactor.execute(AbstractIOReactor.java:121)
> 4XESTACKTRACE at
> org/apache/http/impl/nio/reactor/BaseIOReactor.execute(BaseIOReactor.java:70)
> 4XESTACKTRACE at
> org/apache/http/impl/nio/reactor/AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318)
> 4XESTACKTRACE at java/lang/Thread.run(Thread.java:735)
> I should also note that we're attempting to use 1000 client instances on this
> single system... each with potentially 2 active connections simultaneously...
> there is also virtually no CPU load (i.e. less then 5%) on this system...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
