Re: Deadlock when using SslFilter and ProxyFilter together
If you look at the stacktraces, I sent in my previous mails, you see, that the order, in which the filters are called, is right: When receiving: ... at org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:310) <--- SslFilter at org.apache.mina.filter.ssl.SslFilter.messageReceived(SslFilter.java:534) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943) at org.apache.mina.proxy.filter.ProxyFilter.messageReceived(ProxyFilter.java:153) <--- ProxyFilter ... When sending: --- at org.apache.mina.proxy.filter.ProxyFilter.writeData(ProxyFilter.java:208) < ProxyFilter - waiting to lock <0x00076ef8b1d8> (a org.apache.mina.proxy.handlers.socks.Socks4LogicHandler) at org.apache.mina.proxy.filter.ProxyFilter.filterWrite(ProxyFilter.java:192) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:625) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:953) at org.apache.mina.filter.ssl.SslHandler.flushScheduledEvents(SslHandler.java:316) at org.apache.mina.filter.ssl.SslFilter.filterWrite(SslFilter.java:653) < SslFilter I am only writing the client part of an application. I communicate with a FIX/SSL server behind a SOCKS proxy, so I cannot change the order in which Ssl encryption and proxy framing must be done.
Re: Deadlock when using SslFilter and ProxyFilter together
> The new version I have put on http://people.apache.org/~elecharny should > solve the problem : instead of getting the lock whe we enter in the > flushScheduledEvents() event, we exit immediately, doing nothing but > incrementing the number of scheduled-events, instead of getting the > lock. That means the thread which actually hold this lock will process > all the enqueued messages itself, until the counter get down to 0. > >Can you give this version a try ? Looks very good, I am running my application now for 20 minutes or so, and still have no deadlock. I am using now your snapshot versions without any changes. This is very nice, now it's possible to send a pull request for proxy support to the QuickFixJ people the next time you do a Mina release. Thanks a lot, Norbert
Re: Deadlock when using SslFilter and ProxyFilter together
> seems like you have installed the proxy filter before the ssl filter in
> your chain. Can you do the opposite ?
I don't build the chain manually, I simply use a "ProxyConnector" (from package
"org.apache.mina.proxy")
to make a proxy connection (this also explains, why I had the
SslFilter/autoStart problem in my earlier mail).
"ProxyConnector" does the following in "setConnector":
private void setConnector(final SocketConnector connector) {
if (connector == null) {
throw new IllegalArgumentException("connector cannot be null");
}
this.connector = connector;
String className = ProxyFilter.class.getName();
// Removes an old ProxyFilter instance from the chain
if (connector.getFilterChain().contains(className)) {
connector.getFilterChain().remove(className);
}
// Insert the ProxyFilter as the first filter in the filter chain
builder
connector.getFilterChain().addFirst(className, proxyFilter);
}
Re: Deadlock when using SslFilter and ProxyFilter together
> Ok, but you can insert teh SslFilter *before* the proxyFilter. Once you
> get the ProxyConnector instance, get the connector and do :
Well, I am using a patched QuickFixJ version, in which the control flow is a
bit different.
Here is an excerpt from the code they are using:
CompositeIoFilterChainBuilder ioFilterChainBuilder = new
CompositeIoFilterChainBuilder(
userIoFilterChainBuilder);
boolean useProxy = proxyType != null && proxyPort > 0 &&
socketAddresses[0] instanceof InetSocketAddress;
SslFilter sslFilter = null;
if (sslEnabled) {
boolean autoStart = !useProxy;
sslFilter = installSslFilter(ioFilterChainBuilder, autoStart);
}
ioFilterChainBuilder.addLast(FIXProtocolCodecFactory.FILTER_NAME,
new ProtocolCodecFilter(new FIXProtocolCodecFactory()));
if (useProxy) {
SocketConnector socketConnector = new NioSocketConnector();
socketConnector.setFilterChainBuilder(ioFilterChainBuilder);
ioConnector = ProtocolFactory.createIoProxyConnector(
socketConnector,
(InetSocketAddress) socketAddresses[0], new
InetSocketAddress(proxyHost,
proxyPort), proxyType, proxyVersion, proxyUser,
proxyPassword,
proxyDomain, proxyWorkstation);
} else {
ioConnector =
ProtocolFactory.createIoConnector(socketAddresses[0]);
ioConnector.setFilterChainBuilder(ioFilterChainBuilder);
}
ioHandler = new InitiatorIoHandler(fixSession, sslFilter,
networkingOptions, eventHandlingStrategy);
}
private SslFilter installSslFilter(CompositeIoFilterChainBuilder
ioFilterChainBuilder,
boolean autoStart)
throws GeneralSecurityException {
SslFilter sslFilter = new
SslFilter(SSLContextFactory.getInstance(keyStoreName,
keyStorePassword.toCharArray()), autoStart);
if (enableProtocole != null)
sslFilter.setEnabledProtocols(enableProtocole);
if (cipherSuites != null)
sslFilter.setEnabledCipherSuites(cipherSuites);
sslFilter.setUseClientMode(true);
ioFilterChainBuilder.addLast(SSLSupport.FILTER_NAME, sslFilter);
return sslFilter;
}
Sure, with a lot of time, I could possibly rewrite the code, to swap the
positions of the filters in the filterchain.
But I see little sense in it, since the code is working. The only problem I
have, is that I get this deadlock after a few minutes or so.
At the moment, as a workaround, I simply disabled the "synchronized(handler) {
... }" in ProxyFilter.writeData(),
I think for my special case this will work.
I don't think it is possible, that the filterchains are traverted in the same
order for reading and writing.
Re: Deadlock when using SslFilter and ProxyFilter together
I attached a stacktrace of the two threads, which are in a deadlock.
Jon maybe right, when saying the write stack is threadsafe, the problem is,
that simultaneously
a read operation is going on, and both stacks use the same filters.
Thread "NioProcessor-2":
flushScheduledEvents():308, SslHandler {org.apache.mina.filter.ssl}
messageReceived():534, SslFilter {org.apache.mina.filter.ssl}
callNextMessageReceived():542, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
access$1300():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
messageReceived():943, DefaultIoFilterChain$EntryImpl$1
{org.apache.mina.core.filterchain}
messageReceived():153, ProxyFilter {org.apache.mina.proxy.filter}
callNextMessageReceived():542, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
access$1300():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
messageReceived():943, DefaultIoFilterChain$EntryImpl$1
{org.apache.mina.core.filterchain}
messageReceived():109, IoFilterAdapter {org.apache.mina.core.filterchain}
callNextMessageReceived():542, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
fireMessageReceived():535, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
read():684, AbstractPollingIoProcessor {org.apache.mina.core.polling}
process():638, AbstractPollingIoProcessor {org.apache.mina.core.polling}
process():627, AbstractPollingIoProcessor {org.apache.mina.core.polling}
access$600():68, AbstractPollingIoProcessor {org.apache.mina.core.polling}
run():1096, AbstractPollingIoProcessor$Processor {org.apache.mina.core.polling}
run():64, NamePreservingRunnable {org.apache.mina.util}
runWorker():1142, ThreadPoolExecutor {java.util.concurrent}
run():617, ThreadPoolExecutor$Worker {java.util.concurrent}
run():745, Thread {java.lang}
Thread "FixMessageProcessor(0)::ABFX":
writeData():209, ProxyFilter {org.apache.mina.proxy.filter}
filterWrite():192, ProxyFilter {org.apache.mina.proxy.filter}
callPreviousFilterWrite():625, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1
{org.apache.mina.core.filterchain}
flushScheduledEvents():316, SslHandler {org.apache.mina.filter.ssl}
filterWrite():653, SslFilter {org.apache.mina.filter.ssl}
callPreviousFilterWrite():625, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1
{org.apache.mina.core.filterchain}
filterWrite():321, ProtocolCodecFilter {org.apache.mina.filter.codec}
callPreviousFilterWrite():625, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
access$1500():48, DefaultIoFilterChain {org.apache.mina.core.filterchain}
filterWrite():953, DefaultIoFilterChain$EntryImpl$1
{org.apache.mina.core.filterchain}
filterWrite():877, DefaultIoFilterChain$TailFilter
{org.apache.mina.core.filterchain}
callPreviousFilterWrite():625, DefaultIoFilterChain
{org.apache.mina.core.filterchain}
fireFilterWrite():618, DefaultIoFilterChain {org.apache.mina.core.filterchain}
write():530, AbstractIoSession {org.apache.mina.core.session}
write():474, AbstractIoSession {org.apache.mina.core.session}
send():51, IoSessionResponder {quickfix.mina}
send():2534, Session {quickfix}
sendRaw():2466, Session {quickfix}
send():2519, Session {quickfix}
sendToTarget():635, Session {quickfix}
sendMessageToTarget():529, FixApplication {de.digitec.d3.gateway.fix}
subscribe():52, FixProducerHandle {de.digitec.d3.gateway.fix}
subscribe():134, FixApplication {de.digitec.d3.gateway.fix}
processMessage():288, FixMessageProcessor {de.digitec.d3.gateway.fix}
run():245, FixMessageProcessor {de.digitec.d3.gateway.fix}
run():878, FixApplication$MessageProcessorThread {de.digitec.d3.gateway.fix}
DIGITEC GmbH
Johannes-Brahms-Platz 1
20355 Hamburg
Tel +49 40 23776-147
Fax +49 40 23776-200
www.DIGITEC.de
___
DIGITEC GmbH EDV-Beratung, Soft- und Hardware
Sitz Hamburg - HRB 288 18 Amtsgericht Hamburg
Geschäftsführer: Hans-Joachim Hänel, Hans-Jürgen Joost
Finanzamt Hamburg-Mitte, Steuer-Nr. 48/715/01586 - VAT-Nr. DE 118646977
WICHTIGER HINWEIS
Die Kommunikation mit DIGITEC GmbH per Email dient nur dem Zweck der schnellen
Information.
Rechtsverbindliche Absprachen kommen über dieses Medium nicht zustande !
IMPORTANT NOTICE
Communication with digitec GmbH via email is for quick information purposes
only.
This medium is not to be used for the exchange of legally binding statements !
Re: Deadlock when using SslFilter and ProxyFilter together
Hello, I did a "threaddump" with visualvm in the deadlock situation (not sure if this is what you wanted, but I don't see anything else), it shows the same deadlock situation (at the bottom of this mail), as in my previous mail. 2016-02-10 13:04:58 Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode): "Thread-7" #29 daemon prio=5 os_prio=0 tid=0x1ed49000 nid=0xfb0 runnable [0x1f66e000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) - locked <0x00076ee06830> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.skip(BufferedReader.java:411) - locked <0x00076ee06830> (a java.io.InputStreamReader) at de.digitec.d3.pricing.streaming.impl.StreamingChannelImpl.readMessage(StreamingChannelImpl.java:331) at de.digitec.d3.pricing.streaming.impl.Session.run(Session.java:116) - locked <0x00076ee068c8> (a java.lang.Object) Locked ownable synchronizers: - None "SiDispatcher_ReadLoop" #28 prio=5 os_prio=0 tid=0x1ed48800 nid=0x17fc waiting on condition [0x221de000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00076f04c960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at de.digitec.d3.pricing.streaming.impl.SessionManagerAbstractImpl.readMessage(SessionManagerAbstractImpl.java:49) at de.digitec.d3.pricing.streaming.impl.SiDispatcher.run(SiDispatcher.java:455) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - None "Thread-6" #27 daemon prio=5 os_prio=0 tid=0x1ed47800 nid=0x1674 runnable [0x2202f000] java.lang.Thread.State: RUNNABLE at java.net.DualStackPlainSocketImpl.accept0(Native Method) at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131) at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199) - locked <0x00076ee102c8> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:545) at java.net.ServerSocket.accept(ServerSocket.java:513) at de.digitec.d3.pricing.streaming.impl.SocketAcceptor.accept(SocketAcceptor.java:152) at de.digitec.d3.pricing.streaming.impl.SocketAcceptor.run(SocketAcceptor.java:128) Locked ownable synchronizers: - None "SessionManagerStreamingImpl_Timer" #26 daemon prio=5 os_prio=0 tid=0x1ed47000 nid=0x1bbc in Object.wait() [0x21d9f000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x00076ee18180> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) Locked ownable synchronizers: - None "SiDispatcher_Timer" #25 prio=5 os_prio=0 tid=0x1ed46000 nid=0x12d8 in Object.wait() [0x2030f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at java.util.TimerThread.mainLoop(Timer.java:526) - locked <0x00076ee18350> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) Locked ownable synchronizers: - None "RMI TCP Connection(2)-172.16.125.86" #24 daemon prio=5 os_prio=0 tid=0x1ed45800 nid=0x1704 runnable [0x218ad000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) - locked <0x00076ee12618> (a java.io.BufferedInputStream) at
SslFilter with autoStart=false
Hello,
I am trying to patch QuickFIX/J to use SOCKS Proxy connections.
I found a patch on the QuickFIX/J Jira (QFJ-285), which I used as a start,
and which, after some
tweaking, I got to work with Mina-2.0.10/Mina-2.0.11.
The main problem, when doing so, was, that I had to disable the autostart
feature of the SslFilter
in order to build the filterchain (ProxyFilter, SslFilter, FixProtocolFilter)
without exceptions, and there wasn't a public function
to start the Ssl handshake later on (SslFilter.startSsl only works after the
initial handshake, not if autostart=false).
I solved the problem by making the function "SslFilter.initiateHandshake"
public, and executing the following code
in the event handler "AbstractProxyIoHandler. proxySessionOpened":
@Override
public void proxySessionOpened(IoSession session) throws Exception {
log.info("proxySessionOpened called");
if (sslFilter != null) {
IoFilter.NextFilter nextFilter =
session.getFilterChain().getNextFilter(sslFilter);
sslFilter.initiateHandshake(nextFilter, session);
}
}
Maybe you can incorporate the same change in the Mina code (making
initiateHandshake public, I mean), or provide some other means
to initiate the first SSL handshake, when autostart=false.
Regards.
Norbert
