subject:"Issue with keep\-alive connections, when using APR Connector on Windows and starting Processes from Servlets"

Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets

2012-07-02 Thread Laurent Petit

On Fri, 2012-06-29 at 17:18 +0200, Jeff MAURY wrote:
 Hello,
 
 I found the correct code to look at:
 http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/src/network.c
 It does not set the inherit flag but it is using the apr library that maybe
 set the flag under Windows.
 If this is the case and according to your references, it is likely the
 client will not receive the ack and this is a problem with APR and
 Runtime.exec

Hello,

Seems like it's function apr_socket_create, defined in this APR file:

http://svn.apache.org/repos/asf/apr/apr/trunk/network_io/win32/sockets.c


which apparently already tries to create a non-inherited handle.

The code, for reasons beyond my current understanding, relies at compile
time on the flags APR_HAS_UNICODE_FS and _WIN32_WCE.

It also relies at runtime on IF_WIN_OS_IS_UNICODE, which may (?) be an
indirect indication that the underlying OS needs SetHandleInformation to
be explicitly called.


Jeff, do you have ideas/indications on how I could help push the issue
resolution forward ? We're reaching the limits of my current abilities
to help, not having a windows box with C compiler to do tests, etc.  :-(




You'll find below the relevant code:

#ifdef WIN32
/* Socket handles are never truly inheritable, there are too many
 * bugs associated.  WSADuplicateSocket will copy them, but for our
 * purposes, always transform the socket() created as a
non-inherited
 * handle
 */
#if APR_HAS_UNICODE_FS  !defined(_WIN32_WCE)
IF_WIN_OS_IS_UNICODE {
/* A different approach.  Many users report errors such as 
 * (32538)An operation was attempted on something that is not 
 * a socket.  : Parent: WSADuplicateSocket failed...
 *
 * This appears that the duplicated handle is no longer
recognized
 * as a socket handle.  SetHandleInformation should overcome
that
 * problem by not altering the handle identifier.  But this
won't
 * work on 9x - it's unsupported.
 */
SetHandleInformation((HANDLE) (*new)-socketdes, 
 HANDLE_FLAG_INHERIT, 0);
}
#if APR_HAS_ANSI_FS
/* only if APR_HAS_ANSI_FS  APR_HAS_UNICODE_FS */
ELSE_WIN_OS_IS_ANSI
#endif
#endif
#if APR_HAS_ANSI_FS || defined(_WIN32_WCE)
{
HANDLE hProcess = GetCurrentProcess();
HANDLE dup;
if (DuplicateHandle(hProcess, (HANDLE) (*new)-socketdes,
hProcess, 
dup, 0, FALSE, DUPLICATE_SAME_ACCESS)) {
closesocket((*new)-socketdes);
(*new)-socketdes = (SOCKET) dup;
}
}
#endif

#endif /* def WIN32 */




 
 Jeff
 
 
 On Fri, Jun 29, 2012 at 4:38 PM, Laurent Petit lpe...@yseop.com wrote:
 
  On Fri, 2012-06-29 at 15:27 +0200, Jeff MAURY wrote:
   No,
  
   it does not (yet) make sense because one piece of the puzzle is missing.
   The Local socket is not a TCP socket, it is created from a path rather
  than
   from an IP address.
   So this socket is in no way linked to the browser, at least directly, and
   the missing piece is what is the relation from this socket to the
  browser.
 
  You're right, so the question is still open.
 
  In the mean time, I'm come across interesting google search results,
  among which the first one seems really interesting:
 
 
  Oracle Bug 6428742
  ==
 
  Currently, on Windows, Runtime.exec is implemented by calling
  CreateProcess with the bInheritHandles argument set to TRUE,
  which causes open handles to not be closed if they are not
  explicitly set to be non-inherited.  This is the underlying cause
  of many subtle bugs, such as
 
  6347873: (so) Ports opened with ServerSocketChannel blocks when using
  Runtime.exec
  http://monaco.sfbay/detail.jsf?cr=6347873;
 
 
 
  == Note the if they are not explicitly set to be non-inherited =
  this seems like a game changer, since our implicit assumption was that
  we would just look for code which would have accidentally set the handle
  to be inherited, not the other way around.
 
  http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6428742
 
 
 
  QT Project
  ==
 
  When using spawned processes, sockets opened by the client remain in
  CLOSE_WAIT state after closing the connection. Sockets on the remote
  side would be in FIN_WAIT_2 state and waiting for ACK.
  When the spawned (child) process is closed, then ACK is sent and sockets
  are finally moved to CLOSE state on both sides.
  This seems to be caused by the default handle inheritance on Windows by
  child processes.
 
  https://bugreports.qt-project.org/browse/QTBUG-4465
 
 
 
  Zeromq2 project
  ===
 
  On Windows platform, in order to prevent child processes to inherit
  parent sockets, SetHandleInformation
  (
  http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935(v=vs.85).aspx)
  should be called on the socket handle.
 
  This patch comes out from discussion at:
  https://github.com/zeromq/clrzmq/issues/60;

Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets

2012-06-29 Thread Laurent Petit

Hello Jeff, Konstantin  all,

On Mon, 2012-06-25 at 20:52 +0200, verlag.preis...@t-online.de wrote:
 Hello Jeff  all,
 
  Von: Jeff MAURY jeffma...@jeffmaury.com
  Datum: Mon, 25 Jun 2012 18:46:02 +0200
 
  Konstantin,
  
  your explanations are very interesting but unclear to me: what do you
  call the inactivity timer ? When it is started ? After the request has
  been processed by the servlet ? In that case, I see no difference
  between a servlet that launch a process and another one.
  It seems to me that a process that is launched does not inhererits
  handles from its parent process but it's possible that under Windows,
  it's an option so it would be interesting to watch.
  
  Jeff
  
 
 Sorry, I'm just a normal Tomcat user, and I don't know how exactly the APR 
 connector and its Timeout works, so I am unable to answer that.
 
 
 Howewer, I did some further observations:
 
 -When I perform a request to the servlet that opens wordpad.exe, the TCP 
 connection from Tomcat does not close after the timeout - even when I kill 
 the Tomcat process (java.exe), the TCP connection is still open. If I kill 
 wordpad.exe, then finally the connection is closed/aborted.
 -When I have 1 TCP connection open to Tomcat and the servlet starts the 
 little C program, Task manager shows that it has 11 handles.
 However, when I have 5 TCP connections open to Tomcat, and do the request on 
 one of them, Task maanger shows that the C program has 15 handles - so four 
 more handles when there are four more connections to Tomcat. All of that 5 
 TCP connections don't close until I kill that process.
 
 That seems to me to be an indication that socket handles could be inherited 
 by the child processes that are startet by ProcessBuilder from tomcat.
 
 A msdn article mentions how to create a new process using CreateProcess(); it 
 also mentions that socket handles can be inherited: 
 http://msdn.microsoft.com/en-us/library/windows/desktop/ms724466.aspx
 
 However, as I don't have much knowledge about programming with WinAPIs, I 
 don't know why those handles are inherited (the MSDN article mentions that a 
 handle must be specified as inheritable when created, to allow a child 
 process to inherit it). Maybe someone with more WinAPI/Tomcat Native 
 knowledge can help here.


I also had the vague intuition that this related to handles inheritence.
Your recent tests  research tend to make this hypothesis even more
appealing.

I know nothing about tomcat-native implementation, but I was able to see
that the bInheritHandle member is set to true here in tomcat native's C
code:

lpetit:~/tmp/tomcat-native $ grep bInheritHandle -R *
native/os/win32/ntpipe.c:con-sa.bInheritHandle = TRUE;

(
http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/os/win32/ntpipe.c
 )


Jeff, 

What else could we do to help investigate / fix this issue ?


Cheers,

-- 
Laurent


 
 Regards,
 Konstantin Preißer
 
 
 
 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org
 

-- 
Laurent Petit

Agence +33 (0)4 78 47 07 49

Email lpe...@yseop.com

 



 

Yseop apporte une réponse intelligente et individualisée à chacun de vos
clients

 

www.yseop.com




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets

2012-06-29 Thread Jeff MAURY

That what I guessed but I don't understand everything.
The code you are referencing is related to NTPipes and not sockets. So I'm
not familiar with Tomcat Native implementation, but do you know the global
architecture ?
What I guessed is that the Tomcat native stuff created with the inherit
flag, but even if it does, I don't see why it should not work if the socket
is closed after a timeout.

Regards
Jeff



On Fri, Jun 29, 2012 at 11:22 AM, Laurent Petit lpe...@yseop.com wrote:

 Hello Jeff, Konstantin  all,

 On Mon, 2012-06-25 at 20:52 +0200, verlag.preis...@t-online.de wrote:
  Hello Jeff  all,
 
   Von: Jeff MAURY jeffma...@jeffmaury.com
   Datum: Mon, 25 Jun 2012 18:46:02 +0200
 
   Konstantin,
  
   your explanations are very interesting but unclear to me: what do you
   call the inactivity timer ? When it is started ? After the request has
   been processed by the servlet ? In that case, I see no difference
   between a servlet that launch a process and another one.
   It seems to me that a process that is launched does not inhererits
   handles from its parent process but it's possible that under Windows,
   it's an option so it would be interesting to watch.
  
   Jeff
  
 
  Sorry, I'm just a normal Tomcat user, and I don't know how exactly the
 APR connector and its Timeout works, so I am unable to answer that.
 
 
  Howewer, I did some further observations:
 
  -When I perform a request to the servlet that opens wordpad.exe, the TCP
 connection from Tomcat does not close after the timeout - even when I kill
 the Tomcat process (java.exe), the TCP connection is still open. If I kill
 wordpad.exe, then finally the connection is closed/aborted.
  -When I have 1 TCP connection open to Tomcat and the servlet starts the
 little C program, Task manager shows that it has 11 handles.
  However, when I have 5 TCP connections open to Tomcat, and do the
 request on one of them, Task maanger shows that the C program has 15
 handles - so four more handles when there are four more connections to
 Tomcat. All of that 5 TCP connections don't close until I kill that process.
 
  That seems to me to be an indication that socket handles could be
 inherited by the child processes that are startet by ProcessBuilder from
 tomcat.
 
  A msdn article mentions how to create a new process using
 CreateProcess(); it also mentions that socket handles can be inherited:
 http://msdn.microsoft.com/en-us/library/windows/desktop/ms724466.aspx
 
  However, as I don't have much knowledge about programming with WinAPIs,
 I don't know why those handles are inherited (the MSDN article mentions
 that a handle must be specified as inheritable when created, to allow a
 child process to inherit it). Maybe someone with more WinAPI/Tomcat Native
 knowledge can help here.


 I also had the vague intuition that this related to handles inheritence.
 Your recent tests  research tend to make this hypothesis even more
 appealing.

 I know nothing about tomcat-native implementation, but I was able to see
 that the bInheritHandle member is set to true here in tomcat native's C
 code:

 lpetit:~/tmp/tomcat-native $ grep bInheritHandle -R *
 native/os/win32/ntpipe.c:con-sa.bInheritHandle = TRUE;

 (

 http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/native/os/win32/ntpipe.c)


 Jeff,

 What else could we do to help investigate / fix this issue ?


 Cheers,

 --
 Laurent


 
  Regards,
  Konstantin Preißer
 
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
  For additional commands, e-mail: users-h...@tomcat.apache.org
 

 --
 Laurent Petit

 Agence +33 (0)4 78 47 07 49

 Email lpe...@yseop.com







 Yseop apporte une réponse intelligente et individualisée à chacun de vos
 clients



 www.yseop.com




 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org




-- 
Jeff MAURY


Legacy code often differs from its suggested alternative by actually
working and scaling.
 - Bjarne Stroustrup

http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury

Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets

2012-06-29 Thread Laurent Petit

Hello,

On Fri, 2012-06-29 at 11:45 +0200, Jeff MAURY wrote:

That what I guessed but I don't understand everything.
The code you are referencing is related to NTPipes and not sockets.

I'm not a C expert, but my bet is that the file name is misleading.

Indeed, the

con-sa.bInheritHandle = TRUE;

statement appears in the definition driven by the following macro
expansion:

TCN_IMPLEMENT_CALL(jlong, Local, create)(TCN_STDARGS, jstring name,

jlong pool)

And when the macro call TCN_IMPLEMENT_CALL(jlong, Local, create) is
preprocessed, it gives:

JNIEXPORT RT JNICALL Java_org_apache_tomcat_jni_##Local##_##create

, thus the bInheritHandle = TRUE statement is defined in the following
function signature:

JNIEXPORT RT JNICALL
Java_org_apache_tomcat_jni_##Local##_##create(TCN_STDARGS, jstring name,
jlong pool)

which is a function called by JNDI, and as I guess, by class
org.apache.tomcat.ini.Local, method create() :

http://svn.apache.org/repos/asf/tomcat/native/branches/1.1.x/java/org/apache/tomcat/jni/Local.java

And the javadoc for class Local is: Local socket
And for methode create :
/**

* Create a socket.
* @param path The address of the new socket.
* @param cont The parent pool to use
* @return The new socket that has been set up.
*/

It seems like the native code uses a pool, and my bet is that the pool
is responsible for trying to reuse the socket.

And something like this may happen: the socket handle, inherited by the
child process, puts the socket in such a state that the pool either
can't, either doesn't want to reuse it.

And then what happens is that on the other side of the socket, the
client continues to write HTTP requests, and nobody is listening.

And then when the Tomcat's child process is killed, the socket is
finally closed somehow, the client browser notices it and creates a new
connection to the server and retries the HTTP request.

Does that make sense ?

So I'm
not familiar with Tomcat Native implementation, but do you know the global
architecture ?
What I guessed is that the Tomcat native stuff created with the inherit
flag, but even if it does, I don't see why it should not work if the socket
is closed after a timeout.

Regards
Jeff

On Fri, Jun 29, 2012 at 11:22 AM, Laurent Petit lpe...@yseop.com wrote:

Hello Jeff, Konstantin all,

On Mon, 2012-06-25 at 20:52 +0200, verlag.preis...@t-online.de wrote:
Hello Jeff all,

Von: Jeff MAURY jeffma...@jeffmaury.com
Datum: Mon, 25 Jun 2012 18:46:02 +0200

Konstantin,

your explanations are very interesting but unclear to me: what do you
call the inactivity timer ? When it is started ? After the request has
been processed by the servlet ? In that case, I see no difference
between a servlet that launch a process and another one.
It seems to me that a process that is launched does not inhererits
handles from its parent process but it's possible that under Windows,
it's an option so it would be interesting to watch.

Jeff

Sorry, I'm just a normal Tomcat user, and I don't know how exactly the
APR connector and its Timeout works, so I am unable to answer that.

Howewer, I did some further observations:

-When I perform a request to the servlet that opens wordpad.exe, the TCP
connection from Tomcat does not close after the timeout - even when I kill
the Tomcat process (java.exe), the TCP connection is still open. If I kill
wordpad.exe, then finally the connection is closed/aborted.
-When I have 1 TCP connection open to Tomcat and the servlet starts the
little C program, Task manager shows that it has 11 handles.
However, when I have 5 TCP connections open to Tomcat, and do the
request on one of them, Task maanger shows that the C program has 15
handles - so four more handles when there are four more connections to
Tomcat. All of that 5 TCP connections don't close until I kill that process.

That seems to me to be an indication that socket handles could be
inherited by the child processes that are startet by ProcessBuilder from
tomcat.

A msdn article mentions how to create a new process using
CreateProcess(); it also mentions that socket handles can be inherited:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms724466.aspx

However, as I don't have much knowledge about programming with WinAPIs,
I don't know why those handles are inherited (the MSDN article mentions
that a handle must be specified as inheritable when created, to allow a
child process to inherit it). Maybe someone with more WinAPI/Tomcat Native
knowledge can help here.

I also had the vague intuition that this related to handles inheritence.
Your recent tests research tend to make this hypothesis even more
appealing.

I know nothing about tomcat-native implementation, but I was able to see
that the

Re: Re : Re: Issue with keep-alive connections, when using APR Connector on Windows and starting Processes from Servlets

2012-06-29 Thread Jeff MAURY

No,

it does not (yet) make sense because one piece of the puzzle is missing.
The Local socket is not a TCP socket, it is created from a path rather than
from an IP address.
So this socket is in no way linked to the browser, at least directly, and
the missing piece is what is the relation from this socket to the browser.

Jeff

On Fri, Jun 29, 2012 at 2:40 PM, Laurent Petit lpe...@yseop.com wrote: