Re: Multithreading problem
This won't work for a variety of reasons. One is that an SSL_write may fail because of a negotiation in progress and being able to *read* data from the socket may allow the write to progress. Even in my dreams I didn't imagined that you have try to compile it. Of course it won't work!!! That's why i put this: ...arguments to functions are not exactly as in real implementation and some function are simplifed versions of real one, some functions are omitted... and this: --- PSEUDO CODE and datailed description how it works in my previous post. In this piece of pseudo code, i have tried to show what will happen if SSL_write will return 'SSL_ERROR_WANT_WRITE'. It's only a little part of what complete procedure should deal with (but if one part doesn't work, whole procedure is worth nothing). (There is one bug in this pseudo code, I mean: in case SSL_ERROR_WANT_WRITE: There should be WaitOnWrite = true; not WaitOnWrite = false; but you probably have assumed that.) Again, this pseudo code was only to show one thing. I haven't imagined that you start to optimize it. The most important part not included in this pseudo code is one for reading from the same connection to another user buffer. It should work exactly the same like for example 's_client.c' in OpenSSL soure tree but in windows environment. I would suggest some changes to this loop that should solve your problems: 1) Protect all SSL_read/SSL_write functions with a mutex (per-connection if you want). This will prevent you from calling SSL_read an SSL_write at the same time for the same connection. It will also allow you to synchronize your handling of cases where you need to wait. Really you can give me tons of good advices, but it will be worth nothing if in any line of your advice you say this: Again, the short version is this: Anytime anything changes, retry all operations that were waiting for something. You are doing everything except trying to understand that your knowlage about something has changed comes from windows, and windows notifies you about it in special cases. It is possible that something will change and windows won't notify you. You have to know about it before you block in wait function. Windows sends FD_WRITE event if writing is possible and previous invokation of 'send' has failed with WSAEWOULDBLOCK code (only this code, if it has failed from other reason, FD_WRITE will not be send and waiting for it will cause deadlock)... What words should i use, you will understand this?! Nevermind, i cut this thread ,forget about everything, this is my last post, don't answer to it... Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Multithreading problem
If OpenSSL is unable to perform an operation, that means it tried it and
couldn't do it. If that operation was a write, then when it becomes possible,
you will get a write event. If that operation wasn't a write, then who cares
when a write becomes possible?
Maybe I show it using simple pseudocode (arguments to functions are not exactly
as in real implementation and some function are simplifed versions of real one,
some functions are omitted):
--- PSEUDO CODE
bool WaitOnWrite = false;
int res;
// it associates FD_WRITE event with SockEvent object
WSAEventSelect(sock,SockEvent,FD_WRITE);
for (;;)
{
if (WaitOnWrite)
WaitForMultipleObjects(SockEvent)
else
WaitForMultipleObjects(UserEvent);
res = SSL_write(sock,userBuf,len);
res = SSL_get_error(sock,res);
switch(res)
{
case SSL_ERROR_NONE:
WaitOnWrite = false;
break;
case SSL_ERROR_WANT_WRITE:
WaitOnWrite = false;
break;
}
}
--- PSEUDO CODE END
The code above works like this: When it starts it waits for data from user
(user signals when he wants to send some data by setting UserEvent), then
SSL_write sends data from userBuf, if SS_write will success then procedure is
repeated if not program will wait for FD_WRITE event to occur. But
unfortunately it can deadlock. Why? Because there is clearly specified in
windows help that FD_WRITE event only will be send if previous invoke of 'send'
function will fail with WSAEWOULDBLOCK error. (if you don't issue 'send' or
'send' will fail from other reason, FD_WRITE will not be send!). So I have to
know, what reason 'SSL_write' failed or I can deadlock. If i use nonencrypted
connection(Winsock API) it's not a problem. After 'send' command I can check
error by using 'GetLastError'command. But if I use OpenSSL ('SSL_write'
replaces 'send') the only message I will get is SSL_ERROR_WANT_WRITE. SSL_write
can internaly issue 'send' a couple of times, it can clear or modify windows
last error code, SSL_ERROR_WANT_WRITE can be generated from other reason then
underlaying winsock failur (internal OpenSSL logic), finally 'send' may fail
from other reason then WSAEWOULDBLOCK. Now, I hope, it's clear for you.
Conclusion:
OpenSSL is designed to work with 'select' statement. In windows environment
'select' can not be used, and native windows API for waiting on multiple
objects seems to be incompatibile with openSSL (i should rather say OpenSSL is
incompatibile with windows). Writing applications that perform asynchronous IO
operations (interleaving input and outputs) seems to questionable. That's my
opinion - nobody shows me working code so far.(timeouts and polling are rather
for testing then for serious code)
I don't get it. Why are you back on how to tell when a socket event occured?
I thought you had solved that problem by using events.
No,no. All the time i've tried to show you that I haven't. I have said that I
have an idea, but this is only my idea, it's not simple using of windows events
but rather special mixing windows events with 'select' statements, it has
rather complex logic, it has several drawbacks (in normal usage it generates
sometimes some unnecessary function invokations and when something goes wrong i
have to kill the thread or close connection from other (but i have to do it in
reason of serious failure, not normal usage)) and this is connected with logic
of my application (for other purpouses this method is not suitable - especially
for server code). If I have no choice i will use it, but it's always worth to
try something better...
Lucas
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: Multithreading problem
Don't get me wrong, but: I have really appreciated you for time you have devoted to answer me, but unfortunately it seems like it's rather your desire to be guru for others (i don't want to say ignorance) then real desire to help me, is why you are doing this. (I really don't want be malicious, i'm only trying to prevent some usless conversation). Have you at least read, what i have written ? No !!! You haven't. If you had done this you haven't answered: Retry the write when anything changes. That would include receiving an FD_WRITE event. It doesn't matter what caused the error. Whatever it is will only go away when something changes. There is no way that something can change without you knowing about it. Nothing is going to change until or unless the socket changes state or you make some call into OpenSSL. You can wait for something to change. If nothing ever changes, the write will continue to fail. SSL_ERROR_WANT_WRITE is OpenSSL's way of saying that something has to change. to text in which i have tried to show you that waiting in windows environment for events is comlicated task. If you want to safely wait in blocking function for some condition to be met, two things must happen: 1. The condition must be met. 2. Operating system must tell your function about it. Who are you? Philosopher or programmer? If you are philosopher then yes, you have right, i must be really dumb to not understand that condition will be met if it will be met. (1) But if you are programmer second condition (2) is not such obvious. Really, read my previous post then we will talk. I can give you more information if the one from previous post are not enough. But now i don't want to write if no one is reading. You are making things really complex. If you want complete control over I/O, use BIO pairs. No,no. It doesn't help. Instead of some SSL_ERROR i will get some BIO_ERROR... Am I making things really complex??? Now, I just want to get working code. It's not my fault that it is not as easy as someone may think. Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Multithreading problem
No. I don't like blocking sockets because it's very hard to get them right Experience, how EASY it could be done in nonblocking mode is what i'm currently correlating with your words. ... let's cut this this thread. I've droped idea of using third party components and now i'm writing code at lowest level. But it not takes to long to stuck at some point. Here is my problem: When connection is in non-blocking mode 'SSL_read' and 'SSL_write' can return 'SSL_ERROR_WANT_WRITE' or 'SSL_ERROR_WANT_READ'. This means that command should be repeated some time later. In unix environment you can use 'select' statement to block until desired condition is valid. On unix you can have 2 handles - one socket and one pipe(for communication from another thread), so 'select' blocks until one of the following events is met: 1.'Read' from the socket is signaled(data on the wire) 2.'Read' from the pipe is signaled(user wants to send something) 3.'Write' from to socket is signaled(socket is ready to send data, obtained first from user by pipe) and this example is covered for example in 's_client.c' file in openSSL source tree. (or in examples from ) But unfortunataly it is not valid under windows. 'select' function under windows accepts only sockets from the same provider(TCP provider in my example), so you cannot mix pipes and sockets in the same 'select' statement. So if you want use 'select' you have to choices: 1. use 'select' with time intervals and poll the pipe (LAME) 2. open internal TCP connection from thread to another (VERY LAME) Windows has special API to deals with such kind of architecture. There are two main functions: WSAAsyncSelect and WSAEventSelect (and helper functions). I'm using secon one. It works like this: 1. WSAEventSelect associates network events from some socket with event object. When some those events occurs on the socket (for example: FD_READ,FD_WRITE...) windows signals event object. 2. User creates own event object and signals it when he wants to send some data 3. Communication thread invokes WaitForMultipleObjects function with socket event and user event as arguments. Function blocks until there is something to do (read data, send data, etc.) It works good (for unencrypted sockets of course) but unfortunately in another way then someone may think. 'select' statement returns immediately when desired condition is met - for example you are waiting for some data to arrive and this data arrives 'select' returns, if you will not read this data and you will invoke 'select' again it returns immediately because there is still data to read. The same is for write. 'select' returns if there is possibility to send data. For windows events it works like this: Thread is waiting for read condition to be met - for example it is waiting for some data to arrive. Data arrives, and thread weaks up. It signals that there is some data, but if you will not read this data and You will decide to wait again thread will not wake up !!! Windows supposes that you were noticed about some event and there is no need to notice you again. For write event it is much worst. Windows supposes that you can always write to socket and only when some special error occurs (WSAEWOULDBLOCK) it signals possibility to write when problem disappears. In practice it is more complicated, socket functions (send,recv) change internally some states. It is written in windows help files how it works (check MSDN for detailed explanation). It works good if you have control over issued commands. But if you are using third party components like openSSL which use some functions internally (recv,send) it is very complicated to predict behaviour of code. For example if return value from SSL_write is SSL_ERROR_WANT_WRITE what should I do? if SSL_write failed because send(used internally by openSSL) has returned WSAEWOULDBLOCK then I can wait for FD_WRITE condition to be met, but if it has failed from another reason waiting function will block I don't known which socket functions are issued by openSSL command, how many times and what are the error codes. It,s ill, really ill. I have no force to write (I have some idea how it could be done (mix wit 'select') but first i try to ask someone) David, do you have any reference example how it should be done in windows environment??? Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: RE: Multithreading problem
First... sorry for trash in my post's subjects. I'm using www interface on my email provider site for sending emails and there is no option to change this. (I don't known if this is my mailbox or this mailing list server problem). Actually, it's extremely complicated. For example, what do you do if you call 'write' and it doesn't return in a reasonable amount of time? You cannot use 'select' with blocking sockets. If you do, and your 'write' blocks (say because only a few bytes could be written at that instant), you won't be able to call 'read'. I don't known if it was your exact intention but you have suggested me some problem - that delay in sending packet could cause incoming buffer overflow due to not reading data by a long time, hence data loss. But this is the not problem of idea of blocking as a such but rather using mutexes with blocking sockets. In native socket API delaying in writing blocking socket has no impact on reading one. It is only problem with using mutexes for serializing data which should be done due to openssl non multithreading. I can use 'select' with blocking sockets, it will not block on incoming data even if there are not writing data. Of course, as you have said, a can't use it for reading because it will block. Using 'select' for signaling possible data is not prohibited.(it should be clarifying for other readers) So it works like this: Well... this is not exactly solution to my problem. I have asked about blocking sockets in special context. I'm using opessl by delphi component which is intrinsic designed to work in true blocking mode (which is fine for native socket API, encrypted connection is additional option for it). What you have suggested is some kind of emulation which is rather usless in my case (SSL_read and SSL_write are hardcoded in component code in blocking mode - I thought rather about doing some openSSL API calls before invoking component socket read method, ensuring that the method will be invoked if there are some data on the socket causing it to not block). Doesn't this kind of prove that your assumption (that non-blocking sockets are more complicated) is wrong? Look at all the craziness you have to go through to get blocking sockets to work right. Eeee I'm little bit surprising about your interpretation. I thought all the craziness that i have to deal with is due the fact that OpenSSL is not supporting multithreading. I don't blame anyone for this, maybe it is not as easy as someone who did't implemented this may think. I'm only trying to show my problem and find most suitable and easiest solution. I gain an impression (correct me if I'm wrong) that you are trying to compromise the idea of blocking sockets only because openSSL doesn't support it. Many people (like me) are using native socket API (in which blocking socket are natural and correct working) and suppose that openSSL API would be the same - that's why there are many problems with that.(additionaly, just like in my case, it is not only to change my thinking about using socket but also to change third party libraries). Anyway, thanks for your help David. Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: RE: Multithreading problem
I am very, very new to openssl. There is a good example (Example 5-16, Network Security with Openssl book)) for using nonblocking openssl. It is easy to understand. It uses one thread to handle 2 nonblocking socket. You may have to modify it to handle multithread. At least, you have example to follow. I am trying to modify this example, so one openssl socket is handled by one thread using select ( native API) to monitor socket. Thao Dinh Thanks Thao for suggestion. It is not only problems with writing correct code but also wish to use some components (which I have used for communication without encryption) forces me to raise this problem on this mailing list. But of course examples given by you should be useful for me (if i will implement this(probably i will heve to)). Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Multithreading problem
This problem was raised on this mailing list many times, but the clear solution (in my opinion) was not given. From OpenSSL FAQ: ...an SSL connection may not concurrently be used by multiple threads... This means that I can't have 2 threads, one reading and one writing at the same time from the same socket. My application is basic Jabber communicator (messager) I should to constantly listen on socket for incoming messages and at the same time send messages written by me.(this is not communication model like for example in http: request,response,request,response.etc.) If I use simple TCP connection I create 2 threads one reading, one writing. This is simple,fast and correct.(reding and writing are blocking). But when have to SSL connection this is much more complicated. I'm using Delphi and Indy components. There are sugesstion on mailing list that concurrent socket usage can be avoided by creating non-blocking socket and mutex, which is locked when any thread is using socket. But non-blocking socket is more complex to implement and forces me to not use Indy component, since Indy components are desined to work only in blocking mode (for TCP sockets this is correct design, I've read that Indy 10 has an option in core to work in non-blocking mode but I don't known if this option is exposed to user the same way as in socet API (maybe it was added to other purposes), but I have Indy 9 and don't want to upgrade). So using nonblocking sockets forces me to implement everything in native socket API, using OpenSSL API (currently Indy does it internally) and deal with additional complexivity of nonblocking sockets. Is there any OpenSSL function similar to socket API 'select' (SSL_select) If yes, then I can use blocking sockets. One thread is waiting in blocking SSL_select for incoming messages, If massage will come then this thread will try to acquire mutex and then carry out blocking SSL_read which will not block because there is message on socket. I can't do this with socket API 'select' because it signals any data on socket not exactly data on which SSL_read will not block. (TLS renogotiation or something like this). I hope you known what I mean. (I have tried with SSL_pending but it return 0 even if there are data on socket) Any suggestions? Can someone help me with this? Lucas __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
