Re: Man in the middle proxy - Not working
I have tried so many methods to get the data from the webserver. But in all cases it doesnot seems to be get completed. Can you send me some code snippet to get the whole content. In normal proxying I have used the following method // This is the user defined function which returns true if the source string have "Connection: keep-alive" otherwise false if( !IsConnectionAlive(proxyRecvBuf.buf) ) { do { // Connection is closed here so send output to the source memcpy(sSendBuf,sBuff,iBufferSize); send(OutputSocket,sSendBuf,dwReadDataLen,0); // User defined function for reading the content data from the socket dwReadDataLen = Receive(RequestSock,proxyRecvBuf,proxyRecvOverlapped,dwFlag,WSA_INFINITE,FALSE); if(0 == dwReadDataLen || SOCKET_ERROR == dwReadDataLen) break; }while(true); } else { // Connection alive do { memcpy(sSendBuf,sBuff,iBufferSize); send(OutputSocket,sSendBuf,dwReadDataLen,0); memset(sBuff,0,iBufferSize); // User defined function for reading the content data from the socket // Try to Recieve the data in 1 second interval. dwReadDataLen = Receive(RequestSock,proxyRecvBuf,proxyRecvOverlapped,dwFlag,1000,FALSE); if(0 == dwReadDataLen || SOCKET_ERROR == dwReadDataLen) break; }while(true); } In the 'connect-alive' case I am trying to read the data in 1 second interval. I am not sure about how to achieve this using OpenSSL API's. Please let know is this the right approach to do it in SSL commuincation and also How to set the time out along with a recieve request Thanks, Raj Rajmohan SK - Original Message - From: "Dave Thompson" To: Sent: Saturday, August 21, 2010 6:53 AM Subject: RE: Man in the middle proxy - Not working From: owner-openssl-us...@openssl.org On Behalf Of Raj Sent: Wednesday, 18 August, 2010 06:49 I have tried one more method to read the data from the socket, which was partially successful it is defined as follows do { dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); // Gets the data from the server side SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back to the SSL } while(dwReadDataLen > 0 ); That is my simple until-EOF version, see below. By using this method I am able to read the content data from the server and put it back to my browser. But this method is not consistent though, Sometimes browse request will not get completed and also it takes lot of time complete one browse request That's pretty vague. One off-the-cuff guess: As I alluded to, this method has the limitation that it will only (exit and) close when the server does, so if the request allows keepalive and the server chooses it, you never turn around and see if the client=browser is trying again. Maybe it is. If so, *that* request will never go anywhere. Maybe your browser is timing out that request and retrying on a new (good) connection. Maybe this depends on your browser/version/config or request(s), or even the contents of the response page e.g. script or ActiveX. You could look at the response data (just the headers is enough) to see if keepalive is enabled, and check whether your loop actually exits (i.e. you got EOF, which you *probably* won't *if* the server chose keepalive). Or you could look at both responses and requests on the local side with (I think) ssldump. Or you could use a client which tells you (much) more about the requests it is making; I guess wget might be persuaded. Remember there is a big difference between a webpage and a browser=client request and server response. Typical webpages have CSS, scripts, images, and sometimes frames and objects. One webpage may be 10 or 50 or 200 requests and responses. Often a browser won't show you all of the page, and sometimes even any of it, until all the requests/responses are complete. If this is the problem, you need to either: - get the server to do one response per connection (and close). I know downgrading the request to 1.0 works, and I'm pretty sure replacing or adding as applicable Connection: close on 1.1 does. There may also be server-dependent ways. - recognize the end of the response and close downward (and upward also, since this connection is now orphan). If the browser quickly tries a second request it will get an error, but (much) faster, and more certainly retry (since server async close is a more 'expected' error). - recognize the end of the response and turn around to handle another request (and response etc.). - Replies and quires to the previous posting > For a socket used with openssl directly, I believe OVERLAPPED > will be ignored and is of no use. I think you would have to do > your own 'physical' level either as your own BIO type or
RE: Man in the middle proxy - Not working
> From: owner-openssl-us...@openssl.org On Behalf Of Raj > Sent: Wednesday, 18 August, 2010 06:49 > I have tried one more method to read the data from the > socket, which was partially successful it is defined as follows > do > { > dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); > // Gets the > data from the server side > SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the > data back to > the SSL > } while(dwReadDataLen > 0 ); > That is my simple until-EOF version, see below. > By using this method I am able to read the content data from > the server and > put it back to my browser. But this method is not consistent though, > Sometimes browse request will not get completed and also it > takes lot of > time complete one browse request That's pretty vague. One off-the-cuff guess: As I alluded to, this method has the limitation that it will only (exit and) close when the server does, so if the request allows keepalive and the server chooses it, you never turn around and see if the client=browser is trying again. Maybe it is. If so, *that* request will never go anywhere. Maybe your browser is timing out that request and retrying on a new (good) connection. Maybe this depends on your browser/version/config or request(s), or even the contents of the response page e.g. script or ActiveX. You could look at the response data (just the headers is enough) to see if keepalive is enabled, and check whether your loop actually exits (i.e. you got EOF, which you *probably* won't *if* the server chose keepalive). Or you could look at both responses and requests on the local side with (I think) ssldump. Or you could use a client which tells you (much) more about the requests it is making; I guess wget might be persuaded. Remember there is a big difference between a webpage and a browser=client request and server response. Typical webpages have CSS, scripts, images, and sometimes frames and objects. One webpage may be 10 or 50 or 200 requests and responses. Often a browser won't show you all of the page, and sometimes even any of it, until all the requests/responses are complete. If this is the problem, you need to either: - get the server to do one response per connection (and close). I know downgrading the request to 1.0 works, and I'm pretty sure replacing or adding as applicable Connection: close on 1.1 does. There may also be server-dependent ways. - recognize the end of the response and close downward (and upward also, since this connection is now orphan). If the browser quickly tries a second request it will get an error, but (much) faster, and more certainly retry (since server async close is a more 'expected' error). - recognize the end of the response and turn around to handle another request (and response etc.). > - > Replies and quires to the previous posting > > > For a socket used with openssl directly, I believe OVERLAPPED > > will be ignored and is of no use. I think you would have to do > > your own 'physical' level either as your own BIO type or as > > a BIO_pair looping back to your code (the more usual way). > > Frankly I don't think you're anywhere near ready for that. > > I didn't understand about this, Can you describe this in more > detail, Sorry > for that I am new to this technology > See below for the detail I have, which is not very much. > > You should check for error (<=0) and report/handle it. > I have checked all the error codes of SSL functions in my > application, I have posted only some code snippet to avoid junk data > Good. Mention that with your posted code, to avoid getting sidetracked. > >> SSL_accept(Serverssl); > > This is useless. SSL_accept _creates_ a server-side endpoint; > > it is not applicable to a client-side endpoint. > > I have removed this from my application > > > Also, the data read by SSL_read (like POSIX read or C fread) > > does not get a null terminator byte added, > I have outputted the buffer only for indicative purpose. I > have removed the > code for outputting the buffer > > > That's your problem. SSL_pending only indicates data _already > > received and buffered_ by OpenSSL but not yet read by the app. > } while(SSL_pending(Serverssl)); > > Instead of using the above condition I have opted for > > while(dwReadDataLen > 0 ); > > By using this I was able to read the content data. > See next. > > > For a waited/blocking socket, which is the default as you have here, > > you need to keep reading from the server (and in your case writing > > back to the client) until you've done all the data in the response. > > If you require, or the server chooses, HTTP/1.0 style conn-per-txn > > (also known as connection: close or not-keepalive or not-pipelined, > > and also not-chunked) you can just loop until you receive "EOF" (0) > > from SSL_read, caused by the server closing the connection. > > "EOF" (0) > > I am not sure about EOF(
Re: Man in the middle proxy - Not working
Hi I have created multiple threads for processing the multiple socket request. On each thread I am waiting on a processing a single socket request only May I attach my sample application along with my next posting so that you will get more idea about what I am doing and you can instruct me as well what went wrong in my application. Thanks, Raj Rajmohan SK - Original Message - From: "David Schwartz" To: Sent: Thursday, August 19, 2010 5:51 AM Subject: RE: Man in the middle proxy - Not working Raj wrote: I have tried one more method to read the data from the socket, which was partially successful it is defined as follows do { dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); // Gets the data from the server side SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back to the SSL } while(dwReadDataLen > 0 ); This is the basic idea of how you proxy, but it can't work for a general HTTP proxy. For one thing, it assumes the end of a reply is marked by the close of a connection. This is true for some HTTP requests, but it's not true in general. You can write a proxy two different ways: 1) You can understand the protocol you are parsing and know when it changes directions. Based on this understanding, you can switch from proxying in one direction to proxying in the other. 2) You can avoid having to understand the protocol you are parsing. But in this case, you will not know which side is supposed to send data next, so you must always be ready to proxy in either direction. It seems you do neither of these two things. You try to proxy in only one direction at a time but you don't track the protocol. How do you even know when you've sent the entire request and can even enter this loop? How do you know when you've read the entire reply and can begin reading the next request? Your test condition, 'dwReadDataLen>0' will be true so long as the connection is healthy. It will typically remain healthy even when the reply has been fully sent. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Raj wrote: > I have tried one more method to read the data from the socket, > which was > partially successful it is defined as follows > do > { > dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); // Gets > the > data from the server side > SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back > to > the SSL > } while(dwReadDataLen > 0 ); This is the basic idea of how you proxy, but it can't work for a general HTTP proxy. For one thing, it assumes the end of a reply is marked by the close of a connection. This is true for some HTTP requests, but it's not true in general. You can write a proxy two different ways: 1) You can understand the protocol you are parsing and know when it changes directions. Based on this understanding, you can switch from proxying in one direction to proxying in the other. 2) You can avoid having to understand the protocol you are parsing. But in this case, you will not know which side is supposed to send data next, so you must always be ready to proxy in either direction. It seems you do neither of these two things. You try to proxy in only one direction at a time but you don't track the protocol. How do you even know when you've sent the entire request and can even enter this loop? How do you know when you've read the entire reply and can begin reading the next request? Your test condition, 'dwReadDataLen>0' will be true so long as the connection is healthy. It will typically remain healthy even when the reply has been fully sent. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
Hi Thanks for your valuable suggestion. I didn't understand some points which you described in the previous posting, may because of my lack of exposure to the socket technology. I have tried one more method to read the data from the socket, which was partially successful it is defined as follows do { dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); // Gets the data from the server side SSL_write(SourceSsl,pBuff,dwReadDataLen); // Writes the data back to the SSL } while(dwReadDataLen > 0 ); By using this method I am able to read the content data from the server and put it back to my browser. But this method is not consistent though, Sometimes browse request will not get completed and also it takes lot of time complete one browse request - Replies and quires to the previous posting For a socket used with openssl directly, I believe OVERLAPPED will be ignored and is of no use. I think you would have to do your own 'physical' level either as your own BIO type or as a BIO_pair looping back to your code (the more usual way). Frankly I don't think you're anywhere near ready for that. I didn't understand about this, Can you describe this in more detail, Sorry for that I am new to this technology You should check for error (<=0) and report/handle it. Error on _write especially initial is not common, but if it ever happens, proceeding with other operations will likely cause much greater confusion. I have checked all the error codes of SSL functions in my application, I have posted only some code snippet to avoid junk data SSL_accept(Serverssl); This is useless. SSL_accept _creates_ a server-side endpoint; it is not applicable to a client-side endpoint. I have removed this from my application Also, the data read by SSL_read (like POSIX read or C fread) does not get a null terminator byte added, so outputting pBuff as a C-style string is likely to append garbage, especially on the second or more time through the loop. I have outputted the buffer only for indicative purpose. I have removed the code for outputting the buffer That's your problem. SSL_pending only indicates data _already received and buffered_ by OpenSSL but not yet read by the app. For responses more than one SSL record (max 32kbytes if I recall correctly, and server may choose less) AND (probably) more than the TCP window (varies but typically 2 MTU = about 3kbytes to start) there will be some time delay between receiving the first chunk of the data and the next, and the next and so on. } while(SSL_pending(Serverssl)); Instead of using the above condition I have opted for while(dwReadDataLen > 0 ); By using this I was able to read the content data. For a waited/blocking socket, which is the default as you have here, you need to keep reading from the server (and in your case writing back to the client) until you've done all the data in the response. If you require, or the server chooses, HTTP/1.0 style conn-per-txn (also known as connection: close or not-keepalive or not-pipelined, and also not-chunked) you can just loop until you receive "EOF" (0) from SSL_read, caused by the server closing the connection. "EOF" (0) I am not sure about EOF(0), is that some thing similar to End Of File in C++; If you allow and the server uses HTTP/1.1 keep-alive (or pipelining) and/or chunked data, the situation can get quite a bit more complicated. See RFC 2616. If you use a nonblocking socket (which is supported on Windows as far as I know but is apparently not the same as OVERLAPPED) you can also do your own timeout -- that is, read until EOF or optionally calculated end of the response body, *or* timeout. Since HTTP servers will normally send a complete response within a short time (like at most a few seconds), and if one doesn't a person at a browser usually doesn't want to wait anyway, this can be a good simple compromise. Could you send me some code snippet using 'bio' in SSL, I have seen using 'bio' is some sample applications instead of Sockets Thanks, Raj Rajmohan SK - Original Message - From: "Dave Thompson" To: Sent: Saturday, August 07, 2010 9:06 AM Subject: RE: Man in the middle proxy - Not working From: owner-openssl-us...@openssl.org On Behalf Of Raj Sent: Friday, 06 August, 2010 10:14 I was able to read the content data from the server using SSL_read and put back to the browser by using SSL_write. I don't know whether is a right approach or not. If you are doing an SSL connection to the server then SSL_write to and SSL_read from the server are correct. (And you should since the client is requesting SSL.) SSL_read from and SSL_write back to the client are correct if the client is SSL, and you said it is. For [an .ico] I got the response as follows and I was able to see the icon in
RE: Man in the middle proxy - Not working
> From: owner-openssl-us...@openssl.org On Behalf Of Raj > Sent: Friday, 06 August, 2010 10:14 >I was able to read the content data from the server > using SSL_read > and put back to the browser by using SSL_write. I don't know > whether is a > right approach or not. If you are doing an SSL connection to the server then SSL_write to and SSL_read from the server are correct. (And you should since the client is requesting SSL.) SSL_read from and SSL_write back to the client are correct if the client is SSL, and you said it is. > For [an .ico] I got the response as follows and I was > able to see the > icon in my browser > But for [a .png], which is 42,565 bytes long, I am > receiving the > following output. I understood that there is more to do > inorder to read the > content data, which I am not sure about > Can anybody tell me what else should I do inorder to read > the content > and show it the browser. The following are sending some code snippets > > > RequestSock = > WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); For a socket used with openssl directly, I believe OVERLAPPED will be ignored and is of no use. I think you would have to do your own 'physical' level either as your own BIO type or as a BIO_pair looping back to your code (the more usual way). Frankly I don't think you're anywhere near ready for that. > pHost = gethostbyname(pcTargetURL); > memset(&ClientAddr,0,sizeof(ClientAddr)); > ClientAddr.sin_family = AF_INET; > memcpy(&ClientAddr.sin_addr,pHost->h_addr, > pHost->h_length); > ClientAddr.sin_port = htons(atoi(pcPort)); > if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, > sizeof(SOCKADDR_IN))) > { > closesocket(RequestSock); // Connection failed > return false; > } > > SSL *Serverssl; > Serverssl = SSL_new(m_pSSLCtx); > SSL_set_fd(Serverssl, RequestSock); > iRes = SSL_connect(Serverssl); > if(iRes <= 0 ) > { > ERR_print_errors_fp(stderr); > cout << " connect Failed " << endl; > } >iRes = SSL_write(Serverssl,pcData, strlen(pcData)); You should check for error (<=0) and report/handle it. Error on _write especially initial is not common, but if it ever happens, proceeding with other operations will likely cause much greater confusion. > SSL_accept(Serverssl); This is useless. SSL_accept _creates_ a server-side endpoint; it is not applicable to a client-side endpoint. > do > { > dwReadDataLen = > SSL_read(Serverssl,pBuff,iBufferSize); >SSL_write(SourceSsl,pBuff,dwReadDataLen); >cout << "Read buffer \n" << pBuff << endl; Again check for errors. Especially on the _read side, they are actually quite possible. Also, the data read by SSL_read (like POSIX read or C fread) does not get a null terminator byte added, so outputting pBuff as a C-style string is likely to append garbage, especially on the second or more time through the loop. >} while(SSL_pending(Serverssl)); > That's your problem. SSL_pending only indicates data _already received and buffered_ by OpenSSL but not yet read by the app. For responses more than one SSL record (max 32kbytes if I recall correctly, and server may choose less) AND (probably) more than the TCP window (varies but typically 2 MTU = about 3kbytes to start) there will be some time delay between receiving the first chunk of the data and the next, and the next and so on. For a waited/blocking socket, which is the default as you have here, you need to keep reading from the server (and in your case writing back to the client) until you've done all the data in the response. If you require, or the server chooses, HTTP/1.0 style conn-per-txn (also known as connection: close or not-keepalive or not-pipelined, and also not-chunked) you can just loop until you receive "EOF" (0) from SSL_read, caused by the server closing the connection. If you allow and the server uses HTTP/1.1 keep-alive (or pipelining) and/or chunked data, the situation can get quite a bit more complicated. See RFC 2616. If you use a nonblocking socket (which is supported on Windows as far as I know but is apparently not the same as OVERLAPPED) you can also do your own timeout -- that is, read until EOF or optionally calculated end of the response body, *or* timeout. Since HTTP servers will normally send a complete response within a short time (like at most a few seconds), and if one doesn't a person at a browser usually doesn't want to wait anyway, this can be a good simple compromise. __ OpenSSL Project
Re: Man in the middle proxy - Not working
Hi I was able to read the content data from the server using SSL_read and put back to the browser by using SSL_write. I don't know whether is a right approach or not. I have done the experiment in these two urls 1. https://s-static.ak.facebook.com/rsrc.php/z9Q0Q/hash/8yhim1ep.ico 2. https://s-static.ak.facebook.com/rsrc.php/z8OGI/hash/41j5eq4v.png For the first try I got the response as follows and I was able to see the icon in my browser HTTP/1.1 200 OK Cache-Control: public, max-age=31536000 Content-Length: 318 Content-Type: image/x-icon Expires: Sat, 06 Aug 2011 06:58:14 -0700 Last-Modified: Sat, 01 Jan 2000 00:00:00 GMT P3P: CP="DSP LAW" Pragma: X-Cnection: close Date: Fri, 06 Aug 2010 13:58:14 GMT But for the second link, which is 42,565 bytes long, I am receiving the following output. I understood that there is more to do inorder to read the content data, which I am not sure about HTTP/1.1 200 OK Cache-Control: public, max-age=31536000 Content-Length: 42565 Content-Type: image/png Expires: Sat, 06 Aug 2011 07:04:17 -0700 Last-Modified: Sat, 01 Jan 2000 00:00:00 GMT P3P: CP="DSP LAW" Pragma: X-Cnection: close Date: Fri, 06 Aug 2010 14:04:17 GMT Can anybody tell me what else should I do inorder to read the content and show it the browser. The following are sending some code snippets RequestSock = WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); pHost = gethostbyname(pcTargetURL); memset(&ClientAddr,0,sizeof(ClientAddr)); ClientAddr.sin_family = AF_INET; memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); ClientAddr.sin_port = htons(atoi(pcPort)); if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, sizeof(SOCKADDR_IN))) { closesocket(RequestSock); // Connection failed return false; } SSL *Serverssl; Serverssl = SSL_new(m_pSSLCtx); SSL_set_fd(Serverssl, RequestSock); iRes = SSL_connect(Serverssl); if(iRes <= 0 ) { ERR_print_errors_fp(stderr); cout << " connect Failed " << endl; } iRes = SSL_write(Serverssl,pcData, strlen(pcData)); SSL_accept(Serverssl); do { dwReadDataLen = SSL_read(Serverssl,pBuff,iBufferSize); SSL_write(SourceSsl,pBuff,dwReadDataLen); cout << "Read buffer \n" << pBuff << endl; } while(SSL_pending(Serverssl)); Thanks, Raj Rajmohan SK - Original Message - From: "Raj" To: Sent: Friday, August 06, 2010 10:12 AM Subject: Re: Man in the middle proxy - Not working Hi Can you send me some code snippet which shows how to commutate with webserver and read the content data Thanks, Raj Rajmohan SK - Original Message - From: "Dave Thompson" To: Sent: Friday, August 06, 2010 2:19 AM Subject: RE: Man in the middle proxy - Not working From: owner-openssl-us...@openssl.org On Behalf Of Raj Sent: Thursday, 05 August, 2010 01:06 I will describe my code snippet below The module for connecting to server SOCKET RequestSock; SOCKADDR_IN ClientAddr; RequestSock = WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); I don't know much about 'OVERLAPPED' in Windows, but I think it's something like 'nonblocking' in Unix. pHost = gethostbyname(pcTargetURL); memset(&ClientAddr,0,sizeof(ClientAddr)); int iAddrLen = sizeof(ClientAddr); ClientAddr.sin_family = AF_INET; memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); ClientAddr.sin_port = htons(atoi(pcPort)); if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, sizeof(SOCKADDR_IN))) { closesocket(RequestSock); // Connection failed return false; } WSAOVERLAPPED SendOverlapped; DWORD dwSendDataLen = 0; WSABUF ClientRequestBuf; WSAEVENT SendEvent[1]; ClientRequestBuf.buf = pcData; ClientRequestBuf.len = strlen(pcData); SendEvent[0] = WSACreateEvent(); SendOverlapped.hEvent = SendEvent[0]; iRes = WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag, &SendOverlapped,NULL); // Sending data to the server At this point, the send probably hasn't actually happened. And if you call [WSA]Recv and it returns, it almost certainly hasn't actually been done either. You probably have to do some kind of synchronization with the .hEvent,
Re: Man in the middle proxy - Not working
Hi Can you send me some code snippet which shows how to commutate with webserver and read the content data Thanks, Raj Rajmohan SK - Original Message - From: "Dave Thompson" To: Sent: Friday, August 06, 2010 2:19 AM Subject: RE: Man in the middle proxy - Not working From: owner-openssl-us...@openssl.org On Behalf Of Raj Sent: Thursday, 05 August, 2010 01:06 I will describe my code snippet below The module for connecting to server SOCKET RequestSock; SOCKADDR_IN ClientAddr; RequestSock = WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); I don't know much about 'OVERLAPPED' in Windows, but I think it's something like 'nonblocking' in Unix. pHost = gethostbyname(pcTargetURL); memset(&ClientAddr,0,sizeof(ClientAddr)); int iAddrLen = sizeof(ClientAddr); ClientAddr.sin_family = AF_INET; memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); ClientAddr.sin_port = htons(atoi(pcPort)); if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, sizeof(SOCKADDR_IN))) { closesocket(RequestSock); // Connection failed return false; } WSAOVERLAPPED SendOverlapped; DWORD dwSendDataLen = 0; WSABUF ClientRequestBuf; WSAEVENT SendEvent[1]; ClientRequestBuf.buf = pcData; ClientRequestBuf.len = strlen(pcData); SendEvent[0] = WSACreateEvent(); SendOverlapped.hEvent = SendEvent[0]; iRes = WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag, &SendOverlapped,NULL); // Sending data to the server At this point, the send probably hasn't actually happened. And if you call [WSA]Recv and it returns, it almost certainly hasn't actually been done either. You probably have to do some kind of synchronization with the .hEvent, following whatever Windows rules are applicable. FYI pcPort = 443 pcTargetURL = L"www.facebook.com"; pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n" __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
On 05-08-2010 22:49, Dave Thompson wrote: >> From: owner-openssl-us...@openssl.org On Behalf Of Raj >> Sent: Thursday, 05 August, 2010 01:06 > >> I will describe my code snippet below >> >> The module for connecting to server >> >> SOCKET RequestSock; >> SOCKADDR_IN ClientAddr; >> RequestSock = >> WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); > > I don't know much about 'OVERLAPPED' in Windows, but I think > it's something like 'nonblocking' in Unix. Actually, this is the Win32 level way to do Asynchronous IO (AIO), meaning that the operation will happen immediately or in the background, without the cost of extra threads or buffering. The OS will hold on to your buffer pointer and other arguments and then signal completion by signaling the provided or event or (if no event is given), the socket itself. Once signaled, the buffers are yours again and the success/failure, transmitted length etc. are available. The actual transmission logic will usually happen in DMA, interrupt handlers etc. > >> pHost = gethostbyname(pcTargetURL); >> memset(&ClientAddr,0,sizeof(ClientAddr)); >> int iAddrLen = sizeof(ClientAddr); >> ClientAddr.sin_family = AF_INET; >> memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); >> ClientAddr.sin_port = htons(atoi(pcPort)); >> if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, >> sizeof(SOCKADDR_IN))) >> { >>closesocket(RequestSock); // Connection failed >>return false; >> } >> >> WSAOVERLAPPED SendOverlapped; >> DWORD dwSendDataLen = 0; >> WSABUF ClientRequestBuf; >> WSAEVENT SendEvent[1]; >> ClientRequestBuf.buf = pcData; >> ClientRequestBuf.len = strlen(pcData); >> SendEvent[0] = WSACreateEvent(); >> SendOverlapped.hEvent = SendEvent[0]; >> iRes = >> WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag, >> &SendOverlapped,NULL); >> // Sending data to the server >> > At this point, the send probably hasn't actually happened. > And if you call [WSA]Recv and it returns, it almost certainly > hasn't actually been done either. You probably have to do > some kind of synchronization with the .hEvent, following > whatever Windows rules are applicable. > Yep __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
> From: owner-openssl-us...@openssl.org On Behalf Of Raj > Sent: Thursday, 05 August, 2010 01:06 > I will describe my code snippet below > > The module for connecting to server > > SOCKET RequestSock; > SOCKADDR_IN ClientAddr; > RequestSock = > WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); I don't know much about 'OVERLAPPED' in Windows, but I think it's something like 'nonblocking' in Unix. > pHost = gethostbyname(pcTargetURL); > memset(&ClientAddr,0,sizeof(ClientAddr)); > int iAddrLen = sizeof(ClientAddr); > ClientAddr.sin_family = AF_INET; > memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); > ClientAddr.sin_port = htons(atoi(pcPort)); > if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, > sizeof(SOCKADDR_IN))) > { > closesocket(RequestSock); // Connection failed > return false; > } > > WSAOVERLAPPED SendOverlapped; > DWORD dwSendDataLen = 0; > WSABUF ClientRequestBuf; > WSAEVENT SendEvent[1]; > ClientRequestBuf.buf = pcData; > ClientRequestBuf.len = strlen(pcData); > SendEvent[0] = WSACreateEvent(); > SendOverlapped.hEvent = SendEvent[0]; > iRes = > WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag, > &SendOverlapped,NULL); > // Sending data to the server > At this point, the send probably hasn't actually happened. And if you call [WSA]Recv and it returns, it almost certainly hasn't actually been done either. You probably have to do some kind of synchronization with the .hEvent, following whatever Windows rules are applicable. > FYI > pcPort = 443 > pcTargetURL = L"www.facebook.com"; >pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n" > __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
Hi I will describe my code snippet below The module for connecting to server SOCKET RequestSock; SOCKADDR_IN ClientAddr; RequestSock = WSASocket(AF_INET,SOCK_STREAM,0,NULL,0,WSA_FLAG_OVERLAPPED); pHost = gethostbyname(pcTargetURL); memset(&ClientAddr,0,sizeof(ClientAddr)); int iAddrLen = sizeof(ClientAddr); ClientAddr.sin_family = AF_INET; memcpy(&ClientAddr.sin_addr,pHost->h_addr, pHost->h_length); ClientAddr.sin_port = htons(atoi(pcPort)); if(0 != connect(RequestSock,(SOCKADDR *)&ClientAddr, sizeof(SOCKADDR_IN))) { closesocket(RequestSock); // Connection failed return false; } WSAOVERLAPPED SendOverlapped; DWORD dwSendDataLen = 0; WSABUF ClientRequestBuf; WSAEVENT SendEvent[1]; ClientRequestBuf.buf = pcData; ClientRequestBuf.len = strlen(pcData); SendEvent[0] = WSACreateEvent(); SendOverlapped.hEvent = SendEvent[0]; iRes = WSASend(RequestSock,&ClientRequestBuf,1,&dwSendDataLen,dwFlag,&SendOverlapped,NULL); // Sending data to the server FYI pcPort = 443 pcTargetURL = L"www.facebook.com"; pcData = "GET https://www.facebook.com HTTP/1.0\r\n\r\n" Thanks, Raj Rajmohan SK - Original Message - From: "Dave Thompson" To: Sent: Thursday, August 05, 2010 7:48 AM Subject: RE: Man in the middle proxy - Not working From: owner-openssl-us...@openssl.org On Behalf Of Raj Sent: Wednesday, 04 August, 2010 01:09 Thanks for all the response 1. I was able to do the handshaking successfully with the browser. On receiving the request from the browser I will send "HTTP OK " response back to the browser, I was able to do the handshaking and read the actual GET request. To be clear: I interpret you received CONNECT, sent OK, did SSL handshake between browser and you (SSL_accept), then SSL_read (data which is a) GET request. 2. Then I create a new socket to establish the connection with server. The connection was successful. Sends the request to the server Reads the request from the server (Obviously you mean read response.) When I read the response from the server it always return empty. I don't know what went wrong here. I am reading the data from the socket using 'recv' function. Can anybody tell me what went wrong Is the connection to the server clear, or SSL? If SSL, you must use SSL_{connect,write,read,etc} throughout, with a different SSL* pointer than the one for the client side. And check for errors and report them etc. If clear, either: - you did the send and/or recv wrong; we'd have to look at your code, which you should simplify/trim as much as possible. - the server didn't like the request you sent, or you, strongly enough it just closed the connection. For HTTP this should be rare; most issues with the actual request (such as bad method or resource, unauthorized, bad or prohibited or required body, etc.) have defined HTTP error responses. Something like a firewall or frontend that works at the TCP level might just disconnect you, although in my experience they usually block or reject the initial connection (SYN) or break abruptly (RST), either of which appears to your program as an error return (canonically -1, not 0). Can you contact the people operating the server, and can they check their logs around the time of your attempt? Can you connect to the server from a browser on the machine running your proxy, or at one nearby on the same subnet? In clear, SSL, or both? And do a GET like the one you are (receiving and) forwarding from your client? Successfully? Can you run a monitor like tcpdump or wireshark while running your program, to see what was actually sent to the server and confirm if any data or what flags came back? __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
Hi recv function is returning 0 Thanks, Raj Rajmohan SK - Original Message - From: "David Schwartz" To: Sent: Thursday, August 05, 2010 6:05 AM Subject: RE: Man in the middle proxy - Not working Raj wrote: Thanks for all the response 1. I was able to do the handshaking successfully with the browser. On receiving the request from the browser I will send "HTTP OK " response back to the browser, I was able to do the handshaking and read the actual GET request. 2. Then I create a new socket to establish the connection with server. The connection was successful. Sends the request to the server Reads the request from the server When I read the response from the server it always return empty. What does that mean? Are you doing a blocking read or a non-blocking read? If 'read' returns zero, then the connection was closed by the server. If 'read' returns a number less than zero, there is an error -- tell us what error you are getting. If 'read' returns a number greater than zero, then that is the first part of the response. I don't know what went wrong here. I am reading the data from the socket using 'recv' function. Can anybody tell me what went wrong So, what return value do you get from 'recv'? DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
> From: owner-openssl-us...@openssl.org On Behalf Of Raj > Sent: Wednesday, 04 August, 2010 01:09 > Thanks for all the response > 1. I was able to do the handshaking successfully with > the browser. > On receiving the request from the browser I will send "HTTP > OK " response > back to the browser, I was able to do the handshaking and > read the actual > GET request. To be clear: I interpret you received CONNECT, sent OK, did SSL handshake between browser and you (SSL_accept), then SSL_read (data which is a) GET request. > 2. Then I create a new socket to establish the > connection with > server. The connection was successful. > Sends the request to the server > Reads the request from the server > (Obviously you mean read response.) > When I read the response from the server it always return > empty. I don't > know what went wrong here. I am reading the data from the > socket using > 'recv' function. Can anybody tell me what went wrong > Is the connection to the server clear, or SSL? If SSL, you must use SSL_{connect,write,read,etc} throughout, with a different SSL* pointer than the one for the client side. And check for errors and report them etc. If clear, either: - you did the send and/or recv wrong; we'd have to look at your code, which you should simplify/trim as much as possible. - the server didn't like the request you sent, or you, strongly enough it just closed the connection. For HTTP this should be rare; most issues with the actual request (such as bad method or resource, unauthorized, bad or prohibited or required body, etc.) have defined HTTP error responses. Something like a firewall or frontend that works at the TCP level might just disconnect you, although in my experience they usually block or reject the initial connection (SYN) or break abruptly (RST), either of which appears to your program as an error return (canonically -1, not 0). Can you contact the people operating the server, and can they check their logs around the time of your attempt? Can you connect to the server from a browser on the machine running your proxy, or at one nearby on the same subnet? In clear, SSL, or both? And do a GET like the one you are (receiving and) forwarding from your client? Successfully? Can you run a monitor like tcpdump or wireshark while running your program, to see what was actually sent to the server and confirm if any data or what flags came back? __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Raj wrote: > Thanks for all the response > 1. I was able to do the handshaking successfully with the > browser. > On receiving the request from the browser I will send "HTTP OK " > response > back to the browser, I was able to do the handshaking and read the > actual > GET request. > 2. Then I create a new socket to establish the connection with > server. The connection was successful. > Sends the request to the server > Reads the request from the server > > When I read the response from the server it always return empty. What does that mean? Are you doing a blocking read or a non-blocking read? If 'read' returns zero, then the connection was closed by the server. If 'read' returns a number less than zero, there is an error -- tell us what error you are getting. If 'read' returns a number greater than zero, then that is the first part of the response. > I > don't > know what went wrong here. I am reading the data from the socket using > 'recv' function. Can anybody tell me what went wrong So, what return value do you get from 'recv'? DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
Thanks for all the response 1. I was able to do the handshaking successfully with the browser. On receiving the request from the browser I will send "HTTP OK " response back to the browser, I was able to do the handshaking and read the actual GET request. 2. Then I create a new socket to establish the connection with server. The connection was successful. Sends the request to the server Reads the request from the server When I read the response from the server it always return empty. I don't know what went wrong here. I am reading the data from the socket using 'recv' function. Can anybody tell me what went wrong Thanks, Raj Rajmohan SK - Original Message - From: "David Schwartz" To: Sent: Wednesday, July 28, 2010 1:07 AM Subject: RE: Man in the middle proxy - Not working Rene Hollan: Oh! I totally misunderstood this. I thought OP wanted to MITM SSL sessions (which is possible, if (a) the traffic is decrypted, (b) certs are reissued and resigned, and (c) the client TRUSTS the modified cert chain (typically its root cert)). This is just HTTPS Proxy. In which case other answers about terminating the HTTP connection first are correct. No, you were correct. He does want to MITM SSL sessions. A MITM and a normal proxy operate precisely the same way up until the actual proxying part starts. His problem is earlier, when he establishes the connection to the client, determines what host and port the client wants to talk to, and then switches to his SSL proxy/MITM capability. All those steps are the same. 1) Accept plaintext connection. 2) Wait for client to send request. 3) Confirm CONNECT request, host and port valid. 4) Send 200 reply. 5) Make connection to host and port requested by client. 6) If normal proxying, begin proxying (copy ciphertext between client and server). If MITMing, begin MITMing (do SSL negotiation with both client and plaintext, copy plaintext between client and server). DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Man in the middle proxy - Not working
Hi All Thank you so much for all the response. I have one more doubt. If we do normal proxying on https connection, is it possible to read the https content data, at least the URL Thanks, Raj Rajmohan SK - Original Message - From: "David Schwartz" To: Sent: Wednesday, July 28, 2010 1:07 AM Subject: RE: Man in the middle proxy - Not working Rene Hollan: Oh! I totally misunderstood this. I thought OP wanted to MITM SSL sessions (which is possible, if (a) the traffic is decrypted, (b) certs are reissued and resigned, and (c) the client TRUSTS the modified cert chain (typically its root cert)). This is just HTTPS Proxy. In which case other answers about terminating the HTTP connection first are correct. No, you were correct. He does want to MITM SSL sessions. A MITM and a normal proxy operate precisely the same way up until the actual proxying part starts. His problem is earlier, when he establishes the connection to the client, determines what host and port the client wants to talk to, and then switches to his SSL proxy/MITM capability. All those steps are the same. 1) Accept plaintext connection. 2) Wait for client to send request. 3) Confirm CONNECT request, host and port valid. 4) Send 200 reply. 5) Make connection to host and port requested by client. 6) If normal proxying, begin proxying (copy ciphertext between client and server). If MITMing, begin MITMing (do SSL negotiation with both client and plaintext, copy plaintext between client and server). DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Rene Hollan: > Oh! I totally misunderstood this. > I thought OP wanted to MITM SSL sessions (which is possible, if > (a) the traffic is decrypted, (b) certs are reissued and resigned, > and (c) the client TRUSTS the modified cert chain (typically its > root cert)). > This is just HTTPS Proxy. In which case other answers about > terminating the HTTP connection first are correct. No, you were correct. He does want to MITM SSL sessions. A MITM and a normal proxy operate precisely the same way up until the actual proxying part starts. His problem is earlier, when he establishes the connection to the client, determines what host and port the client wants to talk to, and then switches to his SSL proxy/MITM capability. All those steps are the same. 1) Accept plaintext connection. 2) Wait for client to send request. 3) Confirm CONNECT request, host and port valid. 4) Send 200 reply. 5) Make connection to host and port requested by client. 6) If normal proxying, begin proxying (copy ciphertext between client and server). If MITMing, begin MITMing (do SSL negotiation with both client and plaintext, copy plaintext between client and server). DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Oh! I totally misunderstood this. I thought OP wanted to MITM SSL sessions (which is possible, if (a) the traffic is decrypted, (b) certs are reissued and resigned, and (c) the client TRUSTS the modified cert chain (typically its root cert)). This is just HTTPS Proxy. In which case other answers about terminating the HTTP connection first are correct. -Original Message- From: owner-openssl-us...@openssl.org on behalf of David Schwartz Sent: Tue 7/27/2010 4:19 AM To: openssl-users@openssl.org Subject: RE: Man in the middle proxy - Not working Alexey Drozdov wrote: > Hi! > > When your setup proxy setting for browsers, they using HTTP CONNECT > method for establish pure tcp-connection via proxy (not for local > resources). > It's seems like: > > Client send HTTP-request to proxy > CONNECT remotehost:port HTTP/1.1 > Host: remotehost:port > > And begin wait HTTP-response like: > HTTP/1.1 200 Connection established > > Then browser send initiate ssl handshake over this pure tcp-channel. > > Your proxy get HTTP-request instead ssl-handshake and fail: > 2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy > request:.ssls23_srvr.c:391 > > --- > / Alexey Drozdov In other words, you switched to SSL too early. The way you did it, how would you know what host and port you were supposed to proxy a connection to?! You have to wait and get the CONNECT request from the client to know what host and port they want a connection to. Then send an HTTP 200 reply, and then begin proxying. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Alexey Drozdov wrote: > Hi! > > When your setup proxy setting for browsers, they using HTTP CONNECT > method for establish pure tcp-connection via proxy (not for local > resources). > It's seems like: > > Client send HTTP-request to proxy > CONNECT remotehost:port HTTP/1.1 > Host: remotehost:port > > And begin wait HTTP-response like: > HTTP/1.1 200 Connection established > > Then browser send initiate ssl handshake over this pure tcp-channel. > > Your proxy get HTTP-request instead ssl-handshake and fail: > 2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy > request:.ssls23_srvr.c:391 > > --- > / Alexey Drozdov In other words, you switched to SSL too early. The way you did it, how would you know what host and port you were supposed to proxy a connection to?! You have to wait and get the CONNECT request from the client to know what host and port they want a connection to. Then send an HTTP 200 reply, and then begin proxying. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Man in the middle proxy - Not working
Hi! When your setup proxy setting for browsers, they using HTTP CONNECT method for establish pure tcp-connection via proxy (not for local resources). It's seems like: Client send HTTP-request to proxy CONNECT remotehost:port HTTP/1.1 Host: remotehost:port And begin wait HTTP-response like: HTTP/1.1 200 Connection established Then browser send initiate ssl handshake over this pure tcp-channel. Your proxy get HTTP-request instead ssl-handshake and fail: 2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391 --- / Alexey Drozdov From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] On Behalf Of Raj Sent: Monday, July 26, 2010 12:59 PM To: openssl-users@openssl.org Subject: Man in the middle proxy - Not working Hi All Requirement:- I want to build a man in the middle proxy application. I have experimented so many methods to achieve this. But my application is failing when I tried some https url's from the browser (IE 8 and Firefox 3.7). I have configured my browser proxy settings to '4433' port. My application is listening on this port, when I connect to this port from my browser, with the URL https://localhost:4433 it is working, only a certificate warning is there. When I try to connect to another secured site, the SSL_accept function is returning -1 and my error code is as follows. "2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391:" I am not able to trace out the problem for many days. Anybody please help me to trace out this issue, or send me some sample application. What could be reasons for failure Platforms I am using are : MS Windows XP service pack 2 MS Visual Studio 2008 , VC++ Thanks, Raj Rajmohan SK __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Man in the middle proxy - Not working
Hi All Requirement:- I want to build a man in the middle proxy application. I have experimented so many methods to achieve this. But my application is failing when I tried some https url's from the browser (IE 8 and Firefox 3.7). I have configured my browser proxy settings to '4433' port. My application is listening on this port, when I connect to this port from my browser, with the URL https://localhost:4433 it is working, only a certificate warning is there. When I try to connect to another secured site, the SSL_accept function is returning -1 and my error code is as follows. "2572:error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request:.ssls23_srvr.c:391:" I am not able to trace out the problem for many days. Anybody please help me to trace out this issue, or send me some sample application. What could be reasons for failure Platforms I am using are : MS Windows XP service pack 2 MS Visual Studio 2008 , VC++ Thanks, Raj Rajmohan SK