Async I/o with Handling the Threads in openssl
Hi all, I have created small server application ssl based Data sharing to the Public. i faced Handling the incoming connection. if multiple connections are arrived. i ready accept . if and creating the New thread . data send backandforth . i facing issue if 2 or 3 client has arrived start slowing down . how to i will do handle efficiently. handle the data send / receive. can you please any one Help. Just perfect idea. i waiting for reply ! please Help Me! -- Thanks, udhayakumar
RE: Upgrading OpenSSL on Windows 10
Hi Job, Thanks very much for your reply. Apologies for the lateness of mine. I will ask around and get more information about Powershell and PDQ Inventory. Thanks again, Steven Sent with Proton Mail secure email. --- Original Message --- On Wednesday, November 23rd, 2022 at 5:36 AM, Job Cacka wrote: > Michael's point should be asked and answered first for your environment. > > To find all of the OpenSSL bits used on a windows system you would use > Powershell or a tool that flexes its use like PDQ Inventory. There is a > steep learning curve and it is probably off topic for this group but there > are several different ways to use powershell to gain this information from > different viewpoints (Installed files, registry, event log, etc...). > > Thanks, > Job > > -Original Message- > From: openssl-users openssl-users-boun...@openssl.org On Behalf Of Michael > > Wojcik via openssl-users > Sent: Monday, November 21, 2022 4:18 PM > To: openssl-users@openssl.org > Subject: Re: Upgrading OpenSSL on Windows 10 > > > From: openssl-users openssl-users-boun...@openssl.org on behalf of > > Steven_M.irc via openssl-users openssl-users@openssl.org > > Sent: Monday, November 21, 2022 15:56 > > > However, I am running Windows 10, and since (unlike Linux) every piece > > of software outside of Windows itself needs to be updated > > individually, I don't know how to track down every single application that > > might be using OpenSSL and make sure that the copy of OpenSSL it uses is > up-to-date. > > You don't. There may be applications that have OpenSSL linked statically, or > linked into one of its own DLLs, or just with the OpenSSL DLLs renamed. > > > As many of you would know, under repository-based systems (such as > > most Linux distros), this would not be an issue as I could update every > > single application (system or non-system) at once. > > This is not true in the general case. There are applications which are > available on Linux which do not use the distribution's package manager. > There are applications which use their own OpenSSL build, possibly linked > statically or linked into one of their own shared objects or with the > OpenSSL shared objects renamed. Linux distributions have not magically > solved the problem of keeping all software on the system current. > > > Back to Windows: It is possible, with relatively little effort, to find all > the copies of the OpenSSL DLLs under their usual names on a system, and then > glean from them their version information. With significantly more effort, > you can search for exported OpenSSL symbols within third-party binaries, > which will detect some more instances. With quite a lot of additional > effort, you can winkle out binaries which contain significant portions of > code matching some OpenSSL release (see various research efforts on > function-point and code-block matching, and compare with alignment > strategies in other fields, such as genomics). If your definition of > "OpenSSL in an application" is not too ambitious, this might even be > feasible. > > But to what end? Each application will either be well-supported, in which > case you can find out from the vendor what OpenSSL version it contains and > whether an update is available; or it is not, in which you'll be out of > luck. > > This is true of essentially every software component, most of which are not > as well-maintained or monitored as OpenSSL. Modern software development is > mostly a haphazard hodgepodge of accumulating software of uncertain > provenance and little trustworthiness into enormous systems with > unpredictable behavior and failure modes. I'm not sure OpenSSL versions > should be particularly high on anyone's priority list. > > What are you actually trying to accomplish? What's your task? Your threat > model? > > -- > Michael Wojcik
Re: Upgrading OpenSSL on Windows 10
Hi Michael, Thanks very much for replying to my e-mail/post. I apologize for the lateness of my reply. > This is not true in the general case. There are applications which are > available on Linux which do not use the distribution's package manager. There > are applications which use their own OpenSSL build, possibly linked > statically or linked into one of their own shared objects or with the OpenSSL > shared objects renamed. Linux distributions have not magically solved the > problem of keeping all software on the system current. That's disheartening. My next computer will be running Linux and I was thinking that (as long as I stick to installing software from appropriate repositories) my update worries would be over soon. >It is possible, with relatively little effort, to find all the copies of the >OpenSSL DLLs under their usual names on a system Could you please provide me with a list of the usual names? I've got a lot of libssl DLL's on my system, but I'm not sure if they're part of OpenSSL or some other implementation of SSL. >I'm not sure OpenSSL versions should be particularly high on anyone's priority >list. As I understand it, OpenSSL is responsible for establishing HTTPS connections, the primary protocol for ensuring security and authenticity over the Internet, and you *don't* think OpenSSL versions should be a high priority? I don't understand your lack of alarm here. >What are you actually trying to accomplish? What's your task? Your threat >model? I want to be able to trust the HTTPS connections between my PC and servers on the Internet again; whether I'm using a browser, a software installer (that downloads data from the Internet before installing), a peer-to-peer application, or any other network application. Thank you for your time. Steven
Re: Question about migrating from d2i_ECPrivateKey() to d2i_PrivateKey(EVP_PKEY_EC, ...)
On Tue, Nov 22, 2022 at 11:09:07AM -0600, Nico Williams wrote:
> > Not exactly, PKCS#8-based typing is used in d2i_PKCS8_PRIV_KEY_INFO()
> > (for unencrypted PKCS#8 blobs, so no password callback). The
> > d2i_PrivateKey() function takes an explicit pkey_type instead.
>
> Hmmm, well, d2i_PrivateKey() takes an explicit pkey_type, yes, but it's
> not sufficiently informative for ECDH, being just EVP_PKEY_EC. Or are
> there more informative type values I've not discovered yet? When I call
> d2i_PrivateKey(EVP_PKEY_EC, ...) it wants a PKCS#8 encoded private key.
Actually, it supports *both* the PKCS#8 and the legacy type-specific formats.
The algorithm-specific PEM key formats may not be defined for some newer
key types and are deprecated, so sure, you're supposed to use PKCS#8
whenever possible.
For EC private keys, the underlying legacy DER codecs are:
d2i_ECPrivateKey()
i2d_ECPrivateKey()
with signatures:
EC_KEY *d2i_ECPrivateKey(EC_KEY **a, const unsigned char **in, long len);
int i2d_ECPrivateKey(const EC_KEY *a, unsigned char **out);
for d2i_ECPrivateKey(), you get back an EC_KEY() which you can convert
to an EVP_PKEY via the deprecated (copy vs. take ownership):
EVP_PKEY_set1_EC_KEY(3), or EVP_PKEY_assign_EC_KEY(3)
which direct you to EVP_PKEY_fromdata(3) (much scaffolding...).
However, d2i_PrivateKey(EVP_PKEY_EC, ...) given non-PKCS#8 input
internally tries:
static int old_ec_priv_decode(EVP_PKEY *pkey,
const unsigned char **pder, int derlen)
{
EC_KEY *ec;
if ((ec = d2i_ECPrivateKey(NULL, pder, derlen)) == NULL)
return 0;
EVP_PKEY_assign_EC_KEY(pkey, ec);
return 1;
}
which does the expected thing without much fuss. That said, you
mentioned ECDH in passing, so I'm not sure we're taking about the same
things...
--
Viktor.
Re: TLS1.3 change_cipher_spec as part of application data
I checked this further and the issue was indeed with my code.. I did some recent async io completion handler refactor because of which the bio's socket write completion was triggering the observer's read completion callback.. the records were actually client side write bio buffers which got mixed with the ssl_read data on the read path. Thanks for the pointer, we are doing good. Nothing further is needed at this point. Neelabh On Thu, Nov 24, 2022 at 4:51 PM Neelabh Mam wrote: > I hook an observer for decrypted data immediately after the handshake is > successful (SSL_do_handshake rc 1) and it is this observer which gets the > ccs+list data on the vert next ssl_read cycle. Now, it could be that my > code is at fault here.. But I do see the decrypted dummy ccs and one more > record as part of the list data. Annotated hex dump of the buffer > reproduced below. Btw this is against the latest FZ windows FTPS server > which I think recently moved to GNUtls. As suggested earlier I'll check > my code again and open a github ticket. > > Command : PASV > sock-cc 2570925053120 OnWrite, n 29 > sock-cc 2570925053120 OnRead, n 273 > *New session callback 1* > sock-cc 2570925053120 OnRead, n 68 > Response : 227 Entering Passive Mode (127,0,0,1,249,75) > Command : LIST / > sock-dc 2570925624800 OnConnect, n 0 > sock-dc OnConnect() > sock-cc 2570925053120 OnWrite, n 30 > sock-cc 2570925053120 OnRead, n 51 > Response : 150 Starting data transfer. > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 10 > sock-dc 2570925624800 OnRead, n 133 > sock-dc Handshake failed : -1 2 > sock-dc 2570925624800 OnRead, n 108 > *sock-dc Handshake done. TLS session reused : 1* > sock-dc 2570925624800 OnWrite, n 32 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnWrite, n 32 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnWrite, n 16 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnRead, n 86 > observer OnDataChannelIoCompletion read > sock-cc 2570925053120 OnRead, n 48 > Response : 226 Operation successful > > sock-dc 2570925624800 OnRead, n 816 > sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2 > sock-dc : ssl_shutdown() rc : 1 > sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3 > sock-dc : shutdown(sd_send) > observer OnDataChannelIoCompletion read > sock-dc 2570925624800 OnRead, n 0 > sock-dc shutdown mode : 3 > ¢â╣ª5d²┼l⌐\╒═£tdrwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 > $RECYCLE.BIN > -rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe > -rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe > -rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe > -rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe > -rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe > -rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe > -rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe > drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information > -rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt > > 14 TLS Change Cipher Spec protocol > 03 03 SSL 3.3 (TLS 1.2) > 00 01 length of change cipher spec message > 01 payload (irrelevant) > > 17TLS application data > 03 03 version 1.2 > 00 45 length of payload (69 bytes) > > 14 03 03 00 01 01 *17 03 03 00 45* 4b 60 8c 72 be > 99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc > 2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46 > 8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b > 83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74 > 64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70 > 20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20 > 20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35 > 35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a > 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 > 20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36 > 38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33 > 37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d > 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 > 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38 > 31
Re: TLS1.3 change_cipher_spec as part of application data
I hook an observer for decrypted data immediately after the handshake is successful (SSL_do_handshake rc 1) and it is this observer which gets the ccs+list data on the vert next ssl_read cycle. Now, it could be that my code is at fault here.. But I do see the decrypted dummy ccs and one more record as part of the list data. Annotated hex dump of the buffer reproduced below. Btw this is against the latest FZ windows FTPS server which I think recently moved to GNUtls. As suggested earlier I'll check my code again and open a github ticket. Command : PASV sock-cc 2570925053120 OnWrite, n 29 sock-cc 2570925053120 OnRead, n 273 *New session callback 1* sock-cc 2570925053120 OnRead, n 68 Response : 227 Entering Passive Mode (127,0,0,1,249,75) Command : LIST / sock-dc 2570925624800 OnConnect, n 0 sock-dc OnConnect() sock-cc 2570925053120 OnWrite, n 30 sock-cc 2570925053120 OnRead, n 51 Response : 150 Starting data transfer. sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 10 sock-dc 2570925624800 OnRead, n 133 sock-dc Handshake failed : -1 2 sock-dc 2570925624800 OnRead, n 108 *sock-dc Handshake done. TLS session reused : 1* sock-dc 2570925624800 OnWrite, n 32 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnWrite, n 32 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnWrite, n 16 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnRead, n 86 observer OnDataChannelIoCompletion read sock-cc 2570925053120 OnRead, n 48 Response : 226 Operation successful sock-dc 2570925624800 OnRead, n 816 sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2 sock-dc : ssl_shutdown() rc : 1 sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3 sock-dc : shutdown(sd_send) observer OnDataChannelIoCompletion read sock-dc 2570925624800 OnRead, n 0 sock-dc shutdown mode : 3 ¢â╣ª5d²┼l⌐\╒═£tdrwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 $RECYCLE.BIN -rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe -rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe -rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe -rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe -rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe -rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe -rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information -rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt 14 TLS Change Cipher Spec protocol 03 03 SSL 3.3 (TLS 1.2) 00 01 length of change cipher spec message 01 payload (irrelevant) 17TLS application data 03 03 version 1.2 00 45 length of payload (69 bytes) 14 03 03 00 01 01 *17 03 03 00 45* 4b 60 8c 72 be 99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc 2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46 8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b 83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74 64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35 35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36 38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33 37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38 31 30 34 31 36 20 4f 63 74 20 31 35 20 31 37 3a 33 37 20 61 63 63 65 73 73 63 68 6b 36 34 2e 65 : : goes on full list On Thu, Nov 24, 2022 at 3:15 PM Matt Caswell wrote: > > > On 24/11/2022 07:57, Neelabh Mam wrote: > > Hi, > > > > With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I > > see that immediately after a successful data channel handshake (with > > session reuse), a dummy change_cipher_spec record and a non-application > > data record are sent as part of the directory listing data. Same holds > > true for file downloads as well..(again with session reuse). This seems > > to be in line with the last paragraph
Re: TLS1.3 change_cipher_spec as part of application data
On 24/11/2022 07:57, Neelabh Mam wrote: Hi, With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I see that immediately after a successful data channel handshake (with session reuse), a dummy change_cipher_spec record and a non-application data record are sent as part of the directory listing data. Same holds true for file downloads as well..(again with session reuse). This seems to be in line with the last paragraph in Appendix D.4 of RFC8446 which mandates a change_cipher_spec record to be sent from the server in case of session reuse. I could manually filter out the non-application data records from the actual data.. However, I was wondering if there's some api/setting in openssl that would do that at an API level.. as this looks like it should ideally be part of openssl workflow. But then what I am also not sure about is why the receipt of new session ticket data on the control channel does not bubble up as application data ? Openssl, from what I can tell, seems to be "consuming" it (new session data) internally. The earlier 2 messages however, do get exposed out of ssl_read and eventually become part of application data. The only data you should be getting out of an `SSL_read` call is application data. If you're getting handshake or CCS messages as part of app data then something has gone very surprisingly wrong. I cannot imagine what would cause that. You might want to open a github issue about that to dive into it in more detail. Matt Is it like : the message type warrants one to be exposed (change_cipher_spec) and the other to be handled internally (new session data) ? Could we please advise on openssl's standard operating workflow here ? Also, would I have to add logic to manually separate these 2 non-application records from the application data ? Is that how it is supposed to be ? Thank you. Neelabh
