OK. Point taken 😀 Yes. I do make assumptions based on how this app is used and yes it is C++/MFC on windows going from an old version with no specified coding to UTF.
The certificate data in this case is "regular" ie a PEM payload that is not UTF encoded, just: -----BEGIN CERTIFICATE----- xxxxxxx -----END CERTIFICATE----- Ie one byte per character. Filenames OTOH might be a different kettle of fish. I probably should include a testcase where the path has non-ascii characters. And yes, what I wrote then was completely clear to me, but I realise when rereading that it was probably not, so, let me elaborate: The original code on Windows 32-bit no unicode used the CURLOPT_SSL_CTX_FUNCTION I replaced that with the function in the documentation, but that apparently did not add the key to the context, just the certificate so I needed to add code to handle the certificate The string passed to CURLOPT_SSL_CTX_DATA is a CStringA so it is OK to convert to char, especially assuming that the data is just PEM The example code in https://curl.se/libcurl/c/CURLOPT_SSL_CTX_DATA.html adds the certificate to a store in the context, what my code does is to take the certificate and key, again as CStringA then BIO_new_mem_buf() the do PEM_Read_bio_x590 then SSL_CTX_use_certificate, then do the same for the key, SSL_CTX_use_PrivateKey. Ie it sets the certificate and key directly without going through a store. I am not sure why my way works, but not the store way. I guess the "store" concept was added to OpenSSL along the line and I guess I should read up on that. Yes. String handling in Windows is weird. I sort of understand the performance aspect of using UTF-16, but it makes other things harder. I guess another option would be to use std::string all the way. > (OTOH one could just assume that > all [relevant] ACP encodings are ASCII subset, thus > simply NOT do transcoding since it then ought to be > ASCII-compliant content already anyway). Which is pretty much what I do. All certificate data is assumed to be PEM and coded as ASCII with one byte per character. Thanks for your reply and your pointers. -- Med vänlig hälsning Anders Gustafsson, ingenjör [email protected] | Support +358 18 12060 | Direkt +358 9 315 45 121 | Mobil +358 40506 7099 Pedago interaktiv ab, Nygatan 7 B , AX-22100 MARIEHAMN, ÅLAND, FINLAND >>> Andreas Mohr <[email protected]> 2025-08-20 17:44 >>> Hi, disclaimer: quite experienced in certain areas, yet not too much in others (CURL). TL;DR: thus discussing potential string encoding issues only. On Wed, Aug 20, 2025 at 03:58:14PM +0300, Anders Gustafsson via curl-library wrote: > So, yes this is windows 😀 libcurl/8.15.0-DEV OpenSSL/3.5.2 zlib/1.3.1 > > I had some issues and I just want to check whether I am going about this the > right way. The function calls an > API where the client certificate is used to authenticate the caller so in the > original version I used the > sslctx_function(). To complicate matters does my app support PEM certificates > and keys in two different ways: > 1. As files (Say on a removable secure media) and 2. As strings in the > database for ease of use. > > The first way (filenames) worked right away, ie: > > m_Certificate.Trim(); > if (m_Certificate.IsEmpty()) > curl_easy_setopt(curl, CURLOPT_SSLCERT, > m_CertificateFile.GetString()); Bleeep - ATLMFC CStringT::GetString() encountered. This might thus be dirty "encoding shortcutting" here (simply invoking .GetString() to "quickly" "get at" some "char"-compatibly-typed - hah! - input, rather than doing active transcoding to the actually *correct* encoding spec of some char-typed handling). Thus, consulting this one: > Where m_Certificate and m_Key and regular (char) strings with the PEM coded > data. What would "regular" mean? Considering that CString errfilename; with CT2A(errfilename), one would think that this is a CString[T] with UNICODE config setting (put differently, CStringW) environment, however since you said "regular (char) strings", I am assuming that you are on !UNICODE config (i.e., CStringA). https://manpages.ubuntu.com/manpages/kinetic/man3/CURLOPT_SSLCERT.3.html Yeah nice - that page does not specify at all which encoding char *cert (a filesystem item argument!! - which could be containing all the smileys available in this universe, with only a bit of luck...) is expected to have. Thus on Windows one would tend to assume ACP crap - which of course would mean that it is Unicode-compliance-broken (since: neither UTF16 nor UTF8 nor UTF7 nor UTF-EBCDIC or whatever ;-)). > errno_t fileerr = fopen_s(&errfile, CT2A(errfilename), "w+, ccs=UTF-8"); Unicode-compliance-broken filesystem item handling! CT2A(errfilename) will be wide-typed to CP_ACP transition (in UNICODE config setting), and "nothing" *) (in non-UNICODE). *) BTW *HORRIBLE* atlconv.h comment "// Code page doesn't matter" atlconv.h transcoding protocol breakage!!! Yeah, as if that would be the case for e.g. CP_ACP to CP_UTF8 transcoding, which *is* a valid transcoding use case...) (think of CT2CA(..., CP_UTF8) protocol behaviour **DIFFERENCE**) Thus, your errfilename possibly is ACP (CP_ACP, GetACP()) content, which would be "compatible" since fopen_s() API is equally ACP-specced on Windows (yuck). ...but: then it would be Unicode-compliance-broken (due to being ACP crap, rather than e.g. UTF8 as usually on Linux). Since fopen_s() should have an overload for wide-typed input I'd think, the way to go would at least be CT2W(errfilename) - thereby properly preserving Unicode-compliant (since wide-typed!) encoding (when on UNICODE config setting - and ACP crap on !UNICODE). Or, better do utf8everywhere.org (i.e., std::string[-means-utf8] - to have ensured that **every** string traffic anywhere is Unicode-compliant), and thus do std::string errfilename = "myUtf8InputStuffStringFromSomewhere"; // e.g. std::filesystem **) API fopen_s(... CA2W(errfilename, CP_UTF8) ...); **) rather *horribly* Unicode-compliance-broken (on Windows!) - I digress... "<filesystem>: prevent filesystem::path dangerous conversions to/from default code page encoding" https://github.com/microsoft/STL/issues/909 > In the second scenario, PEM in database, I had some problems and I just > wanted to check that the code I came > up with is sane. Ie the authentication will not happen unless I have both > certificate and key, so: > > if (!m_Certificate.IsEmpty()) > { > curl_easy_setopt(curl, > CURLOPT_SSL_CTX_FUNCTION, sslctx_function); > curl_easy_setopt(curl, CURLOPT_SSL_CTX_DATA, > m_Certificate.GetString()); WARNING CORRUPTION: CURLOPT_SSL_CTX_DATA has a void*-typed ptr arg, thus both .GetString() CString[T] is accepted, *silently*). IOW, once on UNICODE config setting it would be *broken*. So, questionable encoding stuff again. According to https://curl.se/libcurl/c/CURLOPT_SSL_CTX_DATA.html "char *mypem = /* CA cert in PEM format" it seems to *appear* that "PEM format" means some plain ASCII-only payload. Now to be maximally precise one could do const UINT nCP_PEM = 20127 /*CP_ASCII*/ /* these clearly are [to be!] all ASCII-only, right!!!? */; std::string strCertificate = CW2A(CA2W(m_Certificate), nCP_PEM); (this transition expects that m_Certificate has system ACP content, of course) (OTOH one could just assume that all [relevant] ACP encodings are ASCII subset, thus simply NOT do transcoding since it then ought to be ASCII-compliant content already anyway). > Then, below, which seems to work OK. I first used the example here: > https://curl.se/libcurl/c/CURLOPT_SSL_CTX_FUNCTION.html > but that one did not fix my key for me. Yes, this code still leaves allocated > memory in case of errors. "fix my key" - that wording might hint at encoding issues. But perhaps we're talking about a plain CURL certificate config issue only after all. I could not precisely identify (thus: discuss) particular ***) issues in your handling, but I'd hope that this will give you some ideas (*if* it is an encoding issue). ***) well, except for the broken fopen_s() filesystem item handling... Greetings Andreas Mohr -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
