Re: [OpenXPKI-users] format of the .../.well-known/est/cacerts response
Hi Stefan, thanks for digging - turns out it is as I expected =) I did some more tests and openssl fails at 1024 chars line length so I assume its just some sort of "buffer too small" issue and we came up with the same conclusion. I think we will add a switch to control if lines should be wrapped or not as I am pretty sure we will stumble over a client sooner or later that fails the same way. Internally we use the perl MIME::Base64 module which does a stream decoding and does not care on any lenght limits, so that should be pretty safe. best regards Oliver On 12.06.24 14:02, Stefan Goeman wrote: Hi Oliver We looked into this further. It seems that RFC 7030 is updated by RFC 8951. RFC 8951 refers to RFC 4648. We believe that OpenXPKI is probalby doing it correctly. *In section 3.1, RFC 8951 says:* /Note that "base64" as used in the HTTP [//RFC2616 <https://www.rfc-editor.org/rfc/rfc8951.html#RFC2616>//] does not permit CRLF, while the "base64" used in MIME [//RFC2045 <https://www.rfc-editor.org/rfc/rfc8951.html#RFC2045>//] does. This specification clarifies that despite what [//RFC2616 <https://www.rfc-editor.org/rfc/rfc8951.html#RFC2616>//] says, white space including CR, LF, spaces (ASCII 32), and tabs (ASCII 9) /*/SHOULD/*/ be tolerated by receivers. Senders are not required to insert any kind of white space./ *In section 3.1 RFC 4648 says:* /MIME [4 <https://www.rfc-editor.org/rfc/rfc4648#ref-4>] is often used as a reference for base 64 encoding. However, MIME does not define "base 64" per se, but rather a "base 64 Content- Transfer-Encoding" for use within MIME. As such, MIME enforces a limit on line length of base 64-encoded data to 76 characters. MIME inherits the encoding from Privacy Enhanced Mail (PEM) [3 <https://www.rfc-editor.org/rfc/rfc4648#ref-3>], stating that it is "virtually identical"; however, PEM uses a line length of 64 characters. The MIME and PEM limits are both due to limits within SMTP. Implementations MUST NOT add line feeds to base-encoded data unless the specification referring to this document explicitly directs base encoders to add line feeds after a specific number of characters./ On the web site there are some examples that use the "base64" tool. Lukily this tool handles both base64 content with and without line feeds. Refering back to RFC 8951, as OpenXPKI can also be a receiver of base64 content (in case of an enroll or re-enroll request), it should be checked if PKCS#10 with and without line feeds can be handled. If base64 is used to decode this, then I believe it will be OK. Greetings, Stefan. *Van:* Oliver Welter *Verzonden:* woensdag 12 juni 2024 13:16 *Aan:* openxpki-users@lists.sourceforge.net *Onderwerp:* Re: [OpenXPKI-users] format of the .../.well-known/est/cacerts response Hi Stefan, OpenXPKI sends the payload in base64 without wrapping it into line blocks while the EST test server does and it looks like the openssl base64 is not able to handle this encoding. I have looked around a bit and did not really find a normative reference if OpenXPKI is doing it wrong or if this is just a glitch in OpenSSL but I will add this as an RFE so we will implement line wrapping with the next release. best regards Oliver On 12.06.24 09:48, Stefan Goeman wrote: Hello I want to use the following commands to get my ca files in a nicely fomated pem-file: * curl -k --connect-timeout 10 -s https://my_est_server/.well-known/est/cacerts <https://my_est_server/.well-known/est/cacerts> -o cacerts.p7 * openssl base64 -d -in cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem Now, when I use the testest server at https://testrfc7030.com:8443/ <https://testrfc7030.com:8443/> these commands work and I get a nicely looking pem file. When I use these commands with my own openxpki server, the second command does not work. I get the error (on debian 12) "unable to load PKCS7 object". One difference that I notice is that when I run the curl command against the testrfc7030 server I already have a nicely formated looking file, like you expect in a pem-file. When I do the curl command against my own server, there is only one line in the file, and when I open this file in vi I see the message "incomplete last line". I have no idea what this means? BTW, If I use "base64 -d cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem" on my system, this works. So, there is obviously a difference between "openssl base64" and "base64". Much thanks in advance for your help. Greetings, Stefan. ___ OpenXPKI-users mail
Re: [OpenXPKI-users] format of the .../.well-known/est/cacerts response
Hi Oliver We looked into this further. It seems that RFC 7030 is updated by RFC 8951. RFC 8951 refers to RFC 4648. We believe that OpenXPKI is probalby doing it correctly. In section 3.1, RFC 8951 says: Note that "base64" as used in the HTTP [RFC2616<https://www.rfc-editor.org/rfc/rfc8951.html#RFC2616>] does not permit CRLF, while the "base64" used in MIME [RFC2045<https://www.rfc-editor.org/rfc/rfc8951.html#RFC2045>] does. This specification clarifies that despite what [RFC2616<https://www.rfc-editor.org/rfc/rfc8951.html#RFC2616>] says, white space including CR, LF, spaces (ASCII 32), and tabs (ASCII 9) SHOULD be tolerated by receivers. Senders are not required to insert any kind of white space. In section 3.1 RFC 4648 says: MIME [4<https://www.rfc-editor.org/rfc/rfc4648#ref-4>] is often used as a reference for base 64 encoding. However, MIME does not define "base 64" per se, but rather a "base 64 Content- Transfer-Encoding" for use within MIME. As such, MIME enforces a limit on line length of base 64-encoded data to 76 characters. MIME inherits the encoding from Privacy Enhanced Mail (PEM) [3<https://www.rfc-editor.org/rfc/rfc4648#ref-3>], stating that it is "virtually identical"; however, PEM uses a line length of 64 characters. The MIME and PEM limits are both due to limits within SMTP. Implementations MUST NOT add line feeds to base-encoded data unless the specification referring to this document explicitly directs base encoders to add line feeds after a specific number of characters. On the web site there are some examples that use the "base64" tool. Lukily this tool handles both base64 content with and without line feeds. Refering back to RFC 8951, as OpenXPKI can also be a receiver of base64 content (in case of an enroll or re-enroll request), it should be checked if PKCS#10 with and without line feeds can be handled. If base64 is used to decode this, then I believe it will be OK. Greetings, Stefan. Van: Oliver Welter Verzonden: woensdag 12 juni 2024 13:16 Aan: openxpki-users@lists.sourceforge.net Onderwerp: Re: [OpenXPKI-users] format of the .../.well-known/est/cacerts response Hi Stefan, OpenXPKI sends the payload in base64 without wrapping it into line blocks while the EST test server does and it looks like the openssl base64 is not able to handle this encoding. I have looked around a bit and did not really find a normative reference if OpenXPKI is doing it wrong or if this is just a glitch in OpenSSL but I will add this as an RFE so we will implement line wrapping with the next release. best regards Oliver On 12.06.24 09:48, Stefan Goeman wrote: Hello I want to use the following commands to get my ca files in a nicely fomated pem-file: * curl -k --connect-timeout 10 -s https://my_est_server/.well-known/est/cacerts -o cacerts.p7 * openssl base64 -d -in cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem Now, when I use the testest server at https://testrfc7030.com:8443/ these commands work and I get a nicely looking pem file. When I use these commands with my own openxpki server, the second command does not work. I get the error (on debian 12) "unable to load PKCS7 object". One difference that I notice is that when I run the curl command against the testrfc7030 server I already have a nicely formated looking file, like you expect in a pem-file. When I do the curl command against my own server, there is only one line in the file, and when I open this file in vi I see the message "incomplete last line". I have no idea what this means? BTW, If I use "base64 -d cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem" on my system, this works. So, there is obviously a difference between "openssl base64" and "base64". Much thanks in advance for your help. Greetings, Stefan. ___ OpenXPKI-users mailing list OpenXPKI-users@lists.sourceforge.net<mailto:OpenXPKI-users@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/openxpki-users -- Protect your environment - close windows and adopt a penguin! ___ OpenXPKI-users mailing list OpenXPKI-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openxpki-users
Re: [OpenXPKI-users] format of the .../.well-known/est/cacerts response
Hi Stefan, OpenXPKI sends the payload in base64 without wrapping it into line blocks while the EST test server does and it looks like the openssl base64 is not able to handle this encoding. I have looked around a bit and did not really find a normative reference if OpenXPKI is doing it wrong or if this is just a glitch in OpenSSL but I will add this as an RFE so we will implement line wrapping with the next release. best regards Oliver On 12.06.24 09:48, Stefan Goeman wrote: Hello I want to use the following commands to get my ca files in a nicely fomated pem-file: * curl -k --connect-timeout 10 -s https://my_est_server/.well-known/est/cacerts -o cacerts.p7 * openssl base64 -d -in cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem Now, when I use the testest server at https://testrfc7030.com:8443/ these commands work and I get a nicely looking pem file. When I use these commands with my own openxpki server, the second command does not work. I get the error (on debian 12) "unable to load PKCS7 object". One difference that I notice is that when I run the curl command against the testrfc7030 server I already have a nicely formated looking file, like you expect in a pem-file. When I do the curl command against my own server, there is only one line in the file, and when I open this file in vi I see the message "incomplete last line". I have no idea what this means? BTW, If I use "base64 -d cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem" on my system, this works. So, there is obviously a difference between "openssl base64" and "base64". Much thanks in advance for your help. Greetings, Stefan. ___ OpenXPKI-users mailing list OpenXPKI-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openxpki-users -- Protect your environment - close windows and adopt a penguin! ___ OpenXPKI-users mailing list OpenXPKI-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openxpki-users
[OpenXPKI-users] format of the .../.well-known/est/cacerts response
Hello I want to use the following commands to get my ca files in a nicely fomated pem-file: * curl -k --connect-timeout 10 -s https://my_est_server/.well-known/est/cacerts -o cacerts.p7 * openssl base64 -d -in cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem Now, when I use the testest server at https://testrfc7030.com:8443/ these commands work and I get a nicely looking pem file. When I use these commands with my own openxpki server, the second command does not work. I get the error (on debian 12) "unable to load PKCS7 object". One difference that I notice is that when I run the curl command against the testrfc7030 server I already have a nicely formated looking file, like you expect in a pem-file. When I do the curl command against my own server, there is only one line in the file, and when I open this file in vi I see the message "incomplete last line". I have no idea what this means? BTW, If I use "base64 -d cacerts.p7 | openssl pkcs7 -inform DER -outform PEM -print_certs -out cacerts.pem" on my system, this works. So, there is obviously a difference between "openssl base64" and "base64". Much thanks in advance for your help. Greetings, Stefan. ___ OpenXPKI-users mailing list OpenXPKI-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/openxpki-users
