Hi,
PEM didn't make JDK 23 as there were further comments about the API. It
hasn't been updated because of that. Hopefully in June I will be
updating the github repo.
Tony
On 5/18/24 3:55 AM, Karl Scheibelhofer wrote:
Hi Tony,
today, I had a look at the recent sources of the PEM API
implementation - i.e. https://github.com/ascarpino/jdk/commits/pem
<https://urldefense.com/v3/__https://github.com/ascarpino/jdk/commits/pem__;!!ACWV5N9M2RV99hQ!KryKT5MjRlHpyEwcDJR5vP5ST53pFGfYJ80ocGPInq5HkglLO0076V7QKD8hckW5TqF_ZNc1b1mNq6Si7vCKEwRx8XUKiQCF$>
It seems that the PEM API Tests are out of sync. They fail with
JT Harness : Tests that failed
java/security/PEM/PEMDecoderTest.java: Testing PEM decodings
java/security/PEM/PEMEncoderTest.java: Testing PEM decoding
I wanted to update the PEM keystore implementation to the latest code
base. Can you tell about the planned next steps?
Regards, Karl
On Sat, Mar 16, 2024 at 10:36 AM Karl Scheibelhofer
<karl.scheibelhofer...@gmail.com
<mailto:karl.scheibelhofer...@gmail.com>> wrote:
Hi Tony,
find my replies inline...
On Mon, Mar 11, 2024 at 6:13 AM Anthony Scarpino
<anthony.scarp...@oracle.com <mailto:anthony.scarp...@oracle.com>>
wrote:
>
>
>
> On Mar 9, 2024, at 8:09 AM, Karl Scheibelhofer
<karl.scheibelhofer...@gmail.com
<mailto:karl.scheibelhofer...@gmail.com>> wrote:
>
>
> ... try again from from my subscribed mail account...
>
>> Hi Tony,
>>
>> in my jdk fork, I created a branch named pem-feedback-karl.
>>
>> https://github.com/KarlScheibelhofer/jdk/tree/pem-feedback-karl
<https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-feedback-karl__;!!ACWV5N9M2RV99hQ!KryKT5MjRlHpyEwcDJR5vP5ST53pFGfYJ80ocGPInq5HkglLO0076V7QKD8hckW5TqF_ZNc1b1mNq6Si7vCKEwRx8QAJkEvq$>
>>
>> It is based on the pem branch of your jdk fork.
>> In this pem-feedback-karl branch, I did some cleanup without
changing
>> the API. Your tests pass as before.
>>
>> My original pem-keystore implementation for the SUN provider is
in this branch
>>
>> https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore
<https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore__;!!ACWV5N9M2RV99hQ!KryKT5MjRlHpyEwcDJR5vP5ST53pFGfYJ80ocGPInq5HkglLO0076V7QKD8hckW5TqF_ZNc1b1mNq6Si7vCKEwRx8evoXyho$>
>>
>> It did not use the PEM API.
>>
>> In the branch
>>
>>
https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore-pem-api
<https://urldefense.com/v3/__https://github.com/KarlScheibelhofer/jdk/tree/pem-keystore-pem-api__;!!ACWV5N9M2RV99hQ!KryKT5MjRlHpyEwcDJR5vP5ST53pFGfYJ80ocGPInq5HkglLO0076V7QKD8hckW5TqF_ZNc1b1mNq6Si7vCKEwRx8ZydhYWH$>
>>
>> I modified the PEM keystore implementation to use the PEMDecoder
and PEMEncoder.
>> To make it pass all tests, I had to fix some issues with the PEM
api:
>>
>> * fix missing line-breaks when encoding certificates (and CRLs)
>> * use uniform line length for PEM encoding keys and certificates
>
>
> It sounds like I did my repo update to use MimeEncoder after you
had pulled the changes.
In which repo can i see the version using MimeEncoder that you
mentioned? i can't find this in
https://github.com/ascarpino/jdk/commits/pem
<https://urldefense.com/v3/__https://github.com/ascarpino/jdk/commits/pem__;!!ACWV5N9M2RV99hQ!KryKT5MjRlHpyEwcDJR5vP5ST53pFGfYJ80ocGPInq5HkglLO0076V7QKD8hckW5TqF_ZNc1b1mNq6Si7vCKEwRx8XUKiQCF$>
>
>> During this work, I took some notes regarding the PEM api:
>>
>> * Consider decoupling the raw PEM encoding and decoding from
SecurityObject.
>> This would make the API usable for general purpose PEM
encoding and
>> decoding, not just for keys and certificates, as it is now.
>
>
> There has been discussions about adding a generic PEM object that
would have methods to return the header, footer, and PEM text,
instead of processing into a class or KeySpec. Is there something
more “general purpose” you had it mind?
In addition to header, footer and PEM text, for my keystore
implementation, I would need the preceding PEM explanatory text lines,
i.e. the lines before the header line
>
>> * For this PEM keystore implementation it is essential to parse the
>> preceding explanatory text lines.
>> The PEM decoder should support this.
>> As it is now, the keystore implementation must parse the PEM
objects
>> itself, including reading PEM header and footer lines.
>> Having to handle half the work in the application diminishes the
>> value of the PEMDecoder.
>> It only delegates the decoding of certificates and keys to the
PEMDecoder.
>>
>> * PEMDecoder should be able to handle or pass through unknown PEM
>> objects gracefully.
>> Otherwise the application has to check the PEM labels in advance
>> itself, which does not make sense.
>
>
> So do you not have a structured data file? I expected the
application would parse its own metadata, then when the structured
code got to a PEM tag, it would pass the InputStream to PEMDecoder.
>
> I am concerned about a generic PEM object storing an unknown
amount of application metadata and returning it back. I feel
handling non-PEM should be in the scope of the encoder/decoder.
No, there is no structured data file. Just a PEM file which contains
multiple PEM objects, typically certificates and private keys. It is
common practice to have related certificates and keys in a single
file. The explanatory text lines contain the name of the key, because
this is the place specified in PEM RFC to hold meta information like
this. To enable implementing a consistent Java Keystore with PEM
format, I see this necessary. At least, I found no other way for
handling key alias names as required by Java Keystore. If you know
another way, please let me know.
The files that my keystore implementation reads and writes are just
completely PEM format, from first to last line.
>
>>
>> * Though supporting InputStream/OutputStream for reading and writing
>> makes sense,
>> supporting Reader/Writer feels even more essential for robust
>> support for all character encodings.
>> Multi-Byte character encodings won't work with
InputStream/OutputStream.
>
>
> A Reader will read ahead, buffering the input data. I saw this
when I had `decode(Reader)` in the API. It would return the first
PEM object, but the read pointer was at the end of the file, missing
the remaining PEM objects.
An application or a Keystore implementation gets a stream providing
multiple concatenated PEM objects that are in this keystore object. It
needs to get all the PEM objects handing in that stream, i.e. in
essence hand in the input stream and get out a list of pem objects
containing certificates, keys and their names.
>
> Is the multi-byte characters for the keystore metadata? I don’t
see how this affects Base64.
Try feeding in a UTF-16 encoded PEM file. I guess it won't work with
InputStream. Because the current implementation expects one character
in each byte.
>
>>
>> * The standard line separator for PEM is "\r\n".
>> For PEM files stored in a typical linux file system, "\n" is
>> typically used, however.
>> See the output of openssl, for example.
>> Because there are still several command line utilities that do not
>> work well with "\r\n" line breaks.
>> Supporting a different line-separator than "\r\n" is a good
idea in
>> my opinion.
>> Base64.getMimeEncoder also supports selecting a custom line
separator.
>
>
> The standards I saw says the line separator is “\r\n”. I realize
decoders have to be more flexible because many may not follow the
line separators or cut-n-paste removes them. I think having a
configurable line separator for encoding is likely to create more
incompatibility rather than lessen for cross-platform and using with
other applications.
Yes, the PEM standard say"\r\n". But widely used tools like openssl
produce "\n". In practice, this is just the better solution. It goes
better with most other tools and version control.
I would opt for "\r\n" by default, but provide an option for "\n" only.
There must be a reason why Base64.getMimeEncoder supports custom line
separators.
>
>>
>> * The PEMEncoder encodes the predefined SecurityObjects only.
>> There is no way to use it to PEM encoded any other type of object.
>> Consider opening a path to generic use.
>
>
> The generic PEM object I mentioned previously I think fits this
case. It would still be a SecurityObject as I don’t see value in
passing in any object.
I will have a look at it. Can you provide a link to your implementation?
>
>>
>> * If an application has a DER encoded certificate, it has to decode
>> and parse the certificate before
>> it can encode it using PEMEncoder.
>> This is inconvenient.
>
>
> Yeah, there isn’t an EncodedKeySpec equivalent. I’d have to
think if there is an easy way to do this without causing more
problems. Given the purpose is going between Java Objects and PEM,
accepting random data isn’t a goal. Maybe something that can be
addressed with a generic PEM object.
As a fallback simply allow passing in a byte[] that is base64 encoded,
i.e. the result of Certificate.getEncoded().
Thank you!
Karl
...
