Re: [Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

On 01/12/2017 04:35 AM, Jan Cholasta wrote:

On 11.1.2017 00:38, Ben Lipton wrote:


On 01/10/2017 01:58 AM, Jan Cholasta wrote:

On 19.12.2016 21:59, Ben Lipton wrote:


On 12/15/2016 11:11 PM, Ben Lipton wrote:


On 12/12/2016 03:52 AM, Jan Cholasta wrote:

On 5.12.2016 16:48, Ben Lipton wrote:

Hi Jan, thanks for the comments.


On 12/05/2016 04:25 AM, Jan Cholasta wrote:

Hi Ben,

On 3.11.2016 00:12, Ben Lipton wrote:

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on 
new
development work for the CSR autogeneration project, and I 
want to

leave
the project in as organized a state as possible. So, I'm taking
inventory of the work I've done in order to make sure that what's
ready
for review can get reviewed and the ideas that have been 
discussed

get
prototyped or at least recorded so they won't be forgotten.


Thanks, I have some questions and comments, see below.



Code that's ready for review (I will continue to put in as much
time as
needed to help get these ready for submission):

- Current PR: https://github.com/freeipa/freeipa/pull/10


How hard would it be to update the PR to use the "new" interface
from
the design thread? By this I mean that currently there is a 
command

(cert_get_requestdata), which creates a CSR from profile id +
principal + helper, but in the design we discussed a command which
creates a CertificationRequestInfo from profile id + principal +
public key.

Internally it could use the OpenSSL helper, no need to 
implement the
full "new" design. With your build_requestinfo.c code below it 
looks

like it should be pretty straightforward.


This is probably doable with the cffi, but I'm concerned about
usability. A user can run the current command to get a (reusable)
script, and run the script to get a CSR. It works with keys in
both PEM
files and NSS databases already. If we change to outputting a
CertificationRequestInfo, in order to make this usable on the 
command

line, we'll need:
- An additional tool to sign a CSR given a CertificationRequestInfo
(for
both types of key storage).
- A way to extract a SubjectPublicKeyInfo structure from a key 
within

the ipa command (like [1] but we need it for both types of key
storage)
Since as far as I know there's no standard encoding for files
containing
only a CertificationRequestInfo or a SubjectPublicKeyInfo, we'll be
writing and distributing these ourselves. I think that's where
most of
the extra work will come in.


For PEM files, this is easily doable using python-cryptography (to
extract SubjectPublicKeyInfo and sign CertificationRequestInfo) and
PyASN1 (to create a CSR from the CertificationRequestInfo and the
signature).


I didn't realize that python-cryptography knew about
SubjectPublicKeyInfo structures, but indeed this seems to be pretty
straightforward:

key = load_pem_private_key(key_bytes, None, default_backend())
pubkey_info = key.public_key().public_bytes(Encoding.DER,
PublicFormat.SubjectPublicKeyInfo)

Thanks for letting me know this functionality already existed.


I'm currently working on the step of signing the
CertificationRequestInfo and creating a CSR from it. I think I have it
working with pyasn1, but of course the "signature algorithm" for the CSR
needs to be specified and implemented within the code since I'm not
using a library that understands CSRs natively. The code I have
currently always produces CSRs with the sha256WithRSAEncryption
algorithm (DER-encode request info, SHA256, PKCS #1v1.5 padding, RSA
encryption), and the OID for that algorithm is hardcoded in the output
CSR. Is this ok or will we need more flexibility than that?


IMO it's OK for starters.



For NSS databases, this will be trickier and will require calling C
functions, as neither certutil nor python-nss provide a way to a)
address existing keys in the database by key ID b) get
SubjectPublicKeyInfo for a given key.


This can be worked around by:

1. Generating a key + temporary certificate:

n=$(head -c 40 /dev/urandom | base32)
certutil -S -n $n -s CN=$n -x -t ,,

2. Extracting the public key from the certificate:

certutil -L -n $n -a >temp.crt
(extract the public key using python-cryptography)

3. Deleting the temporary certificate:

certutil -D -n $n

4. Importing the newly issued certificate:

certutil -A -n $n -t ,, -a 
Oof, thanks, I'm not sure I would have been able to come up with that.
Can you generate a key without a temporary certificate if you use the
NSS API, or does their model require every key to belong to a cert?


I'm pretty sure it's possible, but it certainly won't be as simple as 
this. I gave up after a few hours of digging into NSS source code and 
not being able to figure out how.




As for encoding, the obvious choice is DER. It does not really 
matter

there is no standard file format, as we won't be transferring 

Re: [Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

On 01/10/2017 01:58 AM, Jan Cholasta wrote:

On 19.12.2016 21:59, Ben Lipton wrote:


On 12/15/2016 11:11 PM, Ben Lipton wrote:


On 12/12/2016 03:52 AM, Jan Cholasta wrote:

On 5.12.2016 16:48, Ben Lipton wrote:

Hi Jan, thanks for the comments.


On 12/05/2016 04:25 AM, Jan Cholasta wrote:

Hi Ben,

On 3.11.2016 00:12, Ben Lipton wrote:

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on new
development work for the CSR autogeneration project, and I want to
leave
the project in as organized a state as possible. So, I'm taking
inventory of the work I've done in order to make sure that what's
ready
for review can get reviewed and the ideas that have been discussed
get
prototyped or at least recorded so they won't be forgotten.


Thanks, I have some questions and comments, see below.



Code that's ready for review (I will continue to put in as much
time as
needed to help get these ready for submission):

- Current PR: https://github.com/freeipa/freeipa/pull/10


How hard would it be to update the PR to use the "new" interface 
from

the design thread? By this I mean that currently there is a command
(cert_get_requestdata), which creates a CSR from profile id +
principal + helper, but in the design we discussed a command which
creates a CertificationRequestInfo from profile id + principal +
public key.

Internally it could use the OpenSSL helper, no need to implement the
full "new" design. With your build_requestinfo.c code below it looks
like it should be pretty straightforward.


This is probably doable with the cffi, but I'm concerned about
usability. A user can run the current command to get a (reusable)
script, and run the script to get a CSR. It works with keys in 
both PEM

files and NSS databases already. If we change to outputting a
CertificationRequestInfo, in order to make this usable on the command
line, we'll need:
- An additional tool to sign a CSR given a CertificationRequestInfo
(for
both types of key storage).
- A way to extract a SubjectPublicKeyInfo structure from a key within
the ipa command (like [1] but we need it for both types of key 
storage)

Since as far as I know there's no standard encoding for files
containing
only a CertificationRequestInfo or a SubjectPublicKeyInfo, we'll be
writing and distributing these ourselves. I think that's where 
most of

the extra work will come in.


For PEM files, this is easily doable using python-cryptography (to
extract SubjectPublicKeyInfo and sign CertificationRequestInfo) and
PyASN1 (to create a CSR from the CertificationRequestInfo and the
signature).


I didn't realize that python-cryptography knew about
SubjectPublicKeyInfo structures, but indeed this seems to be pretty
straightforward:

key = load_pem_private_key(key_bytes, None, default_backend())
pubkey_info = key.public_key().public_bytes(Encoding.DER,
PublicFormat.SubjectPublicKeyInfo)

Thanks for letting me know this functionality already existed.


I'm currently working on the step of signing the 
CertificationRequestInfo and creating a CSR from it. I think I have it 
working with pyasn1, but of course the "signature algorithm" for the CSR 
needs to be specified and implemented within the code since I'm not 
using a library that understands CSRs natively. The code I have 
currently always produces CSRs with the sha256WithRSAEncryption 
algorithm (DER-encode request info, SHA256, PKCS #1v1.5 padding, RSA 
encryption), and the OID for that algorithm is hardcoded in the output 
CSR. Is this ok or will we need more flexibility than that?


For NSS databases, this will be trickier and will require calling C
functions, as neither certutil nor python-nss provide a way to a)
address existing keys in the database by key ID b) get
SubjectPublicKeyInfo for a given key.


This can be worked around by:

1. Generating a key + temporary certificate:

n=$(head -c 40 /dev/urandom | base32)
certutil -S -n $n -s CN=$n -x -t ,,

2. Extracting the public key from the certificate:

certutil -L -n $n -a >temp.crt
(extract the public key using python-cryptography)

3. Deleting the temporary certificate:

certutil -D -n $n

4. Importing the newly issued certificate:

certutil -A -n $n -t ,, -a Oof, thanks, I'm not sure I would have been able to come up with that. 
Can you generate a key without a temporary certificate if you use the 
NSS API, or does their model require every key to belong to a cert?


As for encoding, the obvious choice is DER. It does not really matter
there is no standard file format, as we won't be transferring these
as files anyway.


Agreed. I just meant there aren't tools already because this isn't a
type of file one often needs to process.




Would it be ok to stick with the current design in this PR? I'd feel
much better if we could get the basic functionality into the repo and
then iterate on it rat

Re: [Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

On 12/15/2016 11:11 PM, Ben Lipton wrote:


On 12/12/2016 03:52 AM, Jan Cholasta wrote:

On 5.12.2016 16:48, Ben Lipton wrote:

Hi Jan, thanks for the comments.


On 12/05/2016 04:25 AM, Jan Cholasta wrote:

Hi Ben,

On 3.11.2016 00:12, Ben Lipton wrote:

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on new
development work for the CSR autogeneration project, and I want to 
leave

the project in as organized a state as possible. So, I'm taking
inventory of the work I've done in order to make sure that what's 
ready
for review can get reviewed and the ideas that have been discussed 
get

prototyped or at least recorded so they won't be forgotten.


Thanks, I have some questions and comments, see below.



Code that's ready for review (I will continue to put in as much 
time as

needed to help get these ready for submission):

- Current PR: https://github.com/freeipa/freeipa/pull/10


How hard would it be to update the PR to use the "new" interface from
the design thread? By this I mean that currently there is a command
(cert_get_requestdata), which creates a CSR from profile id +
principal + helper, but in the design we discussed a command which
creates a CertificationRequestInfo from profile id + principal +
public key.

Internally it could use the OpenSSL helper, no need to implement the
full "new" design. With your build_requestinfo.c code below it looks
like it should be pretty straightforward.


This is probably doable with the cffi, but I'm concerned about
usability. A user can run the current command to get a (reusable)
script, and run the script to get a CSR. It works with keys in both PEM
files and NSS databases already. If we change to outputting a
CertificationRequestInfo, in order to make this usable on the command
line, we'll need:
- An additional tool to sign a CSR given a CertificationRequestInfo 
(for

both types of key storage).
- A way to extract a SubjectPublicKeyInfo structure from a key within
the ipa command (like [1] but we need it for both types of key storage)
Since as far as I know there's no standard encoding for files 
containing

only a CertificationRequestInfo or a SubjectPublicKeyInfo, we'll be
writing and distributing these ourselves. I think that's where most of
the extra work will come in.


For PEM files, this is easily doable using python-cryptography (to 
extract SubjectPublicKeyInfo and sign CertificationRequestInfo) and 
PyASN1 (to create a CSR from the CertificationRequestInfo and the 
signature).


I didn't realize that python-cryptography knew about 
SubjectPublicKeyInfo structures, but indeed this seems to be pretty 
straightforward:


key = load_pem_private_key(key_bytes, None, default_backend())
pubkey_info = key.public_key().public_bytes(Encoding.DER, 
PublicFormat.SubjectPublicKeyInfo)


Thanks for letting me know this functionality already existed.


For NSS databases, this will be trickier and will require calling C 
functions, as neither certutil nor python-nss provide a way to a) 
address existing keys in the database by key ID b) get 
SubjectPublicKeyInfo for a given key.


As for encoding, the obvious choice is DER. It does not really matter 
there is no standard file format, as we won't be transferring these 
as files anyway.


Agreed. I just meant there aren't tools already because this isn't a 
type of file one often needs to process.




Would it be ok to stick with the current design in this PR? I'd feel
much better if we could get the basic functionality into the repo and
then iterate on it rather than changing the plan at this point. I can
create a separate PR to change cert_get_requestdata to this new
interface and at the same time add the necessary adapters (bullet 
points

above) to make it user-friendly.


Works for me.


Updated the PR to fix conflicts with master. Had some trouble with CI 
but creating a new PR with the same commits fixed it 
(https://github.com/freeipa/freeipa/pull/337). Not sure if it's fixed 
permanently, so I guess I'll just keep the two PRs synchronized now, 
or we could close the old one.




I would probably just implement the adapters within the
cert_build/cert_request client code unless you think having standalone
tools is valuable. I suppose certmonger is going to need these features
too, but I don't know how well sharing code between them is going to 
work.


cert-request is exactly the place where it should be :-) I wouldn't 
bother with certmonger until we have a server-side csrgen.






- Allow some fields to be specified by the user at creation time:
https://github.com/LiptonB/freeipa/commits/local-user-data


Good idea :-)



- Automation for the full process from getting CSR data to requesting
cert: https://github.com/LiptonB/freeipa/commits/local-cert-build


LGTM, although I would prefer if this was a client-side extension of
cert-request rather than 

Re: [Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

On 12/12/2016 03:52 AM, Jan Cholasta wrote:

On 5.12.2016 16:48, Ben Lipton wrote:

Hi Jan, thanks for the comments.


On 12/05/2016 04:25 AM, Jan Cholasta wrote:

Hi Ben,

On 3.11.2016 00:12, Ben Lipton wrote:

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on new
development work for the CSR autogeneration project, and I want to 
leave

the project in as organized a state as possible. So, I'm taking
inventory of the work I've done in order to make sure that what's 
ready

for review can get reviewed and the ideas that have been discussed get
prototyped or at least recorded so they won't be forgotten.


Thanks, I have some questions and comments, see below.



Code that's ready for review (I will continue to put in as much 
time as

needed to help get these ready for submission):

- Current PR: https://github.com/freeipa/freeipa/pull/10


How hard would it be to update the PR to use the "new" interface from
the design thread? By this I mean that currently there is a command
(cert_get_requestdata), which creates a CSR from profile id +
principal + helper, but in the design we discussed a command which
creates a CertificationRequestInfo from profile id + principal +
public key.

Internally it could use the OpenSSL helper, no need to implement the
full "new" design. With your build_requestinfo.c code below it looks
like it should be pretty straightforward.


This is probably doable with the cffi, but I'm concerned about
usability. A user can run the current command to get a (reusable)
script, and run the script to get a CSR. It works with keys in both PEM
files and NSS databases already. If we change to outputting a
CertificationRequestInfo, in order to make this usable on the command
line, we'll need:
- An additional tool to sign a CSR given a CertificationRequestInfo (for
both types of key storage).
- A way to extract a SubjectPublicKeyInfo structure from a key within
the ipa command (like [1] but we need it for both types of key storage)
Since as far as I know there's no standard encoding for files containing
only a CertificationRequestInfo or a SubjectPublicKeyInfo, we'll be
writing and distributing these ourselves. I think that's where most of
the extra work will come in.


For PEM files, this is easily doable using python-cryptography (to 
extract SubjectPublicKeyInfo and sign CertificationRequestInfo) and 
PyASN1 (to create a CSR from the CertificationRequestInfo and the 
signature).


I didn't realize that python-cryptography knew about 
SubjectPublicKeyInfo structures, but indeed this seems to be pretty 
straightforward:


key = load_pem_private_key(key_bytes, None, default_backend())
pubkey_info = key.public_key().public_bytes(Encoding.DER, 
PublicFormat.SubjectPublicKeyInfo)


Thanks for letting me know this functionality already existed.


For NSS databases, this will be trickier and will require calling C 
functions, as neither certutil nor python-nss provide a way to a) 
address existing keys in the database by key ID b) get 
SubjectPublicKeyInfo for a given key.


As for encoding, the obvious choice is DER. It does not really matter 
there is no standard file format, as we won't be transferring these as 
files anyway.


Agreed. I just meant there aren't tools already because this isn't a 
type of file one often needs to process.




Would it be ok to stick with the current design in this PR? I'd feel
much better if we could get the basic functionality into the repo and
then iterate on it rather than changing the plan at this point. I can
create a separate PR to change cert_get_requestdata to this new
interface and at the same time add the necessary adapters (bullet points
above) to make it user-friendly.


Works for me.


Updated the PR to fix conflicts with master. Had some trouble with CI 
but creating a new PR with the same commits fixed it 
(https://github.com/freeipa/freeipa/pull/337). Not sure if it's fixed 
permanently, so I guess I'll just keep the two PRs synchronized now, or 
we could close the old one.




I would probably just implement the adapters within the
cert_build/cert_request client code unless you think having standalone
tools is valuable. I suppose certmonger is going to need these features
too, but I don't know how well sharing code between them is going to 
work.


cert-request is exactly the place where it should be :-) I wouldn't 
bother with certmonger until we have a server-side csrgen.






- Allow some fields to be specified by the user at creation time:
https://github.com/LiptonB/freeipa/commits/local-user-data


Good idea :-)



- Automation for the full process from getting CSR data to requesting
cert: https://github.com/LiptonB/freeipa/commits/local-cert-build


LGTM, although I would prefer if this was a client-side extension of
cert-request rather than a completely new command.


I did try that at first, but

Re: [Freeipa-devel] Travis CI unexpected PEP8 errors

[Ben Lipton]

On 12/14/2016 03:42 AM, Martin Babinsky wrote:

On 12/14/2016 09:00 AM, Standa Laznicka wrote:

On 12/14/2016 02:53 AM, Ben Lipton wrote:

Hi all,

I'm pretty sure this is unrelated to the CI issues discussed in other
threads recently, but they reminded me that I've been having this odd
issue.

https://travis-ci.org/freeipa/freeipa/jobs/183756995 is the most
recent run on my pull request,
https://github.com/freeipa/freeipa/pull/10. For a while now, every
time the CI runs on my PR, it fails due to several PEP8 errors that
are not detected when I run `git diff master -U0 | pep8 --diff` on my
local copy. In fact, the errors are all in files not touched by my PR,
which doesn't make much sense to me based on the behavior of `git 
diff`.


I noticed that the top of the travis build says:

 - Commit 1f50550
 - #10: Client-side CSR autogeneration
 - Branch master

I'm not sure what the "commit" link actually means, but that commit
seems to have nothing to do with my PR nor the current master. Could
Travis be diffing against the wrong code? Or if not, do you have any
idea what might be causing these failures?

Thanks,
Ben


Hi Ben,

I was going through the Travis CI log of and noticed what might have
caused the issue:

$ cd freeipa/freeipa
$ git fetch origin +refs/pull/109/merge:

It seems that for your pull request #10 (and for some reason for your
pull request only), Travis fetched a different (old) pull request which
it then tried to merge with current master, hence the errors. I don't
think it was testing your code at all.

I do not have an explanation for this other than it might be a Travis
bug, CCing Martin for a better answer.

Standa


Yes indeed for some reason Travis fetches completely wrong PR for 
tests. I have no idea why it does this. I have tried to restart the 
build with the same results.


We will probably have to contact Travis support for this issue. In the 
meanwhile, can you prepare a separate PR from completely new branch 
with the same commits so that we can see if it catches up this time?


Evidently there was something special about my first pull request, since 
the new one works fine (as you pointed out). Thanks for confirming that 
it wasn't something I was doing wrong.


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] Travis CI unexpected PEP8 errors

[Ben Lipton]

Hi all,

I'm pretty sure this is unrelated to the CI issues discussed in other 
threads recently, but they reminded me that I've been having this odd issue.


https://travis-ci.org/freeipa/freeipa/jobs/183756995 is the most recent 
run on my pull request, https://github.com/freeipa/freeipa/pull/10. For 
a while now, every time the CI runs on my PR, it fails due to several 
PEP8 errors that are not detected when I run `git diff master -U0 | pep8 
--diff` on my local copy. In fact, the errors are all in files not 
touched by my PR, which doesn't make much sense to me based on the 
behavior of `git diff`.


I noticed that the top of the travis build says:

 - Commit 1f50550
 - #10: Client-side CSR autogeneration
 - Branch master

I'm not sure what the "commit" link actually means, but that commit 
seems to have nothing to do with my PR nor the current master. Could 
Travis be diffing against the wrong code? Or if not, do you have any 
idea what might be causing these failures?


Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

Hi Jan, thanks for the comments.


On 12/05/2016 04:25 AM, Jan Cholasta wrote:

Hi Ben,

On 3.11.2016 00:12, Ben Lipton wrote:

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on new
development work for the CSR autogeneration project, and I want to leave
the project in as organized a state as possible. So, I'm taking
inventory of the work I've done in order to make sure that what's ready
for review can get reviewed and the ideas that have been discussed get
prototyped or at least recorded so they won't be forgotten.


Thanks, I have some questions and comments, see below.



Code that's ready for review (I will continue to put in as much time as
needed to help get these ready for submission):

- Current PR: https://github.com/freeipa/freeipa/pull/10


How hard would it be to update the PR to use the "new" interface from 
the design thread? By this I mean that currently there is a command 
(cert_get_requestdata), which creates a CSR from profile id + 
principal + helper, but in the design we discussed a command which 
creates a CertificationRequestInfo from profile id + principal + 
public key.


Internally it could use the OpenSSL helper, no need to implement the 
full "new" design. With your build_requestinfo.c code below it looks 
like it should be pretty straightforward.


This is probably doable with the cffi, but I'm concerned about 
usability. A user can run the current command to get a (reusable) 
script, and run the script to get a CSR. It works with keys in both PEM 
files and NSS databases already. If we change to outputting a 
CertificationRequestInfo, in order to make this usable on the command 
line, we'll need:
- An additional tool to sign a CSR given a CertificationRequestInfo (for 
both types of key storage).
- A way to extract a SubjectPublicKeyInfo structure from a key within 
the ipa command (like [1] but we need it for both types of key storage)
Since as far as I know there's no standard encoding for files containing 
only a CertificationRequestInfo or a SubjectPublicKeyInfo, we'll be 
writing and distributing these ourselves. I think that's where most of 
the extra work will come in.


Would it be ok to stick with the current design in this PR? I'd feel 
much better if we could get the basic functionality into the repo and 
then iterate on it rather than changing the plan at this point. I can 
create a separate PR to change cert_get_requestdata to this new 
interface and at the same time add the necessary adapters (bullet points 
above) to make it user-friendly.


I would probably just implement the adapters within the 
cert_build/cert_request client code unless you think having standalone 
tools is valuable. I suppose certmonger is going to need these features 
too, but I don't know how well sharing code between them is going to work.




- Allow some fields to be specified by the user at creation time:
https://github.com/LiptonB/freeipa/commits/local-user-data


Good idea :-)



- Automation for the full process from getting CSR data to requesting
cert: https://github.com/LiptonB/freeipa/commits/local-cert-build


LGTM, although I would prefer if this was a client-side extension of 
cert-request rather than a completely new command.


I did try that at first, but I struggled to figure out the interface for 
the modified cert-request. (Not that the current solution is so great, 
what with the copying of options from cert_request and certreq.) If I 
remember correctly, I was uncertain how to implement parameters that are 
required/invalid based on other parameters: the current cert-request 
takes a signed CSR (required), a principal (required), and a profile ID; 
the new cert-request (what I implemented as cert-build) takes a 
principal (required), a profile ID (required), and a key location 
(required). I can't remember if that was the only problem, but I'll try 
again to merge the commands and get back to you.




Other prototypes and design ideas that aren't ready for submission yet:

- Utility written in C to build a CertificationRequestInfo from a
SubjectPublicKeyInfo and an openssl-style config file. The purpose of
this is to take a config that my code already knows how to generate, and
put it in a form that certmonger can use. This is nearly done and
available at:
https://github.com/LiptonB/freeipa-prototypes/blob/master/build_requestinfo.c 



Nice! As I said above, this could really make implementing the "new" 
csrgen interface simple.





- Ideally it should be possible to use this tool to reimplement the full
cert-request automation (local-cert-build branch) without a dependency
on the certutil/openssl tools. However, I don't think any of the python
crypto libraries have bindings for the functions that deal with
CertificationRequestInfo objects, so I don't think I can do this in the
short term.


You can use python-cffi to 

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 10/20/2016 03:52 PM, Ben Lipton wrote:

On 10/17/2016 02:16 AM, Jan Cholasta wrote:

On 13.10.2016 17:23, Ben Lipton wrote:

Thank you, this was a really helpful clarification of your point.
Comments below. Once again, I'm sorry I missed the email for so long.

Ben

On 09/05/2016 06:52 AM, Jan Cholasta wrote:

On 27.8.2016 22:40, Ben Lipton wrote:

On 08/25/2016 04:11 PM, Rob Crittenden wrote:

Ben Lipton wrote:

On 08/23/2016 03:54 AM, Jan Cholasta wrote:

On 8.8.2016 22:23, Ben Lipton wrote:

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I 
hope

you'll
let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate
requests
easier to generate when using alternate certificate 
profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 








The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be
working on
implementing this design over the next couple of months.
If you
have
the time and interest, please take a look and share any
comments or
concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document
that
covers
the proposed schema additions in a more descriptive way 
(with

diagrams!)
I'm very new to developing with LDAP, so some more 
experienced

eyes on
the proposal would be very helpful, even if you don't have
time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 








if you have a chance.


I finally had a chance to take a look at this, here are some
comments:

1) I don't like how transformation rules are tied to a
particular
helper and have to be duplicated for each of them. They
should be
generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated 
using

python-cryptography's CertificateSigningRequestBuilder [1]
rather
than
openssl or certutil or any other command line tool.

There are lots of tools that users might want to use to 
manage

their
private keys, so I don't know if we can assume that whatever
library we
prefer will actually be able to access the private key to 
sign a

CSR,
which is why I thought it would be useful to support more 
than

one.


python-cryptography has the notion of backends, which allow 
it to

support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on 
PKCS#11

backend [3], which provides support for HSMs and soft-tokens
(like
NSS
databases).

Alternatively, for NSS databases (and other "simple" 
cases), you

can
generate the private key with python-cryptography using the
default
backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for
them.

So, the only thing that's currently lacking is HSM support, 
but

given
that we don't support HSMs in IPA nor in certmonger, I don't
think
it's an issue for now.


The
purpose of the mapping rule is to tie together the
transformation
rules
that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those
rules
are
automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider
`openssl`
and `certutil` the only implementations :-) But I don't think
this
solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules 
shouldn't

really be stored on and processed by the server. The server
should
know the *what* (mapping rules), but not the *how*
(transformation
rules). The *how* is an implementation detail and does not
change in
time, so there's no benefit in handling it on the server. It
should be
handled exclusively on the client, which I believe would also
make
the
whole thing more robust (it would not be possible for a bug on
the
server to break all the clients).

This is a good point. However, for the scope of Ben's project
can we
limit it by openssl and certutil support? Otherwise Ben
wouldn't be
able
to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)




This is turning out to be a common (and, I think, reasonable)
reaction
to the proposal. It is rather complex, and I worry that it
will be
difficult to configure. On the other hand, there is some 
hidden
complexity to enabling a simpler config format, as well. 
One of

the
goal

[Freeipa-devel] CSR autogeneration next steps

[Ben Lipton]

Hi everybody,

Soon I'm going to have to reduce the amount of time I spend on new 
development work for the CSR autogeneration project, and I want to leave 
the project in as organized a state as possible. So, I'm taking 
inventory of the work I've done in order to make sure that what's ready 
for review can get reviewed and the ideas that have been discussed get 
prototyped or at least recorded so they won't be forgotten.


Code that's ready for review (I will continue to put in as much time as 
needed to help get these ready for submission):


- Current PR: https://github.com/freeipa/freeipa/pull/10

- Allow some fields to be specified by the user at creation time: 
https://github.com/LiptonB/freeipa/commits/local-user-data


- Automation for the full process from getting CSR data to requesting 
cert: https://github.com/LiptonB/freeipa/commits/local-cert-build


Other prototypes and design ideas that aren't ready for submission yet:

- Utility written in C to build a CertificationRequestInfo from a 
SubjectPublicKeyInfo and an openssl-style config file. The purpose of 
this is to take a config that my code already knows how to generate, and 
put it in a form that certmonger can use. This is nearly done and 
available at: 
https://github.com/LiptonB/freeipa-prototypes/blob/master/build_requestinfo.c


- Ideally it should be possible to use this tool to reimplement the full 
cert-request automation (local-cert-build branch) without a dependency 
on the certutil/openssl tools. However, I don't think any of the python 
crypto libraries have bindings for the functions that deal with 
CertificationRequestInfo objects, so I don't think I can do this in the 
short term.


- Certmonger "helper" program that takes in the CertificationRequestInfo 
that certmonger generates, calls out to IPA for profile-specific data, 
and returns an updated CertificationRequestInfo built from the data. 
Certmonger doesn't currently support this type of helper, but (if I 
understood correctly) this is the architecture Nalin believed would be 
simplest to fit in. This is not done yet, but I intend to complete it 
soon - it shouldn't require much code beyond what's in build_requestinfo.c.


- Tool to convert an XER-encoded cert extension to DER, given the ASN.1 
description of the extension. This would unblock Jan Cholasta's idea of 
using XSLT for templates rather than text-based formatting. I should be 
able to implement the conversion tool, but it may be a while before I 
have time to demo the full XSLT idea.


So: currently on my to do list are the certmonger helper and the 
XER->DER conversion tool. Do you have any comments about these plans, 
and is there anything else I can do to wrap up the project neatly?


Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 10/17/2016 02:16 AM, Jan Cholasta wrote:

On 13.10.2016 17:23, Ben Lipton wrote:

Thank you, this was a really helpful clarification of your point.
Comments below. Once again, I'm sorry I missed the email for so long.

Ben

On 09/05/2016 06:52 AM, Jan Cholasta wrote:

On 27.8.2016 22:40, Ben Lipton wrote:

On 08/25/2016 04:11 PM, Rob Crittenden wrote:

Ben Lipton wrote:

On 08/23/2016 03:54 AM, Jan Cholasta wrote:

On 8.8.2016 22:23, Ben Lipton wrote:

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I 
hope

you'll
let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate
requests
easier to generate when using alternate certificate 
profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 








The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be
working on
implementing this design over the next couple of months.
If you
have
the time and interest, please take a look and share any
comments or
concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document
that
covers
the proposed schema additions in a more descriptive way 
(with

diagrams!)
I'm very new to developing with LDAP, so some more 
experienced

eyes on
the proposal would be very helpful, even if you don't have
time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 








if you have a chance.


I finally had a chance to take a look at this, here are some
comments:

1) I don't like how transformation rules are tied to a
particular
helper and have to be duplicated for each of them. They
should be
generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated 
using

python-cryptography's CertificateSigningRequestBuilder [1]
rather
than
openssl or certutil or any other command line tool.


There are lots of tools that users might want to use to manage
their
private keys, so I don't know if we can assume that whatever
library we
prefer will actually be able to access the private key to 
sign a

CSR,
which is why I thought it would be useful to support more than
one.


python-cryptography has the notion of backends, which allow 
it to

support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on 
PKCS#11

backend [3], which provides support for HSMs and soft-tokens
(like
NSS
databases).

Alternatively, for NSS databases (and other "simple" cases), 
you

can
generate the private key with python-cryptography using the
default
backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for
them.

So, the only thing that's currently lacking is HSM support, but
given
that we don't support HSMs in IPA nor in certmonger, I don't
think
it's an issue for now.


The
purpose of the mapping rule is to tie together the
transformation
rules
that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those
rules
are
automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider
`openssl`
and `certutil` the only implementations :-) But I don't think
this
solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules 
shouldn't

really be stored on and processed by the server. The server
should
know the *what* (mapping rules), but not the *how*
(transformation
rules). The *how* is an implementation detail and does not
change in
time, so there's no benefit in handling it on the server. It
should be
handled exclusively on the client, which I believe would also
make
the
whole thing more robust (it would not be possible for a bug on
the
server to break all the clients).

This is a good point. However, for the scope of Ben's project
can we
limit it by openssl and certutil support? Otherwise Ben
wouldn't be
able
to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)




This is turning out to be a common (and, I think, reasonable)
reaction
to the proposal. It is rather complex, and I worry that it
will be
difficult to configure. On the other hand, there is some 
hidden
complexity to enabling a simpler config format, as well. 
One of

the
goals of the project as it was presented to me was to 

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

Thank you, this was a really helpful clarification of your point. 
Comments below. Once again, I'm sorry I missed the email for so long.


Ben

On 09/05/2016 06:52 AM, Jan Cholasta wrote:

On 27.8.2016 22:40, Ben Lipton wrote:

On 08/25/2016 04:11 PM, Rob Crittenden wrote:

Ben Lipton wrote:

On 08/23/2016 03:54 AM, Jan Cholasta wrote:

On 8.8.2016 22:23, Ben Lipton wrote:

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I hope
you'll
let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate
requests
easier to generate when using alternate certificate profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 







The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be
working on
implementing this design over the next couple of months. 
If you

have
the time and interest, please take a look and share any
comments or
concerns that you have.

Thanks!

Ben

Just a quick update to say that I've created a new document 
that

covers
the proposed schema additions in a more descriptive way (with
diagrams!)
I'm very new to developing with LDAP, so some more experienced
eyes on
the proposal would be very helpful, even if you don't have
time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 







if you have a chance.


I finally had a chance to take a look at this, here are some
comments:

1) I don't like how transformation rules are tied to a 
particular
helper and have to be duplicated for each of them. They 
should be

generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated using
python-cryptography's CertificateSigningRequestBuilder [1] 
rather

than
openssl or certutil or any other command line tool.


There are lots of tools that users might want to use to manage
their
private keys, so I don't know if we can assume that whatever
library we
prefer will actually be able to access the private key to sign a
CSR,
which is why I thought it would be useful to support more than
one.


python-cryptography has the notion of backends, which allow it to
support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on PKCS#11
backend [3], which provides support for HSMs and soft-tokens 
(like

NSS
databases).

Alternatively, for NSS databases (and other "simple" cases), you
can
generate the private key with python-cryptography using the 
default

backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for 
them.


So, the only thing that's currently lacking is HSM support, but
given
that we don't support HSMs in IPA nor in certmonger, I don't 
think

it's an issue for now.


The
purpose of the mapping rule is to tie together the 
transformation

rules
that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those 
rules

are
automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider 
`openssl`
and `certutil` the only implementations :-) But I don't think 
this

solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules shouldn't
really be stored on and processed by the server. The server 
should
know the *what* (mapping rules), but not the *how* 
(transformation

rules). The *how* is an implementation detail and does not
change in
time, so there's no benefit in handling it on the server. It
should be
handled exclusively on the client, which I believe would also 
make

the
whole thing more robust (it would not be possible for a bug on 
the

server to break all the clients).
This is a good point. However, for the scope of Ben's project 
can we
limit it by openssl and certutil support? Otherwise Ben 
wouldn't be

able
to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)




This is turning out to be a common (and, I think, reasonable)
reaction
to the proposal. It is rather complex, and I worry that it 
will be

difficult to configure. On the other hand, there is some hidden
complexity to enabling a simpler config format, as well. One of
the
goals of the project as it was presented to me was to allow the
creation
of profiles that add certificate extensions *that FreeIPA

Re: [Freeipa-devel] Github review feature

[Ben Lipton]

On 09/16/2016 03:17 AM, Martin Basti wrote:


Sorry for stealing your thread, but you started asking about github 
review emails :)



Standard review inline comments are disabled on purpose, each comment 
generates one email, so we decided that is better after review to 
write a regular comment "NACK, please see inline comments" or so.


I would expecting that the new review feature is sending all comments 
in one batch, but I was wrong. I used the new review feature (with the 
pending comments) but when I sent it I received all comments as single 
notifications again, so again one inline comment = one email


Even worse it is with states of review (approved, required change). I 
didn't received any notification from github related to this (not sure 
if is part of any inline comment message or just not implemented yet). 
This is not documented in their API docs (according David) so we 
cannot use it our tools yet.


Generally adding Labels ACK/Rejected are more visible and filters can 
be made easily.



So for now I would stay with our old workflow and not extend email 
notifications. We can play with new review feature for longer time and 
decide if it is worth to use it (and change email notification 
accordingly)




That all seems reasonable. I hope they will improve the API in the 
future to make this work better, but in the meantime the current process 
is fine. Thanks for looking into it!


Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [freeipa PR#10] Client-side CSR autogeneration (comment)

[Ben Lipton]

On 09/15/2016 02:12 AM, jcholast wrote:

jcholast commented on a pull request

"""
In addition to my inline comments above:

1. "Certificate mapping" does not really evoke "certificate request templating" 
to me, and is also used in the context of mapping identities to certificates. Could we use a more 
suitable name to avoid confusion?
2. The `ipalib.certmapping` module is used only in `ipaclient`, so that's where 
it should be located. It can be moved to `ipalib` later if necessary.
3. I don't think `IPAExtension` deserves it's own module, at least not now.
"""

See the full comment at 
https://github.com/freeipa/freeipa/pull/10#issuecomment-247244120


Tried sending my comments as a "review" (new Github feature) and it 
seems they don't get sent to the list that way. So:


Thanks for the comments! I've fixed the simple ones and replied to the 
rest. Regarding your comments about file organization:


1. I quite agree that certmapping isn't a good name for what this
   turned out to be. With the convention of naming modules after the
   objects they model, perhaps a good name would
   be|certrequest|or|csr|? The command could be renamed to something
   like|certrequest-get-data|(or|certrequest-get-script|).
2. Just to confirm, you're suggesting just moving these classes to
   the|ipaclient.plugins.|module?
3. Seems reasonable, I've moved it into the ipalib module for now. It
   will go wherever the contents of that module end up.

Logistical stuff:

 * Now that this is under review I won't add any more content. Are you
   ok with the two commits about testing being part of this review or
   should I remove them?
 * If you run rebase --autosquash with the latest commit it doesn't
   actually apply cleanly, but I'm trying not to change history while
   it's being reviewed, so I'll do the rebase later on if that's ok?


-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 09/02/2016 05:04 AM, Petr Spacek wrote:

On 2.9.2016 04:19, Ben Lipton wrote:

On 07/27/2016 02:42 PM, Ben Lipton wrote:

On 07/21/2016 11:43 AM, Petr Spacek wrote:

Besides this nit,
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Mapping_Rules#Planned_implementation

sounds reasonable. I like how it prevents bad data from template-injection.

That's what I like about it, too. It does turn out to make things a little
tricky when it comes to writing rules that won't render if the data they
depend on is unavailable. (Because instead of rendering individual rules
which we can drop if they're missing data, we build one big template that
has to handle missing data correctly on its own.) I think it's probably
still worth it, though. I added this to the "Alternatives considered"
section of the above document.

By the way, I just wrote a followup blog post on this subject: describing the
challenges I've had with suppressing rules when the data isn't available, and
wondering if it's worth it. The post is here:
http://blog.benjaminlipton.com/2016/09/01/rule-suppression.html. It might be a
bit of a dense read, but I wanted to have the considerations documented at
least. As always, please let me know if there's anything I can clarify. And if
you do happen to read it and it makes you prefer one solution over the others,
I'd love to hear your opinion.

Hello Ben,

my comments are in-line (text copied from the blog post):


Conclusions

The current implementation is working ok, but the “Declaring data dependencies” 
solution is also appealing. Recording in data rules what data they depend on is 
only slightly more involved than wrapping that reference in ipa.datafield(), 
and could also be useful for other purposes.

I agree that syntax with explicit "if"s is little bit more elaborate. On the
other hand, the explicit condition is easier to read (for me) because I can
see what it is doing directly - I do not remember meaning of magical IPA macros.

I.e. I like version with explicit "if"s more.


Plus, it would get rid of the empty sections in openssl configs, as well as

some of the complex macros.

+1


The extra templating and new tags required to get rid of extra commas and 
newlines don’t seem worth it to me, unless we discover a version of openssl or 
certutil that can’t consume the current output.

I definitely agree.



Finally, I think the number of hoops needing to be jumped through to fine-tune 
the output format hint at this “template interpolation” approach being less 
successful than originally expected. While it was expected that inserting data 
rule templates into syntax rule templates and rendering the whole thing would 
produce similar results to rendering data rules first and inserting the output 
into syntax rules, that is not turning out to be the case. It might be wise to 
reconsider the simpler option - it may be easier to implement reliable jinja2 
template markup escaping than to build templates smart enough to handle any 
combination of data that’s available.

This is certainly something to think about. Personally I think that version
with explicit "if"s is easy to understandad, write, and also it has no risk of
data injection (AFAIK).

Explicit escaping is usually very error prone... but I'm not in position to
judge how user-friendly it would be when compared with other solutions. After
all, goal of the feature is to make life of an average admin easier :-)


I hope this brain-dump will help you somehow :-) Have a nice day!
It very much does, thanks for reading! I was leaning towards the 
"explicit ifs" version but didn't want to add more changes unless the 
difference was meaningful. I will finish up the patch to convert to the 
"explicit ifs" version and make it available in the PR for discussion.


I also agree that implementing escaping for a format we don't control 
makes me nervous. One thing we could do is change the format of the data 
that goes into the final formatter step. 
http://blog.benjaminlipton.com/2016/07/19/csr-generation-templating.html#two-pass-data-interpolation 
assumed that would also be jinja2 (hence the {% section %} tag), but it 
wouldn't have to be. We could create/choose a different format with 
well-defined escaping routines. But it would need to be a structured 
format, so that the openssl formatter can figure out what things go into 
sections and where. And if we're implementing a structured format for 
defining CSR contents, maybe we should skip the openssl format entirely 
and go straight to templating the actual CSR structure, as discussed in 
this thread 
https://www.redhat.com/archives/freeipa-devel/2016-August/msg00652.html 
(towards the bottom of the email). These are just some ideas for the 
future; for now I'm going to set this aside because template 
interpolation seems to be working well eno

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 07/27/2016 02:42 PM, Ben Lipton wrote:

On 07/21/2016 11:43 AM, Petr Spacek wrote:

Besides this nit,
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Mapping_Rules#Planned_implementation
sounds reasonable. I like how it prevents bad data from template-injection.


That's what I like about it, too. It does turn out to make things a 
little tricky when it comes to writing rules that won't render if the 
data they depend on is unavailable. (Because instead of rendering 
individual rules which we can drop if they're missing data, we build 
one big template that has to handle missing data correctly on its 
own.) I think it's probably still worth it, though. I added this to 
the "Alternatives considered" section of the above document.


By the way, I just wrote a followup blog post on this subject: 
describing the challenges I've had with suppressing rules when the data 
isn't available, and wondering if it's worth it. The post is here: 
http://blog.benjaminlipton.com/2016/09/01/rule-suppression.html. It 
might be a bit of a dense read, but I wanted to have the considerations 
documented at least. As always, please let me know if there's anything I 
can clarify. And if you do happen to read it and it makes you prefer one 
solution over the others, I'd love to hear your opinion.


Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] Gettext and CSR generation prompts

[Ben Lipton]

Hi,

I'm working on adding a feature where some fields of an 
automatically-generated certificate request can be provided by the user 
rather than coming directly from the database. In my current 
implementation, rules that ask for user-specified data are just data 
rules with the "prompt" option set to a string; this string is displayed 
to the user to request the data. For example: 
https://github.com/LiptonB/freeipa/blob/local-user-data/install/share/csr/rules/dataEmailUserSpecified.json


I was thinking about internationalization for these prompt strings. We 
don't need to worry about translating strings in rules that admins 
define themselves, but if we want to ship any examples of this type of 
rule with IPA, I was thinking that the prompts might need to be translated.


I can wrap the prompt string in _() when the JSON blob is read in, but 
as the strings are defined in JSON files they still won't be 
automatically incorporated into the po and pot files, so this won't 
really do anything. Can you think of a clean way to include these 
strings when translation files are updated?


To see the full implementation, take a look at:
https://github.com/freeipa/freeipa/compare/master...LiptonB:local-user-data

Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 08/25/2016 04:11 PM, Rob Crittenden wrote:

Ben Lipton wrote:

On 08/23/2016 03:54 AM, Jan Cholasta wrote:

On 8.8.2016 22:23, Ben Lipton wrote:

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I hope
you'll
let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate
requests
easier to generate when using alternate certificate profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 






The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be
working on
implementing this design over the next couple of months. If you
have
the time and interest, please take a look and share any
comments or
concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document that
covers
the proposed schema additions in a more descriptive way (with
diagrams!)
I'm very new to developing with LDAP, so some more experienced
eyes on
the proposal would be very helpful, even if you don't have 
time to

absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 






if you have a chance.


I finally had a chance to take a look at this, here are some
comments:

1) I don't like how transformation rules are tied to a particular
helper and have to be duplicated for each of them. They should be
generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated using
python-cryptography's CertificateSigningRequestBuilder [1] rather
than
openssl or certutil or any other command line tool.

There are lots of tools that users might want to use to manage 
their

private keys, so I don't know if we can assume that whatever
library we
prefer will actually be able to access the private key to sign a
CSR,
which is why I thought it would be useful to support more than 
one.


python-cryptography has the notion of backends, which allow it to
support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on PKCS#11
backend [3], which provides support for HSMs and soft-tokens (like
NSS
databases).

Alternatively, for NSS databases (and other "simple" cases), you 
can

generate the private key with python-cryptography using the default
backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for them.

So, the only thing that's currently lacking is HSM support, but 
given

that we don't support HSMs in IPA nor in certmonger, I don't think
it's an issue for now.


The
purpose of the mapping rule is to tie together the transformation
rules
that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those rules
are
automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider `openssl`
and `certutil` the only implementations :-) But I don't think this
solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules shouldn't
really be stored on and processed by the server. The server should
know the *what* (mapping rules), but not the *how* (transformation
rules). The *how* is an implementation detail and does not 
change in

time, so there's no benefit in handling it on the server. It
should be
handled exclusively on the client, which I believe would also make
the
whole thing more robust (it would not be possible for a bug on the
server to break all the clients).

This is a good point. However, for the scope of Ben's project can we
limit it by openssl and certutil support? Otherwise Ben wouldn't be
able
to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)




This is turning out to be a common (and, I think, reasonable)
reaction
to the proposal. It is rather complex, and I worry that it will be
difficult to configure. On the other hand, there is some hidden
complexity to enabling a simpler config format, as well. One of 
the

goals of the project as it was presented to me was to allow the
creation
of profiles that add certificate extensions *that FreeIPA doesn't
yet
know about*. With the current proposal, one only has to add a rule
generating text that the helper will understand.


... which will be possible only as long as the helper 
understands the

extension. Which it might not, thus the current pro

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 08/23/2016 03:54 AM, Jan Cholasta wrote:

On 8.8.2016 22:23, Ben Lipton wrote:

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I hope 
you'll

let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate 
requests

easier to generate when using alternate certificate profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 





The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be 
working on

implementing this design over the next couple of months. If you
have
the time and interest, please take a look and share any 
comments or

concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document that
covers
the proposed schema additions in a more descriptive way (with
diagrams!)
I'm very new to developing with LDAP, so some more experienced
eyes on
the proposal would be very helpful, even if you don't have time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 





if you have a chance.


I finally had a chance to take a look at this, here are some
comments:

1) I don't like how transformation rules are tied to a particular
helper and have to be duplicated for each of them. They should be
generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated using
python-cryptography's CertificateSigningRequestBuilder [1] rather
than
openssl or certutil or any other command line tool.


There are lots of tools that users might want to use to manage their
private keys, so I don't know if we can assume that whatever
library we
prefer will actually be able to access the private key to sign a 
CSR,

which is why I thought it would be useful to support more than one.


python-cryptography has the notion of backends, which allow it to
support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on PKCS#11
backend [3], which provides support for HSMs and soft-tokens (like 
NSS

databases).

Alternatively, for NSS databases (and other "simple" cases), you can
generate the private key with python-cryptography using the default
backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for them.

So, the only thing that's currently lacking is HSM support, but given
that we don't support HSMs in IPA nor in certmonger, I don't think
it's an issue for now.


The
purpose of the mapping rule is to tie together the transformation
rules
that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those rules 
are

automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider `openssl`
and `certutil` the only implementations :-) But I don't think this
solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules shouldn't
really be stored on and processed by the server. The server should
know the *what* (mapping rules), but not the *how* (transformation
rules). The *how* is an implementation detail and does not change in
time, so there's no benefit in handling it on the server. It 
should be
handled exclusively on the client, which I believe would also make 
the

whole thing more robust (it would not be possible for a bug on the
server to break all the clients).

This is a good point. However, for the scope of Ben's project can we
limit it by openssl and certutil support? Otherwise Ben wouldn't be 
able

to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)



This is turning out to be a common (and, I think, reasonable) 
reaction

to the proposal. It is rather complex, and I worry that it will be
difficult to configure. On the other hand, there is some hidden
complexity to enabling a simpler config format, as well. One of the
goals of the project as it was presented to me was to allow the
creation
of profiles that add certificate extensions *that FreeIPA doesn't 
yet

know about*. With the current proposal, one only has to add a rule
generating text that the helper will understand.


... which will be possible only as long as the helper understands the
extension. Which it might not, thus the current proposal works only
for *some* extensions that FreeIPA doesn&#x

Re: [Freeipa-devel] [PATCH 0004-0012] Automatic CSR generation

[Ben Lipton]

On 08/16/2016 03:04 PM, Martin Kosek wrote:

On 08/16/2016 08:12 PM, Alexander Bokovoy wrote:

On Tue, 16 Aug 2016, Ben Lipton wrote:

On 08/10/2016 08:52 AM, Ben Lipton wrote:

The pull request at https://github.com/LiptonB/freeipa/pull/4/commits has
been brought up to date (with a force push), and also includes 3 more
patches, described below.

The patchset is also attached. To make sure that everything applies, I just
regenerated the whole set, though there may not be meaningful changes.


After a discussion about how to address some of the concerns that have been
voiced about this project, there have been some changes to the project
direction. So, I wanted to provide an update about what the plans are. If you
have objections or feel that I'm not representing it correctly, please let me
know.

Since we have yet to see all the ways people will want to use this feature,
the immediate goal is to provide something that we can iterate on. To make
this easier, we will avoid storing rule data on the server or modifying the
server schema, as those changes would need to be supported long term/handled
correctly on update. I plan to approach this as follows:
- Separate the provider of mapping rules into a separate component from the
generation of a config based on those rules
- Build an alternative rule provider that reads local files rather than
querying IPA
- Move the implementation of CSR config formatting from the server API into a
library (where should this go? ipalib? ipapython?), and then provide a
client-side command that builds a config using the library.

Up to you -- ipapython is traditionally used for very basic dependencies
when nothing is configured and is used by both installers and the
framework, ipalib -- for common code in the framework itself.


- Templates for at least two profiles ("user" profile with
CN=, subject and email address SAN, "service" profile
with CN=, subject and DNS SAN) will be provided. Users
will be able to build custom profiles by putting files in the appropriate
directories on their client machines (but we will not guarantee backward
compatibility for the format of these files).
- If we decide to move forward with storing rules on the server, the library
call can be referenced from the server code, using the rule provider that
pulls rules from the API. However, at that point we may also go in the
direction of making automatic cert generation fully the responsibility of
Dogtag, and keep the CSR-generation approach client-side only.

Comments welcome! Unless the changes are more complex than I anticipate, I
hope to have a prototype of this approach for review by the end of this week.

The summary above looks fine.

+1, this looks good to me too. Thanks Ben, good job!

Martin
it took a little longer than I expected, but the client-side 
implementation is now available for review at 
https://github.com/freeipa/freeipa/pull/10. Please take a look when you 
get a chance.


Thanks!
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0004-0012] Automatic CSR generation

[Ben Lipton]

On 08/10/2016 08:52 AM, Ben Lipton wrote:
The pull request at https://github.com/LiptonB/freeipa/pull/4/commits 
has been brought up to date (with a force push), and also includes 3 
more patches, described below.


The patchset is also attached. To make sure that everything applies, I 
just regenerated the whole set, though there may not be meaningful 
changes.


After a discussion about how to address some of the concerns that have 
been voiced about this project, there have been some changes to the 
project direction. So, I wanted to provide an update about what the 
plans are. If you have objections or feel that I'm not representing it 
correctly, please let me know.


Since we have yet to see all the ways people will want to use this 
feature, the immediate goal is to provide something that we can iterate 
on. To make this easier, we will avoid storing rule data on the server 
or modifying the server schema, as those changes would need to be 
supported long term/handled correctly on update. I plan to approach this 
as follows:
- Separate the provider of mapping rules into a separate component from 
the generation of a config based on those rules
- Build an alternative rule provider that reads local files rather than 
querying IPA
- Move the implementation of CSR config formatting from the server API 
into a library (where should this go? ipalib? ipapython?), and then 
provide a client-side command that builds a config using the library.
- Templates for at least two profiles ("user" profile with 
CN=, subject and email address SAN, "service" 
profile with CN=, subject and DNS SAN) will be 
provided. Users will be able to build custom profiles by putting files 
in the appropriate directories on their client machines (but we will not 
guarantee backward compatibility for the format of these files).
- If we decide to move forward with storing rules on the server, the 
library call can be referenced from the server code, using the rule 
provider that pulls rules from the API. However, at that point we may 
also go in the direction of making automatic cert generation fully the 
responsibility of Dogtag, and keep the CSR-generation approach 
client-side only.


Comments welcome! Unless the changes are more complex than I anticipate, 
I hope to have a prototype of this approach for review by the end of 
this week.


Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Karma Requests for Dogtag 10.3.5-1 and ldapjdk

[Ben Lipton]

On 08/10/2016 12:21 PM, Matthew Harmsen wrote:


*The following candidate builds of Dogtag 10.3.5 and ldapjdk on Fedora 
24, 25, and 26 (rawhide) consist of the following:*


  * *Fedora 24:*
  o *dogtag-pki-10.3.5-1.fc24
*
  o *dogtag-pki-theme-10.3.5-1.fc24
*
  o *pki-core-10.3.5-1.fc24
*
  o *pki-console-10.3.5-1.fc24
*
  o *ldapjdk-4.18-19.fc24
*
  * *Fedora 25:*
  o *dogtag-pki-10.3.5-1.fc25
*
  o *dogtag-pki-theme-10.3.5-1.fc25
*
  o *pki-core-10.3.5-1.fc25
*
  o *pki-console-10.3.5-1.fc25
*
  o *ldapjdk-4.18-19.fc25
*
  * *Fedora 26 (rawhide):*
  o *dogtag-pki-10.3.5-1.fc26
*
  o *dogtag-pki-theme-10.3.5-1.fc26
*
  o *pki-core-10.3.5-1.fc26
*
  o *pki-console-10.3.5-1.fc26
*
  o *ldapjdk-4.18-19.fc26
*

*Please provide Karma for the following builds:*

  * *Fedora 24:*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-059eb8aaee
dogtag-pki-10.3.5-1.fc24
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-9f1baf574f
dogtag-pki-theme-10.3.5-1.fc24
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-4d226a5f7e
pki-core-10.3.5-1.fc24
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-dd16599bc7
pki-console-10.3.5-1.fc24
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-1835df9b39
ldapjdk-4.18-19.fc24

*
  * *Fedora 25:*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-85261e13c5
   dogtag-pki-10.3.5-1.fc25*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-3f0224b152
   dogtag-pki-theme-10.3.5-1.fc25*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-0a384ead60
pki-core-10.3.5-1.fc25
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-849cbeecb1
pki-console-10.3.5-1.fc25
*
  o *https://bodhi.fedoraproject.org/updates/FEDORA-2016-3f6bc9b601
ldapjdk-4.18-19.fc25

*



On Fedora 24 I am unable to upgrade to these packages without manual 
intervention:

[root@vm freeipa]# dnf update --allowerasing --best
Last metadata expiration check: 2:08:29 ago on Wed Aug 10 16:38:24 2016.
Error: nothing provides resteasy-atom-provider >= 3.0.17-1 needed by 
pki-base-java-10.3.5-1.fc24.noarch.
package pki-tools-10.3.5-1.fc24.x86_64 requires pki-base-java = 
10.3.5-1.fc24, but none of the providers can be installed.
nothing provides resteasy-atom-provider >= 3.0.17-1 needed by 
pki-base-java-10.3.5-1.fc24.noarch.
nothing provides resteasy-atom-provider >= 3.0.17-1 needed by 
pki-base-java-10.3.5-1.fc24.noarch.
nothing provides resteasy-atom-provider >= 3.0.17-1 needed by 
pki-base-java-10.3.5-1.fc24.noarch.
nothing provides resteasy-atom-provider >= 3.0.17-1 needed by 
pki-base-java-10.3.5-1.fc24.noarch

[root@vm freeipa]# rpm -q resteasy-atom-provider
resteasy-atom-provider-3.0.6-11.fc24.noarch

Am I doing something wrong, or does the new resteasy need to be added 
back to testing? 
(https://bodhi.fedoraproject.org/updates/FEDORA-2016-d80872c309)


Ben
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0013-0015] Automatic CSR generation - usability improvements

[Ben Lipton]

On 08/10/2016 05:07 AM, Alexander Bokovoy wrote:

On Wed, 10 Aug 2016, Petr Spacek wrote:

On 9.8.2016 22:07, Ben Lipton wrote:

Aaand there's a typo in patch 15. Updated version attached.


Ben,

it would be great if you can always send whole patch set, including the
patches which were not changed from the previous iteration.

It is getting quite hard to follow and mix-and-match patches from 
e-mails into

one patch set.

As a nice to have addition, you can push the whole patch set to 
Github (or

somewhere else) so we can just clone the repo and be done with it :-)

I concur here. I'm a bit lost which patches have what state, jumping
from thread to thread and from response to response. ;)


Ok, I've merged everything into the older thread, and updated the github 
branch with all the changes, old and new. I hope that will be easier to 
follow. Sorry about the confusion!


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0013-0015] Automatic CSR generation - usability improvements

[Ben Lipton]

Aaand there's a typo in patch 15. Updated version attached.

On 08/09/2016 02:22 PM, Ben Lipton wrote:

Hello,

The attached patches improve upon my last patchset to:

0013: Add support for generating a full script that makes a CSR, 
rather than just a config, and use that support to automate the full 
flow from script generation through cert issuance
Usage note: the UI for this could probably use work. I currently have 
the --helper-args param that allows additional data to be passed to 
the helper. Commonly this would be something like:
Certutil: --helper-args '-d /path/to/nss/db' (precreated with certutil 
-N -d /path/to/nss/db)
Openssl: --helper-args 'd /path/to/keyfile' (precreated with openssl 
genrsa -out /path/to/keyfile)

See the commit message for a full command line.

0014: Allow the feature to be used by non-admin users

0015: Improve error handling by reporting a nice message if the 
mapping rules are broken, or if the data required to generate the 
subject DN is missing


These improvements may make it easier to test the other patches.

Thanks,
Ben




From c220b49c2c3a76b58b153dc614d3af216c5cf30d Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Mon, 1 Aug 2016 10:20:16 -0400
Subject: [PATCH] Improve error handling for certificate mapping

All calls to jinja2 will now raise an IPA error type if rendering does
not succeed, so that broken rules will not generate InternalErrors.
Additionally, if the rendering does not generate a subject DN (for
example if the wrong certificate profile for a principal is used), an
exception is raised, as the CSR creation will not succeed.

https://fedorahosted.org/freeipa/ticket/4899
---
 ipalib/errors.py |  9 
 ipaserver/plugins/certmapping.py | 45 +---
 2 files changed, 47 insertions(+), 7 deletions(-)

diff --git a/ipalib/errors.py b/ipalib/errors.py
index 7b4f15dd60ee80719195ba1b9b85d075b10bdf4f..ad46fc351ab7a0c0c3fd3dc4ba99de6f6b4cb0fa 100644
--- a/ipalib/errors.py
+++ b/ipalib/errors.py
@@ -1698,6 +1698,15 @@ class CertificateFormatError(CertificateError):
 format = _('Certificate format error: %(error)s')
 
 
+class CertificateMappingError(CertificateError):
+"""
+**4035** Raised when a valid cert request config can not be generated
+"""
+
+errno = 4303
+format = _('%(reason)s')
+
+
 class MutuallyExclusiveError(ExecutionError):
 """
 **4303** Raised when an operation would result in setting two attributes which are mutually exlusive.
diff --git a/ipaserver/plugins/certmapping.py b/ipaserver/plugins/certmapping.py
index 1d87d1f90de6cc8cb330176df49209dd85581170..36cdd1f2a19ceb5dd223d95ce34eaedc8fe778d7 100644
--- a/ipaserver/plugins/certmapping.py
+++ b/ipaserver/plugins/certmapping.py
@@ -431,11 +431,22 @@ class Formatter(object):
 def _format(self, base_template_name, base_template_params, render_data):
 base_template = self.jinja2.get_template(
 base_template_name, globals=self.passthrough_globals)
-combined_template_source = base_template.render(**base_template_params)
+
+try:
+combined_template_source = base_template.render(**base_template_params)
+except jinja2.UndefinedError:
+raise errors.CertificateMappingError(reason=_(
+'Template error when formatting certificate data'))
+
 self.backend.debug(
 'Formatting with template: %s' % combined_template_source)
 combined_template = self.jinja2.from_string(combined_template_source)
-script = combined_template.render(**render_data)
+
+try:
+script = combined_template.render(**render_data)
+except jinja2.UndefinedError:
+raise errors.CertificateMappingError(reason=_(
+'Template error when formatting certificate data'))
 return dict(script=script)
 
 def _wrap_rule(self, rule, rule_type):
@@ -450,8 +461,12 @@ class Formatter(object):
 self.backend.debug('Syntax rule template: %s' % syntax_rule)
 template = self.jinja2.from_string(
 syntax_rule, globals=self.passthrough_globals)
-prepared_template = self._wrap_rule(
-template.render(datarules=data_rules), 'syntax')
+try:
+rendered = template.render(datarules=data_rules)
+except jinja2.UndefinedError:
+raise errors.CertificateMappingError(reason=_(
+'Template error when formatting certificate data'))
+prepared_template = self._wrap_rule(rendered, 'syntax')
 return prepared_template
 
 
@@ -472,6 +487,12 @@ class OpenSSLFormatter(Formatter):
 rendered = self._format(
 'openssl_base.tmpl',
 {'parameters': parameters, 'extensions': extensions

[Freeipa-devel] [PATCH 0013-0015] Automatic CSR generation - usability improvements

[Ben Lipton]

Hello,

The attached patches improve upon my last patchset to:

0013: Add support for generating a full script that makes a CSR, rather 
than just a config, and use that support to automate the full flow from 
script generation through cert issuance
Usage note: the UI for this could probably use work. I currently have 
the --helper-args param that allows additional data to be passed to the 
helper. Commonly this would be something like:
Certutil: --helper-args '-d /path/to/nss/db' (precreated with certutil 
-N -d /path/to/nss/db)
Openssl: --helper-args 'd /path/to/keyfile' (precreated with openssl 
genrsa -out /path/to/keyfile)

See the commit message for a full command line.

0014: Allow the feature to be used by non-admin users

0015: Improve error handling by reporting a nice message if the mapping 
rules are broken, or if the data required to generate the subject DN is 
missing


These improvements may make it easier to test the other patches.

Thanks,
Ben
From 4108cc1e1dd1751993c3ac40f5f5dfbe18e03ca2 Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Fri, 5 Aug 2016 09:29:13 -0400
Subject: [PATCH 13/15] Automate full cert request flow

Adds `cert-build` command that pulls down a config with
`cert-get-requestdata`, then uses it to build a CSR, and submits the CSR
with `cert-request`. To enable this, the format of the returned configs
has been changed to a bash script, which produces the CSR when executed.

Example usage:
$ ipa cert-build --principal blipton --profile-id userCert --helper certutil \
  --helper-args '-d /tmp/certs'

https://fedorahosted.org/freeipa/ticket/4899
---
 API.txt   | 21 -
 install/share/csrtemplates/certutil_base.tmpl | 12 +-
 install/share/csrtemplates/openssl_base.tmpl  | 19 -
 ipaclient/plugins/cert.py | 61 ++-
 ipaclient/plugins/certmapping.py  | 20 ++---
 ipaserver/plugins/cert.py | 29 +
 ipaserver/plugins/certmapping.py  | 14 +++---
 7 files changed, 161 insertions(+), 15 deletions(-)

diff --git a/API.txt b/API.txt
index d724cad0d2898102b358229e5699c6bbc9d94702..8cbbf4cf4fd70f19062b8b491ed2ef452754d981 100644
--- a/API.txt
+++ b/API.txt
@@ -723,6 +723,21 @@ option: Str('version?')
 output: Entry('result')
 output: Output('summary', type=[, ])
 output: PrimaryKey('value')
+command: cert_build/1
+args: 0,10,3
+option: Flag('add', autofill=True, default=False)
+option: Flag('all', autofill=True, cli_name='all', default=False)
+option: Str('cacn?', autofill=True, cli_name='ca', default=u'ipa')
+option: Str('helper?')
+option: Str('helper_args?')
+option: Principal('principal')
+option: Str('profile_id?')
+option: Flag('raw', autofill=True, cli_name='raw', default=False)
+option: Str('request_type', autofill=True, default=u'pkcs10')
+option: Str('version?')
+output: Entry('result')
+output: Output('summary', type=[, ])
+output: PrimaryKey('value')
 command: cert_find/1
 args: 1,29,4
 arg: Str('criteria?')
@@ -760,8 +775,9 @@ output: ListOfEntries('result')
 output: Output('summary', type=[, ])
 output: Output('truncated', type=[])
 command: cert_get_requestdata/1
-args: 0,4,1
-option: Str('format')
+args: 0,5,1
+option: Str('helper')
+option: Str('out?')
 option: Principal('principal')
 option: Str('profile_id')
 option: Str('version?')
@@ -6455,6 +6471,7 @@ default: caacl_remove_service/1
 default: caacl_remove_user/1
 default: caacl_show/1
 default: cert/1
+default: cert_build/1
 default: cert_find/1
 default: cert_get_requestdata/1
 default: cert_remove_hold/1
diff --git a/install/share/csrtemplates/certutil_base.tmpl b/install/share/csrtemplates/certutil_base.tmpl
index d57273c04acede24af054c16e2128a8405bd2460..6c6425fc02867a3bc6d39313a2e07601819a7708 100644
--- a/install/share/csrtemplates/certutil_base.tmpl
+++ b/install/share/csrtemplates/certutil_base.tmpl
@@ -1,4 +1,14 @@
 {% raw -%}
 {% import "ipa_macros.tmpl" as ipa -%}
 {%- endraw %}
-certutil -R -a {{ options|join(' ') }}
+#!/bin/bash -e
+
+if [[ $# -lt 1 ]]; then
+echo "Usage: $0  [  ]"
+echo "Called as: $0 $@"
+exit 1
+fi
+
+CSR="$1"
+shift
+certutil -R -a -z <(head -c 4096 /dev/urandom) -o "$CSR" {{ options|join(' ') }} "$@"
diff --git a/install/share/csrtemplates/openssl_base.tmpl b/install/share/csrtemplates/openssl_base.tmpl
index bdc58bd4afb2f55df800c19d691dd570883038de..597577bcd3366394f9cb0d1efa4b9dfe5790bc8f 100644
--- a/install/share/csrtemplates/openssl_base.tmpl
+++ b/install/share/csrtemplates/openssl_base

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 07/25/2016 07:45 AM, Jan Cholasta wrote:

On 25.7.2016 13:11, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:

On 20.7.2016 16:05, Ben Lipton wrote:

Hi,

Thanks very much for the feedback! Some responses below; I hope you'll
let me know what you think of my reasoning.


On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate requests
easier to generate when using alternate certificate profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 




The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be working on
implementing this design over the next couple of months. If you 
have

the time and interest, please take a look and share any comments or
concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document that
covers
the proposed schema additions in a more descriptive way (with
diagrams!)
I'm very new to developing with LDAP, so some more experienced 
eyes on

the proposal would be very helpful, even if you don't have time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 




if you have a chance.


I finally had a chance to take a look at this, here are some 
comments:


1) I don't like how transformation rules are tied to a particular
helper and have to be duplicated for each of them. They should be
generic and work with any helper, as helpers are just an
implementation detail and their resulting data is the same.

In fact, I think I would prefer if the CSR was generated using
python-cryptography's CertificateSigningRequestBuilder [1] rather 
than

openssl or certutil or any other command line tool.


There are lots of tools that users might want to use to manage their
private keys, so I don't know if we can assume that whatever 
library we

prefer will actually be able to access the private key to sign a CSR,
which is why I thought it would be useful to support more than one.


python-cryptography has the notion of backends, which allow it to
support multiple crypto implementations. Upstream it currently
supports only OpenSSL [2], but some work has been done on PKCS#11
backend [3], which provides support for HSMs and soft-tokens (like NSS
databases).

Alternatively, for NSS databases (and other "simple" cases), you can
generate the private key with python-cryptography using the default
backend, export it to a file and import the file to the target
database, so you don't actually need the PKCS#11 backend for them.

So, the only thing that's currently lacking is HSM support, but given
that we don't support HSMs in IPA nor in certmonger, I don't think
it's an issue for now.


The
purpose of the mapping rule is to tie together the transformation 
rules

that produce the same data into an object that's
implementation-agnostic, so that profiles referencing those rules are
automatically compatible with all the helper options.


They are implementation-agnostic, as long as you consider `openssl`
and `certutil` the only implementations :-) But I don't think this
solution scales well to other possible implementations.

Anyway, my main grudge is that the transformation rules shouldn't
really be stored on and processed by the server. The server should
know the *what* (mapping rules), but not the *how* (transformation
rules). The *how* is an implementation detail and does not change in
time, so there's no benefit in handling it on the server. It should be
handled exclusively on the client, which I believe would also make the
whole thing more robust (it would not be possible for a bug on the
server to break all the clients).

This is a good point. However, for the scope of Ben's project can we
limit it by openssl and certutil support? Otherwise Ben wouldn't be able
to complete the project in time.


I'm fine with that, but I don't think it's up to me :-)




This is turning out to be a common (and, I think, reasonable) reaction
to the proposal. It is rather complex, and I worry that it will be
difficult to configure. On the other hand, there is some hidden
complexity to enabling a simpler config format, as well. One of the
goals of the project as it was presented to me was to allow the 
creation

of profiles that add certificate extensions *that FreeIPA doesn't yet
know about*. With the current proposal, one only has to add a rule
generating text that the helper will understand.


... which will be possible only as long as the helper understands the
extension. Which it might not, thus the current proposal works only
for *some* extensions that FreeIPA doesn't yet support.

We can go ad infinitum here but with any helper implementation, be it
pyt

Re: [Freeipa-devel] [PATCH 0153] Fix ipa-replica-prepare's error message about missing local CA instanc

[Ben Lipton]

On 08/01/2016 11:38 AM, Petr Spacek wrote:

Hello,

Fix ipa-replica-prepare's error message about missing local CA instance

ipa-replica-prepare must be run on a replica with CA or all the certs
needs to be provided (for CA-less case).

The old messages were utterly confusing because they mixed errors about
missing certs and missing local CA instance into one text.

https://fedorahosted.org/freeipa/ticket/6134




The error message in the patch says "must be ran" instead of "must be run".
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0004-0012] Automatic CSR generation

[Ben Lipton]

On 08/01/2016 11:57 PM, Fraser Tweedale wrote:

On Fri, Jul 29, 2016 at 11:13:16AM -0400, Ben Lipton wrote:

On 07/29/2016 09:39 AM, Petr Spacek wrote:

On 27.7.2016 19:06, Ben Lipton wrote:

Hi all,

I think the automatic CSR generation feature
(https://fedorahosted.org/freeipa/ticket/4899,
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation) is
stable enough to review now. The following are summaries of the attached 
patches:
0004: LDAP schema changes for the new feature
0005: Basic API for new objects and CSR generation
0006: Update install automation to create some default mapping rules
0007: Implement the lookups and text processing that generates the CSR config
0008 and 0009: Implement some actual transformation rules so that the feature
is usable
0010: Add a new cert profile for user certs, with mappings
0011: Implement import/export of cert profiles with mappings
0012: Tests for profile import/export

Generally speaking, later patches depend on earlier ones, but I don't
anticipate any problems from committing earlier patches without later ones.

If you prefer, you can also comment on the pull request version:
https://github.com/LiptonB/freeipa/pull/4. Note that I may force push on this
branch.

Allocation of OIDs for schema change also needs review:
https://code.engineering.redhat.com/gerrit/#/c/80061/

Known issues:
- When the requested principal does not have some of the requested data,
produces funny-looking configs with extra commas, empty sections, etc. They
are still accepted by my copies of openssl and certutil, but they look ugly.
- The new objects don't have any ACIs, so for the moment only admin can run
the new commands.
- Does not yet have support for prompting user for field values, so currently
all data must come from the database.
- All processing happens on the server side. As discussed in a previous
thread, it would be desirable to break this out into a library so it could be
used client-side.

Very excited to hear your thoughts!

Hi Ben,

I wanted to give it a try from user's point of view first, before diving into
implementation details. Here are some observations:

Thanks for giving it a try! This is great feedback.

0. Design pages are using term "helper" and it is used even as option in the
example with smartcards. Please fix either design page or the code so they are
consistent.

Good point. In a previous discussion, Alexander remarked that --helper
sounded too low-level, but I find that --use sounds very generic and
--format doesn't make a lot of sense for ipa cert-request, which never
actually gives you the config that's generated. So if they're all going to
be the same word, which is probably a good idea, I might be leaning towards
--helper, but I'm interested to hear opinions on this.

1. The "ipa cert-request" command is missing options --autofill and --use (aka
helper aka format :-) which are mentioned in the design pages.

Yeah, I haven't managed to implement much of the UI niceness suggested by
the design page. I probably should have mentioned that in the email - all
that I expect to be working at this point is 'ipa cert-get-requestdata' and
CRUD for the mapping rules/profiles themselves.

2. "ipa cert_get_requestdata" command passes even without --profile-id and
generates empty config. I think that this is not expected :-)

More expected than you might think :) I'm guessing what's happening is that
you're passing a user principal and it's defaulting to the caIPAserviceCert
profile, then failing to fill out any of the fields because the data it
needs is not there. I agree this isn't great. I was planning to address this
by having it throw an error if it can't generate at least the subject of the
request, and maybe suggest using a different profile.

I chose to have it default to caIPAserviceCert because that's what ipa
cert-request does, but maybe that's not as predictable as I thought.


In general use I think that 'caIPAserviceCert' is unlikely to be
used a majory of the time, and it is a new command so there are no
compatibility issues; therefore why not make the profile option
mandatory?
Fair enough. Ok, a modified patch that changes this (and fixes some 
label errors I noticed) is attached.



3. "ipa cert_get_requestdata --format=openssl" prints the text to stdout
including label "Configuration file contents:". This is hard to use because
simple redirection like "ipa cert_get_requestdata --format=openssl > config"
will not give you valid OpenSSL config file - it needs hand-editing.

It would be good to add --out parameter you envisioned in the design page.
Please ask jcholast for proper name and semantics :-) With --out option the
command can be used to generate valid config (or script if certutil was 
selected).

Agreed. Another example of the UI not being quite right yet. I'

Re: [Freeipa-devel] [PATCH 0004-0012] Automatic CSR generation

[Ben Lipton]

On 07/29/2016 09:39 AM, Petr Spacek wrote:

On 27.7.2016 19:06, Ben Lipton wrote:

Hi all,

I think the automatic CSR generation feature
(https://fedorahosted.org/freeipa/ticket/4899,
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation) is
stable enough to review now. The following are summaries of the attached 
patches:
0004: LDAP schema changes for the new feature
0005: Basic API for new objects and CSR generation
0006: Update install automation to create some default mapping rules
0007: Implement the lookups and text processing that generates the CSR config
0008 and 0009: Implement some actual transformation rules so that the feature
is usable
0010: Add a new cert profile for user certs, with mappings
0011: Implement import/export of cert profiles with mappings
0012: Tests for profile import/export

Generally speaking, later patches depend on earlier ones, but I don't
anticipate any problems from committing earlier patches without later ones.

If you prefer, you can also comment on the pull request version:
https://github.com/LiptonB/freeipa/pull/4. Note that I may force push on this
branch.

Allocation of OIDs for schema change also needs review:
https://code.engineering.redhat.com/gerrit/#/c/80061/

Known issues:
- When the requested principal does not have some of the requested data,
produces funny-looking configs with extra commas, empty sections, etc. They
are still accepted by my copies of openssl and certutil, but they look ugly.
- The new objects don't have any ACIs, so for the moment only admin can run
the new commands.
- Does not yet have support for prompting user for field values, so currently
all data must come from the database.
- All processing happens on the server side. As discussed in a previous
thread, it would be desirable to break this out into a library so it could be
used client-side.

Very excited to hear your thoughts!

Hi Ben,

I wanted to give it a try from user's point of view first, before diving into
implementation details. Here are some observations:


Thanks for giving it a try! This is great feedback.


0. Design pages are using term "helper" and it is used even as option in the
example with smartcards. Please fix either design page or the code so they are
consistent.
Good point. In a previous discussion, Alexander remarked that --helper 
sounded too low-level, but I find that --use sounds very generic and 
--format doesn't make a lot of sense for ipa cert-request, which never 
actually gives you the config that's generated. So if they're all going 
to be the same word, which is probably a good idea, I might be leaning 
towards --helper, but I'm interested to hear opinions on this.


1. The "ipa cert-request" command is missing options --autofill and --use (aka
helper aka format :-) which are mentioned in the design pages.
Yeah, I haven't managed to implement much of the UI niceness suggested 
by the design page. I probably should have mentioned that in the email - 
all that I expect to be working at this point is 'ipa 
cert-get-requestdata' and CRUD for the mapping rules/profiles themselves.


2. "ipa cert_get_requestdata" command passes even without --profile-id and
generates empty config. I think that this is not expected :-)


More expected than you might think :) I'm guessing what's happening is 
that you're passing a user principal and it's defaulting to the 
caIPAserviceCert profile, then failing to fill out any of the fields 
because the data it needs is not there. I agree this isn't great. I was 
planning to address this by having it throw an error if it can't 
generate at least the subject of the request, and maybe suggest using a 
different profile.


I chose to have it default to caIPAserviceCert because that's what ipa 
cert-request does, but maybe that's not as predictable as I thought.


3. "ipa cert_get_requestdata --format=openssl" prints the text to stdout
including label "Configuration file contents:". This is hard to use because
simple redirection like "ipa cert_get_requestdata --format=openssl > config"
will not give you valid OpenSSL config file - it needs hand-editing.

It would be good to add --out parameter you envisioned in the design page.
Please ask jcholast for proper name and semantics :-) With --out option the
command can be used to generate valid config (or script if certutil was 
selected).
Agreed. Another example of the UI not being quite right yet. I've been 
unsure how to handle this, because of certutil taking a command line and 
openssl a config file. It actually gets even more complicated because, 
as you point out in the next item, openssl also needs a command in 
addition to the config file. I'm interested in thinking about how to 
handle this cleanly from a user perspective. Generating a script, or 
providing the command lines as hints, might be w

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 07/21/2016 11:43 AM, Petr Spacek wrote:

On 20.7.2016 19:25, Ben Lipton wrote:

On 07/20/2016 12:21 PM, Simo Sorce wrote:

On Wed, 2016-07-20 at 12:14 -0400, Ben Lipton wrote:

On 07/20/2016 10:37 AM, Simo Sorce wrote:

On Wed, 2016-07-20 at 10:17 -0400, Ben Lipton wrote:

On 07/20/2016 06:27 AM, Simo Sorce wrote:

On Tue, 2016-07-19 at 16:20 -0400, Ben Lipton wrote:

Hi,

I have updated the design page
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_
Gene
rati
on/Mapping_Rules
with my plan for implementing user-configurable rules for
mapping
IPA
data into certificate requests. In brief: we will use Jinja2
for
templating. Data rules (which map individual data items) and
syntax
rules (which group them into certificate fields) will both be
snippets
of Jinja2 markup. The formatting process will be as follows:

I've finally found some time to read the sub-page Mapping_Rules and for me it
is kind of hard to follow. It would not be understandable without this e-mail
and the blog posts (BTW the blog articles are among best I have seen!).

Most importantly, the explanations in brackets above ["Data rules (which map
individual data items) and (which group them into certificate fields)"] are
missing in the wiki page itself :-)

Could you fold relevant parts of the e-mails and blogs back into the wiki page
so it is self-contained?


Very good point. I may have been assuming knowledge of the whole design 
when reading this document, which doesn't make sense. I did some 
cleanup, plus added some more detail about how things work in the 
implementation I just sent out for review. I hope that will clarify 
things somewhat.

Besides this nit,
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Mapping_Rules#Planned_implementation
sounds reasonable. I like how it prevents bad data from template-injection.


That's what I like about it, too. It does turn out to make things a 
little tricky when it comes to writing rules that won't render if the 
data they depend on is unavailable. (Because instead of rendering 
individual rules which we can drop if they're missing data, we build one 
big template that has to handle missing data correctly on its own.) I 
think it's probably still worth it, though. I added this to the 
"Alternatives considered" section of the above document.

Regarding
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A
, I prefer Option A with separate object for each helper. It is somehow
cleaner and it might be useful to use distinct object classes for each helper 
etc.


+1, I think option B was a bit of premature optimization.



API for ipa cert-get-requestdata sounds good.
API for ipa cert-request makes sense to me as well.

In any case I would recommend you to consult API design with Jan Cholasta
  - he is our API custodian.


BTW I very much like "Alternatives considered" sections, we should have this
for each design!

Good work, I really like the dutiful analysis!

   

Thanks for the feedback, and the kind words!

Ben
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0004-0012] Automatic CSR generation

[Ben Lipton]

Hi all,

I think the automatic CSR generation feature 
(https://fedorahosted.org/freeipa/ticket/4899, 
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation) 
is stable enough to review now. The following are summaries of the 
attached patches:

0004: LDAP schema changes for the new feature
0005: Basic API for new objects and CSR generation
0006: Update install automation to create some default mapping rules
0007: Implement the lookups and text processing that generates the CSR 
config
0008 and 0009: Implement some actual transformation rules so that the 
feature is usable

0010: Add a new cert profile for user certs, with mappings
0011: Implement import/export of cert profiles with mappings
0012: Tests for profile import/export

Generally speaking, later patches depend on earlier ones, but I don't 
anticipate any problems from committing earlier patches without later ones.


If you prefer, you can also comment on the pull request version: 
https://github.com/LiptonB/freeipa/pull/4. Note that I may force push on 
this branch.


Allocation of OIDs for schema change also needs review: 
https://code.engineering.redhat.com/gerrit/#/c/80061/


Known issues:
- When the requested principal does not have some of the requested data, 
produces funny-looking configs with extra commas, empty sections, etc. 
They are still accepted by my copies of openssl and certutil, but they 
look ugly.
- The new objects don't have any ACIs, so for the moment only admin can 
run the new commands.
- Does not yet have support for prompting user for field values, so 
currently all data must come from the database.
- All processing happens on the server side. As discussed in a previous 
thread, it would be desirable to break this out into a library so it 
could be used client-side.


Very excited to hear your thoughts!
Ben
From b2b9d3acd4eb7529f7d6ca5d58ddff546481fdf0 Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Tue, 5 Jul 2016 14:19:35 -0400
Subject: [PATCH 1/9] Add schema to support automatic CSR generation

This adds the schema discussed in
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A, along with containers for the new ipaCertMappingRuleset objectClass. There are no containers for ipaCertFieldMappingRule and ipaCertTransformationRule because they will be stored as child objects of ipaCertProfile and ipaCertMappingRuleset objects respectively.

https://fedorahosted.org/freeipa/ticket/4899
---
 install/share/60certificate-profiles.ldif | 7 +++
 install/share/bootstrap-template.ldif | 6 ++
 install/updates/41-cert-mapping.update| 4 
 install/updates/Makefile.am   | 1 +
 ipalib/constants.py   | 2 ++
 5 files changed, 20 insertions(+)
 create mode 100644 install/updates/41-cert-mapping.update

diff --git a/install/share/60certificate-profiles.ldif b/install/share/60certificate-profiles.ldif
index a87fe667d56768419dacf57103e347e88c945e2a..031cec05fc5b19b38b568c10eaf2120d58326396 100644
--- a/install/share/60certificate-profiles.ldif
+++ b/install/share/60certificate-profiles.ldif
@@ -7,6 +7,13 @@ attributeTypes: (2.16.840.1.113730.3.8.21.1.5 NAME 'ipaCertProfileCategory' DESC
 attributeTypes: (2.16.840.1.113730.3.8.21.1.6 NAME 'ipaCaId' DESC 'Dogtag Authority ID' EQUALITY caseIgnoreMatch ORDERING caseIgnoreOrderingMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'IPA v4.4 Lightweight CAs' )
 attributeTypes: (2.16.840.1.113730.3.8.21.1.7 NAME 'ipaCaIssuerDN' DESC 'Issuer DN' SUP distinguishedName X-ORIGIN 'IPA v4.4 Lightweight CAs' )
 attributeTypes: (2.16.840.1.113730.3.8.21.1.8 NAME 'ipaCaSubjectDN' DESC 'Subject DN' SUP distinguishedName X-ORIGIN 'IPA v4.4 Lightweight CAs' )
+attributeTypes: (2.16.840.1.113730.3.8.21.1.9 NAME 'ipaCertSyntaxMapping' DESC 'Reference to ipaCertMappingRuleset: How to format the specification for this field' SUP distinguishedName EQUALITY distinguishedNameMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 SINGLE-VALUE X-ORIGIN 'IPA v4.5' )
+attributeTypes: (2.16.840.1.113730.3.8.21.1.10 NAME 'ipaCertDataMapping' DESC 'Reference to ipaCertMappingRuleset: How to map data into field values' SUP distinguishedName EQUALITY distinguishedNameMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN 'IPA v4.5' )
+attributeTypes: (2.16.840.1.113730.3.8.21.1.11 NAME 'ipaCertTransformationTemplate' DESC 'How to transform a specific data item' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN 'IPA v4.5' )
+attributeTypes: (2.16.840.1.113730.3.8.21.1.12 NAME 'ipaCertTransformationHelper' DESC 'Helper to which this transformation is targeted' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN 'IPA v4.5' 

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 07/25/2016 11:07 AM, Simo Sorce wrote:

On Mon, 2016-07-25 at 11:04 -0400, Simo Sorce wrote:

On Mon, 2016-07-25 at 10:51 -0400, Ben Lipton wrote:

On 07/25/2016 05:07 AM, Simo Sorce wrote:

On Mon, 2016-07-25 at 10:50 +0200, Jan Cholasta wrote:

Anyway, my main grudge is that the transformation rules shouldn't
really
be stored on and processed by the server. The server should know the
*what* (mapping rules), but not the *how* (transformation rules). The
*how* is an implementation detail and does not change in time, so
there's no benefit in handling it on the server. It should be handled
exclusively on the client, which I believe would also make the whole
thing more robust (it would not be possible for a bug on the server
to
break all the clients).

W/o entering in specific +1 as a general comment on this.
If it can be done on the client, probably better be done there.

Simo.


My thinking was that while the CSR generation must be done on the
client, the retrieval and formatting of the data for the CSR should be
done on the server, so that the functionality is available to all
consumers of the API (ipa command-line, certmonger, Web UI, something
else?). I imagine it would be relatively easy to move the formatting
stuff into the ipa CLI, but all the other clients would then need an
implementation of their own, and so we'd need to worry about
interpreting the templates and generating CSRs in multiple different
languages. It's true that as it stands a bug on the server could break
all the clients, but on the other hand there's only one implementation
to maintain, rather than a different one in each client.

But maybe I'm not seeing the proper priorities here. Perhaps it's more
of a problem because clients are easier to update with bugfixes than the
server? Or maybe the preference for the client is for scalability
reasons? Could you tell me more about why you prefer a client
implementation?

(And yeah, everything here carries a disclaimer of "I probably can't
make any large changes in the remaining 3 weeks of my internship," but I
think it's still good to know and document what the limitations of the
current implementation are.)

Thanks,
Ben

You do not want to have to upgrade the server because tool foobarx
became suddenly the most used. Client tools may change over the time as
well, so if you try to generate stuff on the server you may end up
having to support multiple version with little way of knowing which
version that is.

It is true that multiple client would have to implement "something", but
that something could be a python library+binary that other tools/script
can call or pipe through as needed.

Note, from my pov the code should be more or less the same except it
would run on the client rather than the server. Templates would be
delivered via the same package that delivers the tool/module and admins
would have the option to add more locally, though I am not against
sharing templates via the server if we think that is a good idea in
general (but the same issue vs tools changing and rendering templates
broken with one or another version remain).

Simo.

Ok, I definitely see your point here about making it easier to support 
the shifting versions of the helper utilities. Pulling the formatting 
out into a standalone binary that could be used by different clients 
seems achievable. The Web UI wouldn't be able to use it, I guess, but as 
of now there's no web UI for this feature anyway. I'll make sure this is 
at least documented as a desirable modification.


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 07/25/2016 12:03 PM, Simo Sorce wrote:

On Mon, 2016-07-25 at 18:05 +0300, Alexander Bokovoy wrote:

But maybe I'm not seeing the proper priorities here. Perhaps it's

more

of a problem because clients are easier to update with bugfixes than
the server? Or maybe the preference for the client is for

scalability

reasons? Could you tell me more about why you prefer a client
implementation?

Making client responsible for generating the certificate signing
request serves several purposes where privacy is one of main benefits:
access to private key stays at the client side.

I would definitely veto any scheme where the client must send the
private key to the server. I thought the server would generate the CSR,
but then it would be sent to the client for signing ?

Simo.

The server generates the data and formats it for the helper tool. The 
helper runs on the client and generates the CSR, with signature. I don't 
think we were considering signing anything server-side; in this thread I 
was referring to whether the data should be requested and formatted on 
the server or client side.


--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0003] Fix several small typos

[Ben Lipton]

On 07/18/2016 04:54 PM, Lukas Slebodnik wrote:

On (18/07/16 16:38), Petr Spacek wrote:

On 14.7.2016 16:11, Ben Lipton wrote:

On 07/14/2016 04:09 AM, Alexander Bokovoy wrote:

On Wed, 13 Jul 2016, Ben Lipton wrote:

Nothing too exciting, just fixes a few typos I've noticed in comments.

ACK. However, please file a ticket and mention it in the commit message.

Is it worth the bureaucracy? I do not think so. We will certainly not backport
typo fixes in comments to older branches so I would say that ticket is useless.

Just my two cents.


+1

LS

Necessary or not, as the ticket is created now, any objection to pushing 
the patch?


Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 07/25/2016 05:07 AM, Simo Sorce wrote:

On Mon, 2016-07-25 at 10:50 +0200, Jan Cholasta wrote:

Anyway, my main grudge is that the transformation rules shouldn't
really
be stored on and processed by the server. The server should know the
*what* (mapping rules), but not the *how* (transformation rules). The
*how* is an implementation detail and does not change in time, so
there's no benefit in handling it on the server. It should be handled
exclusively on the client, which I believe would also make the whole
thing more robust (it would not be possible for a bug on the server
to
break all the clients).

W/o entering in specific +1 as a general comment on this.
If it can be done on the client, probably better be done there.

Simo.

My thinking was that while the CSR generation must be done on the 
client, the retrieval and formatting of the data for the CSR should be 
done on the server, so that the functionality is available to all 
consumers of the API (ipa command-line, certmonger, Web UI, something 
else?). I imagine it would be relatively easy to move the formatting 
stuff into the ipa CLI, but all the other clients would then need an 
implementation of their own, and so we'd need to worry about 
interpreting the templates and generating CSRs in multiple different 
languages. It's true that as it stands a bug on the server could break 
all the clients, but on the other hand there's only one implementation 
to maintain, rather than a different one in each client.


But maybe I'm not seeing the proper priorities here. Perhaps it's more 
of a problem because clients are easier to update with bugfixes than the 
server? Or maybe the preference for the client is for scalability 
reasons? Could you tell me more about why you prefer a client 
implementation?


(And yeah, everything here carries a disclaimer of "I probably can't 
make any large changes in the remaining 3 weeks of my internship," but I 
think it's still good to know and document what the limitations of the 
current implementation are.)


Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 07/25/2016 08:12 AM, Alexander Bokovoy wrote:

On Mon, 25 Jul 2016, Jan Cholasta wrote:
This is turning out to be a common (and, I think, reasonable) 
reaction

to the proposal. It is rather complex, and I worry that it will be
difficult to configure. On the other hand, there is some hidden
complexity to enabling a simpler config format, as well. One of the
goals of the project as it was presented to me was to allow the 
creation

of profiles that add certificate extensions *that FreeIPA doesn't yet
know about*. With the current proposal, one only has to add a rule
generating text that the helper will understand.


... which will be possible only as long as the helper understands the
extension. Which it might not, thus the current proposal works only
for *some* extensions that FreeIPA doesn't yet support.

We can go ad infinitum here but with any helper implementation, be it
python-cryptography or anything else, you will need to have a support
there as well.


My point was that the current proposal is not any better than my 
proposal in this regard, as neither of them allows one to use an 
arbitrary extension.

That's true. One of arguments for supporting openssl format was to allow
people to generate CSRs themselves later. E.g. it would be a stepping
stone to some automation around certificates to those who need it.
Turning everything into python-cryptography has less benefit in this
context.


The idea with unknown extensions was to allow mapping
their acceptance to a specific relationship between IPA objects
(optionally) and an input from the CSR. A simplest example would be an
identity rule that would copy an ASN.1 encoded content from the CSR to
the certificate.

That's on the mapping side, not on the CSR generation side, but it 
would

go similarly for the CSR if you would be able to enter unknown but
otherwise correct ASN.1 stream. There is no difference at which helper
type we are talking about because all of them support inserting ASN.1
content.
Along similar lines, on the CSR generation side, certutil has an 
--extGeneric flag, and openssl also has an "ARBITRARY EXTENSIONS" 
feature described in x509v3_config(5). I'm guessing that generating 
those configs automatically would be a bit beyond the current feature 
set of jinja2, but when we allow the requester to specify contents for 
some of the data fields it should be possible to build ok automation 
around that.



With your suggestion,
if there's a mapping between "san_directoryname" and the 
corresponding
API calls or configuration lines, we need some way for users to 
augment
that mapping without changing the code. If there's no mapping, and 
it's
just done with text processing, we need enough in the config 
format to

be able to generate fairly complex structures:

builder = builder.subject_name(x509.Name(u'CN=user,O=EXAMPLE.COM'))
builder =
builder.add_extension(x509.SubjectAlternativeName([x509.RFC822Name(u'u...@example.com'), 



x509.DirectoryName(x509.Name(u'CN=user,O=EXAMPLE.COM'))]), False)

and we need to do it without it being equivalent to calling eval() on
the config attributes. I'm not sure how to achieve this (is it 
safe to
call getattr(x509, extensiontype)(value) where extensiontype and 
value

are user-specified?) and it definitely would have to be tied to a
particular library/tool.


As I pointed out above, this needs to be figured out for the generic
case for both the current proposal and my suggestion.

OTOH, I think we could use GSER encoding of the extension value:

  { rfc822Name:"u...@example.com",
directoryName:rdnSequence:"CN=user,O=EXAMPLE.COM" }

GSER is not really used widely and does not have standardized encoding
rules beyond its own definition. If you want to allow transformation
rules in GSER that mention existing content in IPA objects, you would
need to deal with templating anyway. At this point it becomes 
irrelevant

what you are templating, though.


True, but the goal here is not to avoid templating, but rather to 
avoid implementation-specific bits on the server, and GSER is the 
only thing that is textual, implementation-neutral and, as a bonus, 
standardized.

If we move these bits to the client, the only thing server will need to
deal with is CSR. The client then would definitely need to have ability
to provide alternative CSR sources (e.g. openssl text format) to aid
API users that don't utilize IPA tools directly. At this point GSER use
is irrelevant either.
The GSER format is new to me, but I don't see any particular reason the 
existing design would not be able to produce this format. But: is there 
an existing library that allows creating a CSR from a GSER description? 
If our goal is to provide an easy way to create certificates according 
to a particular profile, and the CSR is still our main way to provide 
that data to the IPA cert-request feature, we need to be able to build 
that CSR. If we have a client that accepts GSER and has at least the CSR 
generation features we need, then may

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 07/20/2016 12:21 PM, Simo Sorce wrote:

On Wed, 2016-07-20 at 12:14 -0400, Ben Lipton wrote:

On 07/20/2016 10:37 AM, Simo Sorce wrote:

On Wed, 2016-07-20 at 10:17 -0400, Ben Lipton wrote:

On 07/20/2016 06:27 AM, Simo Sorce wrote:

On Tue, 2016-07-19 at 16:20 -0400, Ben Lipton wrote:

Hi,

I have updated the design page
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_
Gene
rati
on/Mapping_Rules
with my plan for implementing user-configurable rules for
mapping
IPA
data into certificate requests. In brief: we will use Jinja2
for
templating. Data rules (which map individual data items) and
syntax
rules (which group them into certificate fields) will both be
snippets
of Jinja2 markup. The formatting process will be as follows:
1. Syntax rules will be rendered using Jinja2. Data rules
(rule
text,
not rendered) will be passed as the datarules attribute.
2. Rendered syntax rules will be processed by the Formatter
class
for
the selected CSR generation helper (e.g. openssl or
certutil).
The
formatter combines these partial rules into a full template
for
the
config.
3. The template will be rendered using Jinja2. Relevant data
from
the
IPA database will be available in the context for this
rendering.
4. The final rendered template will be returned to the
caller,
labeled
with its function (e.g. a command line or a config file).

Are there any comments or objections to this approach? Here's
an
example
to show what it might look like in practice.

Example data rules:
email={{subject.email}}
O={{config.ipacertificatesubjectbase}}\nCN={{subject.username
}}

Example syntax rule:
subjectAltName=@{% section %}{{datarules|join('\n')}}{%
endsection %}

Example composed config template:
[ req ]
prompt = no
encrypt_key = no

distinguished_name = {% section
%}O={{config.ipacertificatesubjectbase}}
CN={{subject.username}}{% endsection %}

req_extensions = exts

[ exts ]
subjectAltName=@{% section %}email={{subject.email}}{%
endsection
%}

There's a lot more information about the thinking behind this
at
http://blog.benjaminlipton.com/2016/07/19/csr-generation-temp
lati
ng.h
tml
if you're interested, as well.

Nice work Ben,
it's been really nice to be able to follow your notes on the
blog
post,
one question remains lingering in my head, why jinja2 ?
I know that engine relatively well as I used it in ipsilon, so
I am
not
questioning the choice just asking why specifically jinja2 and
not
something else, potentially language agnostic.

Simo.

Honestly, my reasoning didn't go very far beyond that it seems to
be
widely used and is compatible with python, which is the language
where
the implementation is taking place (in the IPA RPC server). I
thought
about using the built-in python format strings or creating a
simple
domain-specific language, but the likelihood of wanting the
built-in
text processing features (join, replace, maybe even for loops)
seemed
high, and I didn't want to reimplement those features.

Will the additional package dependency be a problem?

I am more concerned a out the ability to process the data (which I
guess is stored in LDAP) by another client, or in the CLI.
Other than that the dependency does not concern me too much
provided
jinja2 templating is stable and has some guarantee that it will be
supportable long term.

If that is not guaranteed it is a problem, we cannot easily swap
out
one language for another once data is stored and used by the
server.
So the most important consideration for me is whether we are
locking
ourselves into something that will be hard to deal with later or
not.

Should the jinja2 project fail by the wayside next year would we be
able to easily replace it with another engine without changing the
templates as stored ?

Simo.


Ah, ok, I understand the concern. For now, the plan is that the
server
will do all the text processing, so I don't really forsee a need for
any
other client to read the mapping rules from LDAP. However, it's true
that templates written in jinja2 would probably need at least minor
changes to be compatible with another templating engine. (Same goes
for
any other choice - a lot of these engines seem to have very similar,
but
not exactly compatible, syntax). I don't really know how to judge
the
long-term viability of the jinja2 project, though it seems to be
recognized by lots of projects (ansible[1], openstack[2], flask[3],
even
django[4] which has its own templating engine).

In any case, if the team prefers it, I'd be comfortable going with a
more minimal DSL that only has the features we know we need. It
might
slightly limit the types of certs that can be generated, but that can
be
iterated on. But it would be another thing to design, build and
maintain. Let me know what you think.

I am ok using jinja2 as long as we realize we may be on the hook for
maintaining it ourselves in the long term. It's probably easier to do
that than to write our own anyway.

Simo.
It might also be w

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 07/20/2016 10:37 AM, Simo Sorce wrote:

On Wed, 2016-07-20 at 10:17 -0400, Ben Lipton wrote:

On 07/20/2016 06:27 AM, Simo Sorce wrote:

On Tue, 2016-07-19 at 16:20 -0400, Ben Lipton wrote:

Hi,

I have updated the design page
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Gene
rati
on/Mapping_Rules
with my plan for implementing user-configurable rules for mapping
IPA
data into certificate requests. In brief: we will use Jinja2 for
templating. Data rules (which map individual data items) and
syntax
rules (which group them into certificate fields) will both be
snippets
of Jinja2 markup. The formatting process will be as follows:
1. Syntax rules will be rendered using Jinja2. Data rules (rule
text,
not rendered) will be passed as the datarules attribute.
2. Rendered syntax rules will be processed by the Formatter class
for
the selected CSR generation helper (e.g. openssl or certutil).
The
formatter combines these partial rules into a full template for
the
config.
3. The template will be rendered using Jinja2. Relevant data from
the
IPA database will be available in the context for this rendering.
4. The final rendered template will be returned to the caller,
labeled
with its function (e.g. a command line or a config file).

Are there any comments or objections to this approach? Here's an
example
to show what it might look like in practice.

Example data rules:
email={{subject.email}}
O={{config.ipacertificatesubjectbase}}\nCN={{subject.username}}

Example syntax rule:
subjectAltName=@{% section %}{{datarules|join('\n')}}{%
endsection %}

Example composed config template:
[ req ]
prompt = no
encrypt_key = no

distinguished_name = {% section
%}O={{config.ipacertificatesubjectbase}}
CN={{subject.username}}{% endsection %}

req_extensions = exts

[ exts ]
subjectAltName=@{% section %}email={{subject.email}}{% endsection
%}

There's a lot more information about the thinking behind this at
http://blog.benjaminlipton.com/2016/07/19/csr-generation-templati
ng.h
tml
if you're interested, as well.

Nice work Ben,
it's been really nice to be able to follow your notes on the blog
post,
one question remains lingering in my head, why jinja2 ?
I know that engine relatively well as I used it in ipsilon, so I am
not
questioning the choice just asking why specifically jinja2 and not
something else, potentially language agnostic.

Simo.

Honestly, my reasoning didn't go very far beyond that it seems to be
widely used and is compatible with python, which is the language
where
the implementation is taking place (in the IPA RPC server). I
thought
about using the built-in python format strings or creating a simple
domain-specific language, but the likelihood of wanting the built-in
text processing features (join, replace, maybe even for loops)
seemed
high, and I didn't want to reimplement those features.

Will the additional package dependency be a problem?

I am more concerned a out the ability to process the data (which I
guess is stored in LDAP) by another client, or in the CLI.
Other than that the dependency does not concern me too much provided
jinja2 templating is stable and has some guarantee that it will be
supportable long term.

If that is not guaranteed it is a problem, we cannot easily swap out
one language for another once data is stored and used by the server.
So the most important consideration for me is whether we are locking
ourselves into something that will be hard to deal with later or not.

Should the jinja2 project fail by the wayside next year would we be
able to easily replace it with another engine without changing the
templates as stored ?

Simo.

Ah, ok, I understand the concern. For now, the plan is that the server 
will do all the text processing, so I don't really forsee a need for any 
other client to read the mapping rules from LDAP. However, it's true 
that templates written in jinja2 would probably need at least minor 
changes to be compatible with another templating engine. (Same goes for 
any other choice - a lot of these engines seem to have very similar, but 
not exactly compatible, syntax). I don't really know how to judge the 
long-term viability of the jinja2 project, though it seems to be 
recognized by lots of projects (ansible[1], openstack[2], flask[3], even 
django[4] which has its own templating engine).


In any case, if the team prefers it, I'd be comfortable going with a 
more minimal DSL that only has the features we know we need. It might 
slightly limit the types of certs that can be generated, but that can be 
iterated on. But it would be another thing to design, build and 
maintain. Let me know what you think.


Ben

[1] 
http://docs.ansible.com/ansible/playbooks_variables.html#using-variables-about-jinja2

[2] http://lists.openstack.org/pipermail/openstack-dev/2013-July/012016.html
[3] http://flask.pocoo.org/docs/0.11/templating/
[4] 
https://docs.djangoproject.com/en/1.9/topics/templates/#django.t

Re: [Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

On 07/20/2016 06:27 AM, Simo Sorce wrote:

On Tue, 2016-07-19 at 16:20 -0400, Ben Lipton wrote:

Hi,

I have updated the design page
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generati
on/Mapping_Rules
with my plan for implementing user-configurable rules for mapping
IPA
data into certificate requests. In brief: we will use Jinja2 for
templating. Data rules (which map individual data items) and syntax
rules (which group them into certificate fields) will both be
snippets
of Jinja2 markup. The formatting process will be as follows:
1. Syntax rules will be rendered using Jinja2. Data rules (rule
text,
not rendered) will be passed as the datarules attribute.
2. Rendered syntax rules will be processed by the Formatter class
for
the selected CSR generation helper (e.g. openssl or certutil). The
formatter combines these partial rules into a full template for the
config.
3. The template will be rendered using Jinja2. Relevant data from
the
IPA database will be available in the context for this rendering.
4. The final rendered template will be returned to the caller,
labeled
with its function (e.g. a command line or a config file).

Are there any comments or objections to this approach? Here's an
example
to show what it might look like in practice.

Example data rules:
email={{subject.email}}
O={{config.ipacertificatesubjectbase}}\nCN={{subject.username}}

Example syntax rule:
subjectAltName=@{% section %}{{datarules|join('\n')}}{% endsection %}

Example composed config template:
[ req ]
prompt = no
encrypt_key = no

distinguished_name = {% section
%}O={{config.ipacertificatesubjectbase}}
CN={{subject.username}}{% endsection %}

req_extensions = exts

[ exts ]
subjectAltName=@{% section %}email={{subject.email}}{% endsection %}

There's a lot more information about the thinking behind this at
http://blog.benjaminlipton.com/2016/07/19/csr-generation-templating.h
tml
if you're interested, as well.

Nice work Ben,
it's been really nice to be able to follow your notes on the blog post,
one question remains lingering in my head, why jinja2 ?
I know that engine relatively well as I used it in ipsilon, so I am not
questioning the choice just asking why specifically jinja2 and not
something else, potentially language agnostic.

Simo.
Honestly, my reasoning didn't go very far beyond that it seems to be 
widely used and is compatible with python, which is the language where 
the implementation is taking place (in the IPA RPC server). I thought 
about using the built-in python format strings or creating a simple 
domain-specific language, but the likelihood of wanting the built-in 
text processing features (join, replace, maybe even for loops) seemed 
high, and I didn't want to reimplement those features.


Will the additional package dependency be a problem?

Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

Hi,

Thanks very much for the feedback! Some responses below; I hope you'll 
let me know what you think of my reasoning.



On 07/20/2016 04:20 AM, Jan Cholasta wrote:

Hi,

On 17.6.2016 00:06, Ben Lipton wrote:

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate requests
easier to generate when using alternate certificate profiles:
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 


The use case for this is described in
https://fedorahosted.org/freeipa/ticket/4899. I will be working on
implementing this design over the next couple of months. If you have
the time and interest, please take a look and share any comments or
concerns that you have.

Thanks!

Ben


Just a quick update to say that I've created a new document that covers
the proposed schema additions in a more descriptive way (with diagrams!)
I'm very new to developing with LDAP, so some more experienced eyes on
the proposal would be very helpful, even if you don't have time to
absorb the full design. Please take a look at
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 


if you have a chance.


I finally had a chance to take a look at this, here are some comments:

1) I don't like how transformation rules are tied to a particular 
helper and have to be duplicated for each of them. They should be 
generic and work with any helper, as helpers are just an 
implementation detail and their resulting data is the same.


In fact, I think I would prefer if the CSR was generated using 
python-cryptography's CertificateSigningRequestBuilder [1] rather than 
openssl or certutil or any other command line tool.


There are lots of tools that users might want to use to manage their 
private keys, so I don't know if we can assume that whatever library we 
prefer will actually be able to access the private key to sign a CSR, 
which is why I thought it would be useful to support more than one. The 
purpose of the mapping rule is to tie together the transformation rules 
that produce the same data into an object that's 
implementation-agnostic, so that profiles referencing those rules are 
automatically compatible with all the helper options.


I do agree that it would be preferable to target APIs rather than 
command-line tools. When looking at the openSSL and NSS APIs I came to 
the conclusion that they would be more difficult than the command-line 
tools to target, as a first implementation. However, I haven't really 
looked at python-cryptography, and maybe it would have been a good choice.


2) The schema seems unnecessarily complex. I think all we need is a 
single new multi-value attribute in certprofile. For your S/MIME 
example, it could be something like:


attr: subjectname=CN={subject.cn},{subject_base}
attr: san_rfc822name={subject.email}
attr: san_directoryname={subject.dn}


This is turning out to be a common (and, I think, reasonable) reaction 
to the proposal. It is rather complex, and I worry that it will be 
difficult to configure. On the other hand, there is some hidden 
complexity to enabling a simpler config format, as well. One of the 
goals of the project as it was presented to me was to allow the creation 
of profiles that add certificate extensions *that FreeIPA doesn't yet 
know about*. With the current proposal, one only has to add a rule 
generating text that the helper will understand. With your suggestion, 
if there's a mapping between "san_directoryname" and the corresponding 
API calls or configuration lines, we need some way for users to augment 
that mapping without changing the code. If there's no mapping, and it's 
just done with text processing, we need enough in the config format to 
be able to generate fairly complex structures:


builder = builder.subject_name(x509.Name(u'CN=user,O=EXAMPLE.COM'))
builder = 
builder.add_extension(x509.SubjectAlternativeName([x509.RFC822Name(u'u...@example.com'), 
x509.DirectoryName(x509.Name(u'CN=user,O=EXAMPLE.COM'))]), False)


and we need to do it without it being equivalent to calling eval() on 
the config attributes. I'm not sure how to achieve this (is it safe to 
call getattr(x509, extensiontype)(value) where extensiontype and value 
are user-specified?) and it definitely would have to be tied to a 
particular library/tool.


Ben



Honza

[1] 
<https://cryptography.io/en/latest/x509/reference/#x-509-csr-certificate-signing-request-builder-object>




--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [DESIGN] Text-based rules for CSR autogeneration using Jinja2

[Ben Lipton]

Hi,

I have updated the design page 
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Mapping_Rules 
with my plan for implementing user-configurable rules for mapping IPA 
data into certificate requests. In brief: we will use Jinja2 for 
templating. Data rules (which map individual data items) and syntax 
rules (which group them into certificate fields) will both be snippets 
of Jinja2 markup. The formatting process will be as follows:
1. Syntax rules will be rendered using Jinja2. Data rules (rule text, 
not rendered) will be passed as the datarules attribute.
2. Rendered syntax rules will be processed by the Formatter class for 
the selected CSR generation helper (e.g. openssl or certutil). The 
formatter combines these partial rules into a full template for the config.
3. The template will be rendered using Jinja2. Relevant data from the 
IPA database will be available in the context for this rendering.
4. The final rendered template will be returned to the caller, labeled 
with its function (e.g. a command line or a config file).


Are there any comments or objections to this approach? Here's an example 
to show what it might look like in practice.


Example data rules:
email={{subject.email}}
O={{config.ipacertificatesubjectbase}}\nCN={{subject.username}}

Example syntax rule:
subjectAltName=@{% section %}{{datarules|join('\n')}}{% endsection %}

Example composed config template:
[ req ]
prompt = no
encrypt_key = no

distinguished_name = {% section %}O={{config.ipacertificatesubjectbase}}
CN={{subject.username}}{% endsection %}

req_extensions = exts

[ exts ]
subjectAltName=@{% section %}email={{subject.email}}{% endsection %}

There's a lot more information about the thinking behind this at 
http://blog.benjaminlipton.com/2016/07/19/csr-generation-templating.html 
if you're interested, as well.


Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH 0003] Fix several small typos

[Ben Lipton]

On 07/14/2016 04:09 AM, Alexander Bokovoy wrote:

On Wed, 13 Jul 2016, Ben Lipton wrote:

Nothing too exciting, just fixes a few typos I've noticed in comments.

ACK. However, please file a ticket and mention it in the commit message.



Thanks, updated patch attached.
From a9916a5ce1447417dcbf2c00a9024775d0a30e17 Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Fri, 8 Jul 2016 11:41:43 -0400
Subject: [PATCH] Fix several small typos

Fixes: https://fedorahosted.org/freeipa/ticket/6085
---
 ipatests/test_xmlrpc/tracker/base.py| 2 +-
 ipatests/test_xmlrpc/tracker/user_plugin.py | 2 +-
 ipatests/test_xmlrpc/xmlrpc_test.py | 5 +++--
 ipatests/util.py| 2 +-
 4 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/ipatests/test_xmlrpc/tracker/base.py b/ipatests/test_xmlrpc/tracker/base.py
index 6a0af510f52aa1d7ccd94450c0848149d9abab48..f7fc55d54b6bcd31308d3eaaea9dffb0228e38bf 100644
--- a/ipatests/test_xmlrpc/tracker/base.py
+++ b/ipatests/test_xmlrpc/tracker/base.py
@@ -289,5 +289,5 @@ class Tracker(object):
  set(expected_updates.keys()))
 
 def check_update(self, result, extra_keys=()):
-"""Check the plugin's `find` command result"""
+"""Check the plugin's `mod` command result"""
 raise NotImplementedError(self._override_me_msg)
diff --git a/ipatests/test_xmlrpc/tracker/user_plugin.py b/ipatests/test_xmlrpc/tracker/user_plugin.py
index 1a85e93327e5d517249fd67e208e83a922509002..10093a9530dbdd57b53e412ad6c4d4f2e6174fd5 100644
--- a/ipatests/test_xmlrpc/tracker/user_plugin.py
+++ b/ipatests/test_xmlrpc/tracker/user_plugin.py
@@ -184,7 +184,7 @@ class UserTracker(Tracker):
 Overriding Tracker method for setting self.attrs correctly;
  * most attributes stores its value in list
  * the rest can be overridden by expected_updates
- * allow deleting parametrs if update value is None
+ * allow deleting parameters if update value is None
 """
 if expected_updates is None:
 expected_updates = {}
diff --git a/ipatests/test_xmlrpc/xmlrpc_test.py b/ipatests/test_xmlrpc/xmlrpc_test.py
index ddfe9c17c5ca110bc5eb66dead618383b861eda9..78d96388d36f586a3f95c7dffd7a072da43b1f81 100644
--- a/ipatests/test_xmlrpc/xmlrpc_test.py
+++ b/ipatests/test_xmlrpc/xmlrpc_test.py
@@ -97,8 +97,9 @@ fuzzy_issuer = Fuzzy(type=six.string_types)
 
 fuzzy_hex = Fuzzy('^0x[0-9a-fA-F]+$', type=six.string_types)
 
-# Matches password - password consists of all printable characters without whitespaces
-# The only exception is space, but space cannot be at the beggingin or end of the pwd
+# Matches password - password consists of all printable characters without
+# whitespaces. The only exception is space, but space cannot be at the
+# beginning or end of the pwd.
 fuzzy_password = Fuzzy('^\S([\S ]*\S)*$')
 
 # Matches generalized time value. Time format is: %Y%m%d%H%M%SZ
diff --git a/ipatests/util.py b/ipatests/util.py
index 118c47a12e0d97907cb559d716989a9ca6c5f304..796f87cbbfb25028a5135394a0a97a6b1f7180b5 100644
--- a/ipatests/util.py
+++ b/ipatests/util.py
@@ -205,7 +205,7 @@ class Fuzzy(object):
 >>> fuzzy.test  # doctest:+ELLIPSIS
  at 0x...>
 
-To aid debugging, `Fuzzy.__repr__()` revealse these kwargs as well:
+To aid debugging, `Fuzzy.__repr__()` reveals these kwargs as well:
 
 >>> fuzzy  # doctest:+ELLIPSIS
 Fuzzy('.+', ,  at 0x...>)
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH 0003] Fix several small typos

[Ben Lipton]

Nothing too exciting, just fixes a few typos I've noticed in comments.

Thanks,
Ben
From 26d9ba08e06a145fa9d67a039d23c3fdb272b23e Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Fri, 8 Jul 2016 11:41:43 -0400
Subject: [PATCH] Fix several small typos

---
 ipatests/test_xmlrpc/tracker/base.py| 2 +-
 ipatests/test_xmlrpc/tracker/user_plugin.py | 2 +-
 ipatests/test_xmlrpc/xmlrpc_test.py | 5 +++--
 ipatests/util.py| 2 +-
 4 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/ipatests/test_xmlrpc/tracker/base.py b/ipatests/test_xmlrpc/tracker/base.py
index 6a0af510f52aa1d7ccd94450c0848149d9abab48..f7fc55d54b6bcd31308d3eaaea9dffb0228e38bf 100644
--- a/ipatests/test_xmlrpc/tracker/base.py
+++ b/ipatests/test_xmlrpc/tracker/base.py
@@ -289,5 +289,5 @@ class Tracker(object):
  set(expected_updates.keys()))
 
 def check_update(self, result, extra_keys=()):
-"""Check the plugin's `find` command result"""
+"""Check the plugin's `mod` command result"""
 raise NotImplementedError(self._override_me_msg)
diff --git a/ipatests/test_xmlrpc/tracker/user_plugin.py b/ipatests/test_xmlrpc/tracker/user_plugin.py
index 1a85e93327e5d517249fd67e208e83a922509002..10093a9530dbdd57b53e412ad6c4d4f2e6174fd5 100644
--- a/ipatests/test_xmlrpc/tracker/user_plugin.py
+++ b/ipatests/test_xmlrpc/tracker/user_plugin.py
@@ -184,7 +184,7 @@ class UserTracker(Tracker):
 Overriding Tracker method for setting self.attrs correctly;
  * most attributes stores its value in list
  * the rest can be overridden by expected_updates
- * allow deleting parametrs if update value is None
+ * allow deleting parameters if update value is None
 """
 if expected_updates is None:
 expected_updates = {}
diff --git a/ipatests/test_xmlrpc/xmlrpc_test.py b/ipatests/test_xmlrpc/xmlrpc_test.py
index ddfe9c17c5ca110bc5eb66dead618383b861eda9..78d96388d36f586a3f95c7dffd7a072da43b1f81 100644
--- a/ipatests/test_xmlrpc/xmlrpc_test.py
+++ b/ipatests/test_xmlrpc/xmlrpc_test.py
@@ -97,8 +97,9 @@ fuzzy_issuer = Fuzzy(type=six.string_types)
 
 fuzzy_hex = Fuzzy('^0x[0-9a-fA-F]+$', type=six.string_types)
 
-# Matches password - password consists of all printable characters without whitespaces
-# The only exception is space, but space cannot be at the beggingin or end of the pwd
+# Matches password - password consists of all printable characters without
+# whitespaces. The only exception is space, but space cannot be at the
+# beginning or end of the pwd.
 fuzzy_password = Fuzzy('^\S([\S ]*\S)*$')
 
 # Matches generalized time value. Time format is: %Y%m%d%H%M%SZ
diff --git a/ipatests/util.py b/ipatests/util.py
index 118c47a12e0d97907cb559d716989a9ca6c5f304..796f87cbbfb25028a5135394a0a97a6b1f7180b5 100644
--- a/ipatests/util.py
+++ b/ipatests/util.py
@@ -205,7 +205,7 @@ class Fuzzy(object):
 >>> fuzzy.test  # doctest:+ELLIPSIS
  at 0x...>
 
-To aid debugging, `Fuzzy.__repr__()` revealse these kwargs as well:
+To aid debugging, `Fuzzy.__repr__()` reveals these kwargs as well:
 
 >>> fuzzy  # doctest:+ELLIPSIS
 Fuzzy('.+', ,  at 0x...>)
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0001: Silence sshd messages during install

[Ben Lipton]

On 07/07/2016 11:19 AM, Ben Lipton wrote:


Thanks for the review! Comments below.


On 07/01/2016 07:42 AM, Martin Basti wrote:




On 29.06.2016 20:46, Ben Lipton wrote:
The attached patch silences some annoying messages I've been getting 
when upgrading the freeipa-client package on F24:

"""
WARNING: 'UseLogin yes' is not supported in Fedora and may cause 
several problems.
This will be fixed by openssh-7.2p2-9.fc24 
(https://bugzilla.redhat.com/show_bug.cgi?id=1350347) so we probably 
shouldn't worry about it.

Could not load host key: /etc/ssh/ssh_host_dsa_key
This is because by default sshd looks for all of 
/etc/ssh/ssh_host_dsa_key, /etc/ssh/ssh_host_ecdsa_key, 
/etc/ssh/ssh_host_ed25519_key and /etc/ssh/ssh_host_rsa_key, but 
Fedora doesn't generate a DSA key by default.

"""

Since the script causing the message only looks at the return code 
from sshd to determine the right options to use, I thought it might 
be ok to discard the output. What do you think?


Ben




Hello, I don't like to hiding errors/warnings. Can you determine and 
solve the root cause?


I definitely agree with this in principle, but in this case the 
purpose of this code is to try different, potentially wrong, 
parameters to sshd until it finds a combination that it accepts. It 
seems like in some environments this would produce error messages that 
aren't actionable and don't indicate any problem for package function, 
which is why I didn't think these messages were necessarily worth 
preserving.


On the other hand, if the code makes the wrong decision about sshd 
version we might be interested in error logs that show why. Can we log 
this to a file instead of the console, maybe?


If you'd prefer just addressing the root cause, a patch that prevents 
the missing host key error is attached, but it won't stop the error 
messages showing up when openssh is an older version.


Thanks,
Ben


Whoops, realized that my patch created a tempfile and didn't delete it. 
Updated.


From 035fbe343876afbdae1c38f0eeebf0492c1a2850 Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Thu, 7 Jul 2016 10:28:04 -0400
Subject: [PATCH] Use existing HostKey config to test sshd

Prevents sshd from producing warning messages on package upgrade because
not all of the default host key files (/etc/ssh/ssh_host_dsa_key,
/etc/ssh/ssh_host_ecdsa_key, /etc/ssh/ssh_host_ed25519_key and
/etc/ssh/ssh_host_rsa_key) are present.
---
 freeipa.spec.in | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index ff27a32eebcc640cdbc8895f47732f06a90c4a1b..abb8b8db2c764eb4ef5733383bb2edbb244af955 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -1007,23 +1007,28 @@ if [ -f '/etc/ssh/sshd_config' -a $restore -ge 2 ]; then
 /^(AuthorizedKeysCommand(User|RunAs)|PubKeyAgentRunAs)[ \t]/ d
 ' /etc/ssh/sshd_config >/etc/ssh/sshd_config.ipanew
 
-if /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody'; then
+# Prevent complaints about missing host keys by using the configured ones
+tmp_config=$(mktemp sshd_config.XX)
+sed -n '/^HostKey[ \t]/ p' /etc/ssh/sshd_config > $tmp_config
+
+if /usr/sbin/sshd -t -f $tmp_config -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody'; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandUser nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f $tmp_config -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody'; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandRunAs nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f $tmp_config -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody'; then
 sed -ri '
 s/^AuthorizedKeysCommand (.+)$/PubKeyAgent \1 %u/
 s/^PubKeyAgent .*$/\0\nPubKeyAgentRunAs nobody/
 ' /etc/ssh/sshd_config.ipanew
 fi
 
+rm -f $tmp_config
 mv -Z /etc/ssh/sshd_config.ipanew /etc/ssh/sshd_config
 chmod 600 /etc/ssh/sshd_config
 
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 0001: Silence sshd messages during install

[Ben Lipton]

Thanks for the review! Comments below.


On 07/01/2016 07:42 AM, Martin Basti wrote:




On 29.06.2016 20:46, Ben Lipton wrote:
The attached patch silences some annoying messages I've been getting 
when upgrading the freeipa-client package on F24:

"""
WARNING: 'UseLogin yes' is not supported in Fedora and may cause 
several problems.
This will be fixed by openssh-7.2p2-9.fc24 
(https://bugzilla.redhat.com/show_bug.cgi?id=1350347) so we probably 
shouldn't worry about it.

Could not load host key: /etc/ssh/ssh_host_dsa_key
This is because by default sshd looks for all of 
/etc/ssh/ssh_host_dsa_key, /etc/ssh/ssh_host_ecdsa_key, 
/etc/ssh/ssh_host_ed25519_key and /etc/ssh/ssh_host_rsa_key, but Fedora 
doesn't generate a DSA key by default.

"""

Since the script causing the message only looks at the return code 
from sshd to determine the right options to use, I thought it might 
be ok to discard the output. What do you think?


Ben




Hello, I don't like to hiding errors/warnings. Can you determine and 
solve the root cause?


I definitely agree with this in principle, but in this case the purpose 
of this code is to try different, potentially wrong, parameters to sshd 
until it finds a combination that it accepts. It seems like in some 
environments this would produce error messages that aren't actionable 
and don't indicate any problem for package function, which is why I 
didn't think these messages were necessarily worth preserving.


On the other hand, if the code makes the wrong decision about sshd 
version we might be interested in error logs that show why. Can we log 
this to a file instead of the console, maybe?


If you'd prefer just addressing the root cause, a patch that prevents 
the missing host key error is attached, but it won't stop the error 
messages showing up when openssh is an older version.


Thanks,
Ben
From afb460c2fe3b8329ae5b8ed9603db8723e79c34a Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Thu, 7 Jul 2016 10:28:04 -0400
Subject: [PATCH] Use existing HostKey config to test sshd

Prevents sshd from producing warning messages on package upgrade because
not all of the default host key files (/etc/ssh/ssh_host_dsa_key,
/etc/ssh/ssh_host_ecdsa_key, /etc/ssh/ssh_host_ed25519_key and
/etc/ssh/ssh_host_rsa_key) are present.
---
 freeipa.spec.in | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index ff27a32eebcc640cdbc8895f47732f06a90c4a1b..4a339869257df4b599b774ec7ac728d43ab33ff5 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -1007,17 +1007,21 @@ if [ -f '/etc/ssh/sshd_config' -a $restore -ge 2 ]; then
 /^(AuthorizedKeysCommand(User|RunAs)|PubKeyAgentRunAs)[ \t]/ d
 ' /etc/ssh/sshd_config >/etc/ssh/sshd_config.ipanew
 
-if /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody'; then
+# Prevent complaints about missing host keys by using the configured ones
+tmp_config=$(mktemp sshd_config.XX)
+sed -n '/^HostKey[ \t]/ p' /etc/ssh/sshd_config > $tmp_config
+
+if /usr/sbin/sshd -t -f $tmp_config -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody'; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandUser nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f $tmp_config -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody'; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandRunAs nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f $tmp_config -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody'; then
 sed -ri '
 s/^AuthorizedKeysCommand (.+)$/PubKeyAgent \1 %u/
 s/^PubKeyAgent .*$/\0\nPubKeyAgentRunAs nobody/
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [WIP] Automatic CSR generation - first steps

[Ben Lipton]

I have added a few new features to the code, including:
- A new certificate profile for user certs
- Import and export of included mapping rules when certificate profiles 
are imported/exported
The updated patches are at 
https://github.com/LiptonB/freeipa/pull/2/commits.


I look forward to hearing your thoughts, either in the pull request or 
here on the mailing list.


Thanks,
Ben

On 06/27/2016 01:44 PM, Ben Lipton wrote:


My email client is playing tricks on me - 
https://github.com/LiptonB/freeipa/pull/2 is the correct link.



On 06/27/2016 01:14 PM, Ben Lipton wrote:

Hi,

I have implemented the core functionality of the automatic CSR 
generation design 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation). 
The code (which should be considered a work in progress) is available 
at https://github.com/LiptonB/freeipa/pull/2, please take a look and 
let me know what you think!


First, a demo, then some notes:

[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format openssl

Debug output: [req]
prompt = no
distinguished_name = sec0
req_extensions = exts

[sec0]
CN=hostname.ipadom.example.com
O=IPADOM.EXAMPLE.COM

[sec1]
DNS=hostname.ipadom.example.com

[exts]
subjectAltName=@sec1


[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format certutil
Debug output: certutil -R -s 
CN=hostname.ipadom.example.com,O=IPADOM.EXAMPLE.COM --extSAN 
dns:hostname.ipadom.example.com



Notes:
- This is implemented using the four-level schema 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A). 
I'm very interested in comments on improving the schema or the way I 
interact with it in the code.
- Only includes rules for one profile at the moment, and it's 
probably not one you'd use (it weirdly puts the FQDN in both Subject 
and SubjectAltName). Think of it as an example to show that 
extensions are supported.
- Right now, transformation rules are implemented in python. 
Migrating them to a scheme where rules are text-based and can be 
added at runtime is a future goal.






-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH] 0001: Silence sshd messages during install

[Ben Lipton]

The attached patch silences some annoying messages I've been getting 
when upgrading the freeipa-client package on F24:

"""
WARNING: 'UseLogin yes' is not supported in Fedora and may cause several 
problems.

Could not load host key: /etc/ssh/ssh_host_dsa_key
"""

Since the script causing the message only looks at the return code from 
sshd to determine the right options to use, I thought it might be ok to 
discard the output. What do you think?


Ben
From bb102411cceb557d9869a384af7d7473483f8d9a Mon Sep 17 00:00:00 2001
From: Ben Lipton 
Date: Wed, 29 Jun 2016 14:44:23 -0400
Subject: [PATCH] Silence sshd messages during install

During install we call sshd with no config file, sometimes leading to it
complaining about missing files or bad config options. Since we're just
looking for the return code to see if the options are correct, we can
discard these error messages.
---
 freeipa.spec.in | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index a4d3c067c6c2c0cf911476d950257170fa74e059..73fd47a7469ea27a3ff2883b513bed2be0d420da 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -1007,17 +1007,17 @@ if [ -f '/etc/ssh/sshd_config' -a $restore -ge 2 ]; then
 /^(AuthorizedKeysCommand(User|RunAs)|PubKeyAgentRunAs)[ \t]/ d
 ' /etc/ssh/sshd_config >/etc/ssh/sshd_config.ipanew
 
-if /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody'; then
+if /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandUser=nobody' 2>/dev/null; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandUser nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f /dev/null -o 'AuthorizedKeysCommand=/usr/bin/sss_ssh_authorizedkeys' -o 'AuthorizedKeysCommandRunAs=nobody' 2>/dev/null; then
 sed -ri '
 s/^PubKeyAgent (.+) %u$/AuthorizedKeysCommand \1/
 s/^AuthorizedKeysCommand .*$/\0\nAuthorizedKeysCommandRunAs nobody/
 ' /etc/ssh/sshd_config.ipanew
-elif /usr/sbin/sshd -t -f /dev/null -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody'; then
+elif /usr/sbin/sshd -t -f /dev/null -o 'PubKeyAgent=/usr/bin/sss_ssh_authorizedkeys %u' -o 'PubKeyAgentRunAs=nobody' 2>/dev/null; then
 sed -ri '
 s/^AuthorizedKeysCommand (.+)$/PubKeyAgent \1 %u/
 s/^PubKeyAgent .*$/\0\nPubKeyAgentRunAs nobody/
-- 
2.5.5

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [WIP] Automatic CSR generation - first steps

[Ben Lipton]

My email client is playing tricks on me - 
https://github.com/LiptonB/freeipa/pull/2 is the correct link.



On 06/27/2016 01:14 PM, Ben Lipton wrote:

Hi,

I have implemented the core functionality of the automatic CSR 
generation design 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation). 
The code (which should be considered a work in progress) is available 
at https://github.com/LiptonB/freeipa/pull/2, please take a look and 
let me know what you think!


First, a demo, then some notes:

[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format openssl

Debug output: [req]
prompt = no
distinguished_name = sec0
req_extensions = exts

[sec0]
CN=hostname.ipadom.example.com
O=IPADOM.EXAMPLE.COM

[sec1]
DNS=hostname.ipadom.example.com

[exts]
subjectAltName=@sec1


[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format certutil
Debug output: certutil -R -s 
CN=hostname.ipadom.example.com,O=IPADOM.EXAMPLE.COM --extSAN 
dns:hostname.ipadom.example.com



Notes:
- This is implemented using the four-level schema 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A). 
I'm very interested in comments on improving the schema or the way I 
interact with it in the code.
- Only includes rules for one profile at the moment, and it's probably 
not one you'd use (it weirdly puts the FQDN in both Subject and 
SubjectAltName). Think of it as an example to show that extensions are 
supported.
- Right now, transformation rules are implemented in python. Migrating 
them to a scheme where rules are text-based and can be added at 
runtime is a future goal.


-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [WIP] Automatic CSR generation - first steps

[Ben Lipton]

Hi,

I have implemented the core functionality of the automatic CSR 
generation design 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation). 
The code (which should be considered a work in progress) is available at 
https://github.com/LiptonB/freeipa/pull/2, please take a look and let me 
know what you think!


First, a demo, then some notes:

[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format openssl

Debug output: [req]
prompt = no
distinguished_name = sec0
req_extensions = exts

[sec0]
CN=hostname.ipadom.example.com
O=IPADOM.EXAMPLE.COM

[sec1]
DNS=hostname.ipadom.example.com

[exts]
subjectAltName=@sec1


[root@ipavm ~]# ipa cert-get-requestdata --principal 
host/hostname.ipadom.example.com --format certutil
Debug output: certutil -R -s 
CN=hostname.ipadom.example.com,O=IPADOM.EXAMPLE.COM --extSAN 
dns:hostname.ipadom.example.com



Notes:
- This is implemented using the four-level schema 
(http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema#Option_A). 
I'm very interested in comments on improving the schema or the way I 
interact with it in the code.
- Only includes rules for one profile at the moment, and it's probably 
not one you'd use (it weirdly puts the FQDN in both Subject and 
SubjectAltName). Think of it as an example to show that extensions are 
supported.
- Right now, transformation rules are implemented in python. Migrating 
them to a scheme where rules are text-based and can be added at runtime 
is a future goal.
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

On 06/14/2016 08:27 AM, Ben Lipton wrote:

Hello all,

I have written up a design proposal for making certificate requests 
easier to generate when using alternate certificate profiles: 
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 
The use case for this is described in 
https://fedorahosted.org/freeipa/ticket/4899. I will be working on 
implementing this design over the next couple of months. If you have 
the time and interest, please take a look and share any comments or 
concerns that you have.


Thanks!

Ben

Just a quick update to say that I've created a new document that covers 
the proposed schema additions in a more descriptive way (with diagrams!) 
I'm very new to developing with LDAP, so some more experienced eyes on 
the proposal would be very helpful, even if you don't have time to 
absorb the full design. Please take a look at 
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation/Schema 
if you have a chance.


Thanks,
Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [Design Review Request] V4/Automatic_Certificate_Request_Generation

[Ben Lipton]

Hello all,

I have written up a design proposal for making certificate requests 
easier to generate when using alternate certificate profiles: 
http://www.freeipa.org/page/V4/Automatic_Certificate_Request_Generation. 
The use case for this is described in 
https://fedorahosted.org/freeipa/ticket/4899. I will be working on 
implementing this design over the next couple of months. If you have the 
time and interest, please take a look and share any comments or concerns 
that you have.


Thanks!

Ben

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code