[Freeipa-users] Replica issue / Certificate Authority

[Christopher Young]

I'm hoping to provide enough information to get some help to a very
important issue that I'm currently having.

I have two IPA servers at a single location that recently had a
replication issue that I eventually resolved by reinitializing one of
the masters/replicas with one that seemed to be the most 'good'.

In any case, somewhere in this process, the new IPA 4.4 was release
with/for CentOS 7.3.

At this moment, regular replication seems to be working properly (in
that I don't have any obvious issues and web interfaces on both
systems seem to be consistent for updates EXCEPT when it comes to the
certificates).

Before I get to the errors, here is the output of some of the commands
that I would expect anyone would need:

--
[root@ipa01 ~]# ipa-replica-manage list
ipa01.passur.local: master
ipa02.passur.local: master
-
[root@ipa01 ~]# ipa-replica-manage list -v ipa01.passur.local
ipa02.passur.local: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully:
Incremental update succeeded
  last update ended: 2016-12-16 20:25:40+00:00
-
[root@ipa01 ~]# ipa-replica-manage list -v ipa02.passur.local
ipa01.passur.local: replica
  last init status: None
  last init ended: 1970-01-01 00:00:00+00:00
  last update status: Error (0) Replica acquired successfully:
Incremental update succeeded
  last update ended: 2016-12-16 20:25:40+00:00
-
[root@ipa01 ~]# ipa-replica-manage list-ruv
Replica Update Vectors:
ipa01.passur.local:389: 4
ipa02.passur.local:389: 6
Certificate Server Replica Update Vectors:
ipa02.passur.local:389: 97
ipa01.passur.local:389: 96
--


After the yum updates were applied to each system, I noticed that the
results of 'ipa-server-upgrade' were quite different.  The 'ipa02'
system went through without errors (this was also the system I used to
reinitialize the other when I had a replication issue recently).



On 'ipa01', I have following at the end of the 'ipaupgrade.log' file:
--
2016-12-14T18:09:26Z ERROR IPA server upgrade failed: Inspect
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2016-12-14T18:09:26Z DEBUG   File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171,
in execute
return_value = self.run()
  File 
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py",
line 46, in run
server.upgrade()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 1863, in upgrade
upgrade_configuration()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 1785, in upgrade_configuration
ca_enable_ldap_profile_subsystem(ca)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py",
line 336, in ca_enable_ldap_profile_subsystem
cainstance.migrate_profiles_to_ldap()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
line 1984, in migrate_profiles_to_ldap
_create_dogtag_profile(profile_id, profile_data, overwrite=False)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py",
line 1990, in _create_dogtag_profile
with api.Backend.ra_certprofile as profile_api:
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py",
line 2060, in __enter__
raise errors.RemoteRetrieveError(reason=_('Failed to authenticate
to CA REST API'))

2016-12-14T18:09:26Z DEBUG The ipa-server-upgrade command failed,
exception: RemoteRetrieveError: Failed to authenticate to CA REST API
2016-12-14T18:09:26Z ERROR Unexpected error - see
/var/log/ipaupgrade.log for details:
RemoteRetrieveError: Failed to authenticate to CA REST API
2016-12-14T18:09:26Z ERROR The ipa-server-upgrade command failed. See
/var/log/ipaupgrade.log for more information
--


In addition, when I go to the IPA web interface on the 'ipa01' system,
I get the following when I try to view any of the certificates:
--
IPA Error 4301: CertificateOperationError

Certificate operation cannot be completed: EXCEPTION (Invalid Credential.)
--


I was wondering if there was a method for taking all the CA
details/tree/what have you from my 'ipa02' system and using it to
repopulate the 'ipa01'.   Since everything else seems to be working
correctly after a reinitialize on 'ipa01', I thought this would be the
safest way, but I'm opening any solutions as I need to get this fixed
ASAP.

Please let me know any additional details that may help OR if there is
a procedure that I could use to quickly and easily recreate 'ipa01'
WITH the certificate authority properly working on both.  I may need
some educate there.


Thanks!

-- Chris

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] FreeIPA 4.2.0: An error has occurred (IPA Error 4301: CertificateOperationError)

[Christopher Young]

I have a similar issue (see my recent list post), and I was wondering
if this was ever fixed?  CA appears to work one system
(master/replica) but not the other.

On Mon, Jun 13, 2016 at 4:41 AM, Petr Vobornik  wrote:
> On 06/12/2016 07:05 PM, dan.finkelst...@high5games.com wrote:
>> The restore I was referring to was a red herring; we ended up wiping the 
>> server
>> and saving ipa-backup files, which was the only way we could successfully
>> reconfigure/reinitialize IPA on the host.
>>
>
> As Rob wrote, please check PKI logs. The most important ones here are:
>
> /var/log/pki/pki-tomcat/ca/selftests.log
> /var/log/pki/pki-tomcat/ca/debug
>
> Debug log usually has additional info for possible cause logged in
> selftest log.
>
>
>> *From: *Rob Crittenden 
>> *Date: *Friday, June 10, 2016 at 17:17
>> *To: *Daniel Finkestein ,
>> "freeipa-users@redhat.com" 
>> *Subject: *Re: [Freeipa-users] FreeIPA 4.2.0: An error has occurred (IPA 
>> Error
>> 4301: CertificateOperationError)
>>
>> dan.finkelst...@high5games.com  wrote:
>>
>> And, from the 'ipactl -d --ignore-service-failures restart' we get this:
>>
>> ipa: DEBUG: stderr=
>>
>> ipa: DEBUG: wait_for_open_ports: localhost [8080, 8443] timeout 300
>>
>> ipa: DEBUG: Waiting until the CA is running
>>
>> ipa: DEBUG: Starting external process
>>
>> ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
>>
>> '--no-check-certificate'
>>
>> 'https://ipa.example.com:8443/ca/admin/ca/getStatus'
>>
>> ipa: DEBUG: Process finished, return code=4
>>
>> ipa: DEBUG: stdout=
>>
>> ipa: DEBUG: stderr=--2016-06-10 15:29:38--
>>
>> https://ipa.example.com:8443/ca/admin/ca/getStatus
>>
>> Resolving ipa.example.com (ipa.example.com)... 10.55.10.31
>>
>> Connecting to ipa.example.com (ipa.example.com)|10.55.10.31|:8443...
>>
>> connected.
>>
>> Unable to establish SSL connection.
>>
>> ipa: DEBUG: The CA status is: check interrupted due to error: Command
>>
>> ''/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate'
>>
>> 'https://ipa.example.com:8443/ca/admin/ca/getStatus'' returned non-zero
>>
>> exit status 4
>>
>> ipa: DEBUG: Waiting for CA to start...
>>
>> ipa: DEBUG: Starting external process
>>
>> ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
>>
>> '--no-check-certificate'
>>
>> 'https://ipa.example.com:8443/ca/admin/ca/getStatus'
>>
>> ipa: DEBUG: Process finished, return code=4
>>
>> ipa: DEBUG: stdout=
>>
>> ipa: DEBUG: stderr=--2016-06-10 15:29:43--
>>
>> https://ipa.example.com:8443/ca/admin/ca/getStatus
>>
>> Resolving ipa.example.com (ipa.example.com)... 10.55.10.31
>>
>> Connecting to ipa.example.com (ipa.example.com)|10.55.10.31|:8443...
>>
>> connected.
>>
>> Unable to establish SSL connection.
>>
>> ipa: DEBUG: The CA status is: check interrupted due to error: Command
>>
>> ''/usr/bin/wget' '-S' '-O' '-' '--timeout=30' '--no-check-certificate'
>>
>> 'https://ipa.example.com:8443/ca/admin/ca/getStatus'' returned non-zero
>>
>> exit status 4
>>
>> ipa: DEBUG: Waiting for CA to start...
>>
>> ipa: DEBUG: Starting external process
>>
>> ipa: DEBUG: args='/usr/bin/wget' '-S' '-O' '-' '--timeout=30'
>>
>> '--no-check-certificate'
>>
>> 'https://ipa.example.com:8443/ca/admin/ca/getStatus'
>>
>> Which leads me to believe that tomcat doesn't have the right 
>> certificate(s).
>>
>> I don't think that's the problem. I'd check the pki logs to see if it
>>
>> started and if not, why. Note that it is quite possible for tomcat to
>>
>> start and the CA to fail because tomcat is just a container.
>>
>> In a previous e-mail you said something about a restore, what was that?
>>
>> rob
>>
>> 
>>
>> *Daniel Alex Finkelstein*| Lead Dev Ops Engineer
>>
>> _dan.finkelst...@h5g.com 
>> _| 
>> 212.604.3447
>>
>> One World Trade Center, New York, NY 10007
>>
>> www.high5games.com 
>>
>> Play High 5 Casino  and Shake
>>
>> the Sky 
>>
>> Follow us on: Facebook , Twitter
>>
>> , YouTube
>>
>> , Linkedin
>>
>> 
>>
>> //
>>
>> /This message and any attachments may contain confidential or privileged
>>
>> information and are only for the use of the intended recipient of this
>>
>> message. If you are not the intended recipient, please notify the sender
>>
>> by return email, and delete or destroy this and all copies of this
>>
>> message and all attachments. Any unauthorized disclosur

Re: [Freeipa-users] Replica issue / Certificate Authority

[Christopher Young]

Ok.  I think I have a 'hint' here, but I could use some help getting this fixed.

Comparing the two IPA servers, I found the following (modified SOME of
the output myself):

on 'ipa02' (the 'good' one):
-
ipa cert-show 1
  Issuing CA: ipa
  Certificate: <<>>
  Subject: CN=Certificate Authority,O=.LOCAL
  Issuer: CN=Certificate Authority,O=.LOCAL
  Not Before: Thu Jan 01 06:23:38 2015 UTC
  Not After: Mon Jan 01 06:23:38 2035 UTC
  Fingerprint (MD5): a6:aa:88:d4:66:e2:70:c1:e3:8c:37:0b:f3:eb:19:7d
  Fingerprint (SHA1):
11:c2:5a:58:bc:77:55:37:39:9b:13:b1:1a:a2:02:50:be:2e:a0:7f
  Serial number: 1
  Serial number (hex): 0x1
  Revoked: False
--


on 'ipa01'
-
ipa cert-show 1
ipa: ERROR: Certificate operation cannot be completed: EXCEPTION
(Invalid Credential.)
-

Thinking about this, I decided to just start checking for
inconsistencies and I noticed the following:

ipa02:
---
[root@ipa02 ~]# certutil -L -d /etc/httpd/alias/ -n ipaCert -a |
openssl x509 -text  | head
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 268304413 (0xffe001d)
Signature Algorithm: sha256WithRSAEncryption
Issuer: O=x.LOCAL, CN=Certificate Authority
Validity
Not Before: Nov 23 18:19:31 2016 GMT
Not After : Nov 13 18:19:31 2018 GMT
Subject: O=x.LOCAL, CN=IPA RA

--
ipa01:
--
[root@ipa01 tmp]# certutil -L -d /etc/httpd/alias/ -n ipaCert -a |
openssl x509 -text  | head
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 7 (0x7)
Signature Algorithm: sha256WithRSAEncryption
Issuer: O=.LOCAL, CN=Certificate Authority
Validity
Not Before: Jan  1 06:24:23 2015 GMT
Not After : Dec 21 06:24:23 2016 GMT
Subject: O=.LOCAL, CN=IPA RA

--

So, it looks like somewhere in the process, the certificate got
renewed but not updated on 'ipa01'?  I apologize as I'm trying to
understand this.  I believe that my end goal is probably still the
same (verify replication and get things working properly on the
'ipa01' system.

Any help is very much appreciated!

-- Chris


On Fri, Dec 16, 2016 at 3:35 PM, Christopher Young
 wrote:
> I'm hoping to provide enough information to get some help to a very
> important issue that I'm currently having.
>
> I have two IPA servers at a single location that recently had a
> replication issue that I eventually resolved by reinitializing one of
> the masters/replicas with one that seemed to be the most 'good'.
>
> In any case, somewhere in this process, the new IPA 4.4 was release
> with/for CentOS 7.3.
>
> At this moment, regular replication seems to be working properly (in
> that I don't have any obvious issues and web interfaces on both
> systems seem to be consistent for updates EXCEPT when it comes to the
> certificates).
>
> Before I get to the errors, here is the output of some of the commands
> that I would expect anyone would need:
>
> --
> [root@ipa01 ~]# ipa-replica-manage list
> ipa01.passur.local: master
> ipa02.passur.local: master
> -
> [root@ipa01 ~]# ipa-replica-manage list -v ipa01.passur.local
> ipa02.passur.local: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (0) Replica acquired successfully:
> Incremental update succeeded
>   last update ended: 2016-12-16 20:25:40+00:00
> -
> [root@ipa01 ~]# ipa-replica-manage list -v ipa02.passur.local
> ipa01.passur.local: replica
>   last init status: None
>   last init ended: 1970-01-01 00:00:00+00:00
>   last update status: Error (0) Replica acquired successfully:
> Incremental update succeeded
>   last update ended: 2016-12-16 20:25:40+00:00
> -
> [root@ipa01 ~]# ipa-replica-manage list-ruv
> Replica Update Vectors:
> ipa01.passur.local:389: 4
> ipa02.passur.local:389: 6
> Certificate Server Replica Update Vectors:
> ipa02.passur.local:389: 97
> ipa01.passur.local:389: 96
> --
>
>
> After the yum updates were applied to each system, I noticed that the
> results of 'ipa-server-upgrade' were quite different.  The 'ipa02'
> system went through without errors (this was also the system I used to
> reinitialize the other when I had a replication issue recently).
>
>
>
> On 'ipa01', I have following at the end of the 'ipaupgrade.log' file:
> --
> 2016-12-14T18:09:26Z ERROR IPA server upgrade failed: Inspect
> /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
> 2016-12-14T18:09:26Z DEBUG   File
> "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171,
> in execu

[Freeipa-users] User certificates with FreeIPA and another question.

[Christopher Young]

Some of this might be rudimentary, so I apologize if this is answered
somewhere, though I've tried to search and have not had much luck...

Basically,  I would like to be able to issue user certificates (Subject:
email=sblblabla@blabla.local) in order to use client SSL security on some
things.  I'm very new to FreeIPA, but have worked with external CAs in the
past for similar requests, however this is my first entry into
creating/running a localized CA within an organization.

I was wondering if this is possible via the command line, and if so, how to
go about submitting the request and receiving the certificate.  Any
guidance or assistance would be greatly appreciated!


Additionally, just as a matter of cleanliness, is there any way possible to
just completely wipe out the existence of a certificate/request from
FreeIPA.  I have done some trial-and-error and obviously have made mistakes
that I'd prefer to clean up after.  I've revoked those certs, however the
perfectionist in me hates seeing them there.  I'm quite certain the answer
is 'no', but I thought I would ask anyway.

Thanks for any assistance,

Chris
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

[Freeipa-users] User certificates with FreeIPA and management

[Christopher Young]

Some of this might be rudimentary, so I apologize if this is answered
somewhere, though I've tried to search and have not had much luck...

Basically,  I would like to be able to issue user certificates (Subject:
email=sblblabla@blabla.local) in order to use client SSL security on some
things.  I'm very new to FreeIPA, but have worked with external CAs in the
past for similar requests, however this is my first entry into
creating/running a localized CA within an organization.

I was wondering if this is possible via the command line, and if so, how to
go about submitting the request and receiving the certificate.  Any
guidance or assistance would be greatly appreciated!


Additionally, just as a matter of cleanliness, is there any way possible to
just completely wipe out the existence of a certificate/request from
FreeIPA.  I have done some trial-and-error and obviously have made mistakes
that I'd prefer to clean up after.  I've revoked those certs, however the
perfectionist in me hates seeing them there.  I'm quite certain the answer
is 'no', but I thought I would ask anyway.

Thanks for any assistance,

Chris
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

Re: [Freeipa-users] User certificates with FreeIPA and another question.

[Christopher Young]

Obvious next question:  Any plans to implement that functionality or advice
on how one might get some level of functionality for this?  Would it be
possible to create another command-line based openssl CA that could issue
these but using IPA as the root CA for those?

I'm just trying to provide a solution for situations where we would like to
utilize client/user cert authentication for situations like secure apache
directory access as well as user VPN certificates.  Any advise or ideas are
great appreciated.

Thanks again!

On Thu, Feb 5, 2015 at 4:09 PM, Rob Crittenden  wrote:

> Christopher Young wrote:
> > Some of this might be rudimentary, so I apologize if this is answered
> > somewhere, though I've tried to search and have not had much luck...
> >
> > Basically,  I would like to be able to issue user certificates (Subject:
> > email=sblblabla@blabla.local) in order to use client SSL security on
> > some things.  I'm very new to FreeIPA, but have worked with external CAs
> > in the past for similar requests, however this is my first entry into
> > creating/running a localized CA within an organization.
>
> IPA doesn't issue user certificates yet, only server certificates.
>
> > I was wondering if this is possible via the command line, and if so, how
> > to go about submitting the request and receiving the certificate.  Any
> > guidance or assistance would be greatly appreciated!
> >
> >
> > Additionally, just as a matter of cleanliness, is there any way possible
> > to just completely wipe out the existence of a certificate/request from
> > FreeIPA.  I have done some trial-and-error and obviously have made
> > mistakes that I'd prefer to clean up after.  I've revoked those certs,
> > however the perfectionist in me hates seeing them there.  I'm quite
> > certain the answer is 'no', but I thought I would ask anyway.
>
> Right, the answer is no. In fact it is a good thing that all
> certificates are accounted for.
>
> rob
>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

Re: [Freeipa-users] User certificates with FreeIPA and another question.

[Christopher Young]

Would anyone happen to have any guides on how one could get through this
process?  I'm a one-man IT shop at the moment, so I'm building up a
tremendous amount of infrastructure at once.  I'm thinking that the option
of creating a subCA with something simple like openssl would be the best
option, but figuring out that process in a minimal amount of time is going
to be tough.

I'm going to try and give myself some reading assignments and push that
forward, but if anyone happens to have a good handle on that
process/commands/etc. and would be interesting in double a couple of hours
of consulting to me, I would be very interested in listening provided we
could come up with a reasonable rate/timeframe.  If anyone is interested,
please contact me directly off-list.

Thanks again.  These answers/ideas have been most helpful.

On Fri, Feb 6, 2015 at 9:30 AM, Martin Kosek  wrote:

> On 02/06/2015 12:53 AM, Christopher Young wrote:
> > Obvious next question:  Any plans to implement that functionality or
> advice
> > on how one might get some level of functionality for this?  Would it be
> > possible to create another command-line based openssl CA that could issue
> > these but using IPA as the root CA for those?
>
> As for FreeIPA plans, we plan to vastly improve our flexibility to process
> certificates in next upstream version - FreeIPA 4.2. In next version, one
> should be able to create other certificate profiles (from FreeIPA default
> service cert profile) or even subCAs to do what you want.
>
> As for current workarounds, you would have to issue and sign a for example
> NSS
> or openssl based subCA and then sign user certs there. But I would leave
> Fraser
> or Jan to tell if this would be really possible.
>
> > I'm just trying to provide a solution for situations where we would like
> to
> > utilize client/user cert authentication for situations like secure apache
> > directory access as well as user VPN certificates.  Any advise or ideas
> are
> > great appreciated.
> >
> > Thanks again!
> >
> > On Thu, Feb 5, 2015 at 4:09 PM, Rob Crittenden 
> wrote:
> >
> >> Christopher Young wrote:
> >>> Some of this might be rudimentary, so I apologize if this is answered
> >>> somewhere, though I've tried to search and have not had much luck...
> >>>
> >>> Basically,  I would like to be able to issue user certificates
> (Subject:
> >>> email=sblblabla@blabla.local) in order to use client SSL security on
> >>> some things.  I'm very new to FreeIPA, but have worked with external
> CAs
> >>> in the past for similar requests, however this is my first entry into
> >>> creating/running a localized CA within an organization.
> >>
> >> IPA doesn't issue user certificates yet, only server certificates.
> >>
> >>> I was wondering if this is possible via the command line, and if so,
> how
> >>> to go about submitting the request and receiving the certificate.  Any
> >>> guidance or assistance would be greatly appreciated!
> >>>
> >>>
> >>> Additionally, just as a matter of cleanliness, is there any way
> possible
> >>> to just completely wipe out the existence of a certificate/request from
> >>> FreeIPA.  I have done some trial-and-error and obviously have made
> >>> mistakes that I'd prefer to clean up after.  I've revoked those certs,
> >>> however the perfectionist in me hates seeing them there.  I'm quite
> >>> certain the answer is 'no', but I thought I would ask anyway.
> >>
> >> Right, the answer is no. In fact it is a good thing that all
> >> certificates are accounted for.
> >>
> >> rob
> >>
> >>
> >
> >
> >
>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

Re: [Freeipa-users] User certificates with FreeIPA and another question.

[Christopher Young]

I actually think I can get this going at this time if I can just figure out
how to submit a subca csr to dogtag, sign it, and acquire it.
Documentation on that seems to be hard to come by, but I'm digging to avoid
eating up this thread (and trying to RTFM where possible).  I still stand
by my request for consulting time if anyone has more intimate knowledge,
however if someone can point me in the best direction for getting a openssl
based subca's csr submitted, signed, acquired, I think I can get the rest
going.  Your help would be greatly, greatly appreciated.

Chris

On Mon, Feb 9, 2015 at 12:18 PM, Christopher Young 
wrote:

> Would anyone happen to have any guides on how one could get through this
> process?  I'm a one-man IT shop at the moment, so I'm building up a
> tremendous amount of infrastructure at once.  I'm thinking that the option
> of creating a subCA with something simple like openssl would be the best
> option, but figuring out that process in a minimal amount of time is going
> to be tough.
>
> I'm going to try and give myself some reading assignments and push that
> forward, but if anyone happens to have a good handle on that
> process/commands/etc. and would be interesting in double a couple of hours
> of consulting to me, I would be very interested in listening provided we
> could come up with a reasonable rate/timeframe.  If anyone is interested,
> please contact me directly off-list.
>
> Thanks again.  These answers/ideas have been most helpful.
>
> On Fri, Feb 6, 2015 at 9:30 AM, Martin Kosek  wrote:
>
>> On 02/06/2015 12:53 AM, Christopher Young wrote:
>> > Obvious next question:  Any plans to implement that functionality or
>> advice
>> > on how one might get some level of functionality for this?  Would it be
>> > possible to create another command-line based openssl CA that could
>> issue
>> > these but using IPA as the root CA for those?
>>
>> As for FreeIPA plans, we plan to vastly improve our flexibility to process
>> certificates in next upstream version - FreeIPA 4.2. In next version, one
>> should be able to create other certificate profiles (from FreeIPA default
>> service cert profile) or even subCAs to do what you want.
>>
>> As for current workarounds, you would have to issue and sign a for
>> example NSS
>> or openssl based subCA and then sign user certs there. But I would leave
>> Fraser
>> or Jan to tell if this would be really possible.
>>
>> > I'm just trying to provide a solution for situations where we would
>> like to
>> > utilize client/user cert authentication for situations like secure
>> apache
>> > directory access as well as user VPN certificates.  Any advise or ideas
>> are
>> > great appreciated.
>> >
>> > Thanks again!
>> >
>> > On Thu, Feb 5, 2015 at 4:09 PM, Rob Crittenden 
>> wrote:
>> >
>> >> Christopher Young wrote:
>> >>> Some of this might be rudimentary, so I apologize if this is answered
>> >>> somewhere, though I've tried to search and have not had much luck...
>> >>>
>> >>> Basically,  I would like to be able to issue user certificates
>> (Subject:
>> >>> email=sblblabla@blabla.local) in order to use client SSL security on
>> >>> some things.  I'm very new to FreeIPA, but have worked with external
>> CAs
>> >>> in the past for similar requests, however this is my first entry into
>> >>> creating/running a localized CA within an organization.
>> >>
>> >> IPA doesn't issue user certificates yet, only server certificates.
>> >>
>> >>> I was wondering if this is possible via the command line, and if so,
>> how
>> >>> to go about submitting the request and receiving the certificate.  Any
>> >>> guidance or assistance would be greatly appreciated!
>> >>>
>> >>>
>> >>> Additionally, just as a matter of cleanliness, is there any way
>> possible
>> >>> to just completely wipe out the existence of a certificate/request
>> from
>> >>> FreeIPA.  I have done some trial-and-error and obviously have made
>> >>> mistakes that I'd prefer to clean up after.  I've revoked those certs,
>> >>> however the perfectionist in me hates seeing them there.  I'm quite
>> >>> certain the answer is 'no', but I thought I would ask anyway.
>> >>
>> >> Right, the answer is no. In fact it is a good thing that all
>> >> certificates are accounted for.
>> >>
>> >> rob
>> >>
>> >>
>> >
>> >
>> >
>>
>>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go To http://freeipa.org for more info on the project

[Freeipa-users] Slave DNS on FreeIPA replica

[Christopher Young]

I have - what I believe to be - a couple of basic questions (I apologize in
advance if these are answered elsewhere, though I've tried to do some
searching ahead of time.):

I recently added an IPA replica to an existing IPA server and noticed that
everything appeared to succeed in the setup.  One observation is that DNS
(bind) was not set up on this new host.  I was wondering if this is normal
behavior, and if so, is there a set of instructions needed to add/create
additional DNS servers for use with FreeIPA?

Ideally, I would like to have DNS running on all IPA hosts.  Additionally,
I plan on adding a pair of caching/slave DNS servers running standing BIND
on remote networks and was wondering what the procedure would be to slave
those zones onto those.  Would that be the same as allowing the transfer
from those IPs and treating them just like any other BIND slave for the
appropriate zones?

I appreciate the clarifications and all the effort that goes into this!

Chris
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Slave DNS on FreeIPA replica

[Christopher Young]

I clearly missed that.  Thanks for the clarification.  As far as adding
additional DNS servers merely to slave the zones, is that more or less the
same as configuring any other bind slave?

On Mon, Apr 6, 2015 at 3:15 PM, Rob Crittenden  wrote:

> Christopher Young wrote:
> > I have - what I believe to be - a couple of basic questions (I apologize
> > in advance if these are answered elsewhere, though I've tried to do some
> > searching ahead of time.):
> >
> > I recently added an IPA replica to an existing IPA server and noticed
> > that everything appeared to succeed in the setup.  One observation is
> > that DNS (bind) was not set up on this new host.  I was wondering if
> > this is normal behavior, and if so, is there a set of instructions
> > needed to add/create additional DNS servers for use with FreeIPA?
> >
> > Ideally, I would like to have DNS running on all IPA hosts.
> > Additionally, I plan on adding a pair of caching/slave DNS servers
> > running standing BIND on remote networks and was wondering what the
> > procedure would be to slave those zones onto those.  Would that be the
> > same as allowing the transfer from those IPs and treating them just like
> > any other BIND slave for the appropriate zones?
> >
> > I appreciate the clarifications and all the effort that goes into this!
>
> DNS and a CA are optional components in a replica. You can add them
> using ipa-dns-install and ipa-ca-install respectively.
>
> To install bind during the replica install process add the option
> --setup-dns.
>
> rob
>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Status on Sub-CAs for FreeIPA v4.2

[Christopher Young]

I, too, am very much in need of user certificates.  If it is possible to
setup an additional FreeIPA server to test this out, then I could help out
in testing the feature.  I obviously don't want to impact my production
environment too much, but it is rather stagnant, so if I can backup the
LDAP db every once in a while, that could work.   Otherwise, I could
possible find some time to set up another instance for testing.  I
definitely need this feature!  Thank you so much for working on it.

Chris

On Mon, Jun 1, 2015 at 6:34 PM, Fraser Tweedale  wrote:

> On Mon, Jun 01, 2015 at 05:19:20PM +0300, Alexander Bokovoy wrote:
> > On Mon, 01 Jun 2015, Thibaut Pouzet wrote:
> > >Hi,
> > >
> > >I am currently trying to use FreeIPA to issue client certificates for
> > >some internal application we have. (More precisely, SSL double
> > >authentication between two of my applications, client side would be
> > >java, server-side would be apache httpd.) I considered two options :
> > >
> > >1. Issue client certificates directly from FreeIPA : It do not seems
> > >that it's currently "supported". I can actually generate a client
> > >certificate by creating a new principal for a host, and use ipa-getcert
> > >to generate a certificate for it. However, this certificate is valid for
> > >both user and server authentication, and I cannot change it.
> > >Furthermore, I cannot change the CN of the certificate, it is the
> > >server's hostname for which the pincipal has been generated. That's a
> > >poor solution.
> > >
> > >
> > >2. Issue a Sub-CA signed by the IPA CA, that I would use with openssl to
> > >do whatever I want to do. I tried to use the dogtag profiles with the
> > >ipa-getcert -T option, but the profiles were ignored when I tried to use
> > >them. And I always got 'regular' certificates.
> > >
> > >I did some research, and found this RFE :
> > >http://www.freeipa.org/page/V4/Sub-CAs
> > >
> > >And this Sub-CA notions seems to be perfect for what I want to do. When
> > >I'm looking at the ticket, it seems that it is quietly sleeping
> > >somewhere, remaining not updated.
> > >
> > >I would love to see this feature in FreeIPA v4.2, has anyone a status on
> > >this RFE and it's current status ?
> > >
> Hi Thibaut,
>
> I'm working on user certificates, profiles and sub-CAs.  User
> certificates and custom profiles are a near-certainty to make 4.2.
> Sub-CAs will not make it into the alpha; hopefully I can finish the
> feature and squeeze it into 4.2 but it's a possibility that sub-CAs
> will arrive in a follow-up release.
>
> Would you be willing to help test all these features and provide
> feedback?  I will soon be preparing a COPR with test builds so if
> you would like to help in this way, I can help you get set up to do
> this.  I (we) would really appreciate your feedback.
>
> Cheers,
> Fraser
>
>
> > Design page is there, the work happens on freeipa-devel@. There are
> > multiple patches in the review process right now. If you are willing to
> > help with testing them, welcome to the development list.
> >
> > --
> > / Alexander Bokovoy
> >
> > --
> > Manage your subscription for the Freeipa-users mailing list:
> > https://www.redhat.com/mailman/listinfo/freeipa-users
> > Go to http://freeipa.org for more info on the project
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Specific rights needed to enroll a new host

[Christopher Young]

I'm trying to develop a process in Ansible to enroll new hosts (as well as
check beforehand to see if the host is already enrolled).  I was wondering
a couple of things:

#1. Has anyone else worked out a process for doing this using a non 'admin'
account?

#2. Is there a simple mechanism (preferably something that could be
automated and thus not require any interactivity), that could be used to
check as to whether a system is enrolled?  I would hope that some type of
simple LDAP search or simple command that could be run to check with easy
return codes.

In particular, I'm trying to avoid using the 'admin' user to enroll hosts
because I'd like to minimize the rights to just the enrollment of new hosts
as well as checking for an existing enrollment.

Any thoughts of feedback that could point me in the best direction would be
greatly appreciated!

Thanks,

Chris
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] 4.2 Packages for RHEL/CentOS 7.1

[Christopher Young]

Do we know what the status of getting these packages prepped and into the
mainstream repos (like EPEL, I suppose)?

I'm just curious as I try and keep my repos minimal on servers (for obvious
reasons), but I would really like to begin testing/using the functionality
in 4.2.

Thanks as always!


Chris
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] 4.2 Packages for RHEL/CentOS 7.1

[Christopher Young]

I recall that original message about the packaging before RHEL 7.2 and
how few of us expressed interest.  I believe I did respond to the
positive that I could use these packages, but I certainly understand
additional effort.  I just hate to be waiting on RH's cycle to get
updates to one of the pieces of my infrastructure where features are
in-demand and getting added more often.  I prefer my base server OS's
to stay as stable as possible, but FreeIPA is an exception for me.  In
any case, I appreciate the effort and the response.

Just so that I'm clear, this basically means that we should wait until
the RHEL 7.2 release (and the following CentOS 7.2 release) before
this will generally available?  I want to make sure I pay attention to
that as it gets released.

Thanks,

Chris

On Thu, Nov 12, 2015 at 3:45 AM, Alexander Bokovoy  wrote:
> On Wed, 11 Nov 2015, Christopher Young wrote:
>>
>> Do we know what the status of getting these packages prepped and into the
>> mainstream repos (like EPEL, I suppose)?
>>
>> I'm just curious as I try and keep my repos minimal on servers (for
>> obvious
>> reasons), but I would really like to begin testing/using the functionality
>> in 4.2.
>
> I believe EPEL's policy prevents you from packaging software which
> exists in RHEL proper. FreeIPA 4.2 is coming with RHEL 7.2, it is
> already published as part of RHEL 7.2 beta in September.
>
> I want to remind  that during this summer I ran few queries here
> (freeipa-users@) and elsewhere to solicit opinions whether people want
> to have FreeIPA 4.2 packages available for CentOS before RHEL 7.2
> release. Very few responses came back and there wasn't any convincing
> feedback that would have justified additional effort to make the
> repository and maintenance reasonable.
>
> https://www.redhat.com/archives/freeipa-users/2015-July/msg00243.html
>
> --
> / Alexander Bokovoy

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Obtaining certificate private keys for Apache/etc.

[Christopher Young]

I've been doing some reading and perhaps I'm confusing myself, but I
couldn't find any definitive guide on how to go about doing what I
think it a pretty simple thing.

My ipa-client installs appear to generate a new TLS/SSL/PKI cert for
each host when they are registered.  I'd like to utilize that
certificate with Apache/tomcat/etc..  I'm aware of how to obtain the
certificate itself, however I'm not clear on how to obtain the private
key (in a format that I can use as well) that was used to generate the
certificate.

Would someone kindly point me in the right direction or ideally just
educate me on the command/options needed to do this.  In particular,
I'm looking to create pem files for both the key and cert for use with
Apache, but it would be useful to understand how to do it for other
stores as well.  (Hint: this would be great to just have in a document
that makes it clear). :)

Thanks again to the freeipa team.  I love this product.

-- Chris

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Obtaining certificate private keys for Apache/etc.

[Christopher Young]

That looks more like a regular SSL/TLS guide.  I was asking about
interacting with FreeIPA (likely via certmonger, I think)
specifically.  If anyone has details on obtaining the default system
cert (and especially the private key) and exporting/converting to
PEMs, I'd greatly appreciate.

I will try and look over your guide regardless soon and provide some
feedback.  At the moment, I'm confused about where the private keys
used to generate the default host PKI certs are stored and how to
extract them.

On Tue, Feb 2, 2016 at 6:50 PM, Filipozzi, Luca  wrote:
> I wrote the following guide for sysadmins at my university in an attempt to
> coalesce in one place what I consider to be the good practices for X.509
> certificate management using OpenSSL.  I've included examples on how to load
> private keys, end-entity certificates and intermediate certificates into
> alternate trust stores (PKCS for IIS and JKS for Java).
>
> https://confluence.id.ubc.ca:8443/display/ITSecurity/how+to+obtain%2C+deploy+and+verify+an+X.509+certificate
>
> Let me know if you have suggestions for improvement.
>
> --
> Luca Filipozzi, UBC IT Enterprise Architecture
>
> On Feb 2, 2016, at 15:43, Christopher Young  wrote:
>
> I've been doing some reading and perhaps I'm confusing myself, but I
> couldn't find any definitive guide on how to go about doing what I
> think it a pretty simple thing.
>
> My ipa-client installs appear to generate a new TLS/SSL/PKI cert for
> each host when they are registered.  I'd like to utilize that
> certificate with Apache/tomcat/etc..  I'm aware of how to obtain the
> certificate itself, however I'm not clear on how to obtain the private
> key (in a format that I can use as well) that was used to generate the
> certificate.
>
> Would someone kindly point me in the right direction or ideally just
> educate me on the command/options needed to do this.  In particular,
> I'm looking to create pem files for both the key and cert for use with
> Apache, but it would be useful to understand how to do it for other
> stores as well.  (Hint: this would be great to just have in a document
> that makes it clear). :)
>
> Thanks again to the freeipa team.  I love this product.
>
> -- Chris
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Obtaining certificate private keys for Apache/etc.

[Christopher Young]

Thanks.  That's good advice and good to know.  I'm going to be trying
to work this into an Ansible role, so having a command listing helps
alot.

That leads to a curious question if anyone has thought about building
an Ansible module(s) for manipulating FreeIPA objects.  I'm going to
do some searching for that.

On Wed, Feb 3, 2016 at 3:12 AM, Martin Kosek  wrote:
> On 02/03/2016 12:42 AM, Christopher Young wrote:
>> I've been doing some reading and perhaps I'm confusing myself, but I
>> couldn't find any definitive guide on how to go about doing what I
>> think it a pretty simple thing.
>>
>> My ipa-client installs appear to generate a new TLS/SSL/PKI cert for
>> each host when they are registered.  I'd like to utilize that
>> certificate with Apache/tomcat/etc..  I'm aware of how to obtain the
>> certificate itself, however I'm not clear on how to obtain the private
>> key (in a format that I can use as well) that was used to generate the
>> certificate.
>>
>> Would someone kindly point me in the right direction or ideally just
>> educate me on the command/options needed to do this.  In particular,
>> I'm looking to create pem files for both the key and cert for use with
>> Apache, but it would be useful to understand how to do it for other
>> stores as well.  (Hint: this would be great to just have in a document
>> that makes it clear). :)
>
> Hi Chris,
>
> I do not think it is a good idea to do what you are doing :-) The host
> certificate does not need to be the same as Web certificate. From FreeIPA 4.1
> (IIRC), it is not even requested by default on all clients.
>
> I would rather recommend generating a separate certificate for the Web UI, we
> have some walkthrough here:
>
> http://www.freeipa.org/page/PKI#Requesting_a_new_certificate
>
>> Thanks again to the freeipa team.  I love this product.
>
> And I love to hear notes from the community like this, very rewarding!

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] YUbiKey for HOTP auth

[Christopher Young]

This is great work.  Could you perhaps write up a Howto of some sort?  I
could definitely use this!
On Mar 12, 2016 11:27 AM, "Brad Bendy"  wrote:

> After doing some more trial and error I got it it to work.
>
> Take the 20 byte secret key, remove the spaces and convert to base 32.
> Also disable OATH Token Identifier in the YubiKey tool.
>
> I used this tool to convert it
> http://tomeko.net/online_tools/hex_to_base32.php?lang=en
>
> Then take that base32 value and insert into the secret field on
> FreeIPA add token screen and your good to go, I used sha1 for
> algorithm.
>
> On Sat, Mar 12, 2016 at 8:47 AM, Brad Bendy  wrote:
> > Hi,
> >
> > YubiKey supports HOTP it appears, but im having a heck of a time
> > getting the token to add FreeIPA. The YubiKey tool gives me the OATH
> > Token which is 6 bytes and the secret key in 20 bytes hex. Ive entered
> > the secret key and OATH token into the "key" field, ive tried all
> > algorithms and get the error of "invalid 'ipatokenotpkey': Non-base32
> > digit found"
> >
> > Am I missing something? Or is this just not possible at all? I can't
> > find any documentation on Google saying how to set these up.
> >
> > Thanks!
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Replication Issues

[Christopher Young]

I've seen similar posts, but in the interest of asking fresh and
trying to understand what is going on, I thought I would ask for
advice on how best to handle this situation.

In the interest of providing some history:
I have three (3) FreeIPA servers.  Everything is running 4.4.0 now.
The originals (orldc-prod-ipa01, orldc-prod-ipa02) were upgraded from
the 3.x branch quite a while back.  Everything had been working fine,
however I ran into a replication issue (that I _think_ may have been a
result of IPv6 being disabled by my default Ansible roles).  I thought
I had resolved that by reinitializing the 2nd replica,
orldc-prod-ipa02.

In any case, I feel like the replication has never been fully stable
since then, and I have all types of errors in messages that indicate
something is off.  I had single introduced a 3rd replica such that the
agreements would look like so:

orldc-prod-ipa01 -> orldc-prod-ipa02 ---> bohdc-prod-ipa01

It feels like orldc-prod-ipa02 & bohdc-prod-ipa01 are out of sync.
I've tried reinitializing them in order but with no positive results.
At this point, I feel like I'm ready to 'bite the bullet' and tear
them down quickly (remove them from IPA, delete the local
DBs/directories) and rebuild them from scratch.

I want to minimize my impact as much as possible (which I can somewhat
do by redirecting LDAP/DNS request via my load-balancers temporarily)
and do this right.

(Getting to the point...)

I'd like advice on the order of operations to do this.  Give the
errors (I'll include samples at the bottom of this message), does it
make sense for me to remove the replicas on bohdc-prod-ipa01 &
orldc-prod-ipa02 (in that order), wipe out any directories/residual
pieces (I'd need some idea of what to do there), and then create new
replicas? -OR-  Should I export/backup the LDAP DB and rebuild
everything from scratch.

I need advice and ideas.  Furthermore, if there is someone with
experience in this that would be interested in making a little money
on the side, let me know, because having an extra brain and set of
hands would be welcome.

DETAILS:
=


ERRORS I see on orldc-prod-ipa01 (the one whose LDAP DB seems the most
up-to-date since my changes are usually directed at it):
--
Mar  6 14:36:24 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:24.434956575 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  6 14:36:25 orldc-prod-ipa01 ipa-dnskeysyncd: ipa : INFO
  LDAP bind...
Mar  6 14:36:25 orldc-prod-ipa01 ipa-dnskeysyncd: ipa : INFO
  Commencing sync process
Mar  6 14:36:26 orldc-prod-ipa01 ipa-dnskeysyncd:
ipa.ipapython.dnssec.keysyncer.KeySyncer: INFO Initial LDAP dump
is done, sychronizing with ODS and BIND
Mar  6 14:36:27 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:27.799519203 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  6 14:36:30 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:30.994760069 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  6 14:36:34 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:34.940115481 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  6 14:36:35 orldc-prod-ipa01 named-pkcs11[32134]: client
10.26.250.66#49635 (56.10.in-addr.arpa): transfer of
'56.10.in-addr.arpa/IN': AXFR-style IXFR started
Mar  6 14:36:35 orldc-prod-ipa01 named-pkcs11[32134]: client
10.26.250.66#49635 (56.10.in-addr.arpa): transfer of
'56.10.in-addr.arpa/IN': AXFR-style IXFR ended
Mar  6 14:36:37 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:37.977875463 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  6 14:36:40 orldc-prod-ipa01 ns-slapd:
[06/Mar/2017:14:36:40.999275184 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote r

Re: [Freeipa-users] Replication Issues

[Christopher Young]

Thank you very much for the response!

To start:

[root@orldc-prod-ipa01 ~]# rpm -qa 389-ds-base
389-ds-base-1.3.5.10-18.el7_3.x86_64


So, I believe a good part of my problem is that I'm not _positive_
which replica is good at this point (though my directory really isn't
that huge).

Do you have any pointers on a good method of comparing the directory
data between them?  I was wondering if anyone knows of any tools to
facilitate that.  I was thinking that it might make sense for me to
dump the DB and restore, but I really don't know that procedure.  As I
mentioned, my directory really isn't that large at all, however I'm
not positive the best bullet-item listed method to proceed.  (I know
I'm not helping things :) )

Would it be acceptable to just 'assume' one of the replicas is good
(taking the risk of whatever missing pieces I'll have to deal with),
completely removing the others, and then rebuilding the replicas from
scratch?

If I go that route, what are the potential pitfalls?


I want to decide on an approach and try and resolve this once and for all.

Thanks again! It really is appreciated as I've been frustrated with
this for a while now.

-- Chris

On Tue, Mar 7, 2017 at 8:45 AM, Mark Reynolds  wrote:
> What version of 389-ds-base are you using?
>
> rpm -qa | grep 389-ds-base
>
>
> comments below..
>
> On 03/06/2017 02:37 PM, Christopher Young wrote:
>
> I've seen similar posts, but in the interest of asking fresh and
> trying to understand what is going on, I thought I would ask for
> advice on how best to handle this situation.
>
> In the interest of providing some history:
> I have three (3) FreeIPA servers.  Everything is running 4.4.0 now.
> The originals (orldc-prod-ipa01, orldc-prod-ipa02) were upgraded from
> the 3.x branch quite a while back.  Everything had been working fine,
> however I ran into a replication issue (that I _think_ may have been a
> result of IPv6 being disabled by my default Ansible roles).  I thought
> I had resolved that by reinitializing the 2nd replica,
> orldc-prod-ipa02.
>
> In any case, I feel like the replication has never been fully stable
> since then, and I have all types of errors in messages that indicate
> something is off.  I had single introduced a 3rd replica such that the
> agreements would look like so:
>
> orldc-prod-ipa01 -> orldc-prod-ipa02 ---> bohdc-prod-ipa01
>
> It feels like orldc-prod-ipa02 & bohdc-prod-ipa01 are out of sync.
> I've tried reinitializing them in order but with no positive results.
> At this point, I feel like I'm ready to 'bite the bullet' and tear
> them down quickly (remove them from IPA, delete the local
> DBs/directories) and rebuild them from scratch.
>
> I want to minimize my impact as much as possible (which I can somewhat
> do by redirecting LDAP/DNS request via my load-balancers temporarily)
> and do this right.
>
> (Getting to the point...)
>
> I'd like advice on the order of operations to do this.  Give the
> errors (I'll include samples at the bottom of this message), does it
> make sense for me to remove the replicas on bohdc-prod-ipa01 &
> orldc-prod-ipa02 (in that order), wipe out any directories/residual
> pieces (I'd need some idea of what to do there), and then create new
> replicas? -OR-  Should I export/backup the LDAP DB and rebuild
> everything from scratch.
>
> I need advice and ideas.  Furthermore, if there is someone with
> experience in this that would be interested in making a little money
> on the side, let me know, because having an extra brain and set of
> hands would be welcome.
>
> DETAILS:
> =
>
>
> ERRORS I see on orldc-prod-ipa01 (the one whose LDAP DB seems the most
> up-to-date since my changes are usually directed at it):
> --
> Mar  6 14:36:24 orldc-prod-ipa01 ns-slapd:
> [06/Mar/2017:14:36:24.434956575 -0500] NSMMReplicationPlugin -
> agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
> (orldc-prod-ipa02:389): The remote replica has a different database
> generation ID than the local database.  You may have to reinitialize
> the remote replica, or the local replica.
> Mar  6 14:36:25 orldc-prod-ipa01 ipa-dnskeysyncd: ipa : INFO
>   LDAP bind...
> Mar  6 14:36:25 orldc-prod-ipa01 ipa-dnskeysyncd: ipa : INFO
>   Commencing sync process
> Mar  6 14:36:26 orldc-prod-ipa01 ipa-dnskeysyncd:
> ipa.ipapython.dnssec.keysyncer.KeySyncer: INFO Initial LDAP dump
> is done, sychronizing with ODS and BIND
> Mar  6 14:36:27 orldc-prod-ipa01 ns-slapd:
> [06/Mar/2017:14:36:27.799519203 -0500] NSMMReplicationPlugin -
> agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
> (o

Re: [Freeipa-users] Replication Issues

[Christopher Young]

I had attempted to do _just_ a re-initialize on orldc-prod-ipa02
(using --from orldc-prod-ipa01), but after it completes, I still end
up with the same errors.  What would be my next course of action?

To clarify the error(s) on orldc-prod-ipa01 are:
-
Mar  7 18:04:53 orldc-prod-ipa01 ns-slapd:
[07/Mar/2017:18:04:53.549127059 -0500] NSMMReplicationPlugin -
agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa02:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.

-


On orldc-prod-ipa02, I get:
-
Mar  7 18:06:00 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:00.290853165 -0500] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa01:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  7 18:06:01 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:01.715691409 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
Mar  7 18:06:01 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:01.720475590 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
Mar  7 18:06:01 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:01.728588145 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
Mar  7 18:06:04 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:04.286539164 -0500] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
(orldc-prod-ipa01:389): The remote replica has a different database
generation ID than the local database.  You may have to reinitialize
the remote replica, or the local replica.
Mar  7 18:06:05 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:05.328239468 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
Mar  7 18:06:05 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:05.330429534 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
Mar  7 18:06:05 orldc-prod-ipa02 ns-slapd:
[07/Mar/2017:18:06:05.333270479 -0500] attrlist_replace - attr_replace
(nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
failed.
-


I'm trying to figure out what my next step(s) would be in this
situation.  Would that be to completely remove those systems are
replicas (orldc-prod-ipa02 and bohdc-prod-ipa01) and then completely
recreate the replicas?

I appreciate all the responses.  I'm still trying to figure out what
options to use for db2ldif, but I'm looking that up to at least try
and look at the DBs.

Thanks,

Chris

On Tue, Mar 7, 2017 at 4:23 PM, Mark Reynolds  wrote:
>
>
> On 03/07/2017 11:29 AM, Christopher Young wrote:
>> Thank you very much for the response!
>>
>> To start:
>> 
>> [root@orldc-prod-ipa01 ~]# rpm -qa 389-ds-base
>> 389-ds-base-1.3.5.10-18.el7_3.x86_64
>> 
> You are on the latest version with the latest replication fixes.
>>
>> So, I believe a good part of my problem is that I'm not _positive_
>> which replica is good at this point (though my directory really isn't
>> that huge).
>>
>> Do you have any pointers on a good method of comparing the directory
>> data between them?  I was wondering if anyone knows of any tools to
>> facilitate that.  I was thinking that it might make sense for me to
>> dump the DB and restore, but I really don't know that procedure.  As I
>> mentioned, my directory really isn't that large at all, however I'm
>> not positive the best bullet-item listed method to proceed.  (I know
>> I'm not helping things :) )
> Heh, well only you know what your data should be.  You can always do a
> db2ldif.pl on each server and compare the ldif files that are
> generated.  Then pick the one you think is the most up to date.
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/html/Administration_Guide/Populating_Directory_Databases-Exporting_Data.html#Exporting-db2ldif
>
> Once you decide on a server, then you need to reinitialize all the other
> servers/replicas from the "good" one. Use " ipa-replica-manage
> re-initialize" for this.
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Identity_Management_Guide/ipa-replica-manage.html#initialize
>
> That's it.
>
> Good luck,
> Mark
>
>>
>> Would it be acceptable to just 'assume' one of the replicas is good

Re: [Freeipa-users] Replication Issues

[Christopher Young]

My replication scheme has things like so:

orldc-prod-ipa01 <--> orldc-prod-ipa02 <--> bohdc-prod-ipa01

I had run re-initialize on orldc-prod-ipa02 (--from orldc-prod-ipa01) AND
re-initialize on bohdc-prod-ipa01 (--from orldc-prod-ipa02).

That is where i'm currently at with the same errors.

Any additional thoughts beyond just destroying 'orldc-prod-ipa02' and
bohdc-prod-ipa01 and re-installing them as new replicas?

As always, many thanks.

On Tue, Mar 7, 2017 at 7:40 PM, Mark Reynolds  wrote:
>
>
> On 03/07/2017 06:08 PM, Christopher Young wrote:
>> I had attempted to do _just_ a re-initialize on orldc-prod-ipa02
>> (using --from orldc-prod-ipa01), but after it completes, I still end
>> up with the same errors. What would be my next course of action?
> I have no idea. Sounds like the reinit did not work if you keep getting
> the same errors, or you reinited the wrong server (or the wrong
> direction) Remember you have to reinit ALL the replicas - not just one.
>>
>> To clarify the error(s) on orldc-prod-ipa01 are:
>> -
>> Mar 7 18:04:53 orldc-prod-ipa01 ns-slapd:
>> [07/Mar/2017:18:04:53.549127059 -0500] NSMMReplicationPlugin -
>> agmt="cn=cloneAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
>> (orldc-prod-ipa02:389): The remote replica has a different database
>> generation ID than the local database. You may have to reinitialize
>> the remote replica, or the local replica.
>> 
>> -
>>
>>
>> On orldc-prod-ipa02, I get:
>> -
>> Mar 7 18:06:00 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:00.290853165 -0500] NSMMReplicationPlugin -
>> agmt="cn=masterAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
>> (orldc-prod-ipa01:389): The remote replica has a different database
>> generation ID than the local database. You may have to reinitialize
>> the remote replica, or the local replica.
>> Mar 7 18:06:01 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:01.715691409 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> Mar 7 18:06:01 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:01.720475590 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> Mar 7 18:06:01 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:01.728588145 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> Mar 7 18:06:04 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:04.286539164 -0500] NSMMReplicationPlugin -
>> agmt="cn=masterAgreement1-orldc-prod-ipa01.passur.local-pki-tomcat"
>> (orldc-prod-ipa01:389): The remote replica has a different database
>> generation ID than the local database. You may have to reinitialize
>> the remote replica, or the local replica.
>> Mar 7 18:06:05 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:05.328239468 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> Mar 7 18:06:05 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:05.330429534 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> Mar 7 18:06:05 orldc-prod-ipa02 ns-slapd:
>> [07/Mar/2017:18:06:05.333270479 -0500] attrlist_replace - attr_replace
>> (nsslapd-referral, ldap://orldc-prod-ipa01.passur.local:389/o%3Dipaca)
>> failed.
>> -
>>
>>
>> I'm trying to figure out what my next step(s) would be in this
>> situation. Would that be to completely remove those systems are
>> replicas (orldc-prod-ipa02 and bohdc-prod-ipa01) and then completely
>> recreate the replicas?
>>
>> I appreciate all the responses. I'm still trying to figure out what
>> options to use for db2ldif, but I'm looking that up to at least try
>> and look at the DBs.
>>
>> Thanks,
>>
>> Chris
>>
>> On Tue, Mar 7, 2017 at 4:23 PM, Mark Reynolds 
wrote:
>>>
>>> On 03/07/2017 11:29 AM, Christopher Young wrote:
>>>> Thank you very much for the response!
>>>>
>>>> To start:
>>>> 
>>>> [root@orldc-prod-ipa01 ~]# rpm -qa 389-ds-base
>>>> 389-ds-base-1.3.5.10-18.el7_3.x86_64
>>>> 
>>> You are on the latest version with the latest replication fixes.
>>>> So, I believe a good part of my problem is that I'm not _positive_
>>>> which replica is good a