Oh you are lucky to have ~150 replication conflicts :)

How did you get those? Did you run upgrade in parallel or did you have some network issues?


You have to manually fix all replication conflicts and the re-run ipa-server-upgrade

Please follow guide I posted previously, sorry :(


Martin


On 12.10.2016 21:30, John Popowitch wrote:
I ran the following on each of my three servers:
kinit admin
ldapsearch -Y GSSAPI -b 'dc=aws,dc=cappex,dc=com' "nsds5ReplConflict=*" \* 
nsds5ReplConflict
There are 49, 57, 49 entries returned by that query on the respective server.
Here is the one related to 'System: Modify Certificate Profile' from  the first 
server:

# CA Administrator + c93bf230-a32311e5-b492895f-f9294e47, privileges, pbac, aws
  .cappex.com
dn: cn=CA Administrator+nsuniqueid=c93bf230-a32311e5-b492895f-f9294e47,cn=priv
  ileges,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Add CA ACL+nsuniqueid=c93bf269-a32311e5-b492895f-f9294e47
  ,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete CA ACL+nsuniqueid=c93bf26d-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Manage CA ACL Membership+nsuniqueid=c93bf271-a32311e5-b49
  2895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify CA ACL+nsuniqueid=c93bf275-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete Certificate Profile+nsuniqueid=c93bf27c-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Import Certificate Profile+nsuniqueid=c93bf280-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify Certificate Profile+nsuniqueid=c93bf284-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
objectClass: groupofnames
objectClass: top
objectClass: nestedgroup
cn: CA Administrator
description: CA Administrator
nsds5ReplConflict: namingConflict cn=CA Administrator,cn=privileges,cn=pbac,dc
  =aws,dc=cappex,dc=com


Here are the related entries from the second server:

# CA Administrator + c93bf230-a32311e5-b492895f-f9294e47, privileges, pbac, aws
  .cappex.com
dn: cn=CA Administrator+nsuniqueid=c93bf230-a32311e5-b492895f-f9294e47,cn=priv
  ileges,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Add CA ACL+nsuniqueid=c93bf269-a32311e5-b492895f-f9294e47
  ,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete CA ACL+nsuniqueid=c93bf26d-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Manage CA ACL Membership+nsuniqueid=c93bf271-a32311e5-b49
  2895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify CA ACL+nsuniqueid=c93bf275-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete Certificate Profile+nsuniqueid=c93bf27c-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Import Certificate Profile+nsuniqueid=c93bf280-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify Certificate Profile+nsuniqueid=c93bf284-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
objectClass: groupofnames
objectClass: top
objectClass: nestedgroup
cn: CA Administrator
description: CA Administrator
nsds5ReplConflict: namingConflict cn=ca administrator,cn=privileges,cn=pbac,dc
  =aws,dc=cappex,dc=com

# System: Modify Certificate Profile + c93bf284-a32311e5-b492895f-f9294e47, per
  missions, pbac, aws.cappex.com
dn: cn=System: Modify Certificate Profile+nsuniqueid=c93bf284-a32311e5-b492895
  f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
member: cn=CA Administrator+nsuniqueid=c93bf230-a32311e5-b492895f-f9294e47,cn=
  privileges,cn=pbac,dc=aws,dc=cappex,dc=com
ipaPermTargetFilter: (objectclass=ipacertprofile)
ipaPermRight: write
ipaPermBindRuleType: permission
ipaPermissionType: V2
ipaPermissionType: MANAGED
ipaPermissionType: SYSTEM
cn: System: Modify Certificate Profile
objectClass: ipapermission
objectClass: top
objectClass: groupofnames
objectClass: ipapermissionv2
ipaPermDefaultAttr: description
ipaPermDefaultAttr: ipacertprofilestoreissued
ipaPermDefaultAttr: cn
ipaPermLocation: cn=certprofiles,cn=ca,dc=aws,dc=cappex,dc=com
nsds5ReplConflict: namingConflict cn=system: modify certificate profile,cn=per
  missions,cn=pbac,dc=aws,dc=cappex,dc=com


And from the third server:

# CA Administrator + c93bf230-a32311e5-b492895f-f9294e47, privileges, pbac, aws
  .cappex.com
dn: cn=CA Administrator+nsuniqueid=c93bf230-a32311e5-b492895f-f9294e47,cn=priv
  ileges,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Add CA ACL+nsuniqueid=c93bf269-a32311e5-b492895f-f9294e47
  ,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete CA ACL+nsuniqueid=c93bf26d-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Manage CA ACL Membership+nsuniqueid=c93bf271-a32311e5-b49
  2895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify CA ACL+nsuniqueid=c93bf275-a32311e5-b492895f-f9294
  e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Delete Certificate Profile+nsuniqueid=c93bf27c-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Import Certificate Profile+nsuniqueid=c93bf280-a32311e5-b
  492895f-f9294e47,cn=permissions,cn=pbac,dc=aws,dc=cappex,dc=com
memberOf: cn=System: Modify Certificate Profile,cn=permissions,cn=pbac,dc=aws,
  dc=cappex,dc=com
objectClass: groupofnames
objectClass: top
objectClass: nestedgroup
cn: CA Administrator
description: CA Administrator
nsds5ReplConflict: namingConflict cn=CA Administrator,cn=privileges,cn=pbac,dc
  =aws,dc=cappex,dc=com


Thank you for sending a link with more info on replication conflicts.
I'm reading it now.
-John



-----Original Message-----
From: Martin Basti [mailto:mba...@redhat.com]
Sent: Wednesday, October 12, 2016 5:46 AM
To: John Popowitch; Alexander Bokovoy
Cc: freeipa-users@redhat.com
Subject: Re: [Freeipa-users] FreeIPA v4.2 stopped working, wants me to run 
ipa-server-upgrade, but has errors



On 11.10.2016 22:01, John Popowitch wrote:
Ah, yes, thank you, Alexander.
I agree it would help if I followed the example better.
It would also help if I understood the example so a little description of what 
each command does would be very helpful.
Sorry, we don't have time to explain everything here. `man ldapsearch` is your 
friend


It looks like that ACI record does exist.
Now how would I remove these LDAP records?
I dig deeper into code, and actually this error is not caused by ACIs,
because it even does not get there. I think that this may be caused by
replication conflict on permission entry that caused the IPA doesn't see
it but DS refuses to add it there.

Can you please check as Directory Manager if there are any replication
conflicts using this command?
ldapsearch -D 'cn=directory manager' -W -b 'dc=aws,dc=cappex,dc=com'
"nsds5ReplConflict=*" \* nsds5ReplConflict

Please check if there is replication conflict on entry 'System: Modify
Certificate Profile'

More info about replication conflicts:
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/html/Administration_Guide/Managing_Replication-Solving_Common_Replication_Conflicts.html

-----Original Message-----
From: Alexander Bokovoy [mailto:aboko...@redhat.com]
Sent: Tuesday, October 11, 2016 2:44 PM
To: John Popowitch
Cc: Martin Basti; freeipa-users@redhat.com
Subject: Re: [Freeipa-users] FreeIPA v4.2 stopped working, wants me to run 
ipa-server-upgrade, but has errors

On ti, 11 loka 2016, John Popowitch wrote:
It doesn't look like there are any entries.

# ldapsearch -x -b 'cn=certprofiles,cn=ca,dc=aws,dc=cappex,dc=com' -s
base aci
'ldapsearch -x' is 'use simple authentication instead of SASL' -- given that 
you didn't specify any identity for simple authentication, you are running an 
anonymous search. Martin asked you to 'kinit' as administrator and then use 
SASL GSSAPI.

ACIs only available for retrieval to administrators. It is not a surprise that 
anonymous access does not see them.

It would be good if you would have followed the example:
Here you have example

kinit admin

ldapsearch -Y GSSAPI -b 'cn=certprofiles,cn=ca,dc=<your>,dc=<suffix>'
-s base aci

On 11.10.2016 17:48, John Popowitch wrote:
Thanks, Martin.
But I'm afraid you've gone beyond my level of LDAP knowledge.
How would I check for that ACI?
-John

From: Martin Basti [mailto:mba...@redhat.com]
Sent: Tuesday, October 11, 2016 10:38 AM
To: John Popowitch;
freeipa-users@redhat.com<mailto:freeipa-users@redhat.com>
Subject: Re: [Freeipa-users] FreeIPA v4.2 stopped working, wants me to
run ipa-server-upgrade, but has errors




On 11.10.2016 17:21, John Popowitch wrote:
I agree that is weird.
Several of the other managed permissions are updated successfully and they are 
very similar.
Yes, I can try to remove the permission manually.
Is there any risk in corrupting or breaking the system?
This is, I believe, one of three IPA servers in a multi-master replication.
And we run our production website (basically our company) off of these servers.
Assuming it's safe enough to do, could I delete that permission via the UI or 
does it need to be directly via LDAP?

Upgrade will re-create permission.

You have to directly using LDAP as Directory Manager

Also please check in: cn=certprofiles,cn=ca,$SUFFIX

if you have this ACI there

aci: (targetattr = "cn || description ||
ipacertprofilestoreissued")(targetfil
ter = "(objectclass=ipacertprofile)")(version 3.0;acl
"permission:System: Mod  ify Certificate Profile";allow (write) groupdn
= "ldap:///cn=System<ldap://cn=System>: Modify C  ertificate
Profile,cn=permissions,cn=pbac,dc=dom-058-017,dc=abc,dc=idm,dc=lab
,dc=eng,dc=brq,dc=redhat,dc=com";)

This may also cause an issue, so if removing of permission itself did
not help (or permission does not exist) you may need to remove this ACI

Martin




From: Martin Basti [mailto:mba...@redhat.com]
Sent: Tuesday, October 11, 2016 9:47 AM
To: John Popowitch;
freeipa-users@redhat.com<mailto:freeipa-users@redhat.com>
Subject: Re: [Freeipa-users] FreeIPA v4.2 stopped working, wants me to
run ipa-server-upgrade, but has errors


That's weird because the code is checking if a permission exists before
it tries to add a new one

Can you try to remove 'System: Modify Certificate Profile' manually from LDAP 
and re-run ipa-server-upgrade?



On 11.10.2016 15:53, John Popowitch wrote:
2016-10-10T19:51:38Z DEBUG Updating managed permission: System: Modify
Certificate Profile 2016-10-10T19:51:38Z DEBUG Destroyed connection
context.ldap2_82077392 2016-10-10T19:51:38Z ERROR Upgrade failed with
This entry already exists 2016-10-10T19:51:38Z DEBUG Traceback (most recent 
call last):
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", line 
306, in __upgrade
     self.modified = (ld.update(self.files) or self.modified)
   File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", 
line 905, in update
     self._run_updates(all_updates)
   File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", 
line 877, in _run_updates
     self._run_update_plugin(update['plugin'])
   File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", 
line 852, in _run_update_plugin
     restart_ds, updates = self.api.Updater[plugin_name]()
   File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 1400, in 
__call__
     return self.execute(**options)
File 
"/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_managed_permissions.py",
 line 433, in execute
     anonymous_read_aci)
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_managed_permissions.py",
 line 529, in update_permission
     ldap.add_entry(entry)
   File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1428, in 
add_entry
     self.conn.add_s(str(entry.dn), attrs.items())
   File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
     self.gen.throw(type, value, traceback)
   File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 938, in 
error_handler
     raise errors.DuplicateEntry()
DuplicateEntry: This entry already exists

2016-10-10T19:51:38Z DEBUG Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 
418, in start_creation
     run_step(full_msg, method)
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 
408, in run_step
     method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", 
line 314, in __upgrade
     raise RuntimeError(e)
RuntimeError: This entry already exists

2016-10-10T19:51:38Z DEBUG   [error] RuntimeError: This entry already exists
2016-10-10T19:51:38Z DEBUG   [cleanup]: stopping directory server
2016-10-10T19:51:38Z DEBUG Starting external process
2016-10-10T19:51:38Z DEBUG args='/bin/systemctl' 'stop' 
'dirsrv@AWS-CAPPEX-COM.service<mailto:dirsrv@AWS-CAPPEX-COM.service>'
2016-10-10T19:51:40Z DEBUG Process finished, return code=0
2016-10-10T19:51:40Z DEBUG stdout= 2016-10-10T19:51:40Z DEBUG stderr=
2016-10-10T19:51:40Z DEBUG   duration: 1 seconds
2016-10-10T19:51:40Z DEBUG   [cleanup]: restoring configuration
2016-10-10T19:51:40Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Saving StateFile to 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Saving StateFile to 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysrestore/sysrestore.state'
2016-10-10T19:51:40Z DEBUG   duration: 0 seconds
2016-10-10T19:51:40Z ERROR IPA server upgrade failed: Inspect 
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2016-10-10T19:51:40Z DEBUG   File 
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
     return_value = self.run()
   File 
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", 
line 50, in run
     raise admintool.ScriptError(str(e))

2016-10-10T19:51:40Z DEBUG The ipa-server-upgrade command failed,
exception: ScriptError: ('IPA upgrade failed.', 1) 2016-10-10T19:51:40Z
ERROR ('IPA upgrade failed.', 1)



From: Martin Basti [mailto:mba...@redhat.com]
Sent: Tuesday, October 11, 2016 1:53 AM
To: John Popowitch;
freeipa-users@redhat.com<mailto:freeipa-users@redhat.com>
Subject: Re: [Freeipa-users] FreeIPA v4.2 stopped working, wants me to
run ipa-server-upgrade, but has errors




On 10.10.2016 23:30, John Popowitch wrote:
Hello FreeIPA community.
I've inherited a group of three FreeIPA v4.2 servers on CentOS 7.2.
I had to reboot one of the servers and now IPA won't run saying, "Upgrade required: 
please run ipa-server-upgrade command."
But when I run ipa-server-upgrade I get an error:
ipa: ERROR: Upgrade failed with This entry already exists When I run it
in debug mode the last action before the error is:
ipa.ipaserver.install.plugins.update_managed_permissions.update_managed
_permissions: DEBUG: Updating managed permission: System: Modify Certificate 
Profile It appears that several of the other managed permissions are processed 
successfully.
When I look in the UI on one of the other servers it appears that this permission 
exists under IPA Server -> Role Based Access Control -> Permissions.
I'm not familiar with FreeIPA so any help would be greatly appreciated.
Thanks in advance.
-John







Hello,

can you post the related part of ipaupgrade.log here?

Martin



--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to