On 09/17/2012 10:27 AM, Michael Mercier wrote: > On 2012-09-10, at 4:35 AM, Petr Spacek wrote: > >> On 09/08/2012 05:03 PM, Dmitri Pal wrote: >>> On 09/07/2012 04:50 PM, Rob Crittenden wrote: >>>> Michael Mercier wrote: >>>>> On 2012-09-07, at 2:47 PM, Dmitri Pal wrote: >>>>> >>>>>> On 09/07/2012 12:42 PM, Michael Mercier wrote: >>>>>>> On 2012-09-07, at 12:14 PM, Dmitri Pal wrote: >>>>>>> >>>>>>>> On 09/06/2012 10:40 AM, Michael Mercier wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I have experienced some odd connectivity issues using MMR with >>>>>>>>> FreeIPA (all systems CentOS 6.3). I have 2 ipa servers >>>>>>>>> (ipaserver / ipaserver2) setup using MMR. >>>>>>>>> >>>>>>>>> [root@ipaserver ~]#ipa-replica-manage list >>>>>>>>> ipaserver.mpls.local: master >>>>>>>>> ipaserver2.mpls.local: master >>>>>>>>> [root@ipaserver ~]# rpm -qa|grep ipa >>>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64 >>>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-server-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch >>>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64 >>>>>>>>> ipa-client-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch >>>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch >>>>>>>>> ipa-python-2.2.0-16.el6.x86_64 >>>>>>>>> >>>>>>>>> >>>>>>>>> [root@ipaserver2 ~]#ipa-replica-manage list >>>>>>>>> ipaserver.mpls.local: master >>>>>>>>> ipaserver2.mpls.local: master >>>>>>>>> [root@ipaserver2 ~]# rpm -qa|grep ipa >>>>>>>>> ipa-client-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-server-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch >>>>>>>>> ipa-python-2.2.0-16.el6.x86_64 >>>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64 >>>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch >>>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64 >>>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch >>>>>>>>> >>>>>>>>> >>>>>>>>> [mike@ipaclient ~]$ rpm -qa|grep ipa >>>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64 >>>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch >>>>>>>>> ipa-python-2.2.0-16.el6.x86_64 >>>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64 >>>>>>>>> ipa-client-2.2.0-16.el6.x86_64 >>>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64 >>>>>>>>> >>>>>>>>> >>>>>>>>> I have a webserver (zenoss) using kerberos authentication. >>>>>>>>> >>>>>>>>> [root@zenoss ~]# rpm -qa|grep ipa >>>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64 >>>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64 >>>>>>>>> ipa-python-2.2.0-16.el6.x86_64 >>>>>>>>> ipa-client-2.2.0-16.el6.x86_64 >>>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch >>>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64 >>>>>>>>> >>>>>>>>> <Location /> >>>>>>>>> SSLRequireSSL >>>>>>>>> AuthType Kerberos >>>>>>>>> AuthName "Kerberos Login" >>>>>>>>> >>>>>>>>> KrbMethodK5Passwd Off >>>>>>>>> KrbAuthRealms MPLS.LOCAL >>>>>>>>> KrbSaveCredentials on >>>>>>>>> KrbServiceName HTTP >>>>>>>>> Krb5KeyTab /etc/http/conf.d/http.keytab >>>>>>>>> >>>>>>>>> AuthLDAPUrl "ldap://ipaserver.mpls.local >>>>>>>>> ipaserver2.mpls.local/dc=mpls,dc=local?krbPrincipalName" >>>>>>>>> RequestHeader set X_REMOTE_USER %{remoteUser}e >>>>>>>>> require ldap-group >>>>>>>>> cn=zenuser,cn=groups,cn=accounts,dc=mpls,dc=local >>>>>>>>> </Location> >>>>>>>>> >>>>>>>>> >>>>>>>>> With both ipaserver and ipaserver2 'up', if I connect to >>>>>>>>> https://zenoss.mpls.local from ipaclient using firefox, I am >>>>>>>>> successfully connected. If on ipaserver I do a 'ifdown eth0' and >>>>>>>>> attempt another connection, it fails. I have also noticed the >>>>>>>>> following: >>>>>>>>> >>>>>>>>> 1. I am unable to use the ipaserver2 management interface when >>>>>>>>> ipaserver is unavailable. >>>>>>>>> 2. It takes a longer period of time to do a kinit >>>>>>>>> >>>>>>>>> If the I then perform: >>>>>>>>> [root@ipaserver ~]#ifup eth0 >>>>>>>>> >>>>>>>>> [root@ipaserver2 ~]#ifdown eth0 >>>>>>>>> >>>>>>>>> [mike@ipaclient ~]$kinit >>>>>>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while >>>>>>>>> getting initial credentials >>>>>>>>> >>>>>>>>> [root@ipaserver2 ~]#ifup eth0 >>>>>>>>> >>>>>>>>> [mike@ipaclient ~]$ kinit >>>>>>>>> Password for mike@MPLS.LOCAL: >>>>>>>>> [mike@ipaclient ~]$ >>>>>>>>> >>>>>>>>> [root@ipaserver2 ~]#ifdown eth0 >>>>>>>>> >>>>>>>>> .. wait number of minutes >>>>>>>>> >>>>>>>>> ipaclient screen locks - type password - after a short delay (~7 >>>>>>>>> seconds) screen unlock compeletes >>>>>>>>> >>>>>>>>> [mike@ipaclient ~]$kinit >>>>>>>>> Password for mike@MPLS.LOCAL: >>>>>>>>> [mike@ipaclient ~]$ >>>>>>>>> >>>>>>>>> Any ideas? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Mike >>>>>>>> This seems to be some DNS problem. >>>>>>>> You client does not see the second replica and might have some name >>>>>>>> resolution timeouts. >>>>>>>> >>>>>>>> Please check your dns setup and krb5.conf on the client. >>>>>>>> >>>>>>>> To help more we need more details about you client configuration >>>>>>>> DNS and >>>>>>>> kerberos. >>>>>>> Hi, >>>>>>> >>>>>>> Additional information... >>>>>>> >>>>>>> [root@zenoss ~]#more /etc/resolv.conf >>>>>>> search mpls.local >>>>>>> domain mpls.local >>>>>>> nameserver 172.16.112.5 >>>>>>> nameserver 172.16.112.8 >>>>>>> >>>>>>> [root@zenoss ~]# more /etc/krb5.conf >>>>>>> #File modified by ipa-client-install >>>>>>> >>>>>>> [libdefaults] >>>>>>> default_realm = MPLS.LOCAL >>>>>>> dns_lookup_realm = true >>>>>>> dns_lookup_kdc = true >>>>>>> rdns = false >>>>>>> ticket_lifetime = 24h >>>>>>> forwardable = yes >>>>>>> >>>>>>> [realms] >>>>>>> MPLS.LOCAL = { >>>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt >>>>>>> } >>>>>>> >>>>>>> [domain_realm] >>>>>>> .mpls.local = MPLS.LOCAL >>>>>>> mpls.local = MPLS.LOCAL >>>>>>> >>>>>>> [root@ipaclient ~]# more /etc/resolv.conf >>>>>>> # Generated by NetworkManager >>>>>>> search mpls.local >>>>>>> nameserver 172.16.112.5 >>>>>>> nameserver 172.16.112.8 >>>>>>> >>>>>>> [root@ipaclient ~]# more /etc/krb5.conf >>>>>>> #File modified by ipa-client-install >>>>>>> >>>>>>> [libdefaults] >>>>>>> default_realm = MPLS.LOCAL >>>>>>> dns_lookup_realm = true >>>>>>> dns_lookup_kdc = true >>>>>>> rdns = false >>>>>>> ticket_lifetime = 24h >>>>>>> forwardable = yes >>>>>>> >>>>>>> [realms] >>>>>>> MPLS.LOCAL = { >>>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt >>>>>>> } >>>>>>> >>>>>>> [domain_realm] >>>>>>> .mpls.local = MPLS.LOCAL >>>>>>> mpls.local = MPLS.LOCAL >>>>>>> >>>>>>> [root@ipaclient ~]# nslookup ipaserver >>>>>>> Server: 172.16.112.5 >>>>>>> Address: 172.16.112.5#53 >>>>>>> >>>>>>> Name: ipaserver.mpls.local >>>>>>> Address: 172.16.112.5 >>>>>>> >>>>>>> [root@ipaserver ~]#ifdown eth0 >>>>>>> >>>>>>> [root@ipaclient ~]# nslookup ipaserver >>>>>>> Server: 172.16.112.8 >>>>>>> Address: 172.16.112.8#53 >>>>>>> >>>>>>> Name: ipaserver.mpls.local >>>>>>> Address: 172.16.112.5 >>>>>>> >>>>>>> [root@ipaclient ~]# nslookup ipaserver2 >>>>>>> Server: 172.16.112.8 >>>>>>> Address: 172.16.112.8#53 >>>>>>> >>>>>>> Name: ipaserver2.mpls.local >>>>>>> Address: 172.16.112.8 >>>>>>> >>>>>>> Copy/paste from the DNS page on ipaserver/ipaserver2 >>>>>>> >>>>>>> @ NS ipaserver.mpls.local. >>>>>>> NS ipaserver2.mpls.local. >>>>>>> _kerberos TXT MPLS.LOCAL >>>>>>> _kerberos-master._tcp SRV 0 100 88 ipaserver >>>>>>> SRV 0 100 88 ipaserver2 >>>>>>> _kerberos-master._udp SRV 0 100 88 ipaserver >>>>>>> SRV 0 100 88 ipaserver2 >>>>>>> _kerberos._tcp SRV 0 100 88 ipaserver >>>>>>> SRV 0 100 88 ipaserver2 >>>>>>> _kerberos._udp SRV 0 100 88 ipaserver >>>>>>> SRV 0 100 88 ipaserver2 >>>>>>> _kpasswd._tcp SRV 0 100 464 ipaserver >>>>>>> SRV 0 100 464 ipaserver2 >>>>>>> _kpasswd._udp SRV 0 100 464 ipaserver >>>>>>> SRV 0 100 464 ipaserver2 >>>>>>> _ldap._tcp SRV 0 100 389 ipaserver >>>>>>> SRV 0 100 389 ipaserver2 >>>>>>> _ntp._udp SRV 0 100 123 ipaserver >>>>>>> SRV 0 100 123 ipaserver2 >>>>>>> ipaclient A 172.16.112.9 >>>>>>> ipaclient2 A 172.16.112.145 >>>>>>> ipaserver A 172.16.112.5 >>>>>>> ipaserver2 A 172.16.112.8 >>>>>>> zenoss A 172.16.112.6 >>>>>>> >>>>>>> Thanks, >>>>>>> Mike >>>>>>> >>>>>> I noticed that there is no domain line in the resolv.conf on the >>>>>> client. >>>>>> AFAIU in this case it would determine the domain by the gethostname and >>>>>> in case of network being down it will fail over to the hosts file. >>>>>> I wonder what is in your /etc/hosts? >>>>>> Dose it have just a short host name? >>>>> [root@ipaclient ~]# more /etc/hosts >>>>> 127.0.0.1 localhost.localdomain localhost >>>>> ::1 localhost6.localdomain6 localhost6 >>>>> >>>>> >>>>> Add domain mpls.local to /etc/resolv.conf >>>>> >>>>> [root@ipaserver ~]#ifdown eth0 >>>>> >>>>> [root@ipaclient ~]# kinit mike >>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting >>>>> initial credentials >>>>> [root@ipaclient ~]# nslookup ipaserver >>>>> Server: 172.16.112.8 >>>>> Address: 172.16.112.8#53 >>>>> >>>>> Name: ipaserver.mpls.local >>>>> Address: 172.16.112.5 >>>>> >>>>> [root@ipaclient ~]# nslookup ipaserver2 >>>>> Server: 172.16.112.8 >>>>> Address: 172.16.112.8#53 >>>>> >>>>> Name: ipaserver2.mpls.local >>>>> Address: 172.16.112.8 >>>>> >>>>> add '172.16.112.9 ipaclient.mpls.local ipaclient' to /etc/hosts >>>>> >>>>> [root@ipaserver ~]#ifup eth0 >>>>> >>>>> [root@ipaclient ~]# kinit mike >>>>> Password for mike@MPLS.LOCAL: >>>>> >>>>> [root@ipaserver ~]#ifdown eth0 >>>>> >>>>> [root@ipaclient ~]# kinit mike >>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting >>>>> initial credentials >>>>> [root@ipaclient ~]# nslookup -type=srv _kerberos-master._tcp >>>>> Server: 172.16.112.8 >>>>> Address: 172.16.112.8#53 >>>>> >>>>> _kerberos-master._tcp.mpls.local service = 0 100 88 >>>>> ipaserver2.mpls.local. >>>>> _kerberos-master._tcp.mpls.local service = 0 100 88 >>>>> ipaserver.mpls.local. >>>>> >>>>> [root@ipaclient ~]# nslookup -type=srv _kerberos-master._udp >>>>> Server: 172.16.112.5 >>>>> Address: 172.16.112.5#53 >>>>> >>>>> _kerberos-master._udp.mpls.local service = 0 100 88 >>>>> ipaserver.mpls.local. >>>>> _kerberos-master._udp.mpls.local service = 0 100 88 >>>>> ipaserver2.mpls.local. >>>>> >>>>> >>>>> [root@ipaclient ~]# kinit mike >>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting >>>>> initial credentials >>>>> >>>>> [root@ipaserver ~]#ifup eth0 >>>>> >>>>> [root@ipaclient ~]# kinit mike >>>>> Password for mike@MPLS.LOCAL: >>>> I'd start with the sssd logs. Is it seeing the main server go offline >>>> and not switching to the second one? Or is it going into offline mode? >>>> >>>> Do you have _srv_ or both servers listed in ipa_server in >>>> /etc/sssd/sssd.conf? >>>> >>>> rob >>>> >>> Rob, may be I am missing something but how SSSD is related in this case? >>> The test is done using kinit not SSSD. >>> >>> It would actually be an interesting test to try the same via SSSD for >>> example do su to mike instead of kinit and see what would happen (watch >>> SSSD logs with high debug level, 8 for example). >>> If that works it would probably mean that kinit does not fail over >>> properly. So this would be a Kerberos kinit bug not IPA/SSSD bug. >>> >> AFAIK there is "sssd_krb5_locator_plugin". This plugin changes Kerberos >> servers dynamically at library level, so kinit should select same server as >> SSSD. >> >> Manual page sssd_krb5_locator_plugin says: >> If the environment variable SSSD_KRB5_LOCATOR_DEBUG is set to any value >> debug messages will be sent to stderr. >> >> You can execute >> SSSD_KRB5_LOCATOR_DEBUG=1 kinit ... > Hello, > > [root@ipaclient ~]# SSSD_KRB5_LOCATOR_DEBUG=1 kinit mike > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[2] locate_service[1] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[2] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[1] locate_service[1] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[1] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] open failed [2][No such file or directory]. > [sssd_krb5_locator] reading kpasswd address failed, using kdc address. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[2] socktype[2] locate_service[2] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[2] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > Password for mike@MPLS.LOCAL: > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] open failed [2][No such file or directory]. > [sssd_krb5_locator] reading kpasswd address failed, using kdc address. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[2] locate_service[2] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[2] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] open failed [2][No such file or directory]. > [sssd_krb5_locator] reading kpasswd address failed, using kdc address. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[1] locate_service[2] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[1] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > > [root@ipaserver2 ~]ifdown eth0 # NOTE: ipaserver2 is 172.16.112.8 > > [root@ipaclient ~]# SSSD_KRB5_LOCATOR_DEBUG=1 kinit mike > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[2] locate_service[1] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[2] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > [sssd_krb5_locator] sssd_krb5_locator_init called > [sssd_krb5_locator] Found [172.16.112.8] in > [/var/lib/sss/pubconf/kdcinfo.MPLS.LOCAL]. > [sssd_krb5_locator] sssd_realm[MPLS.LOCAL] requested realm[MPLS.LOCAL] > family[0] socktype[1] locate_service[1] > [sssd_krb5_locator] addr[172.16.112.8:88] family[2] socktype[1] > [sssd_krb5_locator] [172.16.112.8] used > [sssd_krb5_locator] sssd_krb5_locator_close called > kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting initial > credentials
Jakub, does this make sense to you? > Thanks, > Mike > > > _______________________________________________ > Freeipa-users mailing list > Freeipa-users@redhat.com > https://www.redhat.com/mailman/listinfo/freeipa-users -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. ------------------------------- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ _______________________________________________ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users