Re: Failing krb5-related tests on el6
oops, sorry -- that's already there. On Mon, Nov 7, 2016 at 4:35 PM, Alexey Serbin wrote: > Ah, I see -- thank you for the clarification. > > BTW, could 'ignore_acceptor_hostname' parameter help there? Like adding > the following into the krb5.conf: > > [libdefaults] > ignore_acceptor_hostname = true > > > Best regards, > > Alexey > > On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon wrote: > >> On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin >> wrote: >> >> > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 >> > address. >> > >> >> The issue is the actual Kudu servers, not the KDC. The KDC is on >> 127.0.0.1, >> but we have service principals like kudu/127.2.3.4@TESTREALM that are >> causing the issue. >> >> -Todd >> >> >> > >> > >> > Best regards, >> > >> > Alexey >> > >> > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon wrote: >> > >> > > FWIW it looks like there's already some code out there that can do the >> > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html >> > > >> > > >> > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon >> wrote: >> > > >> > > > Hey folks >> > > > >> > > > I've been looking into why our kerberos-dependent tests are failing >> on >> > > el6 >> > > > and it looks like it will be unfortunately difficult to fix. >> > > > >> > > > The first issue was that krb5 1.10 (on el6) doesn't automatically >> > create >> > > > the directory for a DIR: type ticket cache. That one was easy >> to >> > > fix >> > > > and got minikdc-test passing. >> > > > >> > > > The more insidious issue, though, is the following: >> > > > In miniclusters we start daemons on weird local IP addresses >> > (127.x.y.z) >> > > > which don't have any corresponding domain names. So, we have >> configured >> > > the >> > > > MiniKDC to disable reverse DNS, and are creating service principals >> > kudu/ >> > > > 127.x@krbtest.com. This works great on krb5-1.12. >> > > > >> > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ >> > > > Ticket/Display.html?id=7603) which is preventing this from working >> on >> > > > el6. When 'rdns = false', Kerberos is unable to figure out which >> realm >> > > the >> > > > server is in (and doesn't fall back to the default realm). So, our >> > > > connections are failing with: "Cannot determine realm for numeric >> host >> > > > address". >> > > > >> > > > If we change to 'rdns = true', then it will first try to reverse >> lookup >> > > > the weird loopback, and then when it fails, still give the same >> error >> > > about >> > > > numeric hosts. >> > > > >> > > > It's possible there's some way to explicitly set the target realm, >> but >> > if >> > > > so I can't seem to find it. So, the only workarounds I can think of >> > are: >> > > > >> > > > 1) Use a local build of krb5 1.12 or later on el6 for test >> purposes. I >> > > > verified that if I built krb5 1.14 and put its libraries in >> > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, >> > the >> > > > tests passed fine. >> > > > >> > > > The downside here is that we would lose test coverage of the library >> > > > version actually installed on a lot of end-user systems. So another >> > > option >> > > > might be to test against a patched version of 1.10 (with only the >> fix >> > for >> > > > this numeric hostname issue). >> > > > >> > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries >> > for >> > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that >> > nature. >> > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, >> but I >> > > > can't see any other way to override the resolver on a per-user basis >> > > > ($HOSTALIASES only does forward DNS). >> > > > >> > > > Any other ideas? >> > > > >> > > > -Todd >> > > > -- >> > > > Todd Lipcon >> > > > Software Engineer, Cloudera >> > > > >> > > >> > > >> > > >> > > -- >> > > Todd Lipcon >> > > Software Engineer, Cloudera >> > > >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > >
Re: Failing krb5-related tests on el6
Here's one more workaround which might actually be simplest: it seems like if we LD_PRELOAD (or otherwise override) krb5_get_host_realm(), we can detect this case and return our default KRBTEST.COM realm. I'll give that one a try and see if it works, since it's only 10-15 lines of code vs importing new dependencies/build complexity. -Todd On Mon, Nov 7, 2016 at 6:49 PM, Todd Lipcon wrote: > On Mon, Nov 7, 2016 at 4:35 PM, Alexey Serbin > wrote: > >> Ah, I see -- thank you for the clarification. >> >> BTW, could 'ignore_acceptor_hostname' parameter help there? Like adding >> the following into the krb5.conf: >> >> [libdefaults] >> ignore_acceptor_hostname = true >> > > We've actually already got that there. But it doesn't help -- the issue is > that the client can't figure out what realm it's connecting to in krb 1.10 > > >> >> >> Best regards, >> >> Alexey >> >> On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon wrote: >> >> > On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin >> > wrote: >> > >> > > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 >> > > address. >> > > >> > >> > The issue is the actual Kudu servers, not the KDC. The KDC is on >> 127.0.0.1, >> > but we have service principals like kudu/127.2.3.4@TESTREALM that are >> > causing the issue. >> > >> > -Todd >> > >> > >> > > >> > > >> > > Best regards, >> > > >> > > Alexey >> > > >> > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon >> wrote: >> > > >> > > > FWIW it looks like there's already some code out there that can do >> the >> > > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html >> > > > >> > > > >> > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon >> wrote: >> > > > >> > > > > Hey folks >> > > > > >> > > > > I've been looking into why our kerberos-dependent tests are >> failing >> > on >> > > > el6 >> > > > > and it looks like it will be unfortunately difficult to fix. >> > > > > >> > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically >> > > create >> > > > > the directory for a DIR: type ticket cache. That one was >> easy >> > to >> > > > fix >> > > > > and got minikdc-test passing. >> > > > > >> > > > > The more insidious issue, though, is the following: >> > > > > In miniclusters we start daemons on weird local IP addresses >> > > (127.x.y.z) >> > > > > which don't have any corresponding domain names. So, we have >> > configured >> > > > the >> > > > > MiniKDC to disable reverse DNS, and are creating service >> principals >> > > kudu/ >> > > > > 127.x@krbtest.com. This works great on krb5-1.12. >> > > > > >> > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ >> > > > > Ticket/Display.html?id=7603) which is preventing this from >> working on >> > > > > el6. When 'rdns = false', Kerberos is unable to figure out which >> > realm >> > > > the >> > > > > server is in (and doesn't fall back to the default realm). So, our >> > > > > connections are failing with: "Cannot determine realm for numeric >> > host >> > > > > address". >> > > > > >> > > > > If we change to 'rdns = true', then it will first try to reverse >> > lookup >> > > > > the weird loopback, and then when it fails, still give the same >> error >> > > > about >> > > > > numeric hosts. >> > > > > >> > > > > It's possible there's some way to explicitly set the target realm, >> > but >> > > if >> > > > > so I can't seem to find it. So, the only workarounds I can think >> of >> > > are: >> > > > > >> > > > > 1) Use a local build of krb5 1.12 or later on el6 for test >> purposes. >> > I >> > > > > verified that if I built krb5 1.14 and put its libraries in >> > > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding >> binaries, >> > > the >> > > > > tests passed fine. >> > > > > >> > > > > The downside here is that we would lose test coverage of the >> library >> > > > > version actually installed on a lot of end-user systems. So >> another >> > > > option >> > > > > might be to test against a patched version of 1.10 (with only the >> fix >> > > for >> > > > > this numeric hostname issue). >> > > > > >> > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS >> entries >> > > for >> > > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that >> > > nature. >> > > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, >> but >> > I >> > > > > can't see any other way to override the resolver on a per-user >> basis >> > > > > ($HOSTALIASES only does forward DNS). >> > > > > >> > > > > Any other ideas? >> > > > > >> > > > > -Todd >> > > > > -- >> > > > > Todd Lipcon >> > > > > Software Engineer, Cloudera >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Todd Lipcon >> > > > Software Engineer, Cloudera >> > > > >> > > >> > >> > >> > >> > -- >> > Todd Lipcon >> > Software Engineer, Cloudera >> > >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
Re: Failing krb5-related tests on el6
OK, this new workaround seems to be working on my el6 box. The next issue I ran into is that KRB5_CLIENT_KTNAME is new in 1.11, so we need to explicitly login from the keytab in order for servers to act as a client. Put a WIP patch that addresses both of these here: https://gerrit.cloudera.org/#/c/4990/ Dan's checking to see if this also addresses OSX krb5 test failures. Assuming people are OK with the general direction, will finish this up tomorrow, so we can hopefully have green tests in the 1.1 release. On Mon, Nov 7, 2016 at 7:08 PM, Todd Lipcon wrote: > Here's one more workaround which might actually be simplest: it seems like > if we LD_PRELOAD (or otherwise override) krb5_get_host_realm(), we can > detect this case and return our default KRBTEST.COM realm. > > I'll give that one a try and see if it works, since it's only 10-15 lines > of code vs importing new dependencies/build complexity. > > -Todd > > On Mon, Nov 7, 2016 at 6:49 PM, Todd Lipcon wrote: > >> On Mon, Nov 7, 2016 at 4:35 PM, Alexey Serbin >> wrote: >> >>> Ah, I see -- thank you for the clarification. >>> >>> BTW, could 'ignore_acceptor_hostname' parameter help there? Like adding >>> the following into the krb5.conf: >>> >>> [libdefaults] >>> ignore_acceptor_hostname = true >>> >> >> We've actually already got that there. But it doesn't help -- the issue >> is that the client can't figure out what realm it's connecting to in krb >> 1.10 >> >> >>> >>> >>> Best regards, >>> >>> Alexey >>> >>> On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon wrote: >>> >>> > On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin >>> > wrote: >>> > >>> > > May be, as a workaround we could run our mini_kdc only on at >>> 127.0.0.1 >>> > > address. >>> > > >>> > >>> > The issue is the actual Kudu servers, not the KDC. The KDC is on >>> 127.0.0.1, >>> > but we have service principals like kudu/127.2.3.4@TESTREALM that are >>> > causing the issue. >>> > >>> > -Todd >>> > >>> > >>> > > >>> > > >>> > > Best regards, >>> > > >>> > > Alexey >>> > > >>> > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon >>> wrote: >>> > > >>> > > > FWIW it looks like there's already some code out there that can do >>> the >>> > > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper. >>> html >>> > > > >>> > > > >>> > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon >>> wrote: >>> > > > >>> > > > > Hey folks >>> > > > > >>> > > > > I've been looking into why our kerberos-dependent tests are >>> failing >>> > on >>> > > > el6 >>> > > > > and it looks like it will be unfortunately difficult to fix. >>> > > > > >>> > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically >>> > > create >>> > > > > the directory for a DIR: type ticket cache. That one was >>> easy >>> > to >>> > > > fix >>> > > > > and got minikdc-test passing. >>> > > > > >>> > > > > The more insidious issue, though, is the following: >>> > > > > In miniclusters we start daemons on weird local IP addresses >>> > > (127.x.y.z) >>> > > > > which don't have any corresponding domain names. So, we have >>> > configured >>> > > > the >>> > > > > MiniKDC to disable reverse DNS, and are creating service >>> principals >>> > > kudu/ >>> > > > > 127.x@krbtest.com. This works great on krb5-1.12. >>> > > > > >>> > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ >>> > > > > Ticket/Display.html?id=7603) which is preventing this from >>> working on >>> > > > > el6. When 'rdns = false', Kerberos is unable to figure out which >>> > realm >>> > > > the >>> > > > > server is in (and doesn't fall back to the default realm). So, >>> our >>> > > > > connections are failing with: "Cannot determine realm for numeric >>> > host >>> > > > > address". >>> > > > > >>> > > > > If we change to 'rdns = true', then it will first try to reverse >>> > lookup >>> > > > > the weird loopback, and then when it fails, still give the same >>> error >>> > > > about >>> > > > > numeric hosts. >>> > > > > >>> > > > > It's possible there's some way to explicitly set the target >>> realm, >>> > but >>> > > if >>> > > > > so I can't seem to find it. So, the only workarounds I can think >>> of >>> > > are: >>> > > > > >>> > > > > 1) Use a local build of krb5 1.12 or later on el6 for test >>> purposes. >>> > I >>> > > > > verified that if I built krb5 1.14 and put its libraries in >>> > > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding >>> binaries, >>> > > the >>> > > > > tests passed fine. >>> > > > > >>> > > > > The downside here is that we would lose test coverage of the >>> library >>> > > > > version actually installed on a lot of end-user systems. So >>> another >>> > > > option >>> > > > > might be to test against a patched version of 1.10 (with only >>> the fix >>> > > for >>> > > > > this numeric hostname issue). >>> > > > > >>> > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS >>> entries >>> > > for >>> > > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of th
Re: Failing krb5-related tests on el6
Ah, I see -- thank you for the clarification. BTW, could 'ignore_acceptor_hostname' parameter help there? Like adding the following into the krb5.conf: [libdefaults] ignore_acceptor_hostname = true Best regards, Alexey On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon wrote: > On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin > wrote: > > > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 > > address. > > > > The issue is the actual Kudu servers, not the KDC. The KDC is on 127.0.0.1, > but we have service principals like kudu/127.2.3.4@TESTREALM that are > causing the issue. > > -Todd > > > > > > > > Best regards, > > > > Alexey > > > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon wrote: > > > > > FWIW it looks like there's already some code out there that can do the > > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html > > > > > > > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon wrote: > > > > > > > Hey folks > > > > > > > > I've been looking into why our kerberos-dependent tests are failing > on > > > el6 > > > > and it looks like it will be unfortunately difficult to fix. > > > > > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically > > create > > > > the directory for a DIR: type ticket cache. That one was easy > to > > > fix > > > > and got minikdc-test passing. > > > > > > > > The more insidious issue, though, is the following: > > > > In miniclusters we start daemons on weird local IP addresses > > (127.x.y.z) > > > > which don't have any corresponding domain names. So, we have > configured > > > the > > > > MiniKDC to disable reverse DNS, and are creating service principals > > kudu/ > > > > 127.x@krbtest.com. This works great on krb5-1.12. > > > > > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > > > > Ticket/Display.html?id=7603) which is preventing this from working on > > > > el6. When 'rdns = false', Kerberos is unable to figure out which > realm > > > the > > > > server is in (and doesn't fall back to the default realm). So, our > > > > connections are failing with: "Cannot determine realm for numeric > host > > > > address". > > > > > > > > If we change to 'rdns = true', then it will first try to reverse > lookup > > > > the weird loopback, and then when it fails, still give the same error > > > about > > > > numeric hosts. > > > > > > > > It's possible there's some way to explicitly set the target realm, > but > > if > > > > so I can't seem to find it. So, the only workarounds I can think of > > are: > > > > > > > > 1) Use a local build of krb5 1.12 or later on el6 for test purposes. > I > > > > verified that if I built krb5 1.14 and put its libraries in > > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, > > the > > > > tests passed fine. > > > > > > > > The downside here is that we would lose test coverage of the library > > > > version actually installed on a lot of end-user systems. So another > > > option > > > > might be to test against a patched version of 1.10 (with only the fix > > for > > > > this numeric hostname issue). > > > > > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries > > for > > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that > > nature. > > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, but > I > > > > can't see any other way to override the resolver on a per-user basis > > > > ($HOSTALIASES only does forward DNS). > > > > > > > > Any other ideas? > > > > > > > > -Todd > > > > -- > > > > Todd Lipcon > > > > Software Engineer, Cloudera > > > > > > > > > > > > > > > > -- > > > Todd Lipcon > > > Software Engineer, Cloudera > > > > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
Re: [ANNOUNCE] Jordan Birdsell joining the Kudu PMC
Congratulations and thanks for your hard work!! On Monday, November 7, 2016, Todd Lipcon wrote: > Hi Kudu community, > > Today I'm very pleased to announce that the PMC has voted to add Jordan > Birdsell as a committer and PMC member on the Apache Kudu project! > > Jordan began contributing to Kudu a few months ago and since then has > amassed a great number of contributions. He has almost single-handedly > brought the Python client up to feature parity with the Java and C++ > clients, and has also been actively helping other new users as they get > started with Kudu. > > Please join me in congratulating Jordan! > > -Todd >
Re: Failing krb5-related tests on el6
On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin wrote: > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 > address. > The issue is the actual Kudu servers, not the KDC. The KDC is on 127.0.0.1, but we have service principals like kudu/127.2.3.4@TESTREALM that are causing the issue. -Todd > > > Best regards, > > Alexey > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon wrote: > > > FWIW it looks like there's already some code out there that can do the > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html > > > > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon wrote: > > > > > Hey folks > > > > > > I've been looking into why our kerberos-dependent tests are failing on > > el6 > > > and it looks like it will be unfortunately difficult to fix. > > > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically > create > > > the directory for a DIR: type ticket cache. That one was easy to > > fix > > > and got minikdc-test passing. > > > > > > The more insidious issue, though, is the following: > > > In miniclusters we start daemons on weird local IP addresses > (127.x.y.z) > > > which don't have any corresponding domain names. So, we have configured > > the > > > MiniKDC to disable reverse DNS, and are creating service principals > kudu/ > > > 127.x@krbtest.com. This works great on krb5-1.12. > > > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > > > Ticket/Display.html?id=7603) which is preventing this from working on > > > el6. When 'rdns = false', Kerberos is unable to figure out which realm > > the > > > server is in (and doesn't fall back to the default realm). So, our > > > connections are failing with: "Cannot determine realm for numeric host > > > address". > > > > > > If we change to 'rdns = true', then it will first try to reverse lookup > > > the weird loopback, and then when it fails, still give the same error > > about > > > numeric hosts. > > > > > > It's possible there's some way to explicitly set the target realm, but > if > > > so I can't seem to find it. So, the only workarounds I can think of > are: > > > > > > 1) Use a local build of krb5 1.12 or later on el6 for test purposes. I > > > verified that if I built krb5 1.14 and put its libraries in > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, > the > > > tests passed fine. > > > > > > The downside here is that we would lose test coverage of the library > > > version actually installed on a lot of end-user systems. So another > > option > > > might be to test against a patched version of 1.10 (with only the fix > for > > > this numeric hostname issue). > > > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries > for > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that > nature. > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, but I > > > can't see any other way to override the resolver on a per-user basis > > > ($HOSTALIASES only does forward DNS). > > > > > > Any other ideas? > > > > > > -Todd > > > -- > > > Todd Lipcon > > > Software Engineer, Cloudera > > > > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
[ANNOUNCE] Jordan Birdsell joining the Kudu PMC
Hi Kudu community, Today I'm very pleased to announce that the PMC has voted to add Jordan Birdsell as a committer and PMC member on the Apache Kudu project! Jordan began contributing to Kudu a few months ago and since then has amassed a great number of contributions. He has almost single-handedly brought the Python client up to feature parity with the Java and C++ clients, and has also been actively helping other new users as they get started with Kudu. Please join me in congratulating Jordan! -Todd
Re: Failing krb5-related tests on el6
May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 address. Best regards, Alexey On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon wrote: > FWIW it looks like there's already some code out there that can do the > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon wrote: > > > Hey folks > > > > I've been looking into why our kerberos-dependent tests are failing on > el6 > > and it looks like it will be unfortunately difficult to fix. > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically create > > the directory for a DIR: type ticket cache. That one was easy to > fix > > and got minikdc-test passing. > > > > The more insidious issue, though, is the following: > > In miniclusters we start daemons on weird local IP addresses (127.x.y.z) > > which don't have any corresponding domain names. So, we have configured > the > > MiniKDC to disable reverse DNS, and are creating service principals kudu/ > > 127.x@krbtest.com. This works great on krb5-1.12. > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > > Ticket/Display.html?id=7603) which is preventing this from working on > > el6. When 'rdns = false', Kerberos is unable to figure out which realm > the > > server is in (and doesn't fall back to the default realm). So, our > > connections are failing with: "Cannot determine realm for numeric host > > address". > > > > If we change to 'rdns = true', then it will first try to reverse lookup > > the weird loopback, and then when it fails, still give the same error > about > > numeric hosts. > > > > It's possible there's some way to explicitly set the target realm, but if > > so I can't seem to find it. So, the only workarounds I can think of are: > > > > 1) Use a local build of krb5 1.12 or later on el6 for test purposes. I > > verified that if I built krb5 1.14 and put its libraries in > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, the > > tests passed fine. > > > > The downside here is that we would lose test coverage of the library > > version actually installed on a lot of end-user systems. So another > option > > might be to test against a patched version of 1.10 (with only the fix for > > this numeric hostname issue). > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries for > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that nature. > > We'd have to do this via LD_PRELOAD probably, which is a bummer, but I > > can't see any other way to override the resolver on a per-user basis > > ($HOSTALIASES only does forward DNS). > > > > Any other ideas? > > > > -Todd > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
Re: Failing krb5-related tests on el6
FWIW it looks like there's already some code out there that can do the appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon wrote: > Hey folks > > I've been looking into why our kerberos-dependent tests are failing on el6 > and it looks like it will be unfortunately difficult to fix. > > The first issue was that krb5 1.10 (on el6) doesn't automatically create > the directory for a DIR: type ticket cache. That one was easy to fix > and got minikdc-test passing. > > The more insidious issue, though, is the following: > In miniclusters we start daemons on weird local IP addresses (127.x.y.z) > which don't have any corresponding domain names. So, we have configured the > MiniKDC to disable reverse DNS, and are creating service principals kudu/ > 127.x@krbtest.com. This works great on krb5-1.12. > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > Ticket/Display.html?id=7603) which is preventing this from working on > el6. When 'rdns = false', Kerberos is unable to figure out which realm the > server is in (and doesn't fall back to the default realm). So, our > connections are failing with: "Cannot determine realm for numeric host > address". > > If we change to 'rdns = true', then it will first try to reverse lookup > the weird loopback, and then when it fails, still give the same error about > numeric hosts. > > It's possible there's some way to explicitly set the target realm, but if > so I can't seem to find it. So, the only workarounds I can think of are: > > 1) Use a local build of krb5 1.12 or later on el6 for test purposes. I > verified that if I built krb5 1.14 and put its libraries in > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, the > tests passed fine. > > The downside here is that we would lose test coverage of the library > version actually installed on a lot of end-user systems. So another option > might be to test against a patched version of 1.10 (with only the fix for > this numeric hostname issue). > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries for > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that nature. > We'd have to do this via LD_PRELOAD probably, which is a bummer, but I > can't see any other way to override the resolver on a per-user basis > ($HOSTALIASES only does forward DNS). > > Any other ideas? > > -Todd > -- > Todd Lipcon > Software Engineer, Cloudera > -- Todd Lipcon Software Engineer, Cloudera
