On Mon, Nov 7, 2016 at 4:35 PM, Alexey Serbin <aser...@cloudera.com> wrote:
> Ah, I see -- thank you for the clarification. > > BTW, could 'ignore_acceptor_hostname' parameter help there? Like adding > the following into the krb5.conf: > > [libdefaults] > ignore_acceptor_hostname = true > We've actually already got that there. But it doesn't help -- the issue is that the client can't figure out what realm it's connecting to in krb 1.10 > > > Best regards, > > Alexey > > On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon <t...@cloudera.com> wrote: > > > On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin <aser...@cloudera.com> > > wrote: > > > > > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1 > > > address. > > > > > > > The issue is the actual Kudu servers, not the KDC. The KDC is on > 127.0.0.1, > > but we have service principals like kudu/127.2.3.4@TESTREALM that are > > causing the issue. > > > > -Todd > > > > > > > > > > > > > Best regards, > > > > > > Alexey > > > > > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon <t...@cloudera.com> wrote: > > > > > > > FWIW it looks like there's already some code out there that can do > the > > > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html > > > > > > > > > > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon <t...@cloudera.com> > wrote: > > > > > > > > > Hey folks > > > > > > > > > > I've been looking into why our kerberos-dependent tests are failing > > on > > > > el6 > > > > > and it looks like it will be unfortunately difficult to fix. > > > > > > > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically > > > create > > > > > the directory for a DIR:<path> type ticket cache. That one was easy > > to > > > > fix > > > > > and got minikdc-test passing. > > > > > > > > > > The more insidious issue, though, is the following: > > > > > In miniclusters we start daemons on weird local IP addresses > > > (127.x.y.z) > > > > > which don't have any corresponding domain names. So, we have > > configured > > > > the > > > > > MiniKDC to disable reverse DNS, and are creating service principals > > > kudu/ > > > > > 127.x....@krbtest.com. This works great on krb5-1.12. > > > > > > > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/ > > > > > Ticket/Display.html?id=7603) which is preventing this from working > on > > > > > el6. When 'rdns = false', Kerberos is unable to figure out which > > realm > > > > the > > > > > server is in (and doesn't fall back to the default realm). So, our > > > > > connections are failing with: "Cannot determine realm for numeric > > host > > > > > address". > > > > > > > > > > If we change to 'rdns = true', then it will first try to reverse > > lookup > > > > > the weird loopback, and then when it fails, still give the same > error > > > > about > > > > > numeric hosts. > > > > > > > > > > It's possible there's some way to explicitly set the target realm, > > but > > > if > > > > > so I can't seem to find it. So, the only workarounds I can think of > > > are: > > > > > > > > > > 1) Use a local build of krb5 1.12 or later on el6 for test > purposes. > > I > > > > > verified that if I built krb5 1.14 and put its libraries in > > > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding > binaries, > > > the > > > > > tests passed fine. > > > > > > > > > > The downside here is that we would lose test coverage of the > library > > > > > version actually installed on a lot of end-user systems. So another > > > > option > > > > > might be to test against a patched version of 1.10 (with only the > fix > > > for > > > > > this numeric hostname issue). > > > > > > > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS > entries > > > for > > > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that > > > nature. > > > > > We'd have to do this via LD_PRELOAD probably, which is a bummer, > but > > I > > > > > can't see any other way to override the resolver on a per-user > basis > > > > > ($HOSTALIASES only does forward DNS). > > > > > > > > > > Any other ideas? > > > > > > > > > > -Todd > > > > > -- > > > > > Todd Lipcon > > > > > Software Engineer, Cloudera > > > > > > > > > > > > > > > > > > > > > -- > > > > Todd Lipcon > > > > Software Engineer, Cloudera > > > > > > > > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
