Re: Failing krb5-related tests on el6

[Todd Lipcon]

Here's one more workaround which might actually be simplest: it seems like
if we LD_PRELOAD (or otherwise override) krb5_get_host_realm(), we can
detect this case and return our default KRBTEST.COM realm.

I'll give that one a try and see if it works, since it's only 10-15 lines
of code vs importing new dependencies/build complexity.

-Todd

On Mon, Nov 7, 2016 at 6:49 PM, Todd Lipcon  wrote:

> On Mon, Nov 7, 2016 at 4:35 PM, Alexey Serbin 
> wrote:
>
>> Ah, I see -- thank you for the clarification.
>>
>> BTW, could 'ignore_acceptor_hostname' parameter help there?  Like adding
>> the following into the krb5.conf:
>>
>> [libdefaults]
>>   ignore_acceptor_hostname = true
>>
>
> We've actually already got that there. But it doesn't help -- the issue is
> that the client can't figure out what realm it's connecting to in krb 1.10
>
>
>>
>>
>> Best regards,
>>
>> Alexey
>>
>> On Mon, Nov 7, 2016 at 3:42 PM, Todd Lipcon  wrote:
>>
>> > On Mon, Nov 7, 2016 at 3:37 PM, Alexey Serbin 
>> > wrote:
>> >
>> > > May be, as a workaround we could run our mini_kdc only on at 127.0.0.1
>> > > address.
>> > >
>> >
>> > The issue is the actual Kudu servers, not the KDC. The KDC is on
>> 127.0.0.1,
>> > but we have service principals like kudu/127.2.3.4@TESTREALM that are
>> > causing the issue.
>> >
>> > -Todd
>> >
>> >
>> > >
>> > >
>> > > Best regards,
>> > >
>> > > Alexey
>> > >
>> > > On Sun, Nov 6, 2016 at 9:57 PM, Todd Lipcon 
>> wrote:
>> > >
>> > > > FWIW it looks like there's already some code out there that can do
>> the
>> > > > appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html
>> > > >
>> > > >
>> > > > On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon 
>> wrote:
>> > > >
>> > > > > Hey folks
>> > > > >
>> > > > > I've been looking into why our kerberos-dependent tests are
>> failing
>> > on
>> > > > el6
>> > > > > and it looks like it will be unfortunately difficult to fix.
>> > > > >
>> > > > > The first issue was that krb5 1.10 (on el6) doesn't automatically
>> > > create
>> > > > > the directory for a DIR: type ticket cache. That one was
>> easy
>> > to
>> > > > fix
>> > > > > and got minikdc-test passing.
>> > > > >
>> > > > > The more insidious issue, though, is the following:
>> > > > > In miniclusters we start daemons on weird local IP addresses
>> > > (127.x.y.z)
>> > > > > which don't have any corresponding domain names. So, we have
>> > configured
>> > > > the
>> > > > > MiniKDC to disable reverse DNS, and are creating service
>> principals
>> > > kudu/
>> > > > > 127.x@krbtest.com. This works great on krb5-1.12.
>> > > > >
>> > > > > However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/
>> > > > > Ticket/Display.html?id=7603) which is preventing this from
>> working on
>> > > > > el6. When 'rdns = false', Kerberos is unable to figure out which
>> > realm
>> > > > the
>> > > > > server is in (and doesn't fall back to the default realm). So, our
>> > > > > connections are failing with: "Cannot determine realm for numeric
>> > host
>> > > > > address".
>> > > > >
>> > > > > If we change to 'rdns = true', then it will first try to reverse
>> > lookup
>> > > > > the weird loopback, and then when it fails, still give the same
>> error
>> > > > about
>> > > > > numeric hosts.
>> > > > >
>> > > > > It's possible there's some way to explicitly set the target realm,
>> > but
>> > > if
>> > > > > so I can't seem to find it. So, the only workarounds I can think
>> of
>> > > are:
>> > > > >
>> > > > > 1) Use a local build of krb5 1.12 or later on el6 for test
>> purposes.
>> > I
>> > > > > verified that if I built krb5 1.14 and put its libraries in
>> > > > > LD_LIBRARY_PATH, and modified MiniKDC to use corresponding
>> binaries,
>> > > the
>> > > > > tests passed fine.
>> > > > >
>> > > > > The downside here is that we would lose test coverage of the
>> library
>> > > > > version actually installed on a lot of end-user systems. So
>> another
>> > > > option
>> > > > > might be to test against a patched version of 1.10 (with only the
>> fix
>> > > for
>> > > > > this numeric hostname issue).
>> > > > >
>> > > > > 2) Somehow monkey-patch getnameinfo() to provide reverse DNS
>> entries
>> > > for
>> > > > > 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that
>> > > nature.
>> > > > > We'd have to do this via LD_PRELOAD probably, which is a bummer,
>> but
>> > I
>> > > > > can't see any other way to override the resolver on a per-user
>> basis
>> > > > > ($HOSTALIASES only does forward DNS).
>> > > > >
>> > > > > Any other ideas?
>> > > > >
>> > > > > -Todd
>> > > > > --
>> > > > > Todd Lipcon
>> > > > > Software Engineer, Cloudera
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Todd Lipcon
>> > > > Software Engineer, Cloudera
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Todd Lipcon
>> > Software Engineer, Cloudera
>> >
>>
>
>
>
> --
> Todd Lipcon
> 

Re: [ANNOUNCE] Jordan Birdsell joining the Kudu PMC

[Brock Noland]

Congratulations and thanks for your hard work!!

On Monday, November 7, 2016, Todd Lipcon  wrote:

> Hi Kudu community,
>
> Today I'm very pleased to announce that the PMC has voted to add Jordan
> Birdsell as a committer and PMC member on the Apache Kudu project!
>
> Jordan began contributing to Kudu a few months ago and since then has
> amassed a great number of contributions. He has almost single-handedly
> brought the Python client up to feature parity with the Java and C++
> clients, and has also been actively helping other new users as they get
> started with Kudu.
>
> Please join me in congratulating Jordan!
>
> -Todd
>

Re: Failing krb5-related tests on el6

[Todd Lipcon]

FWIW it looks like there's already some code out there that can do the
appropriate "fake DNS" wrapping: https://cwrap.org/nss_wrapper.html


On Sun, Nov 6, 2016 at 9:13 PM, Todd Lipcon  wrote:

> Hey folks
>
> I've been looking into why our kerberos-dependent tests are failing on el6
> and it looks like it will be unfortunately difficult to fix.
>
> The first issue was that krb5 1.10 (on el6) doesn't automatically create
> the directory for a DIR: type ticket cache. That one was easy to fix
> and got minikdc-test passing.
>
> The more insidious issue, though, is the following:
> In miniclusters we start daemons on weird local IP addresses (127.x.y.z)
> which don't have any corresponding domain names. So, we have configured the
> MiniKDC to disable reverse DNS, and are creating service principals kudu/
> 127.x@krbtest.com. This works great on krb5-1.12.
>
> However, there's a bug in krb5 1.10 (http://krbdev.mit.edu/rt/
> Ticket/Display.html?id=7603) which is preventing this from working on
> el6. When 'rdns = false', Kerberos is unable to figure out which realm the
> server is in (and doesn't fall back to the default realm). So, our
> connections are failing with: "Cannot determine realm for numeric host
> address".
>
> If we change to 'rdns = true', then it will first try to reverse lookup
> the weird loopback, and then when it fails, still give the same error about
> numeric hosts.
>
> It's possible there's some way to explicitly set the target realm, but if
> so I can't seem to find it. So, the only workarounds I can think of are:
>
> 1) Use a local build of krb5 1.12 or later on el6 for test purposes. I
> verified that if I built krb5 1.14 and put its libraries in
> LD_LIBRARY_PATH, and modified MiniKDC to use corresponding binaries, the
> tests passed fine.
>
> The downside here is that we would lose test coverage of the library
> version actually installed on a lot of end-user systems. So another option
> might be to test against a patched version of 1.10 (with only the fix for
> this numeric hostname issue).
>
> 2) Somehow monkey-patch getnameinfo() to provide reverse DNS entries for
> 127.x.y.z mapping to '127-x-y-z.kudu.local' or something of that nature.
> We'd have to do this via LD_PRELOAD probably, which is a bummer, but I
> can't see any other way to override the resolver on a per-user basis
> ($HOSTALIASES only does forward DNS).
>
> Any other ideas?
>
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera