Re: NIFI Multiple Kerberos configuration

[Jeff]

I'll have to set up a test this week and see if I can reproduce this.  If
you'd like, you can file a JIRA [1] with sanitized details of your
krb5.conf and an example flow.

[1] https://issues.apache.org/jira/projects/NIFI/issues

On Sat, Jun 23, 2018 at 3:48 AM Hiroaki Miyanaga 
wrote:

> I tried a similar case last week and it could not access to both cluster at
> the same time.
>
> Try to connect kafka and hadoop managed by their own KDCs.
> I set both KDCs in realms section of krb5.conf.
> But NiFi looks using default realms in krb5.conf.
>
> I find a similar ticket.
>
> https://community.hortonworks.com/questions/149808/unable-to-connect-to-two-kdcs-from-nifi.html
>
>
> On Sat, Jun 23, 2018 at 4:01 AM, Jeff  wrote:
>
> > You can do this by configuring a realm for each KDC to krb5.conf.
> >
> > On Fri, Jun 22, 2018 at 10:37 AM Bryan Bende  wrote:
> >
> > > Java assumes there is one krb5.conf file loaded by the JVM. It looks
> > > for the system property java.security.krb5.conf or falls back to
> > > looking in well-known locations, but still only expects one [1].
> > >
> > > NiFi requires you to set the location in nifi.properties and uses that
> > > value to set the system property above.
> > >
> > > There may be a way to create a single krb5.conf with multiple KDCs,
> > > but I'm not sure exactly how to do it.
> > >
> > > [1]
> > > https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/
> > tutorials/KerberosReq.html
> > >
> > > On Fri, Jun 22, 2018 at 10:10 AM, Milan Das  wrote:
> > > > The problem is krb5.conf. There are two different krb5.conf with two
> > > different kdc server.
> > > > Regards,
> > > > Milan Das
> > > >
> > > > On 6/22/18, 2:04 AM, "Koji Kawamura" 
> wrote:
> > > >
> > > > Hi Milan,
> > > >
> > > > I haven't tried myself, but since NiFi has Kerberos configuration
> > per
> > > > Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able
> > to
> > > > connect multiple Hadoop clusters accessed by different Kerberos
> > > principals
> > > > and keytabs. Principals must resolve domain (realm) correctly, if
> > > both
> > > > Hadoop cluster use the same domain such as 'EXAMPLE.COM', then
> it
> > > will be
> > > > problematic for NiFi to find the right KDC server.
> > > >
> > > > Thanks,
> > > > Koji
> > > >
> > > > On Fri, Jun 22, 2018 at 12:23 AM, Milan Das 
> > > wrote:
> > > >
> > > > > Hello Team,
> > > > >
> > > > > I have very unique problem. We are integration two kerberized
> > > haddop
> > > > > system and they have their own Kerbros setup.
> > > > >
> > > > > Is it possible to two Kerberos kdc configurations in NIFI ?
> > > Integration is
> > > > > Kafka from one Hadoop to Kafka on 2nd Hadoop.
> > > > >
> > > > > Really appreciate any thoughts.
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > > Milan Das
> > > > >
> > > > >
> > > > >
> > > > > [image: ograph]
> > > > >
> > > > > *Milan Das*
> > > > > Sr. System Architect
> > > > >
> > > > > email: m...@interset.com
> > > > > mobile: +1 678 216 5660 <(678)%20216-5660> <(678)%20216-5660>
> > > > >
> > > > > [image: edIn icon] 
> > > > >
> > > > > www.interset.com
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> >
>

Re: NIFI Multiple Kerberos configuration

[Hiroaki Miyanaga]

I tried a similar case last week and it could not access to both cluster at
the same time.

Try to connect kafka and hadoop managed by their own KDCs.
I set both KDCs in realms section of krb5.conf.
But NiFi looks using default realms in krb5.conf.

I find a similar ticket.
https://community.hortonworks.com/questions/149808/unable-to-connect-to-two-kdcs-from-nifi.html


On Sat, Jun 23, 2018 at 4:01 AM, Jeff  wrote:

> You can do this by configuring a realm for each KDC to krb5.conf.
>
> On Fri, Jun 22, 2018 at 10:37 AM Bryan Bende  wrote:
>
> > Java assumes there is one krb5.conf file loaded by the JVM. It looks
> > for the system property java.security.krb5.conf or falls back to
> > looking in well-known locations, but still only expects one [1].
> >
> > NiFi requires you to set the location in nifi.properties and uses that
> > value to set the system property above.
> >
> > There may be a way to create a single krb5.conf with multiple KDCs,
> > but I'm not sure exactly how to do it.
> >
> > [1]
> > https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/
> tutorials/KerberosReq.html
> >
> > On Fri, Jun 22, 2018 at 10:10 AM, Milan Das  wrote:
> > > The problem is krb5.conf. There are two different krb5.conf with two
> > different kdc server.
> > > Regards,
> > > Milan Das
> > >
> > > On 6/22/18, 2:04 AM, "Koji Kawamura"  wrote:
> > >
> > > Hi Milan,
> > >
> > > I haven't tried myself, but since NiFi has Kerberos configuration
> per
> > > Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able
> to
> > > connect multiple Hadoop clusters accessed by different Kerberos
> > principals
> > > and keytabs. Principals must resolve domain (realm) correctly, if
> > both
> > > Hadoop cluster use the same domain such as 'EXAMPLE.COM', then it
> > will be
> > > problematic for NiFi to find the right KDC server.
> > >
> > > Thanks,
> > > Koji
> > >
> > > On Fri, Jun 22, 2018 at 12:23 AM, Milan Das 
> > wrote:
> > >
> > > > Hello Team,
> > > >
> > > > I have very unique problem. We are integration two kerberized
> > haddop
> > > > system and they have their own Kerbros setup.
> > > >
> > > > Is it possible to two Kerberos kdc configurations in NIFI ?
> > Integration is
> > > > Kafka from one Hadoop to Kafka on 2nd Hadoop.
> > > >
> > > > Really appreciate any thoughts.
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Milan Das
> > > >
> > > >
> > > >
> > > > [image: ograph]
> > > >
> > > > *Milan Das*
> > > > Sr. System Architect
> > > >
> > > > email: m...@interset.com
> > > > mobile: +1 678 216 5660 <(678)%20216-5660>
> > > >
> > > > [image: edIn icon] 
> > > >
> > > > www.interset.com
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
>

Re: NIFI Multiple Kerberos configuration

[Jeff]

You can do this by configuring a realm for each KDC to krb5.conf.

On Fri, Jun 22, 2018 at 10:37 AM Bryan Bende  wrote:

> Java assumes there is one krb5.conf file loaded by the JVM. It looks
> for the system property java.security.krb5.conf or falls back to
> looking in well-known locations, but still only expects one [1].
>
> NiFi requires you to set the location in nifi.properties and uses that
> value to set the system property above.
>
> There may be a way to create a single krb5.conf with multiple KDCs,
> but I'm not sure exactly how to do it.
>
> [1]
> https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html
>
> On Fri, Jun 22, 2018 at 10:10 AM, Milan Das  wrote:
> > The problem is krb5.conf. There are two different krb5.conf with two
> different kdc server.
> > Regards,
> > Milan Das
> >
> > On 6/22/18, 2:04 AM, "Koji Kawamura"  wrote:
> >
> > Hi Milan,
> >
> > I haven't tried myself, but since NiFi has Kerberos configuration per
> > Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able to
> > connect multiple Hadoop clusters accessed by different Kerberos
> principals
> > and keytabs. Principals must resolve domain (realm) correctly, if
> both
> > Hadoop cluster use the same domain such as 'EXAMPLE.COM', then it
> will be
> > problematic for NiFi to find the right KDC server.
> >
> > Thanks,
> > Koji
> >
> > On Fri, Jun 22, 2018 at 12:23 AM, Milan Das 
> wrote:
> >
> > > Hello Team,
> > >
> > > I have very unique problem. We are integration two kerberized
> haddop
> > > system and they have their own Kerbros setup.
> > >
> > > Is it possible to two Kerberos kdc configurations in NIFI ?
> Integration is
> > > Kafka from one Hadoop to Kafka on 2nd Hadoop.
> > >
> > > Really appreciate any thoughts.
> > >
> > >
> > >
> > > Regards,
> > >
> > > Milan Das
> > >
> > >
> > >
> > > [image: ograph]
> > >
> > > *Milan Das*
> > > Sr. System Architect
> > >
> > > email: m...@interset.com
> > > mobile: +1 678 216 5660 <(678)%20216-5660>
> > >
> > > [image: edIn icon] 
> > >
> > > www.interset.com
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
>

Re: NIFI Multiple Kerberos configuration

[Bryan Bende]

Java assumes there is one krb5.conf file loaded by the JVM. It looks
for the system property java.security.krb5.conf or falls back to
looking in well-known locations, but still only expects one [1].

NiFi requires you to set the location in nifi.properties and uses that
value to set the system property above.

There may be a way to create a single krb5.conf with multiple KDCs,
but I'm not sure exactly how to do it.

[1] 
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html

On Fri, Jun 22, 2018 at 10:10 AM, Milan Das  wrote:
> The problem is krb5.conf. There are two different krb5.conf with two 
> different kdc server.
> Regards,
> Milan Das
>
> On 6/22/18, 2:04 AM, "Koji Kawamura"  wrote:
>
> Hi Milan,
>
> I haven't tried myself, but since NiFi has Kerberos configuration per
> Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able to
> connect multiple Hadoop clusters accessed by different Kerberos principals
> and keytabs. Principals must resolve domain (realm) correctly, if both
> Hadoop cluster use the same domain such as 'EXAMPLE.COM', then it will be
> problematic for NiFi to find the right KDC server.
>
> Thanks,
> Koji
>
> On Fri, Jun 22, 2018 at 12:23 AM, Milan Das  wrote:
>
> > Hello Team,
> >
> > I have very unique problem. We are integration two kerberized haddop
> > system and they have their own Kerbros setup.
> >
> > Is it possible to two Kerberos kdc configurations in NIFI ? Integration 
> is
> > Kafka from one Hadoop to Kafka on 2nd Hadoop.
> >
> > Really appreciate any thoughts.
> >
> >
> >
> > Regards,
> >
> > Milan Das
> >
> >
> >
> > [image: ograph]
> >
> > *Milan Das*
> > Sr. System Architect
> >
> > email: m...@interset.com
> > mobile: +1 678 216 5660
> >
> > [image: edIn icon] 
> >
> > www.interset.com
> >
> >
> >
> >
> >
>
>
>

Re: NIFI Multiple Kerberos configuration

[Milan Das]

The problem is krb5.conf. There are two different krb5.conf with two different 
kdc server. 
Regards,
Milan Das

On 6/22/18, 2:04 AM, "Koji Kawamura"  wrote:

Hi Milan,

I haven't tried myself, but since NiFi has Kerberos configuration per
Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able to
connect multiple Hadoop clusters accessed by different Kerberos principals
and keytabs. Principals must resolve domain (realm) correctly, if both
Hadoop cluster use the same domain such as 'EXAMPLE.COM', then it will be
problematic for NiFi to find the right KDC server.

Thanks,
Koji

On Fri, Jun 22, 2018 at 12:23 AM, Milan Das  wrote:

> Hello Team,
>
> I have very unique problem. We are integration two kerberized haddop
> system and they have their own Kerbros setup.
>
> Is it possible to two Kerberos kdc configurations in NIFI ? Integration is
> Kafka from one Hadoop to Kafka on 2nd Hadoop.
>
> Really appreciate any thoughts.
>
>
>
> Regards,
>
> Milan Das
>
>
>
> [image: ograph]
>
> *Milan Das*
> Sr. System Architect
>
> email: m...@interset.com
> mobile: +1 678 216 5660
>
> [image: edIn icon] 
>
> www.interset.com
>
>
>
>
>

Re: NIFI Multiple Kerberos configuration

[Koji Kawamura]

Hi Milan,

I haven't tried myself, but since NiFi has Kerberos configuration per
Processor instance, e.g. ListHDFS or PutHDFS, NiFi should be able to
connect multiple Hadoop clusters accessed by different Kerberos principals
and keytabs. Principals must resolve domain (realm) correctly, if both
Hadoop cluster use the same domain such as 'EXAMPLE.COM', then it will be
problematic for NiFi to find the right KDC server.

Thanks,
Koji

On Fri, Jun 22, 2018 at 12:23 AM, Milan Das  wrote:

> Hello Team,
>
> I have very unique problem. We are integration two kerberized haddop
> system and they have their own Kerbros setup.
>
> Is it possible to two Kerberos kdc configurations in NIFI ? Integration is
> Kafka from one Hadoop to Kafka on 2nd Hadoop.
>
> Really appreciate any thoughts.
>
>
>
> Regards,
>
> Milan Das
>
>
>
> [image: ograph]
>
> *Milan Das*
> Sr. System Architect
>
> email: m...@interset.com
> mobile: +1 678 216 5660
>
> [image: edIn icon] 
>
> www.interset.com
>
>
>
>
>

NIFI Multiple Kerberos configuration

[Milan Das]

Hello Team,

I have very unique problem. We are integration two kerberized haddop system and 
they have their own Kerbros setup.

Is it possible to two Kerberos kdc configurations in NIFI ? Integration is 
Kafka from one Hadoop to Kafka on 2nd Hadoop.

Really appreciate any thoughts.

 

Regards,

Milan Das

 

Milan Das
Sr. System Architect
email: m...@interset.com
mobile: +1 678 216 5660
www.interset.com