Can someone give some context on this? I can volunteer myself and try working on this.
Chen On Thu, Jul 2, 2015 at 4:29 AM, Qi Fu <q...@talend.com> wrote: > Hi Yi & Yan, > > Many thanks for your information. I have created a jira for this: > https://issues.apache.org/jira/browse/SAMZA-727 > I'm willing to test it if someone can work on this. > > > -Qi > > ________________________________________ > From: Yi Pan <nickpa...@gmail.com> > Sent: Thursday, July 2, 2015 1:38 AM > To: dev@samza.apache.org > Subject: Re: Security on YARN > > Hi, Yan, > > Your memory serves as well as mine. :) I remember that Chris and I > discussed this Kerberos ticket expiration issue when we were brain storming > on how to access HDFS data in Samza. At high-level, what happens is that > the Kerberos ticket to access a secured Hadoop cluster is issued to Samza > containers at the job start time, and will expire later. For a long-running > Samza job, it does not work. We will need a way to refresh the Kerberos > ticket periodically, which is not supported yet. Chris probably can chime > in with more details. > > -Yi > > On Wed, Jul 1, 2015 at 4:08 PM, Yan Fang <yanfang...@gmail.com> wrote: > > > Hi Qi, > > > > I think this is caused by the fact that Samza currently does not support > > Yarn with Kerberos. Feel free to open a ticket for this feature. > > > > But if my memory serves, there was an issue mentioned about the Kerberos. > > Seems when the Kerberos ticket expires, Samza will have some issues? Can > > not find the resource. Anyone remember this? > > > > Cheers, > > > > Fang, Yan > > yanfang...@gmail.com > > > > On Wed, Jul 1, 2015 at 3:41 AM, Qi Fu <q...@talend.com> wrote: > > > > > Hi all, > > > > > > > > > I'm testing Samza on YARN and I have encountered a problem on the > > security > > > setting of YARN (Kerberos). Here is the detail: > > > > > > 1. My cluster is secured by Kerberos, and I deploy my samza job from > one > > > of the cluster. > > > > > > > > > 2. My config file is in ~/.samza/conf/(yarn-site.xml, core-site.xml, > > > hdfs-site.xml) > > > > > > > > > 3. The job is deployed successfully, and I can get the info such as: > > > > > > ClientHelper [INFO] set package url to scheme: "hdfs" port: -1 > file: > > > "/user/test/samzatest.tar.gz" for application_1435680272316_0003 > > > > > > ClientHelper [INFO] set package size to 212924524 for > > > application_1435680272316_0003 > > > > > > > > > > > > I think the security setting is correct as it can get the file size > > > from HDFS. > > > > > > > > > 4. But I get the error from YARN job manager as following: > > > > > > > > > Application application_1435680272316_0003 failed 2 times due to AM > > > Container for appattempt_1435680272316_0003_000002 exited with > exitCode: > > > -1000 > > > > > > For more detailed output, check application tracking page: > > > http://cdh-namenode:8088/proxy/application_1435680272316_0003/Then, > > click > > > on links to logs of each attempt. > > > > > > Diagnostics: Failed on local exception: java.io.IOException: > > > org.apache.hadoop.security.AccessControlException: Client cannot > > > authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: > > > "talend-cdh-datanode8/62.210.141.237"; destination host is: > > > "talend-cdh-namenode":8020; > > > > > > java.io.IOException: Failed on local exception: java.io.IOException: > > > org.apache.hadoop.security.AccessControlException: Client cannot > > > authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: > > > "cdh-datanode8/62.210.141.237"; destination host is: > > "cdh-namenode":8020; > > > > > > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772) > > > > > > ...... > > > > > > > > > > > > Anyone knows how to solve this? > > > > > > > > > Qi FU > > > > > > -- Chen Song
