[jira] [Commented] (OOZIE-3385) The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one
[
https://issues.apache.org/jira/browse/OOZIE-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694121#comment-16694121
]
LuGuangMing commented on OOZIE-3385:
@[~andras.piros] hello sir, I submit workflow by interface AuthOozieClient.doas
to proxy user instead of CLI. I have not set property job.properties,
user.name is my input proxy user in workflow.xml. This bug occurs when oozie
server reading workflow.xml defined Information , my log print
"DFSClient.toString()" info that DFSClient inner ugi is right ,
"{color:#FF}_2018-11-14 00:00:00,493 INFO
[CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520)
USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446]
JOB[0002497-180928143722290-oozie-root-C]
ACTION[0002497-180928143722290-oozie-root-C@1354]_ _hdfs client user,
DFSClient_{color}{color:#FF}_[clientName=DFSClient_NONMAPREDUCE_-515910437_325,
ugi=platform (auth:PROXY) via oozie/nsplatfor...@dc1.fh.com
(auth:KERBEROS)]_{color}*",* but name node check permission return user is
wrong others, "{color:#FF}_XException,
org.apache.oozie.command.CommandException: E0710: Could not read the workflow
definition, Permission denied: user=dbzq04, access=READ,
inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":platform:supergroup:-rw---{color:#33}",
which details could be look the attachments.{color}_{color}
> The situation multi user submit workflows , occasionally, occur the HDFS
> visitor user become another one
> -
>
> Key: OOZIE-3385
> URL: https://issues.apache.org/jira/browse/OOZIE-3385
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: LuGuangMing
>Priority: Blocker
> Attachments: oozie-server-error.log,
> part_source_HadoopAccessorService.txt, part_source_WorkflowAppService.txt
>
>
> The situation multi user submit workflows , occasionally, occur the HDFS
> visitor user become another one . for example, I need submit a workflow by
> proxy user "{color:#ff}platform{color}" via user oozie (kerberos) , an
> error occur in oozie source code WorkflowAppService.readDefinition read
> workflow.xml.
> *2018-11-14 00:00:00,497 ERROR
> [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517)
> {color:#ff}USER[platform]{color} GROUP[-] TOKEN[]
> APP[myBulkload-Scheduler-CS_TTL-1539689446]
> JOB[0002497-180928143722290-oozie-root-C]
> ACTION[0002497-180928143722290-oozie-root-C@1354] XException,
> org.apache.oozie.command.CommandException: E0710: Could not read the workflow
> definition, Permission denied: user={color:#ff}dbzq04{color},
> access=READ,
> inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":{color:#ff}platform{color}:supergroup:-rw---*
> note: user "{color:#ff}dbzq04{color}" also submit some workflow at
> before, but current submit the workflow of user is user
> {color:#ff}platform. In order to prove current user is platform , I
> insert some logs at oozie source code {color}
>
>
> {code:java}
> /** org.apache.oozie.service.HadoopAccessorService */
> public FileSystem createFileSystem(String user, final URI uri, final
> Configuration conf) throws HadoopAccessorException {
> //.omit..
> try {
>UserGroupInformation ugi = getUGI(user);
>LOG.info("current user="+ugi); //-- my insert log, to print proxy ugi
> info
>return ugi.doAs(new PrivilegedExceptionAction() {
>public FileSystem run() throws Exception {
> FileSystem fs = FileSystem.get(uri, conf);
> //-- my insert log, to print fs inner ugi info
> if(fs instanceof DistributedFileSystem){
> LOG.info("hdfs client user,
> "+((DistributedFileSystem)fs).getClient().toString());
> }
> return fs;
>}
> });
> }catch (InterruptedException ex) {
>throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }catch (IOException ex) {
>throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }
> }{code}
> *my log print result follows:*
>
> 2018-11-14 00:00:00,492 INFO
> [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520)
> USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446]
> JOB[0002497-180928143722290-oozie-root-C]
> ACTION[0002497-180928143722290-oozie-root-C@1354] *current user=platform
> (auth:PROXY) via oozie/nsplatfor...@dc1.fh.com (auth:KERBEROS)*
> 2018-11-14 00:00:00,493 INFO
> [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520)
> USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446]
>
[jira] [Commented] (OOZIE-3385) The situation multi user submit workflows , occasionally, occur the HDFS visitor user become another one
[
https://issues.apache.org/jira/browse/OOZIE-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691555#comment-16691555
]
Andras Piros commented on OOZIE-3385:
-
[~luguangming] the submitting user [is defined by the {{user.name}}
property|https://oozie.apache.org/docs/5.0.0/WorkflowFunctionalSpec.html#a6_User_Propagation]
of the submitted {{workflow.xml}} / {{job.properties}}. While submitting a
workflow / coordinator / bundle job, setting the HTTP parameter
{{oozie.user.name}} would override the one given in the workflow configuration.
There are other possibilities of [user
authentication|https://oozie.apache.org/docs/5.0.0/AG_Install.html#Oozie_User_Authentication_Configuration],
though. I'd recommend to do {{klist}} just before job submission - you might
have been authenticated as another user. Checking the exact CLI command might
also be helpful.
> The situation multi user submit workflows , occasionally, occur the HDFS
> visitor user become another one
> -
>
> Key: OOZIE-3385
> URL: https://issues.apache.org/jira/browse/OOZIE-3385
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: LuGuangMing
>Priority: Blocker
> Attachments: oozie-server-error.log,
> part_source_HadoopAccessorService.txt, part_source_WorkflowAppService.txt
>
>
> The situation multi user submit workflows , occasionally, occur the HDFS
> visitor user become another one . for example, I need submit a workflow by
> proxy user "{color:#ff}platform{color}" via user oozie (kerberos) , an
> error occur in oozie source code WorkflowAppService.readDefinition read
> workflow.xml.
> *2018-11-14 00:00:00,497 ERROR
> [CallableQueue-42]org.apache.oozie.command.wf.SubmitXCommand(517)
> {color:#ff}USER[platform]{color} GROUP[-] TOKEN[]
> APP[myBulkload-Scheduler-CS_TTL-1539689446]
> JOB[0002497-180928143722290-oozie-root-C]
> ACTION[0002497-180928143722290-oozie-root-C@1354] XException,
> org.apache.oozie.command.CommandException: E0710: Could not read the workflow
> definition, Permission denied: user={color:#ff}dbzq04{color},
> access=READ,
> inode="/phoebus/_fileservice/users/nsplatform/platform/workflows/DataLoadWF-1427-1129/workflow.xml":{color:#ff}platform{color}:supergroup:-rw---*
> note: user "{color:#ff}dbzq04{color}" also submit some workflow at
> before, but current submit the workflow of user is user
> {color:#ff}platform. In order to prove current user is platform , I
> insert some logs at oozie source code {color}
>
>
> {code:java}
> /** org.apache.oozie.service.HadoopAccessorService */
> public FileSystem createFileSystem(String user, final URI uri, final
> Configuration conf) throws HadoopAccessorException {
> //.omit..
> try {
>UserGroupInformation ugi = getUGI(user);
>LOG.info("current user="+ugi); //-- my insert log, to print proxy ugi
> info
>return ugi.doAs(new PrivilegedExceptionAction() {
>public FileSystem run() throws Exception {
> FileSystem fs = FileSystem.get(uri, conf);
> //-- my insert log, to print fs inner ugi info
> if(fs instanceof DistributedFileSystem){
> LOG.info("hdfs client user,
> "+((DistributedFileSystem)fs).getClient().toString());
> }
> return fs;
>}
> });
> }catch (InterruptedException ex) {
>throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }catch (IOException ex) {
>throw new HadoopAccessorException(ErrorCode.E0902, ex.getMessage(), ex);
> }
> }{code}
> *my log print result follows:*
>
> 2018-11-14 00:00:00,492 INFO
> [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520)
> USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446]
> JOB[0002497-180928143722290-oozie-root-C]
> ACTION[0002497-180928143722290-oozie-root-C@1354] *current user=platform
> (auth:PROXY) via oozie/nsplatfor...@dc1.fh.com (auth:KERBEROS)*
> 2018-11-14 00:00:00,493 INFO
> [CallableQueue-42]org.apache.oozie.service.HadoopAccessorService(520)
> USER[platform] GROUP[-] TOKEN[] APP[myBulkload-Scheduler-CS_TTL-1539689446]
> JOB[0002497-180928143722290-oozie-root-C]
> ACTION[0002497-180928143722290-oozie-root-C@1354] *hdfs client user,
> DFSClient[clientName=DFSClient_NONMAPREDUCE_-515910437_325, ugi=platform
> (auth:PROXY) via oozie/nsplatfor...@dc1.fh.com (auth:KERBEROS)]*
>
> *over above the proves Indicate at visited HDFS path of user has been
> altered, where did user "**{color:#ff}dbzq04" come from? please help me
> check this problem--{color}***
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
