[jira] [Commented] (HDFS-10863) hadoop superusergroup supergroup issue

2016-09-23 Thread John Zhuge (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517702#comment-15517702
 ] 

John Zhuge commented on HDFS-10863:
---

Hi [~jbigdata.fr], thanks for reporting the issue. You can always post any 
question to u...@hadoop.apache.org.

Please read {{Overview}} of [HDFS Permissions Guide 
(2.7.3)|https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html]:

bq. When a file or directory is created, its owner is the user identity of the 
client process, and its group is the group of the parent directory (the BSD 
rule).

Property {{dfs.permissions.superusergroup}} sets the super user group whose 
members are all considered super users. In you case, any member of group 
{{hadoop}} becomes HDFS super user who will bypass any permission checking. 
Please set it with care.

> hadoop superusergroup supergroup issue
> --
>
> Key: HDFS-10863
> URL: https://issues.apache.org/jira/browse/HDFS-10863
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.7.2
> Environment: $ hadoop version
> Hadoop 2.7.2
>Reporter: www.jbigdata.fr
>Priority: Minor
>
> I want to match my unix user to HDFS: hduser:hadoop.
> For the user I use the VE.
> $ echo $HADOOP_HDFS_USER
> hduser
> For the group I use the hdfs-site.xml :
> 
> dfs.permissions.superusergroup
> hadoop
> 
> The namenode log file show the parameter user/group values.
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner 
> = hduser (auth:SIMPLE)
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup  
> = hadoop
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled 
> = true
> Everything seems to be OK, but when I copy file form FS to HDFS the group is 
> not correct. It keeps the supergroup default value.
> Thoses shell commands show the issue:
> $ ll /srv/downloads/zk.tar
> -rw-r--r-- 1 hduser hadoop 41984000 Aug 18 13:25 /srv/downloads/zk.tar
> $ hdfs dfs -put /srv/downloads/zk.tar /tmp
> $ hdfs dfs -ls /tmp/zk.tar
> -rw-r--r--   2 hduser supergroup   41984000 2016-09-14 12:47 /tmp/zk.tar
> I have:
> -rw-r--r-- 2 hduser supergroup 41984000 2016-09-14 12:47 /tmp/zk.tar
> I expect :
> -rw-r--r-- 2 hduser hadoop 41984000 2016-09-14 12:47 /tmp/zk.tar
> Why the HDFS group is not the value of the dfs.permissions.superusergroup 
> property ?
> @jbigdata.fr



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10863) hadoop superusergroup supergroup issue

2016-09-22 Thread www.jbigdata.fr (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513897#comment-15513897
 ] 

www.jbigdata.fr commented on HDFS-10863:


Hi,

First of all, I classify this issue as a bug because I didn't know to put
"need help" in the JIRA.

My env : Ubuntu 16.04, hadoop 2.7.2, 3 DataNode (a DataNode is 2.7.3 I want
to test "compatibility").

$ uname -a
Linux jbd-vm01 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:01:55 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux

$ hadoop version
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
>From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using
/home/hduser/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar

core-file:


name
MyDistrib@JBD-2.7.2


fs.defaultFS
hdfs://jbd-vm01.jbdata.fr:9000


hadoop.tmp.dir

/srv/work/hdfs272/tmp



fs.ftp.host
jbd-vm03.jbdata.fr


fs.ftp.host.port
21


ha.zookeeper.quorum
jbd-vm01.jbdata.fr,jbd-vm03.jbdata.fr


hadoop.registry.zk.quorum
jbd-vm01.jbdata.fr:2181,jbd-vm03.jbdata.fr:2181


hadoop.http.staticuser.user
hduser



hadoop.proxyuser.hduser.groups
*


hadoop.proxyuser.hduser.hosts
*



hdfs-file


dfs.replication

2


dfs.permissions.superusergroup
hadoop



dfs.namenode.http-address
jbd-vm01.jbdata.fr:50070


dfs.namenode.https-address
jbd-vm01.jbdata.fr:50470


dfs.namenode.name.dir
file:/srv/data/hdfs272/nn1,file:/srv/data/hdfs272/nn2



dfs.namenode.secondary.http-address
jbd-vm03.jbdata.fr:50090


dfs.namenode.secondary.https-address
jbd-vm03.jbdata.fr:50091


dfs.namenode.checkpoint.dir
file:/srv/data/hdfs272/cp1,file:/srv/data/hdfs272/cp2


Need other thing ?

I'm "hard" testing the nfs gateway and I have the same kind of issue with
the group, but it's maybe another JIRA to open.

$ ll /srv/hdfs_nfs/tmp/zk.tar
rw-rr- 1 hduser 2584148964 41984000 Sep 21 06:01 /srv/hdfs_nfs/tmp/zk.tar

Maybe this issue disapear with the 3.0.0...

@JBD 





> hadoop superusergroup supergroup issue
> --
>
> Key: HDFS-10863
> URL: https://issues.apache.org/jira/browse/HDFS-10863
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.7.2
> Environment: $ hadoop version
> Hadoop 2.7.2
>Reporter: www.jbigdata.fr
>Priority: Minor
>
> I want to match my unix user to HDFS: hduser:hadoop.
> For the user I use the VE.
> $ echo $HADOOP_HDFS_USER
> hduser
> For the group I use the hdfs-site.xml :
> 
> dfs.permissions.superusergroup
> hadoop
> 
> The namenode log file show the parameter user/group values.
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner 
> = hduser (auth:SIMPLE)
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup  
> = hadoop
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled 
> = true
> Everything seems to be OK, but when I copy file form FS to HDFS the group is 
> not correct. It keeps the supergroup default value.
> Thoses shell commands show the issue:
> $ ll /srv/downloads/zk.tar
> -rw-r--r-- 1 hduser hadoop 41984000 Aug 18 13:25 /srv/downloads/zk.tar
> $ hdfs dfs -put /srv/downloads/zk.tar /tmp
> $ hdfs dfs -ls /tmp/zk.tar
> -rw-r--r--   2 hduser supergroup   41984000 2016-09-14 12:47 /tmp/zk.tar
> I have:
> -rw-r--r-- 2 hduser supergroup 41984000 2016-09-14 12:47 /tmp/zk.tar
> I expect :
> -rw-r--r-- 2 hduser hadoop 41984000 2016-09-14 12:47 /tmp/zk.tar
> Why the HDFS group is not the value of the dfs.permissions.superusergroup 
> property ?
> @jbigdata.fr



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10863) hadoop superusergroup supergroup issue

2016-09-21 Thread www.jbigdata.fr (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512336#comment-15512336
 ] 

www.jbigdata.fr commented on HDFS-10863:


Hi,

First of all, I classify this issue as a bug because I didn't know to put "need 
help" in the JIRA.

My env : Ubuntu 16.04, hadoop 2.7.2, 3 DataNode (a DataNode is 2.7.3 I want to 
test "compatibility").

$ uname -a
Linux jbd-vm01 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:01:55 UTC 2016 
x86_64 x86_64 x86_64 GNU/Linux

$ hadoop version
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 
b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
>From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using 
/home/hduser/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar

core-file:

  
name
MyDistrib@JBD-2.7.2
  
  
fs.defaultFS
hdfs://jbd-vm01.jbdata.fr:9000
  
  
hadoop.tmp.dir

/srv/work/hdfs272/tmp
  
  
  
fs.ftp.host
jbd-vm03.jbdata.fr
  
  
fs.ftp.host.port
21
  
  
ha.zookeeper.quorum
jbd-vm01.jbdata.fr,jbd-vm03.jbdata.fr
  
  
hadoop.registry.zk.quorum
jbd-vm01.jbdata.fr:2181,jbd-vm03.jbdata.fr:2181
  
  
hadoop.http.staticuser.user
hduser
  
  
  
hadoop.proxyuser.hduser.groups
*
  
  
hadoop.proxyuser.hduser.hosts
*
  


hdfs-file

  
dfs.replication

2
  
  
dfs.permissions.superusergroup
hadoop
  
  
  
dfs.namenode.http-address
jbd-vm01.jbdata.fr:50070
  
  
dfs.namenode.https-address
jbd-vm01.jbdata.fr:50470
  
  
dfs.namenode.name.dir
file:/srv/data/hdfs272/nn1,file:/srv/data/hdfs272/nn2
  
  
  
dfs.namenode.secondary.http-address
jbd-vm03.jbdata.fr:50090
  
  
dfs.namenode.secondary.https-address
jbd-vm03.jbdata.fr:50091
  
  
dfs.namenode.checkpoint.dir
file:/srv/data/hdfs272/cp1,file:/srv/data/hdfs272/cp2
  

Need other thing ?

I'm "hard" testing the nfs gateway and I have the same kind of issue with the 
group, but it's maybe another JIRA to open.

$ ll /srv/hdfs_nfs/tmp/zk.tar
-rw-r--r-- 1 hduser 2584148964 41984000 Sep 21 06:01 /srv/hdfs_nfs/tmp/zk.tar

Maybe this issue disapear with the 3.0.0...

> hadoop superusergroup supergroup issue
> --
>
> Key: HDFS-10863
> URL: https://issues.apache.org/jira/browse/HDFS-10863
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.7.2
> Environment: $ hadoop version
> Hadoop 2.7.2
>Reporter: www.jbigdata.fr
>Priority: Minor
>
> I want to match my unix user to HDFS: hduser:hadoop.
> For the user I use the VE.
> $ echo $HADOOP_HDFS_USER
> hduser
> For the group I use the hdfs-site.xml :
> 
> dfs.permissions.superusergroup
> hadoop
> 
> The namenode log file show the parameter user/group values.
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner 
> = hduser (auth:SIMPLE)
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup  
> = hadoop
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled 
> = true
> Everything seems to be OK, but when I copy file form FS to HDFS the group is 
> not correct. It keeps the supergroup default value.
> Thoses shell commands show the issue:
> $ ll /srv/downloads/zk.tar
> -rw-r--r-- 1 hduser hadoop 41984000 Aug 18 13:25 /srv/downloads/zk.tar
> $ hdfs dfs -put /srv/downloads/zk.tar /tmp
> $ hdfs dfs -ls /tmp/zk.tar
> -rw-r--r--   2 hduser supergroup   41984000 2016-09-14 12:47 /tmp/zk.tar
> I have:
> -rw-r--r-- 2 hduser supergroup 41984000 2016-09-14 12:47 /tmp/zk.tar
> I expect :
> -rw-r--r-- 2 hduser hadoop 41984000 2016-09-14 12:47 /tmp/zk.tar
> Why the HDFS group is not the value of the dfs.permissions.superusergroup 
> property ?
> @jbigdata.fr



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10863) hadoop superusergroup supergroup issue

2016-09-19 Thread Yulei Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15505776#comment-15505776
 ] 

Yulei Li commented on HDFS-10863:
-

According to your setup, I found  my linux user matched hdfs:
[hadoop@hadoop ~]$ ll  test
-rw-rw-r--. 1 hadoop hadoop 5 9月  20 14:25 test
[hadoop@hadoop ~]$ hdfs dfs -ls /tmp
Found 1 items
-rw-r--r--   1 hadoop hadoop  5 2016-09-20 14:30 /tmp/test
My OS is CentOS7, and hadoop version is hadoop 3.0.0, I don't think it is a 
bug. Could you show some more information about your environment and 
configuration?

> hadoop superusergroup supergroup issue
> --
>
> Key: HDFS-10863
> URL: https://issues.apache.org/jira/browse/HDFS-10863
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.7.2
> Environment: $ hadoop version
> Hadoop 2.7.2
>Reporter: www.jbigdata.fr
>Priority: Minor
>
> I want to match my unix user to HDFS: hduser:hadoop.
> For the user I use the VE.
> $ echo $HADOOP_HDFS_USER
> hduser
> For the group I use the hdfs-site.xml :
> 
> dfs.permissions.superusergroup
> hadoop
> 
> The namenode log file show the parameter user/group values.
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner 
> = hduser (auth:SIMPLE)
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup  
> = hadoop
> INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled 
> = true
> Everything seems to be OK, but when I copy file form FS to HDFS the group is 
> not correct. It keeps the supergroup default value.
> Thoses shell commands show the issue:
> $ ll /srv/downloads/zk.tar
> -rw-r--r-- 1 hduser hadoop 41984000 Aug 18 13:25 /srv/downloads/zk.tar
> $ hdfs dfs -put /srv/downloads/zk.tar /tmp
> $ hdfs dfs -ls /tmp/zk.tar
> -rw-r--r--   2 hduser supergroup   41984000 2016-09-14 12:47 /tmp/zk.tar
> I have:
> -rw-r--r-- 2 hduser supergroup 41984000 2016-09-14 12:47 /tmp/zk.tar
> I expect :
> -rw-r--r-- 2 hduser hadoop 41984000 2016-09-14 12:47 /tmp/zk.tar
> Why the HDFS group is not the value of the dfs.permissions.superusergroup 
> property ?
> @jbigdata.fr



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org