Re: Permission Denied
Thanks to all that helped. I've now got my configuration running. All of the configuration files that once referenced localhost now reference my hostname (AccumuloTN). The big deal that I omitted seeing in *any* of the various blogs and instructions was to create a linux directory with the same name as my accumulo user (that is the linux-user-name) and change the ownership of it to accumulo:superuser. So, now not only is it working (from within another linux userid on that machine), I've run code on my Windows machine that connects to my cloud machine and is able to fetch data. Dave Patterson On Mon, Mar 2, 2015 at 11:47 AM, Sean Busbey bus...@cloudera.com wrote: Splitting into three unix users is a good idea. Generally, none of the linux users should need access to any of the local resources owned by the others. (that is, the user running the accumulo processes shouldn't be able to interfere with the backing files used by the HDFS processes). By default, the linux user that drives a particular process will be resolved to a Hadoop user by the NameNode process. Presuming your Accumulo services are running under the linux user accumulo, you should ensure that user exists on the linux node that runs the NameNode. The main issue with running init as the hadoop user is that by default it's likely going to write the accumulo directories as owned by the user that created them. Presuming you are using Accumulo because you have security requirements, the common practice is to make sure only the user that runs Accumulo processes can write to /accumulo and that only that user can read /accumulo/tables and /accumulo/wal. This ensures that other users with access to the HDFS cluster won't be able to bypass the cell-level access controls provided by Accumulo. While you are setting up HDFS directories, you should also create a home directory for the user that runs Accumulo processes. If your HDFS instance is set to use the trash feature (either in server configs or the client configs made available to Accumulo), then by default Accumulo will attempt to use it. Without a home directory, this will result in failures. Alternatively, you can ensure Accumulo doesn't rely on the trash feature by setting gc.trash.ignore in your accumulo-site.xml. One other note: I edited the accumulo-site.xml so it now has property nameinstance.volumes/name valuehdfs://localhost:9000/accumulo/value descriptioncomma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo/description /property You will save yourself headache later if you stick with fully qualified domain names for all HDFS, ZooKeeper, and Accumulo connections. -- Sean On Mon, Mar 2, 2015 at 8:13 AM, David Patterson patt...@gmail.com wrote: David, Thanks for the information. I've issued those two commands in my hadoop shell and still get the same error when I try to initialize accumulo in *its* shell. : 2015-03-02 13:30:41,175 [init.Initialize] FATAL: Failed to initialize filesystem org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=WRITE, inode=/accumulo: accumulo.supergroup:supergroup:drwxr-xr-x My comment that I had 3 users was meant in a linux sense, not in a hadoop sense. So (to borrow terminoloy from RDF or XML) is there something I have to do in my hadoop setup (running under linix:hadoop) or my accumulo setup (running under linux:accumulo) so that the accumuulo I/O gets processed as from someone in the hadoop:supergroup? I tried running the accumulo init from the linux:hadoop user and it worked. I'm not sure if any permissions/etc were hosed by doing it there. I'll see. Thanks for you help. (By the way, is it wrong or a bad idea to split the work into three linux:users, or should it all be done in one linux:user space?) Dave Patterson On Sun, Mar 1, 2015 at 8:35 PM, dlmarion dlmar...@comcast.net wrote: hadoop fs -mkdir /accumulo hadoop fs -chown accumulo:supergroup /accumulo Original message From: David Patterson patt...@gmail.com Date:03/01/2015 7:04 PM (GMT-05:00) To: user@hadoop.apache.org Cc: Subject: Re: Permission Denied David, Thanks for the reply. Taking the questions in the opposite order, my accumulo-site.xml does not have volumes specified. I edited the accumulo-site.xml so it now has property nameinstance.volumes/name valuehdfs://localhost:9000/accumulo/value descriptioncomma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo/description /property and got the same error. How can I precreate /accumulo ? Dave Patterson On Sun, Mar 1, 2015 at 3:50 PM, david marion dlmar...@hotmail.com wrote: It looks like / is owned by hadoop.supergroup and the perms are 755. You could precreate /accumulo and chown it appropriately, or set the perms for / to 775. Init is trying to create /accumulo in hdfs as the accumulo
Re: Permission Denied
David, Thanks for the information. I've issued those two commands in my hadoop shell and still get the same error when I try to initialize accumulo in *its* shell. : 2015-03-02 13:30:41,175 [init.Initialize] FATAL: Failed to initialize filesystem org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=WRITE, inode=/accumulo: accumulo.supergroup:supergroup:drwxr-xr-x My comment that I had 3 users was meant in a linux sense, not in a hadoop sense. So (to borrow terminoloy from RDF or XML) is there something I have to do in my hadoop setup (running under linix:hadoop) or my accumulo setup (running under linux:accumulo) so that the accumuulo I/O gets processed as from someone in the hadoop:supergroup? I tried running the accumulo init from the linux:hadoop user and it worked. I'm not sure if any permissions/etc were hosed by doing it there. I'll see. Thanks for you help. (By the way, is it wrong or a bad idea to split the work into three linux:users, or should it all be done in one linux:user space?) Dave Patterson On Sun, Mar 1, 2015 at 8:35 PM, dlmarion dlmar...@comcast.net wrote: hadoop fs -mkdir /accumulo hadoop fs -chown accumulo:supergroup /accumulo Original message From: David Patterson patt...@gmail.com Date:03/01/2015 7:04 PM (GMT-05:00) To: user@hadoop.apache.org Cc: Subject: Re: Permission Denied David, Thanks for the reply. Taking the questions in the opposite order, my accumulo-site.xml does not have volumes specified. I edited the accumulo-site.xml so it now has property nameinstance.volumes/name valuehdfs://localhost:9000/accumulo/value descriptioncomma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo/description /property and got the same error. How can I precreate /accumulo ? Dave Patterson On Sun, Mar 1, 2015 at 3:50 PM, david marion dlmar...@hotmail.com wrote: It looks like / is owned by hadoop.supergroup and the perms are 755. You could precreate /accumulo and chown it appropriately, or set the perms for / to 775. Init is trying to create /accumulo in hdfs as the accumulo user and your perms dont allow it. Do you have instance.volumes set in accumulo-site.xml? Original message From: David Patterson patt...@gmail.com Date:03/01/2015 3:36 PM (GMT-05:00) To: user@hadoop.apache.org Cc: Subject: Permission Denied I'm trying to create an Accumulo/Hadoop/Zookeeper configuration on a single (Ubuntu) machine, with Hadoop 2.6.0, Zookeeper 3.4.6 and Accumulo 1.6.1. I've got 3 userids for these components that are in the same group and no other users are in that group. I have zookeeper running, and hadoop as well. Hadoop's core-site.xml file has the hadoop.tmp.dir set to /app/hadoop/tmp.The /app/hadoop/tmp directory is owned by the hadoop user and has permissions that allow other members of the group to write (drwxrwxr-x). When I try to initialize Accumulo, with bin/accumulo init, I get FATAL: Failed to initialize filesystem. org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=WRITE, inode=/:hadoop:supergroup:drwxr-xr-x So, my main question is which directory do I need to give group-write permission so the accumulo user can write as needed so it can initialize? The second problem is that the Accumulo init reports [Configuration.deprecation] INFO : fs.default.name is deprecated. Instead use fs.defaultFS. However, the hadoop core-site.xml file contains: namefs.defaultFS/name valuehdfs://localhost:9000/value Is there somewhere else that this value (fs.default.name) is specified? Could it be due to Accumulo having a default value and not getting the override from hadoop because of the problem listed above? Thanks Dave Patterson patt...@gmail.com
Re: Permission Denied
David, Thanks for the reply. Taking the questions in the opposite order, my accumulo-site.xml does not have volumes specified. I edited the accumulo-site.xml so it now has property nameinstance.volumes/name valuehdfs://localhost:9000/accumulo/value descriptioncomma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo/description /property and got the same error. How can I precreate /accumulo ? Dave Patterson On Sun, Mar 1, 2015 at 3:50 PM, david marion dlmar...@hotmail.com wrote: It looks like / is owned by hadoop.supergroup and the perms are 755. You could precreate /accumulo and chown it appropriately, or set the perms for / to 775. Init is trying to create /accumulo in hdfs as the accumulo user and your perms dont allow it. Do you have instance.volumes set in accumulo-site.xml? Original message From: David Patterson patt...@gmail.com Date:03/01/2015 3:36 PM (GMT-05:00) To: user@hadoop.apache.org Cc: Subject: Permission Denied I'm trying to create an Accumulo/Hadoop/Zookeeper configuration on a single (Ubuntu) machine, with Hadoop 2.6.0, Zookeeper 3.4.6 and Accumulo 1.6.1. I've got 3 userids for these components that are in the same group and no other users are in that group. I have zookeeper running, and hadoop as well. Hadoop's core-site.xml file has the hadoop.tmp.dir set to /app/hadoop/tmp.The /app/hadoop/tmp directory is owned by the hadoop user and has permissions that allow other members of the group to write (drwxrwxr-x). When I try to initialize Accumulo, with bin/accumulo init, I get FATAL: Failed to initialize filesystem. org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=WRITE, inode=/:hadoop:supergroup:drwxr-xr-x So, my main question is which directory do I need to give group-write permission so the accumulo user can write as needed so it can initialize? The second problem is that the Accumulo init reports [Configuration.deprecation] INFO : fs.default.name is deprecated. Instead use fs.defaultFS. However, the hadoop core-site.xml file contains: namefs.defaultFS/name valuehdfs://localhost:9000/value Is there somewhere else that this value (fs.default.name) is specified? Could it be due to Accumulo having a default value and not getting the override from hadoop because of the problem listed above? Thanks Dave Patterson patt...@gmail.com
Permission Denied
I'm trying to create an Accumulo/Hadoop/Zookeeper configuration on a single (Ubuntu) machine, with Hadoop 2.6.0, Zookeeper 3.4.6 and Accumulo 1.6.1. I've got 3 userids for these components that are in the same group and no other users are in that group. I have zookeeper running, and hadoop as well. Hadoop's core-site.xml file has the hadoop.tmp.dir set to /app/hadoop/tmp.The /app/hadoop/tmp directory is owned by the hadoop user and has permissions that allow other members of the group to write (drwxrwxr-x). When I try to initialize Accumulo, with bin/accumulo init, I get FATAL: Failed to initialize filesystem. org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=WRITE, inode=/:hadoop:supergroup:drwxr-xr-x So, my main question is which directory do I need to give group-write permission so the accumulo user can write as needed so it can initialize? The second problem is that the Accumulo init reports [Configuration.deprecation] INFO : fs.default.name is deprecated. Instead use fs.defaultFS. However, the hadoop core-site.xml file contains: namefs.defaultFS/name valuehdfs://localhost:9000/value Is there somewhere else that this value (fs.default.name) is specified? Could it be due to Accumulo having a default value and not getting the override from hadoop because of the problem listed above? Thanks Dave Patterson patt...@gmail.com
