Fwd: HDFS multi-tenancy and federation
Hi, I would like to know the following. 1) Can there be multiple namespaces in a single namenode? is it recommended? (I'm having a multi-tenant environment in mind) 2) Let's say I have a federated namespace/namenodes. There are two namenodes A /namespace A1 and namenode B/namespace B1, and have 3 datanodes. Can someone from namespace A1, access the datanode's data in anyway (hacking) belonging to namespace B1. If not how is it handled? After going through a lot of reference, my understanding on HDFS multi-tenancy and federation is that for multi-tenancy what we could do is use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a namespace per tenant. The issue here is that a namenode (active namenode, passive namenode and secondary namenode) has to be assigned per tenant. Is there any other way that multi tenancy can be achieved? On federation, let's say I have a namenode for /marketing and another for /finance. Lets say that marketing bears the most load. How can we load balance this? is it possible? Appreciate any help on this. Regards, Shani.
Re: HDFS multi-tenancy and federation
Hi, Any help on this please? On Mon, Feb 3, 2014 at 12:14 PM, Shani Ranasinghe wrote: > > Hi, > I would like to know the following. > > 1) Can there be multiple namespaces in a single namenode? is it > recommended? (I'm having a multi-tenant environment in mind) > > 2) Let's say I have a federated namespace/namenodes. There are two > namenodes A /namespace A1 and namenode B/namespace B1, and have 3 > datanodes. Can someone from namespace A1, access the datanode's data in > anyway (hacking) belonging to namespace B1. If not how is it handled? > > After going through a lot of reference, my understanding on HDFS > multi-tenancy and federation is that for multi-tenancy what we could do is > use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a > namespace per tenant. The issue here is that a namenode (active namenode, > passive namenode and secondary namenode) has to be assigned per tenant. Is > there any other way that multi tenancy can be achieved? > > On federation, let's say I have a namenode for /marketing and another for > /finance. Lets say that marketing bears the most load. How can we load > balance this? is it possible? > > Appreciate any help on this. > > Regards, > Shani. > > > >
Re: HDFS multi-tenancy and federation
Hi Shani, I haven't done any implementation on HDFS federation, but as far as I know, 1 namenode can handle only 1 namespace at this time. I hope that helps. Regards Prav On Wed, Feb 5, 2014 at 8:05 AM, Shani Ranasinghe wrote: > Hi, > > Any help on this please? > > > > On Mon, Feb 3, 2014 at 12:14 PM, Shani Ranasinghe wrote: > >> >> Hi, >> I would like to know the following. >> >> 1) Can there be multiple namespaces in a single namenode? is it >> recommended? (I'm having a multi-tenant environment in mind) >> >> 2) Let's say I have a federated namespace/namenodes. There are two >> namenodes A /namespace A1 and namenode B/namespace B1, and have 3 >> datanodes. Can someone from namespace A1, access the datanode's data in >> anyway (hacking) belonging to namespace B1. If not how is it handled? >> >> After going through a lot of reference, my understanding on HDFS >> multi-tenancy and federation is that for multi-tenancy what we could do is >> use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a >> namespace per tenant. The issue here is that a namenode (active namenode, >> passive namenode and secondary namenode) has to be assigned per tenant. Is >> there any other way that multi tenancy can be achieved? >> >> On federation, let's say I have a namenode for /marketing and another for >> /finance. Lets say that marketing bears the most load. How can we load >> balance this? is it possible? >> >> Appreciate any help on this. >> >> Regards, >> Shani. >> >> >> >> >
Re: HDFS multi-tenancy and federation
Hi, Thanks for the information. Can I have an answer for the question 2 please? Appreciate any help. On Wed, Feb 5, 2014 at 1:41 PM, praveenesh kumar wrote: > Hi Shani, > > I haven't done any implementation on HDFS federation, but as far as I > know, 1 namenode can handle only 1 namespace at this time. I hope that > helps. > > Regards > Prav > > > On Wed, Feb 5, 2014 at 8:05 AM, Shani Ranasinghe > wrote: > >> Hi, >> >> Any help on this please? >> >> >> >> On Mon, Feb 3, 2014 at 12:14 PM, Shani Ranasinghe >> wrote: >> >>> >>> Hi, >>> I would like to know the following. >>> >>> 1) Can there be multiple namespaces in a single namenode? is it >>> recommended? (I'm having a multi-tenant environment in mind) >>> >>> 2) Let's say I have a federated namespace/namenodes. There are two >>> namenodes A /namespace A1 and namenode B/namespace B1, and have 3 >>> datanodes. Can someone from namespace A1, access the datanode's data in >>> anyway (hacking) belonging to namespace B1. If not how is it handled? >>> >>> After going through a lot of reference, my understanding on HDFS >>> multi-tenancy and federation is that for multi-tenancy what we could do is >>> use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a >>> namespace per tenant. The issue here is that a namenode (active namenode, >>> passive namenode and secondary namenode) has to be assigned per tenant. Is >>> there any other way that multi tenancy can be achieved? >>> >>> On federation, let's say I have a namenode for /marketing and another >>> for /finance. Lets say that marketing bears the most load. How can we load >>> balance this? is it possible? >>> >>> Appreciate any help on this. >>> >>> Regards, >>> Shani. >>> >>> >>> >>> >> >
Re: HDFS multi-tenancy and federation
Federation is just a namenode namespace management capability. It is designed to control namenode management and to provide scalability for namenode. I don't think it poses any security or restrictions on accesssing the HDFS filesystem. I guess this link would help for your question 2 - http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/ViewFs.html On Tue, Jul 15, 2014 at 10:06 AM, Shani Ranasinghe wrote: > Hi, > > Thanks for the information. > > Can I have an answer for the question 2 please? Appreciate any help. > > > On Wed, Feb 5, 2014 at 1:41 PM, praveenesh kumar > wrote: > >> Hi Shani, >> >> I haven't done any implementation on HDFS federation, but as far as I >> know, 1 namenode can handle only 1 namespace at this time. I hope that >> helps. >> >> Regards >> Prav >> >> >> On Wed, Feb 5, 2014 at 8:05 AM, Shani Ranasinghe >> wrote: >> >>> Hi, >>> >>> Any help on this please? >>> >>> >>> >>> On Mon, Feb 3, 2014 at 12:14 PM, Shani Ranasinghe >>> wrote: >>> >>>> >>>> Hi, >>>> I would like to know the following. >>>> >>>> 1) Can there be multiple namespaces in a single namenode? is it >>>> recommended? (I'm having a multi-tenant environment in mind) >>>> >>>> 2) Let's say I have a federated namespace/namenodes. There are two >>>> namenodes A /namespace A1 and namenode B/namespace B1, and have 3 >>>> datanodes. Can someone from namespace A1, access the datanode's data in >>>> anyway (hacking) belonging to namespace B1. If not how is it handled? >>>> >>>> After going through a lot of reference, my understanding on HDFS >>>> multi-tenancy and federation is that for multi-tenancy what we could do is >>>> use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a >>>> namespace per tenant. The issue here is that a namenode (active namenode, >>>> passive namenode and secondary namenode) has to be assigned per tenant. Is >>>> there any other way that multi tenancy can be achieved? >>>> >>>> On federation, let's say I have a namenode for /marketing and another >>>> for /finance. Lets say that marketing bears the most load. How can we load >>>> balance this? is it possible? >>>> >>>> Appreciate any help on this. >>>> >>>> Regards, >>>> Shani. >>>> >>>> >>>> >>>> >>> >> >
Re: HDFS multi-tenancy and federation
Hi Praveenesh, Thank you for pointing this out. Will go through this. On Tue, Jul 15, 2014 at 3:36 PM, praveenesh kumar wrote: > Federation is just a namenode namespace management capability. It is > designed to control namenode management and to provide scalability for > namenode. I don't think it poses any security or restrictions on accesssing > the HDFS filesystem. I guess this link would help for your question 2 - > http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/ViewFs.html > > > > > > On Tue, Jul 15, 2014 at 10:06 AM, Shani Ranasinghe > wrote: > >> Hi, >> >> Thanks for the information. >> >> Can I have an answer for the question 2 please? Appreciate any help. >> >> >> On Wed, Feb 5, 2014 at 1:41 PM, praveenesh kumar >> wrote: >> >>> Hi Shani, >>> >>> I haven't done any implementation on HDFS federation, but as far as I >>> know, 1 namenode can handle only 1 namespace at this time. I hope that >>> helps. >>> >>> Regards >>> Prav >>> >>> >>> On Wed, Feb 5, 2014 at 8:05 AM, Shani Ranasinghe >>> wrote: >>> >>>> Hi, >>>> >>>> Any help on this please? >>>> >>>> >>>> >>>> On Mon, Feb 3, 2014 at 12:14 PM, Shani Ranasinghe >>>> wrote: >>>> >>>>> >>>>> Hi, >>>>> I would like to know the following. >>>>> >>>>> 1) Can there be multiple namespaces in a single namenode? is it >>>>> recommended? (I'm having a multi-tenant environment in mind) >>>>> >>>>> 2) Let's say I have a federated namespace/namenodes. There are two >>>>> namenodes A /namespace A1 and namenode B/namespace B1, and have 3 >>>>> datanodes. Can someone from namespace A1, access the datanode's data in >>>>> anyway (hacking) belonging to namespace B1. If not how is it handled? >>>>> >>>>> After going through a lot of reference, my understanding on HDFS >>>>> multi-tenancy and federation is that for multi-tenancy what we could do is >>>>> use file/folder permissions (u,g,o) and ACL's. Or we could dedicate a >>>>> namespace per tenant. The issue here is that a namenode (active namenode, >>>>> passive namenode and secondary namenode) has to be assigned per tenant. >>>>> Is >>>>> there any other way that multi tenancy can be achieved? >>>>> >>>>> On federation, let's say I have a namenode for /marketing and another >>>>> for /finance. Lets say that marketing bears the most load. How can we load >>>>> balance this? is it possible? >>>>> >>>>> Appreciate any help on this. >>>>> >>>>> Regards, >>>>> Shani. >>>>> >>>>> >>>>> >>>>> >>>> >>> >> >