Re: [Gluster-devel] Data classification proposal

2014-06-27 Thread Jeff Darcy
> Sounds like a metadata server would fix this! > > ( Yes, this is trolling hard. Ignore. ;> ) Fortunately I'm on vacation now, so my head didn't explode. ;) ___ Gluster-devel mailing list Gluster-devel@gluster.org http://supercolony.gluster.org/mailm

Re: [Gluster-devel] Data classification proposal

2014-06-27 Thread Justin Clift
On 27/06/2014, at 8:39 AM, Xavier Hernandez wrote: > On Thursday 26 June 2014 12:52:13 Dan Lambright wrote: >> I don't think brick splitting implemented by LVM would affect directory >> browsing any more than adding an additional brick would, >> > > Yes, splitting a brick in LVM should be the sam

Re: [Gluster-devel] Data classification proposal

2014-06-27 Thread Vivek Agarwal
uster Compliance Feature" Regards, Vivek Just a thought. Shyam - Original Message - From: "Dan Lambright" To: "Jeff Darcy" Cc: "Gluster Devel" Sent: Monday, June 23, 2014 4:48:13 PM Subject: Re: [Gluster-devel] Data classification proposal A frustrat

Re: [Gluster-devel] Data classification proposal

2014-06-27 Thread Xavier Hernandez
On Thursday 26 June 2014 12:52:13 Dan Lambright wrote: > I don't think brick splitting implemented by LVM would affect directory > browsing any more than adding an additional brick would, > Yes, splitting a brick in LVM should be the same than adding a normal brick. The main problem I see is tha

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Shyamsundar Ranganathan
split this view for the client in question. Just a thought. Shyam - Original Message - > From: "Dan Lambright" > To: "Jeff Darcy" > Cc: "Gluster Devel" > Sent: Monday, June 23, 2014 4:48:13 PM > Subject: Re: [Gluster-devel] Data classif

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Dan Lambright
; Sent: Thursday, June 26, 2014 12:01:16 PM Subject: Re: [Gluster-devel] Data classification proposal On 26/06/2014, at 4:54 PM, Dan Lambright wrote: > Implementing brick splitting using LVM would allow you to treat each logical > volume (split) as an independent brick. Each split would ha

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Dan Lambright
" To: "Krishnan Parthasarathi" Cc: "Gluster Devel" Sent: Thursday, June 26, 2014 11:13:48 AM Subject: Re: [Gluster-devel] Data classification proposal > > > For the short-term, wouldn't it be OK to disallow adding bricks that > > > is not a multiple

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Justin Clift
On 26/06/2014, at 4:54 PM, Dan Lambright wrote: > Implementing brick splitting using LVM would allow you to treat each logical > volume (split) as an independent brick. Each split would have its own > .glusterfs subdirectory. I think this would help with taking snapshots as > well. Would brick

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Shyamsundar Ranganathan
> > > For the short-term, wouldn't it be OK to disallow adding bricks that > > > is not a multiple of group-size? > > > > In the *very* short term, yes. However, I think that will quickly > > become an issue for users who try to deploy erasure coding because those > > group sizes will be quite la

Re: [Gluster-devel] Data classification proposal

2014-06-26 Thread Xavier Hernandez
On Wednesday 25 June 2014 11:42:10 Jeff Darcy wrote: > > How space will be allocated to each new sub-brick ? some sort of thin- > > provisioning or will it be distributed evenly on each split ? > > That's left to the user. The latest proposal, based on discussion of > the first, is here: > > htt

Re: [Gluster-devel] Data classification proposal

2014-06-25 Thread Jeff Darcy
> If I understand correctly the proposed data-classification > architecture, each server will have a number of bricks that will be > dynamically modified as needed: as more data-classifying conditions > are defined, a new layer of translators will be added (a new DHT or > AFR, or something else) an

Re: [Gluster-devel] Data classification proposal

2014-06-25 Thread Xavier Hernandez
On Wednesday 25 June 2014 08:35:05 Jeff Darcy wrote: > > For the short-term, wouldn't it be OK to disallow adding bricks that > > is not a multiple of group-size? > > In the *very* short term, yes. However, I think that will quickly > become an issue for users who try to deploy erasure coding bec

Re: [Gluster-devel] Data classification proposal

2014-06-25 Thread Krishnan Parthasarathi
- Original Message - > > For the short-term, wouldn't it be OK to disallow adding bricks that > > is not a multiple of group-size? > > In the *very* short term, yes. However, I think that will quickly > become an issue for users who try to deploy erasure coding because those > group size

Re: [Gluster-devel] Data classification proposal

2014-06-25 Thread Jeff Darcy
> For the short-term, wouldn't it be OK to disallow adding bricks that > is not a multiple of group-size? In the *very* short term, yes. However, I think that will quickly become an issue for users who try to deploy erasure coding because those group sizes will be quite large. As soon as we impl

Re: [Gluster-devel] Data classification proposal

2014-06-24 Thread Krishnan Parthasarathi
Jeff, - Original Message - > > Am I right if I understood that the value for media-type is not > > interpreted beyond the scope of matching rules? That is to say, we > > don't need/have any notion of media-types that type check internally > > for forming (sub)volumes using the rules specif

Re: [Gluster-devel] Data classification proposal

2014-06-24 Thread Jeff Darcy
> Its possible to express your example using lists if their entries are allowed > to overlap. I see that you wanted a way to express a matrix (overlapping > rules) with gluster's tree-like syntax as backdrop. > > A polytree may be a better term than matrix (DAG without cycles), i.e. when > there a

Re: [Gluster-devel] Data classification proposal

2014-06-24 Thread Dan Lambright
uster Devel" Sent: Monday, June 23, 2014 7:11:44 PM Subject: Re: [Gluster-devel] Data classification proposal > Rather than using the keyword "unclaimed", my instinct was to > explicitly list which bricks have not been "claimed". Perhaps you > have something more su

Re: [Gluster-devel] Data classification proposal

2014-06-24 Thread Jeff Darcy
> Am I right if I understood that the value for media-type is not > interpreted beyond the scope of matching rules? That is to say, we > don't need/have any notion of media-types that type check internally > for forming (sub)volumes using the rules specified. Exactly. To us it's just an opaque ID

Re: [Gluster-devel] Data classification proposal

2014-06-24 Thread Krishnan Parthasarathi
Jeff, I have a few questions regarding the rules syntax and how they apply. I think this is different in spirit from the discussion Dan has started and keeping it separate. See questions inline. - Original Message - > One of the things holding up our data classification efforts (which inc

Re: [Gluster-devel] Data classification proposal

2014-06-23 Thread Jeff Darcy
> Rather than using the keyword "unclaimed", my instinct was to > explicitly list which bricks have not been "claimed". Perhaps you > have something more subtle in mind, it is not apparent to me from your > response. Can you provide an example of why it is necessary and a list > could not be provi

Re: [Gluster-devel] Data classification proposal

2014-06-23 Thread Dan Lambright
not do file name filtering, the "filter-condition" option did. I'm ok with that. As far as the "user stories" idea goes, that seems like a good next step. - Original Message - From: "Jeff Darcy" To: "Dan Lambright" Cc: "Gluster Devel&quo

Re: [Gluster-devel] Data classification proposal

2014-06-23 Thread Jeff Darcy
> A frustrating aspect of Linux is the complexity of /etc configuration file's > formats (rsyslog.conf, logrotate, cron, yum repo files, etc) In that spirit > I would simplify the "select" in the data classification proposal (copied > below) to only accept a list of bricks/sub-tiers with wild-cards

Re: [Gluster-devel] Data classification proposal

2014-06-23 Thread Dan Lambright
A frustrating aspect of Linux is the complexity of /etc configuration file's formats (rsyslog.conf, logrotate, cron, yum repo files, etc) In that spirit I would simplify the "select" in the data classification proposal (copied below) to only accept a list of bricks/sub-tiers with wild-cards '*',