Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Venky Shankar
On Thu, Feb 04, 2016 at 11:34:04AM +0530, Shyam wrote: > On 02/04/2016 09:38 AM, Vijay Bellur wrote: > >On 02/03/2016 11:34 AM, Venky Shankar wrote: > >>On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: > Problem is with workloads which know the files that need to be read >

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Jeff Darcy
> Even with compound fops It will still require two sequential network > operations from dht2. One to MDC and one to DC So I don't think it helps. There are still two hops, but making it a compound op keeps the server-to-server communication in the compounding translator (which should already be

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Ira Cooper
Pranith Kumar Karampuri writes: > On 02/03/2016 07:54 PM, Jeff Darcy wrote: >>> Problem is with workloads which know the files that need to be read >>> without readdir, like hyperlinks (webserver), swift objects etc. These >>> are two I know of which will have this problem,

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-04 Thread Pranith Kumar Karampuri
On 02/03/2016 07:54 PM, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem, which can't be improved because we don't have metadata, data

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Pranith Kumar Karampuri
The file data would be located based on its GFID, so before the *first* lookup/stat for a file, there is no way to know it's GFID. NOTE: Instead of a name hash the GFID hash is used, to get immunity against renames and the like, as a name hash could change the location information for the file

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/03/2016 07:54 PM, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem, which can't be improved because we don't have metadata, data

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Jeff Darcy
> Problem is with workloads which know the files that need to be read > without readdir, like hyperlinks (webserver), swift objects etc. These > are two I know of which will have this problem, which can't be improved > because we don't have metadata, data co-located. I have been trying to > think

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Venky Shankar
On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: > > Problem is with workloads which know the files that need to be read > > without readdir, like hyperlinks (webserver), swift objects etc. These > > are two I know of which will have this problem, which can't be improved > > because we

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Vijay Bellur
On 02/03/2016 11:34 AM, Venky Shankar wrote: On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are two I know of which will have this problem,

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-03 Thread Shyam
On 02/04/2016 09:38 AM, Vijay Bellur wrote: On 02/03/2016 11:34 AM, Venky Shankar wrote: On Wed, Feb 03, 2016 at 09:24:06AM -0500, Jeff Darcy wrote: Problem is with workloads which know the files that need to be read without readdir, like hyperlinks (webserver), swift objects etc. These are

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Raghavendra Gowdappa
- Original Message - > From: "Pranith Kumar Karampuri" <pkara...@redhat.com> > To: "Jeff Darcy" <jda...@redhat.com> > Cc: "Gluster Devel" <gluster-devel@gluster.org> > Sent: Tuesday, February 2, 2016 7:52:25 PM > Subjec

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Pranith Kumar Karampuri
On 02/03/2016 11:49 AM, Pranith Kumar Karampuri wrote: On 02/03/2016 09:20 AM, Shyam wrote: On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Shyam
On 02/02/2016 06:22 PM, Jeff Darcy wrote: Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is

Re: [Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-02 Thread Jeff Darcy
> Background: Quick-read + open-behind xlators are developed to help > in small file workload reads like apache webserver, tar etc to get the > data of the file in lookup FOP itself. What happens is, when a lookup > FOP is executed, GF_CONTENT_KEY is added in xdata with max-length and >

[Gluster-devel] regarding GF_CONTENT_KEY and dht2 - perf with small files

2016-02-01 Thread Pranith Kumar Karampuri
hi, Background: Quick-read + open-behind xlators are developed to help in small file workload reads like apache webserver, tar etc to get the data of the file in lookup FOP itself. What happens is, when a lookup FOP is executed, GF_CONTENT_KEY is added in xdata with max-length and posix