Re: [Gluster-devel] [Gluster-users] gluster volume heal info split brain command not showing files in split-brain

2016-03-19 Thread ABHISHEK PALIWAL 
Hi Anuradha,

Please confirm me, this is bug in glusterfs or we need to do something at
our end.

Because this problem is stopping our development.

Regards,
Abhishek

On Thu, Mar 17, 2016 at 1:54 PM, ABHISHEK PALIWAL 
wrote:

> Hi Anuradha,
>
> But in this case I need to do tail on each file which is time taking
> process and other end I can't pause my module until these file is getting
> healed.
>
> Any how I need the output of the split-brain to resolve this problem.
>
> Regards,
> Abhishek
>
> On Wed, Mar 16, 2016 at 6:21 PM, ABHISHEK PALIWAL  > wrote:
>
>> Hi Anuradha,
>>
>> The issue is resolved but we have one more issue something similar to
>> this one in which the file is not getting sync after the steps followed,
>> mentioned in the link which you shared in the previous mail.
>>
>> And problem is that why split-brain command is not showing split-brain
>> entries.
>>
>> Regards,
>> Abhishek
>>
>> On Wed, Mar 16, 2016 at 6:06 PM, Anuradha Talur 
>> wrote:
>>
>>>
>>>
>>> - Original Message -
>>> > From: "Anuradha Talur" 
>>> > To: "ABHISHEK PALIWAL" 
>>> > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>>> > Sent: Wednesday, March 16, 2016 5:32:26 PM
>>> > Subject: Re: [Gluster-users] gluster volume heal info split brain
>>> command not showing files in split-brain
>>> >
>>> >
>>> >
>>> > - Original Message -
>>> > > From: "ABHISHEK PALIWAL" 
>>> > > To: "Anuradha Talur" 
>>> > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>>> > > Sent: Wednesday, March 16, 2016 4:39:26 PM
>>> > > Subject: Re: [Gluster-users] gluster volume heal info split brain
>>> command
>>> > > not showing files in split-brain
>>> > >
>>> > > Hi Anuradha,
>>> > >
>>> > > I am doing the same which is mentioned in the link you shared and It
>>> has
>>> > > been resolved the issue.
>>> > >
>>> > > But my question is if it is the split-brain scenario then why the
>>> command
>>> > > "gluster volume heal info split-brain"
>>> > >  not showing these files in the output even not the parent directory
>>> is
>>> > > present in split-brain.
>>> > >
>>> > > Please find the requested logs
>>>
>>> Abhishek,
>>>
>>> Yes, ideally it should show. I will look into it. The only reason I can
>>> think of, is when parent directory did not have any pending markers to
>>> indicate split-brain; which is why I asked getfattr output for the parent
>>> directory too. But if the issue is resolved, there isn't much info we can
>>> get out of it. Thanks for sharing the logs. Will see what could have caused
>>> this.
>>>
>>> > Abhishek,
>>> >
>>> > Yes, ideally it should show. I will look into it.
>>> > I saw another case with this issue. The parent directory did not have
>>> any pe
>>> > >
>>> > > On Wed, Mar 16, 2016 at 4:20 PM, Anuradha Talur 
>>> wrote:
>>> > >
>>> > > > Hi Abhishek,
>>> > > >
>>> > > > The files that are reporting i/o error have gfid-mismatch. This
>>> situation
>>> > > > is called directory or entry split-brain. You can find steps to
>>> resolve
>>> > > > this kind of split brain here :
>>> > > >
>>> https://gluster.readthedocs.org/en/latest/Troubleshooting/split-brain/ .
>>> > > >
>>> > > > Ideally, the parent directories of these files have to be listed
>>> in heal
>>> > > > info split-brain output. Can you please get extended attributes of
>>> parent
>>> > > > directories of the files that show i/o error (Same getfattr
>>> command that
>>> > > > you previously used.) ?
>>> > > >
>>> > > > - Original Message -
>>> > > > > From: "ABHISHEK PALIWAL" 
>>> > > > > To: "Anuradha Talur" 
>>> > > > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>>> > > > > Sent: Thursday, March 10, 2016 11:22:35 AM
>>> > > > > Subject: Re: [Gluster-users] gluster volume heal info split brain
>>> > > > command not showing files in split-brain
>>> > > > >
>>> > > > > Hi Anuradha,
>>> > > > >
>>> > > > > Please find the glusterfs and glusterd logs directory as an
>>> attachment.
>>> > > > >
>>> > > > > Regards,
>>> > > > > Abhishek
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On Wed, Mar 9, 2016 at 5:54 PM, ABHISHEK PALIWAL <
>>> > > > abhishpali...@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hi Anuradha,
>>> > > > > >
>>> > > > > > Sorry for late reply.
>>> > > > > >
>>> > > > > > Please find the requested logs below:
>>> > > > > >
>>> > > > > > Remote: 10.32.0.48
>>> > > > > > Local : 10.32.1.144
>>> > > > > >
>>> > > > > > Local:
>>> > > > > > #gluster volume heal c_glusterfs info split-brain
>>> > > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
>>> > > > > > Number of entries in split-brain: 0
>>> > > > > >
>>> > > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
>>> > > > > > Number of entries in split-brain: 0
>>> > > > > >
>>> > > > > > Remote:
>>> > > > > > #gluster

Re: [Gluster-devel] On backporting fixes

2016-03-19 Thread Raghavendra Talur 
On Wed, Mar 16, 2016 at 11:59 PM, Atin Mukherjee  wrote:

> -Atin
> Sent from one plus one
>
> On 16-Mar-2016 11:32 am, "Raghavendra Talur"  wrote:
> >
> > Hi,
> >
> > Lot many fixes to tests were found to be not back ported to 3.7 and
> other release branches.
> > This causes tests to fail only in those branches and leaves the
> maintainers puzzled.
> >
> > Also, this seems to be the case with back porting code fixes too.
> >
> > I copied all the changes to tests/ dir on master to tests/ dir on 3.7
> branch and posted a patch at http://review.gluster.org/#/c/13683/ .
> >
> > This is failing for ./tests/bugs/distribute/bug-860663.t test :
> > [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> > tar: Removing leading `/' from member names
> > ./tests/bugs/distribute/bug-860663.t ..
> > 1..15
> > ok 1, LINENUM:23
> > ok 2, LINENUM:24
> > ok 3, LINENUM:26
> > ok 4, LINENUM:27
> > ok 5, LINENUM:30
> > ok 6, LINENUM:32
> > ok 7, LINENUM:35
> > not ok 8 , LINENUM:40
> > FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance
> patchy fix-layout start
> > ok 9, LINENUM:42
> > ok 10, LINENUM:43
> > ok 11, LINENUM:45
> > ok 12, LINENUM:47
> > ok 13, LINENUM:50
> > ok 14, LINENUM:51
> > ok 15, LINENUM:55
> > Failed 1/15 subtests
> >
> > Given that it is a simple rebalance command that is failing I am
> assuming that a critical patch has not been back ported to 3.7, correct me
> if I am wrong.
> >
> > I request every developer to take responsibility of back porting patches.
> >
> >
> > Corollary question: Our test-framework is now capable of disabling tests
> for certain OS, certain branch etc. I would like to propose that we stop
> having tests in main git repo. This will remove need to back port test only
> fixes.
> Where are you going to host these .t files then?


Proposal was a separate git repo for tests. It seems very unlikely to
maintain it separately though.
Basically what I am looking for is making tests/ dir independent of git
branch but still part of the git repo.


>
> >
> >
> > Thanks,
> > Raghavendra Talur
> >
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] gluster volume heal info split brain command not showing files in split-brain

2016-03-19 Thread Anuradha Talur 


- Original Message -
> From: "Anuradha Talur" 
> To: "ABHISHEK PALIWAL" 
> Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> Sent: Wednesday, March 16, 2016 5:32:26 PM
> Subject: Re: [Gluster-users] gluster volume heal info split brain command not 
> showing files in split-brain
> 
> 
> 
> - Original Message -
> > From: "ABHISHEK PALIWAL" 
> > To: "Anuradha Talur" 
> > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> > Sent: Wednesday, March 16, 2016 4:39:26 PM
> > Subject: Re: [Gluster-users] gluster volume heal info split brain command
> > not showing files in split-brain
> > 
> > Hi Anuradha,
> > 
> > I am doing the same which is mentioned in the link you shared and It has
> > been resolved the issue.
> > 
> > But my question is if it is the split-brain scenario then why the command
> > "gluster volume heal info split-brain"
> >  not showing these files in the output even not the parent directory is
> > present in split-brain.
> > 
> > Please find the requested logs

Abhishek,

Yes, ideally it should show. I will look into it. The only reason I can think 
of, is when parent directory did not have any pending markers to indicate 
split-brain; which is why I asked getfattr output for the parent directory too. 
But if the issue is resolved, there isn't much info we can get out of it. 
Thanks for sharing the logs. Will see what could have caused this.

> Abhishek,
> 
> Yes, ideally it should show. I will look into it.
> I saw another case with this issue. The parent directory did not have any pe
> > 
> > On Wed, Mar 16, 2016 at 4:20 PM, Anuradha Talur  wrote:
> > 
> > > Hi Abhishek,
> > >
> > > The files that are reporting i/o error have gfid-mismatch. This situation
> > > is called directory or entry split-brain. You can find steps to resolve
> > > this kind of split brain here :
> > > https://gluster.readthedocs.org/en/latest/Troubleshooting/split-brain/ .
> > >
> > > Ideally, the parent directories of these files have to be listed in heal
> > > info split-brain output. Can you please get extended attributes of parent
> > > directories of the files that show i/o error (Same getfattr command that
> > > you previously used.) ?
> > >
> > > - Original Message -
> > > > From: "ABHISHEK PALIWAL" 
> > > > To: "Anuradha Talur" 
> > > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> > > > Sent: Thursday, March 10, 2016 11:22:35 AM
> > > > Subject: Re: [Gluster-users] gluster volume heal info split brain
> > > command not showing files in split-brain
> > > >
> > > > Hi Anuradha,
> > > >
> > > > Please find the glusterfs and glusterd logs directory as an attachment.
> > > >
> > > > Regards,
> > > > Abhishek
> > > >
> > > >
> > > >
> > > > On Wed, Mar 9, 2016 at 5:54 PM, ABHISHEK PALIWAL <
> > > abhishpali...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Anuradha,
> > > > >
> > > > > Sorry for late reply.
> > > > >
> > > > > Please find the requested logs below:
> > > > >
> > > > > Remote: 10.32.0.48
> > > > > Local : 10.32.1.144
> > > > >
> > > > > Local:
> > > > > #gluster volume heal c_glusterfs info split-brain
> > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
> > > > > Number of entries in split-brain: 0
> > > > >
> > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
> > > > > Number of entries in split-brain: 0
> > > > >
> > > > > Remote:
> > > > > #gluster volume heal c_glusterfs info split-brain
> > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
> > > > > Number of entries in split-brain: 0
> > > > >
> > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
> > > > > Number of entries in split-brain: 0
> > > > >
> > > > > auto-sync.sh.
> > > > > Here you can see that i/o error is detected. Below is the required
> > > > > meta
> > > > > data from both the bricks.
> > > > >
> > > > > 1)
> > > > > stat: cannot stat '/mnt/c//public_html/cello/ior_files/nameroot.ior':
> > > > > Input/output error
> > > > > Remote:
> > > > >
> > > > > getfattr -d -m . -e hex
> > > > > opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > trusted.afr.dirty=0x
> > > > > trusted.bit-rot.version=0x000256ded2f6000ad80f
> > > > > trusted.gfid=0x771221a7bb3c4f1aade40ce9e38a95ee
> > > > >
> > > > > Local:
> > > > >
> > > > > getfattr -d -m . -e hex
> > > > > opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > trusted.bit-rot.version=0x000256ded38f000e3a51
> > > > > trusted.gfid=0x8ea33f46703c4e2d95c09153c1b858fd
> > > > >
> > > > >
> > > > > 2)
> > > > > stat: cannot stat '/mnt/c//security/corbasecurity': Input/output
> > > > > error
> > > > > Remote:
> > > > >
> > > > > getfattr -d -m . -e hex

Re: [Gluster-devel] Location of distaf tests

2016-03-19 Thread Niels de Vos 
On Wed, Mar 09, 2016 at 08:26:44PM +0530, M S Vishwanath Bhat wrote:
> On 9 March 2016 at 19:39, Kaushal M  wrote:
> 
> > On Wed, Mar 9, 2016 at 7:02 PM, M S Vishwanath Bhat 
> > wrote:
> > > Hi,
> > >
> > > When we were discussing about the readiness of distaf for upstream test
> > > automation, this question came up, That we should have a process or
> > workflow
> > > for proposing, reviewing and including the tests somewhere.
> > >
> > > Right now the tests are part of distaf repository
> > > (github.com/gluster/distaf) itself. And contributing to distaf is by
> > sending
> > > a PR. But we want this to be included in gerrit so that review and
> > > contributing process becomes much easier. But the question still
> > remains...
> > > where? Right now I can think of below options.
> > >
> > > * Use the same distaf repo in github for tests as well.
> > > * Create a separate repo distaf_gluster_tests (or something similar) and
> > > have all the tests there.
> > > * Or have a tests/distaf/ directory inside glusterfs repository. And this
> > > tests can be bundled in a rpm and distributed. This directory will have
> > both
> > > the test cases and related library functions.
> >
> > I prefer this approach. It makes it easier for developers to submit
> > tests along with their changes, as is the case with our regression
> > tests now.
> >
> > By library functions, I'm assuming you mean helper libraries related
> > to gluster, which will be used in the tests which will be written.
> >
> 
> Yes, I mean helper functions which are related to gluster. The framework
> itself will be  made a python package. At least that's the plan.

+1 for the tests in tests/distaf/ . I think I would like the libs to be
part of the main distaf project. That would make it easier to run distaf
for other projects that integrate with Gluster (like Samba,
NFS-Ganesha and the like). The upstream projects can that use distaf for
their integration with Gluster if they choose so.

> > I'm also in favor of including them here as well. This will help keep
> > DiSTAF free of an Gluster specific cruft and allow it to be (possibly)
> > reusable by others.
> >
> 
> The recent changes makes it specific to gluster, but very easy to make it
> generic.

Starting Gluster specific is fine, later on when things are in use and
working the way we need it should be trivial to make it more generic.
Just keep a generic way for adding new functionalities for now, and file
issues in GitHub for the parts that need to get refactored.

The sooner we can start using distaf and gain (more) real world
experience the better.

Thanks,
Niels


> Best Regards,
> Vishwanath
> 
> 
> >
> > >
> > > Please let us know what your preferred option is. If you have any other
> > > ideas, please let us know as well.
> > >
> > > Best Regards,
> > > Vishwanath
> > >
> > >
> > > ___
> > > Gluster-devel mailing list
> > > Gluster-devel@gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-devel
> >

> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel



signature.asc
Description: PGP signature
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Arbiter brick size estimation

2016-03-19 Thread Oleksandr Natalenko 
And for 256b inode:

(597904 - 33000) / (1066036 - 23) == 530 bytes per inode.

So I still consider 1k to be good estimation for average workload.

Regards,
  Oleksandr.

On четвер, 17 березня 2016 р. 09:58:14 EET Ravishankar N wrote:
> Looks okay to me Oleksandr. You might want to make a github gist of your
> tests+results as a reference for others.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7.9 update

2016-03-19 Thread Raghavendra Gowdappa 
[1] changes the retry behavior of write-behind in case of flush failures. Do 
you think it needs to be called out in release notes?

[1] http://review.gluster.org/12594

regards,
Raghavendra

- Original Message -
> From: "Vijay Bellur" 
> To: "Gluster Devel" , "Niels de Vos" 
> , "Raghavendra Bhat"
> , "Dan Lambright" , "Nithya 
> Balachandran" 
> Sent: Friday, March 18, 2016 6:52:25 AM
> Subject: Re: [Gluster-devel] 3.7.9 update
> 
> A quick update - 3.7.9 has been tagged in the repository. Will send out
> an announcement once the packages and release notes are ready.
> 
> Thanks,
> Vijay
> 
> 
> On 03/13/2016 01:20 PM, Vijay Bellur wrote:
> > Hey All,
> >
> > I have been running tests with the latest HEAD of release-3.7  on a 2x2
> > distributed replicated volume. Here are some updates:
> >
> > - Write Performance has seen an improvement as seen by running
> > perf-test.sh [1]
> >
> >
> > v3.7.9 with FUSE client
> >
> > TestnameTime
> > emptyfiles_create   961.83
> > emptyfiles_delete   600.08
> > smallfiles_create   1508.38
> > smallfiles_rewrite  1325.60
> > smallfiles_read 598.50
> > smallfiles_reread   384.65
> > smallfiles_delete   623.66
> > largefile_create18.33
> > largefile_rewrite   19.17
> > largefile_read  11.44
> > largefile_reread0.31
> > largefile_delete0.66
> > directory_crawl_create  981.21
> > directory_crawl 30.64
> > directory_recrawl   28.01
> > metadata_modify 1117.92
> > directory_crawl_delete  423.08
> >
> > v3.7.8 with FUSE client
> >
> > TestnameTime
> > emptyfiles_create   953.87
> > emptyfiles_delete   577.46
> > smallfiles_create   1837.33
> > smallfiles_rewrite  2349.37
> > smallfiles_read 604.22
> > smallfiles_reread   394.48
> > smallfiles_delete   629.74
> > largefile_create73.86
> > largefile_rewrite   76.23
> > largefile_read  11.36
> > largefile_reread0.31
> > largefile_delete0.65
> > directory_crawl_create  985.16
> > directory_crawl 31.10
> > directory_recrawl   26.94
> > metadata_modify 1422.60
> > directory_crawl_delete  382.57
> >
> > Hopefully this addresses the write performance drop we observed with 3.7.8.
> >
> > - Regular file system test tools like iozone, dbench etc. are running
> > fine with the fuse client.
> >
> > - Rolling upgrade from 3.7.8 to the latest release-3.7 HEAD worked fine
> > with I/O happening from a fuse client.
> >
> > - There is a memory leak in FUSE client that I observed while running
> > perf-test.sh. A statedump revealed that there was a ref leak on several
> > inodes. I have sent a possible patch [2] which addressed problems in my
> > test setup. This does need careful review and more testing. Given the
> > memory leaks we have been observing with fuse, I feel that it would be
> > good to review mount/fuse for possible leaks and run more tests before
> > releasing 3.7.9. I am looking at pushing out tagging by 2-3 days to mid
> > week to accomplish this. Niels, Raghavendra - can you provide additional
> > help with reviewing here?
> >
> > - Tiering has seen a lot of patches in 3.7.9. Dan, Nithya - can you
> > please assist in preparation of release notes by summarizing the changes
> > and providing inputs on the general readiness of tiering?
> >
> > Thanks,
> > Vijay
> >
> > [1] https://github.com/avati/perf-test/blob/master/perf-test.sh
> >
> > [2] http://review.gluster.org/#/c/13689/
> >
> >
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Arbiter brick size estimation

2016-03-19 Thread Oleksandr Natalenko 
Ravi, I will definitely arrange the results into some short handy 
document and post it here.


Also, @JoeJulian on IRC suggested me to perform this test on XFS bricks 
with inode size of 256b and 1k:


===
22:38 <@JoeJulian> post-factum: Just wondering what 256 byte inodes 
might look like for that. And, by the same token, 1k inodes.

22:39 < post-factum> JoeJulian: should I try 1k inodes instead?
22:41 <@JoeJulian> post-factum: Doesn't hurt to try. My expectation is 
that disk usage will go up despite inode usage going down.

22:41 < post-factum> JoeJulian: ok, will check that
22:41 <@JoeJulian> post-factum: and with 256, I'm curious if inode usage 
will stay close to the same while disk usage goes down.

===

Here are the results for 1k:

(1171336 - 33000) / (1066036 - 23) == 1068 bytes per inode.

Disk usage is indeed higher (1.2G), but inodes usage is the same.

Will test with 256b inode now.

17.03.2016 06:28, Ravishankar N wrote:

Looks okay to me Oleksandr. You might want to make a github gist of
your tests+results as a reference for others.

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Arbiter brick size estimation

2016-03-19 Thread Oleksandr Natalenko 
OK, I've repeated the test with the following hierarchy:

* 10 top-level folders with 10 second-level folders each;
* 10 000 files in each second-level folder.

So, this composes 10×10×1=1M files and 100 folders

Initial brick used space: 33 M
Initial inodes count: 24

After test:

* each brick in replica took 18G, and the arbiter brick took 836M;
* inodes count: 1066036

So:

(836 - 33) / (1066036 - 24) == 790 bytes per inode.

So, yes, it is slightly bigger value than with previous test due to, I guess, 
lots of files in one folder, but it is still too far from 4k. Given a good 
engineer should consider 30% reserve, the ratio is about 1k per stored inode.

Correct me if I'm missing something (regarding average workload and not corner 
cases).

Test script is here: [1]

Regards,
  Oleksandr.

[1] http://termbin.com/qlvz

On вівторок, 8 березня 2016 р. 19:13:05 EET Ravishankar N wrote:
> On 03/05/2016 03:45 PM, Oleksandr Natalenko wrote:
> > In order to estimate GlusterFS arbiter brick size, I've deployed test
> > setup
> > with replica 3 arbiter 1 volume within one node. Each brick is located on
> > separate HDD (XFS with inode size == 512). Using GlusterFS v3.7.6 +
> > memleak
> > patches. Volume options are kept default.
> > 
> > Here is the script that creates files and folders in mounted volume: [1]
> > 
> > The script creates 1M of files of random size (between 1 and 32768 bytes)
> > and some amount of folders. After running it I've got 1036637 folders.
> > So, in total it is 2036637 files and folders.
> > 
> > The initial used space on each brick is 42M . After running script I've
> > got:
> > 
> > replica brick 1 and 2: 19867168 kbytes == 19G
> > arbiter brick: 1872308 kbytes == 1.8G
> > 
> > The amount of inodes on each brick is 3139091. So here goes estimation.
> > 
> > Dividing arbiter used space by files+folders we get:
> > 
> > (1872308 - 42000)/2036637 == 899 bytes per file or folder
> > 
> > Dividing arbiter used space by inodes we get:
> > 
> > (1872308 - 42000)/3139091 == 583 bytes per inode
> > 
> > Not sure about what calculation is correct.
> 
> I think the first one is right because you still haven't used up all the
> inodes.(2036637 used vs. the max. permissible 3139091). But again this
> is an approximation because not all files would be 899 bytes. For
> example if there are a thousand files present in a directory, then du
>  would be more than du  because the directory will take
> some disk space to store the dentries.
> 
> >   I guess we should consider the one
> > 
> > that accounts inodes because of .glusterfs/ folder data.
> > 
> > Nevertheless, in contrast, documentation [2] says it should be 4096 bytes
> > per file. Am I wrong with my calculations?
> 
> The 4KB is a conservative estimate considering the fact that though the
> arbiter brick does not store data, it still keeps a copy of both user
> and gluster xattrs. For example, if the application sets a lot of
> xattrs, it can consume a data block if they cannot be accommodated on
> the inode itself.  Also there is the .glusterfs folder like you said
> which would take up some space. Here is what I tried on an XFS brick:
> [root@ravi4 brick]# touch file
> 
> [root@ravi4 brick]# ls -l file
> -rw-r--r-- 1 root root 0 Mar  8 12:54 file
> 
> [root@ravi4 brick]# du file
> *0   file**
> *
> [root@ravi4 brick]# for i in {1..100}
> 
>  > do
>  > setfattr -n user.value$i -v value$i file
>  > done
> 
> [root@ravi4 brick]# ll -l file
> -rw-r--r-- 1 root root 0 Mar  8 12:54 file
> 
> [root@ravi4 brick]# du -h file
> *4.0Kfile**
> *
> Hope this helps,
> Ravi
> 
> > Pranith?
> > 
> > [1] http://termbin.com/ka9x
> > [2]
> > http://gluster.readthedocs.org/en/latest/Administrator%20Guide/arbiter-vo
> > lumes-and-quorum/ ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel


___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

2016-03-19 Thread Raghavendra Talur 
On Mar 17, 2016 7:50 AM, "Pranith Kumar Karampuri" 
wrote:
>
>
>
> On 03/16/2016 11:46 PM, Raghavendra Talur wrote:
>>
>>
>>
>> On Wed, Mar 16, 2016 at 11:39 AM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:
>>>
>>>
>>>
>>> On 03/16/2016 11:31 AM, Raghavendra Talur wrote:

 Hi,

 Lot many fixes to tests were found to be not back ported to 3.7 and
other release branches.
 This causes tests to fail only in those branches and leaves the
maintainers puzzled.

 Also, this seems to be the case with back porting code fixes too.

 I copied all the changes to tests/ dir on master to tests/ dir on 3.7
branch and posted a patch at http://review.gluster.org/#/c/13683/ .

 This is failing for ./tests/bugs/distribute/bug-860663.t test :
 [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
 tar: Removing leading `/' from member names
 ./tests/bugs/distribute/bug-860663.t ..
 1..15
 ok 1, LINENUM:23
 ok 2, LINENUM:24
 ok 3, LINENUM:26
 ok 4, LINENUM:27
 ok 5, LINENUM:30
 ok 6, LINENUM:32
 ok 7, LINENUM:35
 not ok 8 , LINENUM:40
 FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance
patchy fix-layout start
 ok 9, LINENUM:42
 ok 10, LINENUM:43
 ok 11, LINENUM:45
 ok 12, LINENUM:47
 ok 13, LINENUM:50
 ok 14, LINENUM:51
 ok 15, LINENUM:55
 Failed 1/15 subtests

 Given that it is a simple rebalance command that is failing I am
assuming that a critical patch has not been back ported to 3.7, correct me
if I am wrong.

 I request every developer to take responsibility of back porting
patches.


 Corollary question: Our test-framework is now capable of disabling
tests for certain OS, certain branch etc. I would like to propose that we
stop having tests in main git repo. This will remove need to back port test
only fixes.
>>>
>>>
>>> Some times what I do is to enhance existing test to handle extra cases
based on new code that is added on master. Until the code-fix is not
backported to lower versions, the tests are not valid. Should we mark such
tests disabled when we do enhancements to .t files?
>>
>>
>> Yes, this case would pose a problem and hence any new code change which
introduces a feature should be tested by a new test(not by modifying
existing test) and bug fix should be immediately backported along with test
fix.
>
>
> As Atin mentioned in another mail, it is better to have code + test files
which test code together.

Makes sense,  so proactive back porting of fixes is what we require.

>
> Pranith
>
>>
>>>
>>>
>>> Pranith



 Thanks,
 Raghavendra Talur



 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>>
>>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] gluster volume heal info split brain command not showing files in split-brain

2016-03-19 Thread Anuradha Talur 


- Original Message -
> From: "ABHISHEK PALIWAL" 
> To: "Anuradha Talur" 
> Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> Sent: Thursday, March 17, 2016 4:00:58 PM
> Subject: Re: [Gluster-users] gluster volume heal info split brain command not 
> showing files in split-brain
> 
> Hi Anuradha,
> 
> Please confirm me, this is bug in glusterfs or we need to do something at
> our end.
> 
> Because this problem is stopping our development.
Hi Abhishek,

When you say file is not getting sync, do you mean that the files are not in 
sync after healing or that the existing GFID mismatch that you tried to heal 
failed?
In one of the previous mails, you said that the GFID mismatch problem is 
resolved, is it not so?

To your question about finding the files in split-brain, can you try running 
gluster volume heal  info? Heal info is also supposed to show
the files in split-brain.

If the GFID mismatch is not resolved yet, it would really help understand the 
underlying problem if you give the output of getfattr -m. -de hex 
.
> 
> Regards,
> Abhishek
> 
> On Thu, Mar 17, 2016 at 1:54 PM, ABHISHEK PALIWAL 
> wrote:
> 
> > Hi Anuradha,
> >
> > But in this case I need to do tail on each file which is time taking
> > process and other end I can't pause my module until these file is getting
> > healed.
> >
> > Any how I need the output of the split-brain to resolve this problem.
> >
> > Regards,
> > Abhishek
> >
> > On Wed, Mar 16, 2016 at 6:21 PM, ABHISHEK PALIWAL  > > wrote:
> >
> >> Hi Anuradha,
> >>
> >> The issue is resolved but we have one more issue something similar to
> >> this one in which the file is not getting sync after the steps followed,
> >> mentioned in the link which you shared in the previous mail.
> >>
> >> And problem is that why split-brain command is not showing split-brain
> >> entries.
> >>
> >> Regards,
> >> Abhishek
> >>
> >> On Wed, Mar 16, 2016 at 6:06 PM, Anuradha Talur 
> >> wrote:
> >>
> >>>
> >>>
> >>> - Original Message -
> >>> > From: "Anuradha Talur" 
> >>> > To: "ABHISHEK PALIWAL" 
> >>> > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> >>> > Sent: Wednesday, March 16, 2016 5:32:26 PM
> >>> > Subject: Re: [Gluster-users] gluster volume heal info split brain
> >>> command not showing files in split-brain
> >>> >
> >>> >
> >>> >
> >>> > - Original Message -
> >>> > > From: "ABHISHEK PALIWAL" 
> >>> > > To: "Anuradha Talur" 
> >>> > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> >>> > > Sent: Wednesday, March 16, 2016 4:39:26 PM
> >>> > > Subject: Re: [Gluster-users] gluster volume heal info split brain
> >>> command
> >>> > > not showing files in split-brain
> >>> > >
> >>> > > Hi Anuradha,
> >>> > >
> >>> > > I am doing the same which is mentioned in the link you shared and It
> >>> has
> >>> > > been resolved the issue.
> >>> > >
> >>> > > But my question is if it is the split-brain scenario then why the
> >>> command
> >>> > > "gluster volume heal info split-brain"
> >>> > >  not showing these files in the output even not the parent directory
> >>> is
> >>> > > present in split-brain.
> >>> > >
> >>> > > Please find the requested logs
> >>>
> >>> Abhishek,
> >>>
> >>> Yes, ideally it should show. I will look into it. The only reason I can
> >>> think of, is when parent directory did not have any pending markers to
> >>> indicate split-brain; which is why I asked getfattr output for the parent
> >>> directory too. But if the issue is resolved, there isn't much info we can
> >>> get out of it. Thanks for sharing the logs. Will see what could have
> >>> caused
> >>> this.
> >>>
> >>> > Abhishek,
> >>> >
> >>> > Yes, ideally it should show. I will look into it.
> >>> > I saw another case with this issue. The parent directory did not have
> >>> any pe
> >>> > >
> >>> > > On Wed, Mar 16, 2016 at 4:20 PM, Anuradha Talur 
> >>> wrote:
> >>> > >
> >>> > > > Hi Abhishek,
> >>> > > >
> >>> > > > The files that are reporting i/o error have gfid-mismatch. This
> >>> situation
> >>> > > > is called directory or entry split-brain. You can find steps to
> >>> resolve
> >>> > > > this kind of split brain here :
> >>> > > >
> >>> https://gluster.readthedocs.org/en/latest/Troubleshooting/split-brain/ .
> >>> > > >
> >>> > > > Ideally, the parent directories of these files have to be listed
> >>> in heal
> >>> > > > info split-brain output. Can you please get extended attributes of
> >>> parent
> >>> > > > directories of the files that show i/o error (Same getfattr
> >>> command that
> >>> > > > you previously used.) ?
> >>> > > >
> >>> > > > - Original Message -
> >>> > > > > From: "ABHISHEK PALIWAL" 
> >>> > > > > To: "Anuradha Talur" 
> >>> > > > > Cc:

Re: [Gluster-devel] netbsd regression failure in open-behind.t

2016-03-19 Thread Emmanuel Dreyfus 
On Fri, Mar 18, 2016 at 09:08:04AM -0400, Prasanna Kumar Kalever wrote:

gluster volume top $V0 open | grep -w "$F0" >/dev/null 2>&1
TEST [ $? -eq 0 ];

What do we expect here and what do we get?

I note that the test fails either if glustrer volume top fails, 
ot if its output does not contain $F0 (why not use fgrep "$F0" ?)

What happens? Removing  >/dev/null 2>&1 above may be insighful.

-- 
Emmanuel Dreyfus
m...@netbsd.org
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] gluster volume heal info split brain command not showing files in split-brain

2016-03-19 Thread ABHISHEK PALIWAL 
Hi Anuradha,

But in this case I need to do tail on each file which is time taking
process and other end I can't pause my module until these file is getting
healed.

Any how I need the output of the split-brain to resolve this problem.

Regards,
Abhishek

On Wed, Mar 16, 2016 at 6:21 PM, ABHISHEK PALIWAL 
wrote:

> Hi Anuradha,
>
> The issue is resolved but we have one more issue something similar to this
> one in which the file is not getting sync after the steps followed,
> mentioned in the link which you shared in the previous mail.
>
> And problem is that why split-brain command is not showing split-brain
> entries.
>
> Regards,
> Abhishek
>
> On Wed, Mar 16, 2016 at 6:06 PM, Anuradha Talur  wrote:
>
>>
>>
>> - Original Message -
>> > From: "Anuradha Talur" 
>> > To: "ABHISHEK PALIWAL" 
>> > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>> > Sent: Wednesday, March 16, 2016 5:32:26 PM
>> > Subject: Re: [Gluster-users] gluster volume heal info split brain
>> command not showing files in split-brain
>> >
>> >
>> >
>> > - Original Message -
>> > > From: "ABHISHEK PALIWAL" 
>> > > To: "Anuradha Talur" 
>> > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>> > > Sent: Wednesday, March 16, 2016 4:39:26 PM
>> > > Subject: Re: [Gluster-users] gluster volume heal info split brain
>> command
>> > > not showing files in split-brain
>> > >
>> > > Hi Anuradha,
>> > >
>> > > I am doing the same which is mentioned in the link you shared and It
>> has
>> > > been resolved the issue.
>> > >
>> > > But my question is if it is the split-brain scenario then why the
>> command
>> > > "gluster volume heal info split-brain"
>> > >  not showing these files in the output even not the parent directory
>> is
>> > > present in split-brain.
>> > >
>> > > Please find the requested logs
>>
>> Abhishek,
>>
>> Yes, ideally it should show. I will look into it. The only reason I can
>> think of, is when parent directory did not have any pending markers to
>> indicate split-brain; which is why I asked getfattr output for the parent
>> directory too. But if the issue is resolved, there isn't much info we can
>> get out of it. Thanks for sharing the logs. Will see what could have caused
>> this.
>>
>> > Abhishek,
>> >
>> > Yes, ideally it should show. I will look into it.
>> > I saw another case with this issue. The parent directory did not have
>> any pe
>> > >
>> > > On Wed, Mar 16, 2016 at 4:20 PM, Anuradha Talur 
>> wrote:
>> > >
>> > > > Hi Abhishek,
>> > > >
>> > > > The files that are reporting i/o error have gfid-mismatch. This
>> situation
>> > > > is called directory or entry split-brain. You can find steps to
>> resolve
>> > > > this kind of split brain here :
>> > > >
>> https://gluster.readthedocs.org/en/latest/Troubleshooting/split-brain/ .
>> > > >
>> > > > Ideally, the parent directories of these files have to be listed in
>> heal
>> > > > info split-brain output. Can you please get extended attributes of
>> parent
>> > > > directories of the files that show i/o error (Same getfattr command
>> that
>> > > > you previously used.) ?
>> > > >
>> > > > - Original Message -
>> > > > > From: "ABHISHEK PALIWAL" 
>> > > > > To: "Anuradha Talur" 
>> > > > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
>> > > > > Sent: Thursday, March 10, 2016 11:22:35 AM
>> > > > > Subject: Re: [Gluster-users] gluster volume heal info split brain
>> > > > command not showing files in split-brain
>> > > > >
>> > > > > Hi Anuradha,
>> > > > >
>> > > > > Please find the glusterfs and glusterd logs directory as an
>> attachment.
>> > > > >
>> > > > > Regards,
>> > > > > Abhishek
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, Mar 9, 2016 at 5:54 PM, ABHISHEK PALIWAL <
>> > > > abhishpali...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Anuradha,
>> > > > > >
>> > > > > > Sorry for late reply.
>> > > > > >
>> > > > > > Please find the requested logs below:
>> > > > > >
>> > > > > > Remote: 10.32.0.48
>> > > > > > Local : 10.32.1.144
>> > > > > >
>> > > > > > Local:
>> > > > > > #gluster volume heal c_glusterfs info split-brain
>> > > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
>> > > > > > Number of entries in split-brain: 0
>> > > > > >
>> > > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
>> > > > > > Number of entries in split-brain: 0
>> > > > > >
>> > > > > > Remote:
>> > > > > > #gluster volume heal c_glusterfs info split-brain
>> > > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
>> > > > > > Number of entries in split-brain: 0
>> > > > > >
>> > > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
>> > > > > > Number of entries in split-brain: 0
>> > > > > >
>> > > > > > auto-sync.sh.
>> > > > > > Here you can see that i/o error is detected. Below is the
>> required
>> > > > > > meta
>> > > > > >

Re: [Gluster-devel] [Gluster-users] gluster volume heal info split brain command not showing files in split-brain

2016-03-19 Thread ABHISHEK PALIWAL 
Hi Anuradha,

The issue is resolved but we have one more issue something similar to this
one in which the file is not getting sync after the steps followed,
mentioned in the link which you shared in the previous mail.

And problem is that why split-brain command is not showing split-brain
entries.

Regards,
Abhishek

On Wed, Mar 16, 2016 at 6:06 PM, Anuradha Talur  wrote:

>
>
> - Original Message -
> > From: "Anuradha Talur" 
> > To: "ABHISHEK PALIWAL" 
> > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> > Sent: Wednesday, March 16, 2016 5:32:26 PM
> > Subject: Re: [Gluster-users] gluster volume heal info split brain
> command not showing files in split-brain
> >
> >
> >
> > - Original Message -
> > > From: "ABHISHEK PALIWAL" 
> > > To: "Anuradha Talur" 
> > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> > > Sent: Wednesday, March 16, 2016 4:39:26 PM
> > > Subject: Re: [Gluster-users] gluster volume heal info split brain
> command
> > > not showing files in split-brain
> > >
> > > Hi Anuradha,
> > >
> > > I am doing the same which is mentioned in the link you shared and It
> has
> > > been resolved the issue.
> > >
> > > But my question is if it is the split-brain scenario then why the
> command
> > > "gluster volume heal info split-brain"
> > >  not showing these files in the output even not the parent directory is
> > > present in split-brain.
> > >
> > > Please find the requested logs
>
> Abhishek,
>
> Yes, ideally it should show. I will look into it. The only reason I can
> think of, is when parent directory did not have any pending markers to
> indicate split-brain; which is why I asked getfattr output for the parent
> directory too. But if the issue is resolved, there isn't much info we can
> get out of it. Thanks for sharing the logs. Will see what could have caused
> this.
>
> > Abhishek,
> >
> > Yes, ideally it should show. I will look into it.
> > I saw another case with this issue. The parent directory did not have
> any pe
> > >
> > > On Wed, Mar 16, 2016 at 4:20 PM, Anuradha Talur 
> wrote:
> > >
> > > > Hi Abhishek,
> > > >
> > > > The files that are reporting i/o error have gfid-mismatch. This
> situation
> > > > is called directory or entry split-brain. You can find steps to
> resolve
> > > > this kind of split brain here :
> > > >
> https://gluster.readthedocs.org/en/latest/Troubleshooting/split-brain/ .
> > > >
> > > > Ideally, the parent directories of these files have to be listed in
> heal
> > > > info split-brain output. Can you please get extended attributes of
> parent
> > > > directories of the files that show i/o error (Same getfattr command
> that
> > > > you previously used.) ?
> > > >
> > > > - Original Message -
> > > > > From: "ABHISHEK PALIWAL" 
> > > > > To: "Anuradha Talur" 
> > > > > Cc: gluster-us...@gluster.org, gluster-devel@gluster.org
> > > > > Sent: Thursday, March 10, 2016 11:22:35 AM
> > > > > Subject: Re: [Gluster-users] gluster volume heal info split brain
> > > > command not showing files in split-brain
> > > > >
> > > > > Hi Anuradha,
> > > > >
> > > > > Please find the glusterfs and glusterd logs directory as an
> attachment.
> > > > >
> > > > > Regards,
> > > > > Abhishek
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 9, 2016 at 5:54 PM, ABHISHEK PALIWAL <
> > > > abhishpali...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Anuradha,
> > > > > >
> > > > > > Sorry for late reply.
> > > > > >
> > > > > > Please find the requested logs below:
> > > > > >
> > > > > > Remote: 10.32.0.48
> > > > > > Local : 10.32.1.144
> > > > > >
> > > > > > Local:
> > > > > > #gluster volume heal c_glusterfs info split-brain
> > > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
> > > > > > Number of entries in split-brain: 0
> > > > > >
> > > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
> > > > > > Number of entries in split-brain: 0
> > > > > >
> > > > > > Remote:
> > > > > > #gluster volume heal c_glusterfs info split-brain
> > > > > > Brick 10.32.1.144:/opt/lvmdir/c2/brick
> > > > > > Number of entries in split-brain: 0
> > > > > >
> > > > > > Brick 10.32.0.48:/opt/lvmdir/c2/brick
> > > > > > Number of entries in split-brain: 0
> > > > > >
> > > > > > auto-sync.sh.
> > > > > > Here you can see that i/o error is detected. Below is the
> required
> > > > > > meta
> > > > > > data from both the bricks.
> > > > > >
> > > > > > 1)
> > > > > > stat: cannot stat
> '/mnt/c//public_html/cello/ior_files/nameroot.ior':
> > > > > > Input/output error
> > > > > > Remote:
> > > > > >
> > > > > > getfattr -d -m . -e hex
> > > > > > opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > > # file:
> opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior
> > > > > > trusted.afr.dirty=0x
> > > > > >

Re: [Gluster-devel] Added specs for filesnapshots based on reflinks

2016-03-19 Thread Prasanna Kumar Kalever 
On Thursday, March 17, 2016 11:25:33 AM, Jiffin Tony Thottan wrote:
> On 16/03/16 17:10, Prasanna Kumar Kalever wrote:
> > On Wednesday, March 16, 2016 3:55:52 PM, Niels de Vos wrote:
> >> On Wed, Mar 16, 2016 at 04:00:12AM -0400, Prasanna Kumar Kalever wrote:
> >>> Hi,
> >>>
> >>> Regarding existing file snapshot:
> >>> 
> >>> Currently we have qemu-block xlator which helps in creating a file based
> >>> snapshots,
> >>> the biggest limitation about this feature is that, it allows creation of
> >>> snapshots
> >>> only for qcow2 file types (QEMU specific).
> >>>
> >>> what about other VM image formats (say VMDK, VDI ...)?
> >>> what about general file formats (txt, docx, md ... another 1)?
> >>>
> >>>
> >>> Improvement:
> >>> 
> >>> Suggested a feature that allows you to take a snapshot on any kind of
> >>> file
> >>> with the
> >>> help of reflinks.
> >>>
> >>> Here is the link to specs page (which also has a POC);
> >>> https://github.com/gluster/glusterfs-specs/compare/master...pkalever:master
> >>>
> >>> I need all of your blessings to move this into action.
> >> We do not use GitHub pull requests for the glusterfs-specs repository.
> >> See the main README for sending your change to Gerrit.
> >>
> >> I've left some comments in the GitHub patch. It's tricky to find where
> >> the comments are, but I think this link should do it:
> >>
> >>
> >> https://github.com/gluster/glusterfs-specs/commit/eb00fae40686bf5693242f389bbd9d0b7e0baede#diff-189cd0b816a5a125cfaccf7434dd6d57R46
> > Thanks Niels,
> >
> > I have posted it at,
> > http://review.gluster.org/#/c/13752/
> >
> > Also replied back on your comments on github.
> > It will great if we can update the same in gerrit.
> >
> > --
> > Prasanna
> >
> 
> Hi Prasanna,
> 
> Are you targeting file snapshot feature for 3.8 or 3.8.x release ?
> If yes can please create pull request for adding it to 3.8 roadmap[1].
> 
> [1]
> https://github.com/gluster/glusterweb/edit/master/source/community/roadmap/3.8/index.md
> 

Thanks Jiffin, 3.8 will be too early for this to go in.
However we cannot add sanpshot feature in 3.8.x releases,
so ideally this should go in 4.0 or 3.9 if there is a plan.

Feel free to review the http://review.gluster.org/#/c/13752/
Any further Improvements are most welcome.

--
Prasanna
 
> --
> Jiffin
> >> Thanks,
> >> Niels
> >>
> >> ___
> >> Gluster-devel mailing list
> >> Gluster-devel@gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-devel
> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Location of distaf tests

2016-03-19 Thread Jonathan Holloway 


- Original Message -


From: "M S Vishwanath Bhat"  
To: "Niels de Vos"  
Cc: "Gluster Devel"  
Sent: Thursday, March 17, 2016 8:18:23 AM 
Subject: Re: [Gluster-devel] Location of distaf tests 



On 17 March 2016 at 10:50, Niels de Vos < nde...@redhat.com > wrote: 



On Wed, Mar 09, 2016 at 08:26:44PM +0530, M S Vishwanath Bhat wrote: 
> On 9 March 2016 at 19:39, Kaushal M < kshlms...@gmail.com > wrote: 
> 
> > On Wed, Mar 9, 2016 at 7:02 PM, M S Vishwanath Bhat < msvb...@gmail.com > 
> > wrote: 
> > > Hi, 
> > > 
> > > When we were discussing about the readiness of distaf for upstream test 
> > > automation, this question came up, That we should have a process or 
> > workflow 
> > > for proposing, reviewing and including the tests somewhere. 
> > > 
> > > Right now the tests are part of distaf repository 
> > > ( github.com/gluster/distaf ) itself. And contributing to distaf is by 
> > sending 
> > > a PR. But we want this to be included in gerrit so that review and 
> > > contributing process becomes much easier. But the question still 
> > remains... 
> > > where? Right now I can think of below options. 
> > > 
> > > * Use the same distaf repo in github for tests as well. 
> > > * Create a separate repo distaf_gluster_tests (or something similar) and 
> > > have all the tests there. 
> > > * Or have a tests/distaf/ directory inside glusterfs repository. And this 
> > > tests can be bundled in a rpm and distributed. This directory will have 
> > both 
> > > the test cases and related library functions. 
> > 
> > I prefer this approach. It makes it easier for developers to submit 
> > tests along with their changes, as is the case with our regression 
> > tests now. 
> > 
> > By library functions, I'm assuming you mean helper libraries related 
> > to gluster, which will be used in the tests which will be written. 
> > 
> 
> Yes, I mean helper functions which are related to gluster. The framework 
> itself will be made a python package. At least that's the plan. 

+1 for the tests in tests/distaf/ . I think I would like the libs to be 
part of the main distaf project. That would make it easier to run distaf 
for other projects that integrate with Gluster (like Samba, 
NFS-Ganesha and the like). The upstream projects can that use distaf for 
their integration with Gluster if they choose so. 




My concern with having libs in main distaf project is that, libs and test cases 
are very much tied together. So when someone writes a testcase, they write 
and/or modify the related gluster libs along with it. And the one needs to go 
back and forth between the testcase and gluster libs while writing and 
debugging the case. If they are in separate repo/package, it makes it bit 
difficult (not impossible) for test case writer. 

If everyone is of the same opinion that gluster libs should be part of main 
distaf repo, I can still do it when I package it next week. 




I'm now thinking we should go ahead and keep it three distinct projects: 
- Gluster: testcase--and supporting functions that are not a good fit for the 
libs 
- distaf-libs-gluster: core libarary and utility functions 
- distaf: framework only 

Here's some of my reasoning... 
- We're already talking about not putting the libs in the Gluster repo as an 
option, so from a convenience perspective it's not as relevant which repo is 
being used (distaf or distaf-gluster-libs). 
- And if we're already talking about the possibility of a refactor to split the 
libs out down the road, it's not much different to talk refactor to roll the 
libs into the distaf project should they become a problem to maintain. 
- With that said, if looking to have distaf adopted by a wider non-Gluster 
audience, we'll want to split the libs out down the road anyway. It would also 
lay the foundation for others (distaf-libs-). 
- The Maintainer/Reviewer/Contributor demographic isn't necessarily the same 
across all three and the Maintainer/Reviewer will most likely play a larger 
role initially as we build out the libs and figure out what goes where, what 
works, etc. 
I'm just thinking it would be easier to manage that separately in the 
beginning. 
- If we are including the libs in the distaf project, how will the libs be 
consumed? Roll them in the distaf package? Separate package? Git clone and 
setup.py? 
I like the concept of separate distaf and distaf-libs-gluster packages that can 
be versioned and maintained separately. It keeps them flexible and clean/clear 
of each other, and any framework changes that are required by a library change 
can be managed through the packaging. 
We can still do that if the libs are in the same repo, but in my opinion it's 
not as clean and the value add in having them combined is minimal. 
- It shouldn't take that long to create the new project and may save us some 
inevitable headache down the road. 

Either way, I agree with Niels that the sooner the

Re: [Gluster-devel] regression failures in bug-860663.t

2016-03-19 Thread Prasanna Kumar Kalever 
Can you paste the link here ? 

-Prasanna ​ 
- Original Message -

> From: "Sakshi Bansal" 
> To: "Prasanna Kumar Kalever" 
> Cc: "Gluster Devel" 
> Sent: Wednesday, March 16, 2016 3:35:04 PM
> Subject: Re: [Gluster-devel] regression failures in bug-860663.t

> There is a fix for the test case that went after this regression failure. I
> suppose a rebase should fix the issue.

> - Original Message -

> From: "Prasanna Kumar Kalever" 
> To: "Sakshi Bansal" 
> Cc: "Gluster Devel" 
> Sent: Wednesday, March 16, 2016 3:14:35 PM
> Subject: Re: [Gluster-devel] regression failures in bug-860663.t

> I have noticed it now, a mail thread was created by rtalur with the subject
> "[Gluster-devel] On backporting fixes"

> below are some of the commits failed at bug-860663.t:

> https://build.gluster.org/job/rackspace-regression-2GB-triggered/18598/console
> https://build.gluster.org/job/rackspace-regression-2GB-triggered/18603/console

> -Prasanna ​

> - Original Message -

> > From: "Sakshi Bansal" 
> > To: "Prasanna Kumar Kalever" 
> > Cc: "Gluster Devel" 
> > Sent: Wednesday, March 16, 2016 3:04:25 PM
> > Subject: Re: [Gluster-devel] regression failures in bug-860663.t

> > Hi,

> > Can you send any one patch for which you saw the failure.

> > - Original Message -

> > From: "Prasanna Kumar Kalever" 
> > To: "Gluster Devel" 
> > Sent: Wednesday, March 16, 2016 2:54:45 PM
> > Subject: [Gluster-devel] regression failures in bug-860663.t

> > Hi,

> > Noticed too many regression failures in
> > ./tests/bugs/distribute/bug-860663.t

> > Is someone working on this?

> > -Prasanna ​

> > ___
> > Gluster-devel mailing list
> > Gluster-devel@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

2016-03-19 Thread Atin Mukherjee 
-Atin
Sent from one plus one
On 16-Mar-2016 11:32 am, "Raghavendra Talur"  wrote:
>
> Hi,
>
> Lot many fixes to tests were found to be not back ported to 3.7 and other
release branches.
> This causes tests to fail only in those branches and leaves the
maintainers puzzled.
>
> Also, this seems to be the case with back porting code fixes too.
>
> I copied all the changes to tests/ dir on master to tests/ dir on 3.7
branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>
> This is failing for ./tests/bugs/distribute/bug-860663.t test :
> [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> tar: Removing leading `/' from member names
> ./tests/bugs/distribute/bug-860663.t ..
> 1..15
> ok 1, LINENUM:23
> ok 2, LINENUM:24
> ok 3, LINENUM:26
> ok 4, LINENUM:27
> ok 5, LINENUM:30
> ok 6, LINENUM:32
> ok 7, LINENUM:35
> not ok 8 , LINENUM:40
> FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance patchy
fix-layout start
> ok 9, LINENUM:42
> ok 10, LINENUM:43
> ok 11, LINENUM:45
> ok 12, LINENUM:47
> ok 13, LINENUM:50
> ok 14, LINENUM:51
> ok 15, LINENUM:55
> Failed 1/15 subtests
>
> Given that it is a simple rebalance command that is failing I am assuming
that a critical patch has not been back ported to 3.7, correct me if I am
wrong.
>
> I request every developer to take responsibility of back porting patches.
>
>
> Corollary question: Our test-framework is now capable of disabling tests
for certain OS, certain branch etc. I would like to propose that we stop
having tests in main git repo. This will remove need to back port test only
fixes.
Where are you going to host these .t files then?
>
>
> Thanks,
> Raghavendra Talur
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

2016-03-19 Thread Atin Mukherjee 
-Atin
Sent from one plus one
On 17-Mar-2016 12:02 am, "Raghavendra Talur"  wrote:
>
>
>
> On Wed, Mar 16, 2016 at 11:59 PM, Atin Mukherjee <
atin.mukherje...@gmail.com> wrote:
>>
>> -Atin
>> Sent from one plus one
>>
>>
>> On 16-Mar-2016 11:32 am, "Raghavendra Talur"  wrote:
>> >
>> > Hi,
>> >
>> > Lot many fixes to tests were found to be not back ported to 3.7 and
other release branches.
>> > This causes tests to fail only in those branches and leaves the
maintainers puzzled.
>> >
>> > Also, this seems to be the case with back porting code fixes too.
>> >
>> > I copied all the changes to tests/ dir on master to tests/ dir on 3.7
branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>> >
>> > This is failing for ./tests/bugs/distribute/bug-860663.t test :
>> > [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
>> > tar: Removing leading `/' from member names
>> > ./tests/bugs/distribute/bug-860663.t ..
>> > 1..15
>> > ok 1, LINENUM:23
>> > ok 2, LINENUM:24
>> > ok 3, LINENUM:26
>> > ok 4, LINENUM:27
>> > ok 5, LINENUM:30
>> > ok 6, LINENUM:32
>> > ok 7, LINENUM:35
>> > not ok 8 , LINENUM:40
>> > FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance
patchy fix-layout start
>> > ok 9, LINENUM:42
>> > ok 10, LINENUM:43
>> > ok 11, LINENUM:45
>> > ok 12, LINENUM:47
>> > ok 13, LINENUM:50
>> > ok 14, LINENUM:51
>> > ok 15, LINENUM:55
>> > Failed 1/15 subtests
>> >
>> > Given that it is a simple rebalance command that is failing I am
assuming that a critical patch has not been back ported to 3.7, correct me
if I am wrong.
>> >
>> > I request every developer to take responsibility of back porting
patches.
>> >
>> >
>> > Corollary question: Our test-framework is now capable of disabling
tests for certain OS, certain branch etc. I would like to propose that we
stop having tests in main git repo. This will remove need to back port test
only fixes.
>> Where are you going to host these .t files then?
>
>
> Proposal was a separate git repo for tests. It seems very unlikely to
maintain it separately though.
> Basically what I am looking for is making tests/ dir independent of git
branch but still part of the git repo.
-1, tests can't be independent. You may see a test passing in one branch
but failing in other where there is no delta on the test file but some
other code changes cause this behaviour change. If developers can be
proactive on sending the complete backports I don't see any issue in having
the tests in the repo itself.
>
>>
>>
>> >
>> >
>> > Thanks,
>> > Raghavendra Talur
>> >
>> >
>> > ___
>> > Gluster-devel mailing list
>> > Gluster-devel@gluster.org
>> > http://www.gluster.org/mailman/listinfo/gluster-devel
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Added specs for filesnapshots based on reflinks

2016-03-19 Thread Prasanna Kumar Kalever 
On Wednesday, March 16, 2016 3:55:52 PM, Niels de Vos wrote:
> On Wed, Mar 16, 2016 at 04:00:12AM -0400, Prasanna Kumar Kalever wrote:
> > Hi,
> > 
> > Regarding existing file snapshot:
> > 
> > Currently we have qemu-block xlator which helps in creating a file based
> > snapshots,
> > the biggest limitation about this feature is that, it allows creation of
> > snapshots
> > only for qcow2 file types (QEMU specific).
> > 
> > what about other VM image formats (say VMDK, VDI ...)?
> > what about general file formats (txt, docx, md ... another 1)?
> > 
> > 
> > Improvement:
> > 
> > Suggested a feature that allows you to take a snapshot on any kind of file
> > with the
> > help of reflinks.
> > 
> > Here is the link to specs page (which also has a POC);
> > https://github.com/gluster/glusterfs-specs/compare/master...pkalever:master
> > 
> > I need all of your blessings to move this into action.
> 
> We do not use GitHub pull requests for the glusterfs-specs repository.
> See the main README for sending your change to Gerrit.
> 
> I've left some comments in the GitHub patch. It's tricky to find where
> the comments are, but I think this link should do it:
> 
>   
> https://github.com/gluster/glusterfs-specs/commit/eb00fae40686bf5693242f389bbd9d0b7e0baede#diff-189cd0b816a5a125cfaccf7434dd6d57R46

Thanks Niels,

I have posted it at,
http://review.gluster.org/#/c/13752/

Also replied back on your comments on github.
It will great if we can update the same in gerrit.

--
Prasanna

> 
> Thanks,
> Niels
> 
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] On backporting fixes

2016-03-19 Thread Raghavendra Talur 
On Wed, Mar 16, 2016 at 11:31 AM, Raghavendra Talur 
wrote:

> Hi,
>
> Lot many fixes to tests were found to be not back ported to 3.7 and other
> release branches.
> This causes tests to fail only in those branches and leaves the
> maintainers puzzled.
>
> Also, this seems to be the case with back porting code fixes too.
>
> I copied all the changes to tests/ dir on master to tests/ dir on 3.7
> branch and posted a patch at http://review.gluster.org/#/c/13683/ .
>
> This is failing for ./tests/bugs/distribute/bug-860663.t test :
> [10:39:11] Running tests in file ./tests/bugs/distribute/bug-860663.t
> tar: Removing leading `/' from member names
> ./tests/bugs/distribute/bug-860663.t ..
> 1..15
> ok 1, LINENUM:23
> ok 2, LINENUM:24
> ok 3, LINENUM:26
> ok 4, LINENUM:27
> ok 5, LINENUM:30
> ok 6, LINENUM:32
> ok 7, LINENUM:35
> not ok 8 , LINENUM:40
> FAILED COMMAND: ! gluster --mode=script --wignore volume rebalance patchy
> fix-layout start
> ok 9, LINENUM:42
> ok 10, LINENUM:43
> ok 11, LINENUM:45
> ok 12, LINENUM:47
> ok 13, LINENUM:50
> ok 14, LINENUM:51
> ok 15, LINENUM:55
> Failed 1/15 subtests
>
> Given that it is a simple rebalance command that is failing I am assuming
> that a critical patch has not been back ported to 3.7, correct me if I am
> wrong.
>

In this particular case, the fix has already been posted on 3.7 branch and
just not merged yet. Link http://review.gluster.org/#/c/13537/3
Thanks Sakshi .


>
> I request every developer to take responsibility of back porting patches.
>
>
> Corollary question: Our test-framework is now capable of disabling tests
> for certain OS, certain branch etc. I would like to propose that we stop
> having tests in main git repo. This will remove need to back port test only
> fixes.
>
>
> Thanks,
> Raghavendra Talur
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Thoughts on enhancing replace-brick

2016-03-19 Thread Peter Portante 
Hi Folks,

There is a review posted, http://review.gluster.org/#/c/12250, to which I 
tacked on a review comment for an update to the replace-brick command. The gist 
of it is at https://gist.github.com/portante/248407dbfb29c2515fc3

What do folks think of such a proposal?

Thanks!

-peter

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel