Re: [Gluster-devel] 3.7 regressions on NetBSD

[Nithya Balachandran]

On Sat, Jul 23, 2016 at 9:45 AM, Nithya Balachandran 
wrote:

>
>
> On Fri, Jul 22, 2016 at 9:07 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com> wrote:
>>
>>> I am playing with the following diff, let me see.
>>>
>>> diff --git a/tests/volume.rc b/tests/volume.rc
>>> index 331a802..b288508 100644
>>> --- a/tests/volume.rc
>>> +++ b/tests/volume.rc
>>> @@ -579,7 +579,9 @@ function num_graphs
>>>  function get_aux()
>>>  {
>>>  ##Check if a auxiliary mount is there
>>> -df -h 2>&1 | sed 's#/build/install##' | grep -e
>>> "[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" -
>>> +local rundir=$(gluster --print-statedumpdir)
>>> +local pid=$(cat ${rundir}/${V0}.pid)
>>> +pidof glusterfs 2>&1 | grep -w $pid
>>>
>>>  if [ $? -eq 0 ]
>>>  then
>>>
>>
>> Based on what I saw in code, this seems to get the job done. Comments
>> welcome:
>> http://review.gluster.org/14988
>>
>>
> Nice work Pranith :)
> All, once this is backported to release-3.7, any patches on release-3.7
> patches will need to be rebased so they will pass the NetBSD regression.
>

I am suddenly confused about this - will the patches need to be rebased or
with the next run automatically include the changes once Pranith's fix is
merged?

>
>>>
>>> On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran <
>>> nbala...@redhat.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
 pkara...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
> nbala...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy 
>> wrote:
>>
>>> > I attempted to get us more space on NetBSD by creating a new
>>> partition called
>>> > /data and putting /build as a symlink to /data/build. This has
>>> caused
>>> > problems
>>> > with tests/basic/quota.t. It's marked as bad for master, but not
>>> for
>>> > release-3.7. This is possibly because we have a hard-coded grep for
>>> > /build/install against df -h.
>>>
>>> For the benefit of anyone else looking at this, the grep actually
>>> seems to be
>>> in volume.rc and not in the test itself.
>>>
>>
>> That's right -  it appears to have been done to exclude the install
>> path components from the df output which is what is being done to find 
>> the
>> aux mount. Is there a better way to figure out if the aux mount is 
>> running?
>>
>>>
>>> > Nithya has spent the last 2 days debugging
>>> > without much success. What's a good way forward here? Mark the
>>> test as
>>> > failing for 3.7?
>>>
>>
>> Right. Something went wrong with the system and it refused to run the
>> tests after a while.
>>
>>
>>>
>>> I don't think so.  There are 13 tests that use the affected function
>>> (get_aux).  Do we want to disable 13 tests?  I think we actually need
>>> to fix the function instead.  It seems to me that the check we're
>>> making is very hacky in two ways:
>>>
>>>Checking for both /run and /var/run instead of using
>>> GLUSTERD_WORKDIR
>>>
>>>Excluding /build/install for no obvious reason at all
>>>
>>
>> This looks like it was done to remove the /build/install components
>> from the df -h outputs. Changing the path to /data/build/install broke 
>> this
>> as it did not strip the "/data" from the paths.
>> It did work when I changed the sed to act on /data/build/install but
>> hardcoded paths are not a good approach.
>>
>
> Give me some time, I can send out a patch to print out the default run
> directory if that helps?
> something similar to 'gluster --print-logdir'. What shall we call
> this? 'gluster --print-rundir'? it will
>
>

 This path might be available as an env variable - but is there a better
 way to figure out the aux mount without bothering with df -h?

>
>>> These auxiliary mounts should be in a much more specific place, and
>>> we
>>> should check for that instead of looking for any that might exist.
>>> Who
>>> knows where that place is?  I've copied Raghavendra G as the quota
>>> maintainer, since that seems like our best bet.
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Pranith
>


>>>
>>>
>>> --
>>> Pranith
>>>
>>
>>
>>
>> --
>> Pranith
>>
>

[Gluster-devel] spurious failure in tests/basic/gfapi/libgfapi-fini-hang.t

[Pranith Kumar Karampuri]

I see both of your names in git blame output.
https://build.gluster.org/job/rackspace-regression-2GB-triggered/22439/console
has more information about the failure. This failure happened on
http://review.gluster.org/#/c/14985/ which changes only .t files so I
believe the reason for the failure to be something else. Could you please
take a look to find if there is a problem with the test or if it caught a
race in some recent code change?

-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Frank Rothenstein]

The point is that even if all other backend storage filesystems do
correctly untill 3.7.11 there was no error on ZFS. Something happened
nobody ever could explain in the release of 3.7.12 that makes FUSE-
mount _in ovirt_ (it partly uses dd with iflag=direct  , using
iflag=direct yourself gives also errors on the FUSE-mounts ) unusable.

So 3.7.11 is the last usable version when using ZFS on bricks, afaik.
Am Freitag, den 22.07.2016, 07:52 +1000 schrieb Lindsay Mathieson:
> 
> On 22/07/2016 6:14 AM, David Gossage
>   wrote:
> 
> 
> 
> >   https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.4
> > 
> >   
> > 
> >   
> > 
> >   
> > New asynchronous I/O (AIO)
> >   support.
> >   
> 
> 
> Only for  ZVOLS I think, not datasets.
> 
> 
> 
>   
> 
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users



 

__
BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
Sandhufe 2
18311 Ribnitz-Damgarten

Telefon: 03821-700-0
Fax:   03821-700-240

E-Mail: i...@bodden-kliniken.de   Internet: http://www.bodden-kliniken.de

Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 
079/133/40188
Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski

Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten 
bestimmt. Wenn Sie nicht der vorge- 
sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten Sie 
bitte, dass jede Form der Veröf- 
fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail 
unzulässig ist. Wir bitten Sie, sofort den 
Absender zu informieren und die E-Mail zu löschen. 


 Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
*** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Open Source Storage Summit

[Amye Scavarda]

This was a little late being announced officially, but EMC{code} is hosting
Open Source Storage Summits for the next two LinuxCons: North American and
Europe.

https://www.linux.com/blog/announcing-open-source-storage-summit

At LinuxCon Japan, I was asked to speak for the Gluster project, and it
ended up being really valuable: people from Fujitsu wanted to have an
in-depth conversation about how to contribute code back to the Gluster
project, and I walked them through it, which was really exciting and
unexpected from a day that was mostly talks.

Other nice thing: They're really looking for Gluster participation! I know
they've reached out to Ceph as well to be able to get more speakers at the
next two events. These events are primarily geared to users that are coming
from a traditional storage environment looking to get started in open
source storage. They're not expecting very technical deep dives into niche
areas, but good demos would be welcomed here.

If you're interested in this, let me know, they're actively trying to
recruit speakers for both Toronto and Berlin.
Thanks!

-- amye


-- 
Amye Scavarda | a...@redhat.com | Gluster Community Lead
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 10:25 PM, Jeff Darcy  wrote:

> > Based on what I saw in code, this seems to get the job done. Comments
> > welcome:
> > http://review.gluster.org/14988
>
> Good thinking.  Thanks, Pranith!
>

Nitya clarified my doubts as well on IRC :-).


-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[David Gossage]

On Fri, Jul 22, 2016 at 9:37 AM, Vijay Bellur  wrote:

> On Fri, Jul 22, 2016 at 10:03 AM, Samuli Heinonen 
> wrote:
> > Here is a quick way how to test this:
> > GlusterFS 3.7.13 volume with default settings with brick on ZFS dataset.
> gluster-test1 is server and gluster-test2 is client mounting with FUSE.
> >
> > Writing file with oflag=direct is not ok:
> > [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct
> count=1 bs=1024000
> > dd: failed to open ‘file’: Invalid argument
> >
> > Enable network.remote-dio on Gluster Volume:
> > [root@gluster-test1 gluster]# gluster volume set gluster
> network.remote-dio enable
> > volume set: success
> >
> > Writing small file with oflag=direct is ok:
> > [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct
> count=1 bs=1024000
> > 1+0 records in
> > 1+0 records out
> > 1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s
> >
> > Writing bigger file with oflag=direct is ok:
> > [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct
> count=100 bs=1M
> > 100+0 records in
> > 100+0 records out
> > 104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s
> >
> > Enable Sharding on Gluster Volume:
> > [root@gluster-test1 gluster]# gluster volume set gluster features.shard
> enable
> > volume set: success
> >
> > Writing small file  with oflag=direct is ok:
> > [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct
> count=1 bs=1M
> > 1+0 records in
> > 1+0 records out
> > 1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s
> >
> > Writing bigger file with oflag=direct is not ok:
> > [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct
> count=100 bs=1M
> > dd: error writing ‘file3’: Operation not permitted
> > dd: closing output file ‘file3’: Operation not permitted
> >
>
>
> Thank you for these tests! would it be possible to share the brick and
> client logs?
>

Not sure if his tests are same as my setup but here is what I end up with

Volume Name: glustershard
Type: Replicate
Volume ID: 0cc4efb6-3836-4caa-b24a-b3afb6e407c3
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 192.168.71.10:/gluster1/shard1/1
Brick2: 192.168.71.11:/gluster1/shard2/1
Brick3: 192.168.71.12:/gluster1/shard3/1
Options Reconfigured:
features.shard-block-size: 64MB
features.shard: on
server.allow-insecure: on
storage.owner-uid: 36
storage.owner-gid: 36
cluster.server-quorum-type: server
cluster.quorum-type: auto
network.remote-dio: enable
cluster.eager-lock: enable
performance.stat-prefetch: off
performance.io-cache: off
performance.quick-read: off
cluster.self-heal-window-size: 1024
cluster.background-self-heal-count: 16
nfs.enable-ino32: off
nfs.addr-namelookup: off
nfs.disable: on
performance.read-ahead: off
performance.readdir-ahead: on



 dd if=/dev/zero
of=/rhev/data-center/mnt/glusterSD/192.168.71.11\:_glustershard/
oflag=direct count=100 bs=1M
81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/ __DIRECT_IO_TEST__
 .trashcan/
[root@ccengine2 ~]# dd if=/dev/zero of=/rhev/data-center/mnt/glusterSD/
192.168.71.11\:_glustershard/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test
oflag=direct count=100 bs=1M
dd: error writing
‘/rhev/data-center/mnt/glusterSD/192.168.71.11:_glustershard/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test’:
Operation not permitted

creates the 64M file in expected location then the shard is 0

# file: gluster1/shard1/1/81e19cd3-ae45-449c-b716-ec3e4ad4c2f0/images/test
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.bit-rot.version=0x0200579231f3000e16e7
trusted.gfid=0xec6de302b35f427985639ca3e25d9df0
trusted.glusterfs.shard.block-size=0x0400
trusted.glusterfs.shard.file-size=0x0401


# file: gluster1/shard1/1/.shard/ec6de302-b35f-4279-8563-9ca3e25d9df0.1
security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
trusted.afr.dirty=0x
trusted.gfid=0x2bfd3cc8a727489b9a0474241548fe80


> Regards,
> Vijay
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
[2016-07-22 16:26:44.645166] I [MSGID: 100030] [glusterfsd.c:2338:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.13 (args: /usr/sbin/glusterfs --volfile-server=192.168.71.11 --volfile-server=192.168.71.10 --volfile-server=192.168.71.12 --volfile-id=/glustershard /rhev/data-center/mnt/glusterSD/192.168.71.11:_glustershard)
[2016-07-22 16:26:44.655674] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2016-07-22 16:26:44.661967] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2016-07-22 16:26:44.662718] I [MSGID: 114020] 

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Jeff Darcy]

> Based on what I saw in code, this seems to get the job done. Comments
> welcome:
> http://review.gluster.org/14988

Good thinking.  Thanks, Pranith!
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 8:12 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> I am playing with the following diff, let me see.
>
> diff --git a/tests/volume.rc b/tests/volume.rc
> index 331a802..b288508 100644
> --- a/tests/volume.rc
> +++ b/tests/volume.rc
> @@ -579,7 +579,9 @@ function num_graphs
>  function get_aux()
>  {
>  ##Check if a auxiliary mount is there
> -df -h 2>&1 | sed 's#/build/install##' | grep -e
> "[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" -
> +local rundir=$(gluster --print-statedumpdir)
> +local pid=$(cat ${rundir}/${V0}.pid)
> +pidof glusterfs 2>&1 | grep -w $pid
>
>  if [ $? -eq 0 ]
>  then
>

Based on what I saw in code, this seems to get the job done. Comments
welcome:
http://review.gluster.org/14988


>
> On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
>> pkara...@redhat.com> wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran <
>>> nbala...@redhat.com> wrote:
>>>


 On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:

> > I attempted to get us more space on NetBSD by creating a new
> partition called
> > /data and putting /build as a symlink to /data/build. This has caused
> > problems
> > with tests/basic/quota.t. It's marked as bad for master, but not for
> > release-3.7. This is possibly because we have a hard-coded grep for
> > /build/install against df -h.
>
> For the benefit of anyone else looking at this, the grep actually
> seems to be
> in volume.rc and not in the test itself.
>

 That's right -  it appears to have been done to exclude the install
 path components from the df output which is what is being done to find the
 aux mount. Is there a better way to figure out if the aux mount is running?

>
> > Nithya has spent the last 2 days debugging
> > without much success. What's a good way forward here? Mark the test
> as
> > failing for 3.7?
>

 Right. Something went wrong with the system and it refused to run the
 tests after a while.


>
> I don't think so.  There are 13 tests that use the affected function
> (get_aux).  Do we want to disable 13 tests?  I think we actually need
> to fix the function instead.  It seems to me that the check we're
> making is very hacky in two ways:
>
>Checking for both /run and /var/run instead of using
> GLUSTERD_WORKDIR
>
>Excluding /build/install for no obvious reason at all
>

 This looks like it was done to remove the /build/install components
 from the df -h outputs. Changing the path to /data/build/install broke this
 as it did not strip the "/data" from the paths.
 It did work when I changed the sed to act on /data/build/install but
 hardcoded paths are not a good approach.

>>>
>>> Give me some time, I can send out a patch to print out the default run
>>> directory if that helps?
>>> something similar to 'gluster --print-logdir'. What shall we call this?
>>> 'gluster --print-rundir'? it will
>>>
>>>
>>
>> This path might be available as an env variable - but is there a better
>> way to figure out the aux mount without bothering with df -h?
>>
>>>
> These auxiliary mounts should be in a much more specific place, and we
> should check for that instead of looking for any that might exist.  Who
> knows where that place is?  I've copied Raghavendra G as the quota
> maintainer, since that seems like our best bet.
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>


 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel

>>>
>>>
>>>
>>> --
>>> Pranith
>>>
>>
>>
>
>
> --
> Pranith
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 4:46 PM, Nigel Babu  wrote:

> Hello,
>
> I attempted to get us more space on NetBSD by creating a new partition
> called
> /data and putting /build as a symlink to /data/build. This has caused
> problems
> with tests/basic/quota.t. It's marked as bad for master, but not for
> release-3.7. This is possibly because we have a hard-coded grep for
> /build/install against df -h. Nithya has spent the last 2 days debugging
> without much success. What's a good way forward here? Mark the test as
> failing
> for 3.7?
>

Dude, she spent only 2 hours on this. Yesterday was another netbsd problem.
She is not that bad you know :-P.


>
> We're going to have to accept that some of our tests might be in a
> different
> drive as we try to get more disk space for our machines. How can we make
> our
> tests more resilient from breakage due to regular expressions?
>
> --
> nigelb
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

I am playing with the following diff, let me see.

diff --git a/tests/volume.rc b/tests/volume.rc
index 331a802..b288508 100644
--- a/tests/volume.rc
+++ b/tests/volume.rc
@@ -579,7 +579,9 @@ function num_graphs
 function get_aux()
 {
 ##Check if a auxiliary mount is there
-df -h 2>&1 | sed 's#/build/install##' | grep -e
"[[:space:]]/run/gluster/${V0}$" -e "[[:space:]]/var/run/gluster/${V0}$" -
+local rundir=$(gluster --print-statedumpdir)
+local pid=$(cat ${rundir}/${V0}.pid)
+pidof glusterfs 2>&1 | grep -w $pid

 if [ $? -eq 0 ]
 then


On Fri, Jul 22, 2016 at 7:44 PM, Nithya Balachandran 
wrote:

>
>
> On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran > > wrote:
>>
>>>
>>>
>>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:
>>>
 > I attempted to get us more space on NetBSD by creating a new
 partition called
 > /data and putting /build as a symlink to /data/build. This has caused
 > problems
 > with tests/basic/quota.t. It's marked as bad for master, but not for
 > release-3.7. This is possibly because we have a hard-coded grep for
 > /build/install against df -h.

 For the benefit of anyone else looking at this, the grep actually seems
 to be
 in volume.rc and not in the test itself.

>>>
>>> That's right -  it appears to have been done to exclude the install path
>>> components from the df output which is what is being done to find the aux
>>> mount. Is there a better way to figure out if the aux mount is running?
>>>

 > Nithya has spent the last 2 days debugging
 > without much success. What's a good way forward here? Mark the test as
 > failing for 3.7?

>>>
>>> Right. Something went wrong with the system and it refused to run the
>>> tests after a while.
>>>
>>>

 I don't think so.  There are 13 tests that use the affected function
 (get_aux).  Do we want to disable 13 tests?  I think we actually need
 to fix the function instead.  It seems to me that the check we're
 making is very hacky in two ways:

Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR

Excluding /build/install for no obvious reason at all

>>>
>>> This looks like it was done to remove the /build/install components from
>>> the df -h outputs. Changing the path to /data/build/install broke this as
>>> it did not strip the "/data" from the paths.
>>> It did work when I changed the sed to act on /data/build/install but
>>> hardcoded paths are not a good approach.
>>>
>>
>> Give me some time, I can send out a patch to print out the default run
>> directory if that helps?
>> something similar to 'gluster --print-logdir'. What shall we call this?
>> 'gluster --print-rundir'? it will
>>
>>
>
> This path might be available as an env variable - but is there a better
> way to figure out the aux mount without bothering with df -h?
>
>>
 These auxiliary mounts should be in a much more specific place, and we
 should check for that instead of looking for any that might exist.  Who
 knows where that place is?  I've copied Raghavendra G as the quota
 maintainer, since that seems like our best bet.
 ___
 Gluster-devel mailing list
 Gluster-devel@gluster.org
 http://www.gluster.org/mailman/listinfo/gluster-devel

>>>
>>>
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>>
>> --
>> Pranith
>>
>
>


-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Vijay Bellur]

On Fri, Jul 22, 2016 at 10:03 AM, Samuli Heinonen  wrote:
> Here is a quick way how to test this:
> GlusterFS 3.7.13 volume with default settings with brick on ZFS dataset. 
> gluster-test1 is server and gluster-test2 is client mounting with FUSE.
>
> Writing file with oflag=direct is not ok:
> [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 
> bs=1024000
> dd: failed to open ‘file’: Invalid argument
>
> Enable network.remote-dio on Gluster Volume:
> [root@gluster-test1 gluster]# gluster volume set gluster network.remote-dio 
> enable
> volume set: success
>
> Writing small file with oflag=direct is ok:
> [root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 
> bs=1024000
> 1+0 records in
> 1+0 records out
> 1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s
>
> Writing bigger file with oflag=direct is ok:
> [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 
> bs=1M
> 100+0 records in
> 100+0 records out
> 104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s
>
> Enable Sharding on Gluster Volume:
> [root@gluster-test1 gluster]# gluster volume set gluster features.shard enable
> volume set: success
>
> Writing small file  with oflag=direct is ok:
> [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=1 
> bs=1M
> 1+0 records in
> 1+0 records out
> 1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s
>
> Writing bigger file with oflag=direct is not ok:
> [root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 
> bs=1M
> dd: error writing ‘file3’: Operation not permitted
> dd: closing output file ‘file3’: Operation not permitted
>


Thank you for these tests! would it be possible to share the brick and
client logs?

Regards,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Jeff Darcy]

> > Excluding /build/install for no obvious reason at all
> 
> This looks like it was done to remove the /build/install components from the
> df -h outputs. Changing the path to /data/build/install broke this as it did
> not strip the "/data" from the paths.
> It did work when I changed the sed to act on /data/build/install but
> hardcoded paths are not a good approach.

I agree.  What I'm worried about is that we needed to do this because
something in our code is using /build/install (or perhaps $PWD which
happens to be there) when it should be using $WORKDIR or something
like that.  If so, it sort of seems like a bug and we should probably
fix it independently of how we fix the immediate quota/NetBSD problem.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran 
wrote:

>
>
> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:
>
>> > I attempted to get us more space on NetBSD by creating a new partition
>> called
>> > /data and putting /build as a symlink to /data/build. This has caused
>> > problems
>> > with tests/basic/quota.t. It's marked as bad for master, but not for
>> > release-3.7. This is possibly because we have a hard-coded grep for
>> > /build/install against df -h.
>>
>> For the benefit of anyone else looking at this, the grep actually seems
>> to be
>> in volume.rc and not in the test itself.
>>
>
> That's right -  it appears to have been done to exclude the install path
> components from the df output which is what is being done to find the aux
> mount. Is there a better way to figure out if the aux mount is running?
>
>>
>> > Nithya has spent the last 2 days debugging
>> > without much success. What's a good way forward here? Mark the test as
>> > failing for 3.7?
>>
>
> Right. Something went wrong with the system and it refused to run the
> tests after a while.
>
>
>>
>> I don't think so.  There are 13 tests that use the affected function
>> (get_aux).  Do we want to disable 13 tests?  I think we actually need
>> to fix the function instead.  It seems to me that the check we're
>> making is very hacky in two ways:
>>
>>Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR
>>
>>Excluding /build/install for no obvious reason at all
>>
>
> This looks like it was done to remove the /build/install components from
> the df -h outputs. Changing the path to /data/build/install broke this as
> it did not strip the "/data" from the paths.
> It did work when I changed the sed to act on /data/build/install but
> hardcoded paths are not a good approach.
>

Give me some time, I can send out a patch to print out the default run
directory if that helps?
something similar to 'gluster --print-logdir'. What shall we call this?
'gluster --print-rundir'? it will


>
>> These auxiliary mounts should be in a much more specific place, and we
>> should check for that instead of looking for any that might exist.  Who
>> knows where that place is?  I've copied Raghavendra G as the quota
>> maintainer, since that seems like our best bet.
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Nithya Balachandran]

On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:
>>
>>> > I attempted to get us more space on NetBSD by creating a new partition
>>> called
>>> > /data and putting /build as a symlink to /data/build. This has caused
>>> > problems
>>> > with tests/basic/quota.t. It's marked as bad for master, but not for
>>> > release-3.7. This is possibly because we have a hard-coded grep for
>>> > /build/install against df -h.
>>>
>>> For the benefit of anyone else looking at this, the grep actually seems
>>> to be
>>> in volume.rc and not in the test itself.
>>>
>>
>> That's right -  it appears to have been done to exclude the install path
>> components from the df output which is what is being done to find the aux
>> mount. Is there a better way to figure out if the aux mount is running?
>>
>>>
>>> > Nithya has spent the last 2 days debugging
>>> > without much success. What's a good way forward here? Mark the test as
>>> > failing for 3.7?
>>>
>>
>> Right. Something went wrong with the system and it refused to run the
>> tests after a while.
>>
>>
>>>
>>> I don't think so.  There are 13 tests that use the affected function
>>> (get_aux).  Do we want to disable 13 tests?  I think we actually need
>>> to fix the function instead.  It seems to me that the check we're
>>> making is very hacky in two ways:
>>>
>>>Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR
>>>
>>>Excluding /build/install for no obvious reason at all
>>>
>>
>> This looks like it was done to remove the /build/install components from
>> the df -h outputs. Changing the path to /data/build/install broke this as
>> it did not strip the "/data" from the paths.
>> It did work when I changed the sed to act on /data/build/install but
>> hardcoded paths are not a good approach.
>>
>
> Give me some time, I can send out a patch to print out the default run
> directory if that helps?
> something similar to 'gluster --print-logdir'. What shall we call this?
> 'gluster --print-rundir'? it will
>
>

This path might be available as an env variable - but is there a better way
to figure out the aux mount without bothering with df -h?

>
>>> These auxiliary mounts should be in a much more specific place, and we
>>> should check for that instead of looking for any that might exist.  Who
>>> knows where that place is?  I've copied Raghavendra G as the quota
>>> maintainer, since that seems like our best bet.
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Pranith
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Notifications (was Re: GF_PARENT_DOWN on SIGKILL)

[Jeff Darcy]

> I don't think we need any list traversal because notify sends it down
> the graph.

Good point.  I think we need to change that, BTW.  Relying on
translators to propagate notifications has proven very fragile, as many
of those events are overloaded to mean very different things to
different translators (e.g. just being up vs. having quorum) with
different rules for when they should or should not be propagated.  Going
forward, I think we can save ourselves a lot of headaches by treating
notification as an infrastructure responsibility, and changing
translators to use something else (e.g. IPC fops or upcalls) for their
internal needs.

But that's a different issue.  For now, just pushing one PARENT_DOWN
to the top of the graph should suffice.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 7:42 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

>
>
> On Fri, Jul 22, 2016 at 7:39 PM, Nithya Balachandran 
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:
>>
>>> > I attempted to get us more space on NetBSD by creating a new partition
>>> called
>>> > /data and putting /build as a symlink to /data/build. This has caused
>>> > problems
>>> > with tests/basic/quota.t. It's marked as bad for master, but not for
>>> > release-3.7. This is possibly because we have a hard-coded grep for
>>> > /build/install against df -h.
>>>
>>> For the benefit of anyone else looking at this, the grep actually seems
>>> to be
>>> in volume.rc and not in the test itself.
>>>
>>
>> That's right -  it appears to have been done to exclude the install path
>> components from the df output which is what is being done to find the aux
>> mount. Is there a better way to figure out if the aux mount is running?
>>
>>>
>>> > Nithya has spent the last 2 days debugging
>>> > without much success. What's a good way forward here? Mark the test as
>>> > failing for 3.7?
>>>
>>
>> Right. Something went wrong with the system and it refused to run the
>> tests after a while.
>>
>>
>>>
>>> I don't think so.  There are 13 tests that use the affected function
>>> (get_aux).  Do we want to disable 13 tests?  I think we actually need
>>> to fix the function instead.  It seems to me that the check we're
>>> making is very hacky in two ways:
>>>
>>>Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR
>>>
>>>Excluding /build/install for no obvious reason at all
>>>
>>
>> This looks like it was done to remove the /build/install components from
>> the df -h outputs. Changing the path to /data/build/install broke this as
>> it did not strip the "/data" from the paths.
>> It did work when I changed the sed to act on /data/build/install but
>> hardcoded paths are not a good approach.
>>
>
> Give me some time, I can send out a patch to print out the default run
> directory if that helps?
> something similar to 'gluster --print-logdir'. What shall we call this?
> 'gluster --print-rundir'? it will
>

Wait, it seems like 'gluster --print-statedumpdir' already prints
'/var/run/gluster', is this the directory we want?


>
>
>>
>>> These auxiliary mounts should be in a much more specific place, and we
>>> should check for that instead of looking for any that might exist.  Who
>>> knows where that place is?  I've copied Raghavendra G as the quota
>>> maintainer, since that seems like our best bet.
>>> ___
>>> Gluster-devel mailing list
>>> Gluster-devel@gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>> ___
>> Gluster-devel mailing list
>> Gluster-devel@gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-devel
>>
>
>
>
> --
> Pranith
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Nithya Balachandran]

On Fri, Jul 22, 2016 at 7:31 PM, Jeff Darcy  wrote:

> > I attempted to get us more space on NetBSD by creating a new partition
> called
> > /data and putting /build as a symlink to /data/build. This has caused
> > problems
> > with tests/basic/quota.t. It's marked as bad for master, but not for
> > release-3.7. This is possibly because we have a hard-coded grep for
> > /build/install against df -h.
>
> For the benefit of anyone else looking at this, the grep actually seems to
> be
> in volume.rc and not in the test itself.
>

That's right -  it appears to have been done to exclude the install path
components from the df output which is what is being done to find the aux
mount. Is there a better way to figure out if the aux mount is running?

>
> > Nithya has spent the last 2 days debugging
> > without much success. What's a good way forward here? Mark the test as
> > failing for 3.7?
>

Right. Something went wrong with the system and it refused to run the tests
after a while.


>
> I don't think so.  There are 13 tests that use the affected function
> (get_aux).  Do we want to disable 13 tests?  I think we actually need
> to fix the function instead.  It seems to me that the check we're
> making is very hacky in two ways:
>
>Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR
>
>Excluding /build/install for no obvious reason at all
>

This looks like it was done to remove the /build/install components from
the df -h outputs. Changing the path to /data/build/install broke this as
it did not strip the "/data" from the paths.
It did work when I changed the sed to act on /data/build/install but
hardcoded paths are not a good approach.

>
> These auxiliary mounts should be in a much more specific place, and we
> should check for that instead of looking for any that might exist.  Who
> knows where that place is?  I've copied Raghavendra G as the quota
> maintainer, since that seems like our best bet.
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Jeff Darcy]

> I attempted to get us more space on NetBSD by creating a new partition called
> /data and putting /build as a symlink to /data/build. This has caused
> problems
> with tests/basic/quota.t. It's marked as bad for master, but not for
> release-3.7. This is possibly because we have a hard-coded grep for
> /build/install against df -h.

For the benefit of anyone else looking at this, the grep actually seems to be
in volume.rc and not in the test itself.

> Nithya has spent the last 2 days debugging
> without much success. What's a good way forward here? Mark the test as
> failing for 3.7?

I don't think so.  There are 13 tests that use the affected function
(get_aux).  Do we want to disable 13 tests?  I think we actually need
to fix the function instead.  It seems to me that the check we're
making is very hacky in two ways:

   Checking for both /run and /var/run instead of using GLUSTERD_WORKDIR

   Excluding /build/install for no obvious reason at all

These auxiliary mounts should be in a much more specific place, and we
should check for that instead of looking for any that might exist.  Who
knows where that place is?  I've copied Raghavendra G as the quota
maintainer, since that seems like our best bet.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Samuli Heinonen]

Here is a quick way how to test this:
GlusterFS 3.7.13 volume with default settings with brick on ZFS dataset. 
gluster-test1 is server and gluster-test2 is client mounting with FUSE.

Writing file with oflag=direct is not ok:
[root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 
bs=1024000
dd: failed to open ‘file’: Invalid argument

Enable network.remote-dio on Gluster Volume:
[root@gluster-test1 gluster]# gluster volume set gluster network.remote-dio 
enable
volume set: success

Writing small file with oflag=direct is ok:
[root@gluster-test2 gluster]# dd if=/dev/zero of=file oflag=direct count=1 
bs=1024000
1+0 records in
1+0 records out
1024000 bytes (1.0 MB) copied, 0.0103793 s, 98.7 MB/s

Writing bigger file with oflag=direct is ok:
[root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 
bs=1M
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 1.10583 s, 94.8 MB/s

Enable Sharding on Gluster Volume:
[root@gluster-test1 gluster]# gluster volume set gluster features.shard enable
volume set: success

Writing small file  with oflag=direct is ok:
[root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=1 
bs=1M
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0115247 s, 91.0 MB/s

Writing bigger file with oflag=direct is not ok:
[root@gluster-test2 gluster]# dd if=/dev/zero of=file3 oflag=direct count=100 
bs=1M
dd: error writing ‘file3’: Operation not permitted
dd: closing output file ‘file3’: Operation not permitted

-samuli


> On 22 Jul 2016, at 16:12, Vijay Bellur  wrote:
> 
> 2016-07-22 1:54 GMT-04:00 Frank Rothenstein 
> :
>> The point is that even if all other backend storage filesystems do correctly
>> untill 3.7.11 there was no error on ZFS. Something happened nobody ever
>> could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ (it
>> partly uses dd with iflag=direct  , using iflag=direct yourself gives also
>> errors on the FUSE-mounts ) unusable.
>> 
>> So 3.7.11 is the last usable version when using ZFS on bricks, afaik.
>> 
> 
> Can you please share the exact dd command that causes this problem?
> 
> Thanks,
> Vijay
> ___
> Gluster-users mailing list
> gluster-us...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[David Gossage]

On Fri, Jul 22, 2016 at 8:12 AM, Vijay Bellur  wrote:

> 2016-07-22 1:54 GMT-04:00 Frank Rothenstein <
> f.rothenst...@bodden-kliniken.de>:
> > The point is that even if all other backend storage filesystems do
> correctly
> > untill 3.7.11 there was no error on ZFS. Something happened nobody ever
> > could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_
> (it
> > partly uses dd with iflag=direct  , using iflag=direct yourself gives
> also
> > errors on the FUSE-mounts ) unusable.
> >
> > So 3.7.11 is the last usable version when using ZFS on bricks, afaik.
> >
>
> Can you please share the exact dd command that causes this problem?
>

I want to say it was this one though my logs for vdsm going back that far
have rolled off

 /usr/bin/dd
if=/rhev/data-center/mnt/glusterSD/ccgl1.gl.local:GLUSTER1/7c73a8dd-a72e-4556-ac88-7f6813131e64/dom_md/metadata
iflag=direct of=/dev/null bs=4096 count=1

>
> Thanks,
> Vijay
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[David Gossage]

On Fri, Jul 22, 2016 at 8:23 AM, Samuli Heinonen 
wrote:

>
> > On 21 Jul 2016, at 20:48, David Gossage 
> wrote:
> >
> > Wonder if this may be related at all
> >
> > * #1347553: O_DIRECT support for sharding
> > https://bugzilla.redhat.com/show_bug.cgi?id=1347553
> >
> > Is it possible to downgrade from 3.8 back to 3.7.x
> >
> > Building test box right now anyway but wondering.
> >
>
> Have you been able to do any testing yet?
>

Had time to get box built up to point of getting zfs pool made before left
work yesterday.  hope to have working gluster volumes on various fs
backends by end of day.

Figure 1 volume on xfs with sharding, 1 volume on zfs sharded, 1 on zfs no
shards, and maybe ill make 1 zvol with xfs on top of it also to see what
happens


> "O_DIRECT support for sharding" has been also included in 3.7.12. Is this
> problem occurring only when sharding is enabled? Is it possible that it
> requires direct I/O all the way to the bricks with sharding even when
> network.remote-dio is enabled?
>

It's possible though at time of update when I had issues getting it to stay
connected to ovirt I had just turned sharding on and as of yet had no
sharded images.  I also turned feature off during tests with no benefit in
stability.

>
> Best regards,
> Samuli Heinonen
>
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Pranith Kumar Karampuri]

On Fri, Jul 22, 2016 at 7:07 PM, Jeff Darcy  wrote:

> > Gah! sorry sorry, I meant to send the mail as SIGTERM. Not SIGKILL. So
> xavi
> > and I were wondering why cleanup_and_exit() is not sending GF_PARENT_DOWN
> > event.
>
> OK, then that grinding sound you hear is my brain shifting gears.  ;)  It
> seems that cleanup_and_exit will call xlator.fini in some few cases, but
> it doesn't do anything that would send notify events.  I'll bet the answer
> to "why" is just that nobody thought of it or got around to it.  The next
> question I'd ask is: can you do what you need to do from ec.fini instead?
> That would require enabling it in should_call_fini as well, but otherwise
> seems pretty straightforward.
>
> If the answer to that question is no, then things get more complicated.
> Can we do one loop that sends GF_EVENT_PARENT_DOWN events, then another
> that calls fini?  Can we just do a basic list traversal (as we do now for
> fini) or do we need to do something more complicated to deal with cluster
> translators?  I think a separate loop doing basic list traversal would
> work, even with brick multiplexing, so it's probably worth just coding it
> up as an experiment.
>

I don't think we need any list traversal because notify sends it down the
graph. I guess I will start the experiment then.


-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Jeff Darcy]

> Gah! sorry sorry, I meant to send the mail as SIGTERM. Not SIGKILL. So xavi
> and I were wondering why cleanup_and_exit() is not sending GF_PARENT_DOWN
> event.

OK, then that grinding sound you hear is my brain shifting gears.  ;)  It
seems that cleanup_and_exit will call xlator.fini in some few cases, but
it doesn't do anything that would send notify events.  I'll bet the answer
to "why" is just that nobody thought of it or got around to it.  The next
question I'd ask is: can you do what you need to do from ec.fini instead?
That would require enabling it in should_call_fini as well, but otherwise
seems pretty straightforward.

If the answer to that question is no, then things get more complicated.
Can we do one loop that sends GF_EVENT_PARENT_DOWN events, then another
that calls fini?  Can we just do a basic list traversal (as we do now for
fini) or do we need to do something more complicated to deal with cluster
translators?  I think a separate loop doing basic list traversal would
work, even with brick multiplexing, so it's probably worth just coding it
up as an experiment.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Samuli Heinonen]

> On 21 Jul 2016, at 20:48, David Gossage  wrote:
> 
> Wonder if this may be related at all
> 
> * #1347553: O_DIRECT support for sharding
> https://bugzilla.redhat.com/show_bug.cgi?id=1347553
> 
> Is it possible to downgrade from 3.8 back to 3.7.x 
> 
> Building test box right now anyway but wondering.
> 

Have you been able to do any testing yet?

"O_DIRECT support for sharding" has been also included in 3.7.12. Is this 
problem occurring only when sharding is enabled? Is it possible that it 
requires direct I/O all the way to the bricks with sharding even when 
network.remote-dio is enabled?

Best regards,
Samuli Heinonen

___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Support to reclaim locks (posix) provided lkowner & range matches

[Ira Cooper]

SMB3 with persistent handles has very similar needs.

Michael,

You may also want to review this.

Cheers,

-Ira

- Original Message -
> Hi,
> 
> In certain scenarios (esp.,in highly available environments), the
> application may have to fail-over/connect to a different glusterFS
> client while the I/O is happening. In such cases until there is a ping
> timer expiry and glusterFS server cleans up the locks held by the older
> glusterFS client, the application will not be able to reclaim their lost
> locks. To avoid that we need support in Gluster to let clients reclaim
> the existing locks provided lkwoner and the lock range matches.
> 
> One of the applications which shall benefit from this support is
> NFS-Ganesha. NFS clients try to reclaim their post server reboot.
> 
> I have made relevant changes (WIP) on the server side to have this
> support [1]. The changes include -
> 
> * A new CLI option is provided "features.locks-reclaim-lock" to enable
> this support.
> 
> * Assuming below is done on the client-side (gfapi) - TODO
> While re-trying the lock request, application has to notify the
> glusterFS client that it is a reclaim request. Client on receiving such
> request should set a boolean "reclaim-lock" in the xdata passed to lock
> request.
> 
> * On the server-side -
>- A new field 'reclaim' is added to 'posix_lock_t' to note if it is
> to be reclaimed.
>- While processing LOCK fop, if the "reclaim-lock" is set in the
> xdata received, reclaim field will be enabled in the new posix lock created.
>- While checking for conflicting locks (in 'same_owner()'), if the
> reclaim field is set, comparison will be done for lkowner and lock
> ranges instead of comparing both lkwoner and client UID.
>- Later it will fall through '__insert_and_merge' and the old lock
> will be updated with the details of the new lock created (along with
> client details).
> 
> For client-side support, I am thinking if we can integrate with the new
> lock API being introduced as part of mandatory lock support in gfapi [2]
> 
> Kindly take a look and provide your comments/suggestions.
> 
> The changes seemed minimal and hence I haven't added it as 3.9 release
> feature. But if you feel it is a feature candidate, please let me know.
> I shall open up a feature page.
> 
> Thanks,
> Soumya
> 
> [1] http://review.gluster.org/#/c/14986/
> [2] http://review.gluster.org/11177
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Vijay Bellur]

2016-07-22 1:54 GMT-04:00 Frank Rothenstein :
> The point is that even if all other backend storage filesystems do correctly
> untill 3.7.11 there was no error on ZFS. Something happened nobody ever
> could explain in the release of 3.7.12 that makes FUSE-mount _in ovirt_ (it
> partly uses dd with iflag=direct  , using iflag=direct yourself gives also
> errors on the FUSE-mounts ) unusable.
>
> So 3.7.11 is the last usable version when using ZFS on bricks, afaik.
>

Can you please share the exact dd command that causes this problem?

Thanks,
Vijay
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Pranith Kumar Karampuri]

http://review.gluster.org/14980, this is where we have all the context
about why I sent out this mail. Basically the tests were failing because
umount is racing with version-updation xattrop. While I fixed the test to
handle that race, xavi was wondering why GF_PARENT_DOWN event didn't come.
I found that in cleanup_and_exit() we don't send this event. We are only
calling 'fini()'. So wondering if any one knows why this is so.

On Fri, Jul 22, 2016 at 6:37 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> It is only calling fini() apart from that not much.
>
> On Fri, Jul 22, 2016 at 6:36 PM, Pranith Kumar Karampuri <
> pkara...@redhat.com> wrote:
>
>> Gah! sorry sorry, I meant to send the mail as SIGTERM. Not SIGKILL. So
>> xavi and I were wondering why cleanup_and_exit() is not sending
>> GF_PARENT_DOWN event.
>>
>> On Fri, Jul 22, 2016 at 6:24 PM, Jeff Darcy  wrote:
>>
>>> > Does anyone know why GF_PARENT_DOWN is not triggered on SIGKILL? It
>>> will give
>>> > a chance for xlators to do any cleanup they need to do. For example ec
>>> can
>>> > complete the delayed xattrops.
>>>
>>> Nothing is triggered on SIGKILL.  SIGKILL is explicitly defined to
>>> terminate a
>>> process *immediately*.  Among other things, this means it can not be
>>> ignored or
>>> caught, to preclude handlers doing something that might delay
>>> termination.
>>>
>>>
>>> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>>>
>>> Since at least 4.2BSD and SVr2 (the first version of UNIX that I worked
>>> on)
>>> there have even been distinct kernel code paths to ensure special
>>> handling of
>>> SIGKILL.  There's nothing we can do about SIGKILL except be prepared to
>>> deal
>>> with it the same way we'd deal with the entire machine crashing.
>>>
>>> If you mean why is there nothing we can do on a *server* in response to
>>> SIGKILL on a *client*, that's a slightly more interesting question.  It's
>>> possible that the unique nature of SIGKILL puts connections into a
>>> different state than either system failure (on the more abrupt side) or
>>> clean shutdown (less abrupt).  If so, we probably need to take a look at
>>> the socket/RPC code or perhaps even protocol/server to see why these
>>> connections are not being cleaned up and shut down in a timely fashion.
>>>
>>
>>
>>
>> --
>> Pranith
>>
>
>
>
> --
> Pranith
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Pranith Kumar Karampuri]

It is only calling fini() apart from that not much.

On Fri, Jul 22, 2016 at 6:36 PM, Pranith Kumar Karampuri <
pkara...@redhat.com> wrote:

> Gah! sorry sorry, I meant to send the mail as SIGTERM. Not SIGKILL. So
> xavi and I were wondering why cleanup_and_exit() is not sending
> GF_PARENT_DOWN event.
>
> On Fri, Jul 22, 2016 at 6:24 PM, Jeff Darcy  wrote:
>
>> > Does anyone know why GF_PARENT_DOWN is not triggered on SIGKILL? It
>> will give
>> > a chance for xlators to do any cleanup they need to do. For example ec
>> can
>> > complete the delayed xattrops.
>>
>> Nothing is triggered on SIGKILL.  SIGKILL is explicitly defined to
>> terminate a
>> process *immediately*.  Among other things, this means it can not be
>> ignored or
>> caught, to preclude handlers doing something that might delay termination.
>>
>>
>> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>>
>> Since at least 4.2BSD and SVr2 (the first version of UNIX that I worked
>> on)
>> there have even been distinct kernel code paths to ensure special
>> handling of
>> SIGKILL.  There's nothing we can do about SIGKILL except be prepared to
>> deal
>> with it the same way we'd deal with the entire machine crashing.
>>
>> If you mean why is there nothing we can do on a *server* in response to
>> SIGKILL on a *client*, that's a slightly more interesting question.  It's
>> possible that the unique nature of SIGKILL puts connections into a
>> different state than either system failure (on the more abrupt side) or
>> clean shutdown (less abrupt).  If so, we probably need to take a look at
>> the socket/RPC code or perhaps even protocol/server to see why these
>> connections are not being cleaned up and shut down in a timely fashion.
>>
>
>
>
> --
> Pranith
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Pranith Kumar Karampuri]

Gah! sorry sorry, I meant to send the mail as SIGTERM. Not SIGKILL. So xavi
and I were wondering why cleanup_and_exit() is not sending GF_PARENT_DOWN
event.

On Fri, Jul 22, 2016 at 6:24 PM, Jeff Darcy  wrote:

> > Does anyone know why GF_PARENT_DOWN is not triggered on SIGKILL? It will
> give
> > a chance for xlators to do any cleanup they need to do. For example ec
> can
> > complete the delayed xattrops.
>
> Nothing is triggered on SIGKILL.  SIGKILL is explicitly defined to
> terminate a
> process *immediately*.  Among other things, this means it can not be
> ignored or
> caught, to preclude handlers doing something that might delay termination.
>
>
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04
>
> Since at least 4.2BSD and SVr2 (the first version of UNIX that I worked on)
> there have even been distinct kernel code paths to ensure special handling
> of
> SIGKILL.  There's nothing we can do about SIGKILL except be prepared to
> deal
> with it the same way we'd deal with the entire machine crashing.
>
> If you mean why is there nothing we can do on a *server* in response to
> SIGKILL on a *client*, that's a slightly more interesting question.  It's
> possible that the unique nature of SIGKILL puts connections into a
> different state than either system failure (on the more abrupt side) or
> clean shutdown (less abrupt).  If so, we probably need to take a look at
> the socket/RPC code or perhaps even protocol/server to see why these
> connections are not being cleaned up and shut down in a timely fashion.
>



-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Jeff Darcy]

> Does anyone know why GF_PARENT_DOWN is not triggered on SIGKILL? It will give
> a chance for xlators to do any cleanup they need to do. For example ec can
> complete the delayed xattrops.

Nothing is triggered on SIGKILL.  SIGKILL is explicitly defined to terminate a
process *immediately*.  Among other things, this means it can not be ignored or
caught, to preclude handlers doing something that might delay termination.

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_04

Since at least 4.2BSD and SVr2 (the first version of UNIX that I worked on)
there have even been distinct kernel code paths to ensure special handling of
SIGKILL.  There's nothing we can do about SIGKILL except be prepared to deal
with it the same way we'd deal with the entire machine crashing.

If you mean why is there nothing we can do on a *server* in response to
SIGKILL on a *client*, that's a slightly more interesting question.  It's
possible that the unique nature of SIGKILL puts connections into a
different state than either system failure (on the more abrupt side) or
clean shutdown (less abrupt).  If so, we probably need to take a look at
the socket/RPC code or perhaps even protocol/server to see why these
connections are not being cleaned up and shut down in a timely fashion.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] Support to reclaim locks (posix) provided lkowner & range matches

[Soumya Koduri]

Hi,

In certain scenarios (esp.,in highly available environments), the 
application may have to fail-over/connect to a different glusterFS 
client while the I/O is happening. In such cases until there is a ping 
timer expiry and glusterFS server cleans up the locks held by the older 
glusterFS client, the application will not be able to reclaim their lost 
locks. To avoid that we need support in Gluster to let clients reclaim 
the existing locks provided lkwoner and the lock range matches.


One of the applications which shall benefit from this support is 
NFS-Ganesha. NFS clients try to reclaim their post server reboot.


I have made relevant changes (WIP) on the server side to have this 
support [1]. The changes include -


* A new CLI option is provided "features.locks-reclaim-lock" to enable 
this support.


* Assuming below is done on the client-side (gfapi) - TODO
While re-trying the lock request, application has to notify the 
glusterFS client that it is a reclaim request. Client on receiving such 
request should set a boolean "reclaim-lock" in the xdata passed to lock 
request.


* On the server-side -
  - A new field 'reclaim' is added to 'posix_lock_t' to note if it is 
to be reclaimed.
  - While processing LOCK fop, if the "reclaim-lock" is set in the 
xdata received, reclaim field will be enabled in the new posix lock created.
  - While checking for conflicting locks (in 'same_owner()'), if the 
reclaim field is set, comparison will be done for lkowner and lock 
ranges instead of comparing both lkwoner and client UID.

  - Later it will fall through '__insert_and_merge' and the old lock
will be updated with the details of the new lock created (along with 
client details).


For client-side support, I am thinking if we can integrate with the new 
lock API being introduced as part of mandatory lock support in gfapi [2]


Kindly take a look and provide your comments/suggestions.

The changes seemed minimal and hence I haven't added it as 3.9 release 
feature. But if you feel it is a feature candidate, please let me know. 
I shall open up a feature page.


Thanks,
Soumya

[1] http://review.gluster.org/#/c/14986/
[2] http://review.gluster.org/11177
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] 3.7 regressions on NetBSD

[Atin Mukherjee]

On Fri, Jul 22, 2016 at 4:46 PM, Nigel Babu  wrote:

> Hello,
>
> I attempted to get us more space on NetBSD by creating a new partition
> called
> /data and putting /build as a symlink to /data/build. This has caused
> problems
> with tests/basic/quota.t. It's marked as bad for master, but not for
> release-3.7. This is possibly because we have a hard-coded grep for
> /build/install against df -h. Nithya has spent the last 2 days debugging
> without much success. What's a good way forward here? Mark the test as
> failing
> for 3.7?
>

Yes certainly till we get to solve it, otherwise all the patches to 3.7
will be blocked.


>
> We're going to have to accept that some of our tests might be in a
> different
> drive as we try to get more disk space for our machines. How can we make
> our
> tests more resilient from breakage due to regular expressions?
>
> --
> nigelb
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 

--Atin
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] Question on merging zfs snapshot support into the mainline glusterfs

[Rajesh Joseph]

On Thu, Jul 21, 2016 at 3:07 AM, Vijay Bellur  wrote:

> On 07/19/2016 11:01 AM, Atin Mukherjee wrote:
>
>>
>>
>> On Tue, Jul 19, 2016 at 7:29 PM, Rajesh Joseph > > wrote:
>>
>>
>>
>> On Tue, Jul 19, 2016 at 11:23 AM, > > wrote:
>>
>> __
>> Hi Rajesh,
>>
>> I'd thought about moving the zfs specific implementation to
>> something like
>>
>> xlators/mgmt/glusterd/src/plugins/zfs-specifs-stuffs for the
>> inital go. Could you let me know if this works or in sync with
>> what you'd thought about?
>>
>> Sriram
>>
>>
>> Hi Sriram,
>>
>> Sorry, I was not able to send much time on this. I would prefer you
>> move the code to
>>
>> xlators/mgmt/glusterd/plugins/src/zfs-specifs-stuffs
>>
>>
>>
>> How about having it under
>> xlators/mgmt/glusterd/plugins/snapshot/src/zfs-specifs-stuffs such that
>> in future if we have to write plugins for other features they can be
>> segregated?
>>
>>
> It would be nicer to avoid "specific-stuff" or similar from the naming. We
> can probably leave it at xlators/mgmt/glusterd/plugins/snapshot/src/zfs.
> The naming would be sufficient to indicate that code is specific to zfs
> snapshots.
>

I don't think the directory would be named "zfs-specific_stuffs, instead
zfs specific source file will come directly under
"xlators/mgmt/glusterd/plugins/snapshot/src/".
I think I should have been more clear, my bad.

-Rajesh
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

[Gluster-devel] GF_PARENT_DOWN on SIGKILL

[Pranith Kumar Karampuri]

Does anyone know why GF_PARENT_DOWN is not triggered on SIGKILL? It will
give a chance for xlators to do any cleanup they need to do. For example ec
can complete the delayed xattrops.

-- 
Pranith
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[David Gossage]

2016-07-22 2:32 GMT-05:00 Frank Rothenstein <
f.rothenst...@bodden-kliniken.de>:

> I can't tell myself, I'm using the ovirt-4.0-centos-gluster37 repo
> (from ovirt-release40). I have a second gluster-cluster as storage, I
> didn't dare to upgrade, as it simply works...not as an ovirt/vm storage.
>
> Am Freitag, den 22.07.2016, 08:28 +0200 schrieb Gandalf Corvotempesta:
>
> Il 22 lug 2016 07:54, "Frank Rothenstein" <
> f.rothenst...@bodden-kliniken.de> ha scritto:
> >
> > So 3.7.11 is the last usable version when using ZFS on bricks, afaik.
>
> Even with 3.8 this issue is present?
>
>
I believe Lindsay may have tested 3.8 and found is their still as well.



>
>
> --
>
>
>
>
> __
> BODDEN-KLINIKEN Ribnitz-Damgarten GmbH
> Sandhufe 2
> 18311 Ribnitz-Damgarten
>
> Telefon: 03821-700-0
> Fax:   03821-700-240
>
> E-Mail: i...@bodden-kliniken.de   Internet: http://www.bodden-kliniken.de
>
>
> Sitz: Ribnitz-Damgarten, Amtsgericht: Stralsund, HRB 2919, Steuer-Nr.: 
> 079/133/40188
>
> Aufsichtsratsvorsitzende: Carmen Schröter, Geschäftsführer: Dr. Falko Milski
>
>
> Der Inhalt dieser E-Mail ist ausschließlich für den bezeichneten Adressaten 
> bestimmt. Wenn Sie nicht der vorge-
>
> sehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, beachten 
> Sie bitte, dass jede Form der Veröf-
>
> fentlichung, Vervielfältigung oder Weitergabe des Inhalts dieser E-Mail 
> unzulässig ist. Wir bitten Sie, sofort den
> Absender zu informieren und die E-Mail zu löschen.
>
>
>  Bodden-Kliniken Ribnitz-Damgarten GmbH 2016
> *** Virenfrei durch Kerio Mail Server und Sophos Antivirus ***
>
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] 3.7.13 & proxmox/qemu

[Gandalf Corvotempesta]

Il 22 lug 2016 07:54, "Frank Rothenstein" 
ha scritto:
>
> So 3.7.11 is the last usable version when using ZFS on bricks, afaik.

Even with 3.8 this issue is present?
___
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel