Re: [Gluster-users] bitrot log messages

[Kotresh Hiremath Ravishankar]

Hi Jackie,

"gluster vol bitrot  status" should show the corrupted files gfid.
If you want to get the info from logs, it will be logged as below.

[2016-10-26 05:21:20.767774] A [MSGID: 118023] 
[bit-rot-scrub.c:246:bitd_compare_ckum] 0-master-bit-rot-0: CORRUPTION 
DETECTED: Object /dir1/file1 {Brick: /bricks/brick1/b10 | GFID: 
b912c776-7253-4418-a35f-a37661d673b6}
[2016-10-26 05:21:20.770294] A [MSGID: 118024] 
[bit-rot-scrub.c:266:bitd_compare_ckum] 0-master-bit-rot-0: Marking /dir1/file1 
[GFID: b912c776-7253-4418-a35f-a37661d673b6 | Brick: /bricks/brick1/b10] as 
corrupted..


Thanks and Regards,
Kotresh H R

- Original Message -
> From: "Jackie Tung" 
> To: gluster-users@gluster.org
> Sent: Wednesday, October 26, 2016 2:14:28 AM
> Subject: [Gluster-users] bitrot log messages
> 
> Hi,
> 
> Redhat documentation says that things will get logged to bitd.log, and
> scrub.log. These files are pretty big - even when we only take the “ E “ log
> level lines.
> 
> https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/chap-Detecting_Data_Corruption.html
> 
> Anyone know which particular message patterns we should be actively looking
> for to detect bitrot?
> 
> Thanks,
> Jackie
> 
> 
> 
> The information in this email is confidential and may be legally privileged.
> It is intended solely for the addressee. Access to this email by anyone else
> is unauthorized. If you are not the intended recipient, any disclosure,
> copying, distribution or any action taken or omitted to be taken in reliance
> on it, is prohibited and may be unlawful.
> 
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Problem with add-brick

[Raghavendra Talur]

Top posting because there are multiple questions

1. Atin, it is expected to fail if you don't have RDMA device or if it is
not configured.

2. Rafi and Dennis,
I was not able to determine from logs if it really is a RDMA bug. The brick
logs suggest that brick started and even accepted clients. We should look
at the brick log more deeply to see if there is a config issue somewhere.

As to why RDMA listener did not initiate on fs2 and fs3, we need to get
brick logs for them too.

Thanks,
Raghavendra Talur


On Fri, Sep 30, 2016 at 3:43 PM, Mohammed Rafi K C 
wrote:

> It seems like an actual bug, if you can file a bug in bugzilla, that
> would be great.
>
>
> At least I don't see workaround for this issue, may  be till the next
> update is available with fix, you can use either rdma alone or tcp alone
> volume.
>
> Let me know whether this is acceptable, if so I can give you the steps to
> change the transport of an existing volume.
>
>
> Regards
>
> Rafi KC
>
> On 09/30/2016 10:35 AM, Mohammed Rafi K C wrote:
>
>
>
> On 09/30/2016 02:35 AM, Dennis Michael wrote:
>
>
> Are there any workarounds to this?  RDMA is configured on my servers.
>
>
>
> By this, I assume your rdma setup/configuration over IPoIB is working fine.
>
> Can you tell us what machine you are using and whether SELinux is
> configured on the machine or not.
>
> Also I couldn't see any logs attached here.
>
> Rafi KC
>
>
>
> Dennis
>
> On Thu, Sep 29, 2016 at 7:19 AM, Atin Mukherjee 
> wrote:
>
>> Dennis,
>>
>> Thanks for sharing the logs.
>>
>> It seems like a volume configured created with tcp,rdma transport fails
>> to start (atleast in my local set up). The issue here is although the brick
>> process comes up, but glusterd receives a non zero ret code from the runner
>> interface which spawns the brick process(es).
>>
>> Raghavendra Talur/Rafi,
>>
>> Is this an intended behaviour if rdma device is not configured? Please
>> chime in with your thoughts
>>
>>
>> On Wed, Sep 28, 2016 at 10:22 AM, Atin Mukherjee < 
>> amukh...@redhat.com> wrote:
>>
>>> Dennis,
>>>
>>> It seems like that add-brick has definitely failed and the entry is not
>>> committed into glusterd store. volume status and volume info commands are
>>> referring the in-memory data for fs4 (which exist) but post a restart they
>>> are no longer available. Could you run glusterd with debug log enabled
>>> (systemctl stop glusterd; glusterd -LDEBUG) and provide us cmd_history.log,
>>> glusterd log along with fs4 brick log files to further analyze the issue?
>>> Regarding the missing RDMA ports for fs2, fs3 brick can you cross check if
>>> glusterfs-rdma package is installed on both the nodes?
>>>
>>> On Wed, Sep 28, 2016 at 7:14 AM, Ravishankar N <
>>> ravishan...@redhat.com> wrote:
>>>
 On 09/27/2016 10:29 PM, Dennis Michael wrote:



 [root@fs4 bricks]# gluster volume info

 Volume Name: cees-data
 Type: Distribute
 Volume ID: 27d2a59c-bdac-4f66-bcd8-e6124e53a4a2
 Status: Started
 Number of Bricks: 4
 Transport-type: tcp,rdma
 Bricks:
 Brick1: fs1:/data/brick
 Brick2: fs2:/data/brick
 Brick3: fs3:/data/brick
 Brick4: fs4:/data/brick
 Options Reconfigured:
 features.quota-deem-statfs: on
 features.inode-quota: on
 features.quota: on
 performance.readdir-ahead: on
 [root@fs4 bricks]# gluster volume status
 Status of volume: cees-data
 Gluster process TCP Port  RDMA Port  Online
  Pid
 
 --
 Brick fs1:/data/brick   49152 49153  Y
   1878
 Brick fs2:/data/brick   49152 0  Y
   1707
 Brick fs3:/data/brick   49152 0  Y
   4696
 Brick fs4:/data/brick   N/A   N/AN
   N/A
 NFS Server on localhost 2049  0  Y
   13808
 Quota Daemon on localhost   N/A   N/AY
   13813
 NFS Server on fs1   2049  0  Y
   6722
 Quota Daemon on fs1 N/A   N/AY
   6730
 NFS Server on fs3   2049  0  Y
   12553
 Quota Daemon on fs3 N/A   N/AY
   12561
 NFS Server on fs2   2049  0  Y
   11702
 Quota Daemon on fs2 N/A   N/AY
   11710

 Task Status of Volume cees-data
 
 --
 There are no active volume tasks

 [root@fs4 bricks]# ps auxww | grep gluster
 root 13791  0.0  0.0 701472 19768 ?Ssl  09:06   0:00
 

[Gluster-users] gluster and ovirt

[Thing]

Hi,

Has anyone added 3 gluster nodes to ovirt?  I dont seem to be able to find
much documentation on how to do this and hence failing.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] bitrot log messages

[Jackie Tung]

Hi,

Redhat documentation says that things will get logged to bitd.log, and 
scrub.log.  These files are pretty big - even when we only take the “ E “ log 
level lines.

https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/chap-Detecting_Data_Corruption.html
 


Anyone know which particular message patterns we should be actively looking for 
to detect bitrot?

Thanks,
Jackie
-- 
 

The information in this email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this email 
by anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] gluster0:group1 not matching up with mounted directory

[Joe Julian]

Your volumes named "group0" and "group1" are not replicating, according 
to the volume info you included in your original email. They're both 
distribute volumes with no replication.


On 10/20/2016 08:55 PM, Cory Sanders wrote:

Niels,

Thanks for your answer.  Can you look at the du examples below.  Right now I am 
concerned with gluster0:group0 and group1

They are not replicating properly.  They are supposed to replicate across 3 of 
my 5 nodes.  Not shown here are nodes 2 and 3.

Thanks!


root@node0:/data/brick1# du -h -d 2
382G./group0/.glusterfs
8.0K./group0/images
0   ./group0/template
0   ./group0/dump
382G./group0
0   ./group1/.glusterfs
0   ./group1/images
0   ./group1/template
0   ./group1/dump
0   ./group1
382G.

root@node1:/data/brick1# du -h -d 2
148G./gluster/.glusterfs
4.0K./gluster/images
0   ./gluster/template
0   ./gluster/dump
0   ./gluster/private
148G./gluster
0   ./safe/images
0   ./safe/template
0   ./safe/dump
0   ./safe
314G./group0/.glusterfs
4.0K./group0/images
0   ./group0/template
0   ./group0/dump
314G./group0
182G./group1/.glusterfs
0   ./group1/images
0   ./group1/template
0   ./group1/dump
182G./group1
643G.
root@node4:/data/brick1# du -h -d 2
3.2T./machines0/.glusterfs
0   ./machines0/images
0   ./machines0/template
76K ./machines0/dump
0   ./machines0/private
3.2T./machines0
196G./group1/.glusterfs
0   ./group1/images
0   ./group1/template
0   ./group1/dump
196G./group1
255G./group0/.glusterfs
4.0K./group0/images
0   ./group0/template
0   ./group0/dump
255G./group0
1.5T./backups/.glusterfs
0   ./backups/images
0   ./backups/template
28K ./backups/dump
1.5T./backups
5.1T.
root@node4:/data/brick1#

-Original Message-
From: Niels de Vos [mailto:nde...@redhat.com]
Sent: Tuesday, October 18, 2016 1:28 AM
To: Cory Sanders 
Cc: gluster-users@gluster.org
Subject: Re: [Gluster-users] gluster0:group1 not matching up with mounted 
directory

On Tue, Oct 18, 2016 at 04:57:29AM +, Cory Sanders wrote:

I have volumes set up like this:
gluster> volume info

Volume Name: machines0
Type: Distribute
Volume ID: f602dd45-ddab-4474-8308-d278768f1e00
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: gluster4:/data/brick1/machines0

Volume Name: group1
Type: Distribute
Volume ID: cb64c8de-1f76-46c8-8136-8917b1618939
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: gluster1:/data/brick1/group1

Volume Name: backups
Type: Replicate
Volume ID: d7cb93c4-4626-46fd-b638-65fd244775ae
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: gluster3:/data/brick1/backups
Brick2: gluster4:/data/brick1/backups

Volume Name: group0
Type: Distribute
Volume ID: 0c52b522-5b04-480c-a058-d863df9ee949
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: gluster0:/data/brick1/group0

My problem is that when I do a disk free, group1 is filled up:

root@node0:~# df -h
Filesystem  Size  Used Avail Use% Mounted on
udev 10M 0   10M   0% /dev
tmpfs   3.2G  492K  3.2G   1% /run
/dev/mapper/pve-root 24G   12G   11G  52% /
tmpfs   5.0M 0  5.0M   0% /run/lock
tmpfs   6.3G   56M  6.3G   1% /run/shm
/dev/mapper/pve-data 48G  913M   48G   2% /var/lib/vz
/dev/sda1   495M  223M  248M  48% /boot
/dev/sdb1   740G  382G  359G  52% /data/brick1
/dev/fuse30M   64K   30M   1% /etc/pve
gluster0:group0 740G  382G  359G  52% /mnt/pve/group0
16.xx.xx.137:backups  1.9T  1.6T  233G  88% /mnt/pve/backups
node4:machines0 7.3T  5.1T  2.3T  70% /mnt/pve/machines0
gluster0:group1 740G  643G   98G  87% /mnt/pve/group1
gluster2:/var/lib/vz1.7T  182G  1.5T  11% /mnt/pve/node2local

When I do a du -h in the respective directories, this is what I get.
They don't match up with what a df -h shows.  Gluster0:group0 shows
the right amount of disk free, but gluster0:group1 is too fat and does
not correspond to what is in /mnt/pve/group1

du and df work a little different:
  - du: crawl the directory structure and calculate the size
  - df: call the statfs() function that resturns information directly
from the (superblock of the) filesystem

This means, that all 'df' calls are routed to the bricks that are used for the 
Gluster volume. Those bricks then call statfs() on behalf of the Gluster client 
(fuse mountpoint), and the Gluster client uses the values returned by the 
bricks to calculate the 'fake' output for 'df'.

Now, on your environment you seem to have the RAID1 filesystem mounted on 
/data/brick1 (/dev/sdb1 in the above 'df' output). All of the bricks are also located 
under /data/brick1/. This means that all 'df'
commands will 

Re: [Gluster-users] Epel Repo Link not accessible

[Maxence Sartiaux]

Hello,

You may have three packages

centos-release-gluster36
centos-release-gluster37
centos-release-gluster38

from the CentOS extra repository

- Original Message -
From: "aparna" 
To: "gluster-users" 
Sent: Monday, October 24, 2016 4:29:27 PM
Subject: [Gluster-users] Epel Repo Link not accessible

Hi All,

Just wondering if someone can help me. I was trying to access the below 
link :

Link: 
http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo

But didn't find anything. Looking forward for your response.

Thanks
Aparna
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Bad sectors on brick

[Ravishankar N]

On 10/25/2016 05:42 PM, Gandalf Corvotempesta wrote:

2016-10-24 16:13 GMT+02:00 Niels de Vos :

Yes, correct. But note that different filesystems can handle bad sectors
differently, read-only filesystems is the most common default though.

In 'man 8 mount' the option "errors=" describes the different values for
ext2/3/4. Configuring it to "continue" will most likely cause data
corruption or other bad problems and is definitely not advised ;-)

 From the opennebula forum:

"I think your notion of kernel behaviour during disk failures is not
correct. First of all, this is heavily filesystem dependent. Moreover,
when the bad sector is in the file data (as opposed to filesystem
metadata), read(2)returns something like ENXIO, and the filesystem
continues operating. When the bad sector is in the filesystem
metadata, most filesystems remount themselves read-only (AFAIK with
ext*fs, the exact behaviour can be set via tune2fs(8) as "remount
r/only", "panic", and "ignore" for the brave :-). When the disk is bad
to the point of generating unplug/replug sequence (e.g. SATA channel
reset), the filesystem starts returning ENXIO for all operations, but
it is still mounted. For systemd-based distributions, systemd
sometimes detects an unplugged disk (if it is mounted via /etc/fstab
entry), and umount(2)s it.

The kernel itself does not disable the disk, nor it remounts it r/only
in response to all types of failure."

so, how gluster handle a ENXIO ?

There is no special handling for specific errors.

i) If a particular syscall fails in the brick process, the errno is
propagated back to the application.

ii) If the posix health-checker thread [1], which runs 
periodically,fails in its
I/O, it kills the brick process, subsequent to which any I/O on that 
brick will

fail with ENOTCONN.

-Ravi

[1] 
https://github.com/gluster/glusterfs/blob/v3.8.5/xlators/storage/posix/src/posix-helpers.c#L1809

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Bad sectors on brick

[Gandalf Corvotempesta]

2016-10-24 16:13 GMT+02:00 Niels de Vos :
> Yes, correct. But note that different filesystems can handle bad sectors
> differently, read-only filesystems is the most common default though.
>
> In 'man 8 mount' the option "errors=" describes the different values for
> ext2/3/4. Configuring it to "continue" will most likely cause data
> corruption or other bad problems and is definitely not advised ;-)

>From the opennebula forum:

"I think your notion of kernel behaviour during disk failures is not
correct. First of all, this is heavily filesystem dependent. Moreover,
when the bad sector is in the file data (as opposed to filesystem
metadata), read(2)returns something like ENXIO, and the filesystem
continues operating. When the bad sector is in the filesystem
metadata, most filesystems remount themselves read-only (AFAIK with
ext*fs, the exact behaviour can be set via tune2fs(8) as "remount
r/only", "panic", and "ignore" for the brave :-). When the disk is bad
to the point of generating unplug/replug sequence (e.g. SATA channel
reset), the filesystem starts returning ENXIO for all operations, but
it is still mounted. For systemd-based distributions, systemd
sometimes detects an unplugged disk (if it is mounted via /etc/fstab
entry), and umount(2)s it.

The kernel itself does not disable the disk, nor it remounts it r/only
in response to all types of failure."

so, how gluster handle a ENXIO ?
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume with only one node

[Maxence Sartiaux]

Thank you everyone !

On Tue, 2016-10-25 at 14:27 +0530, Pranith Kumar Karampuri wrote:
> 
> 
> On Tue, Oct 25, 2016 at 2:07 PM, Oleksandr Natalenko
>  wrote:
> > Hello.
> > 
> > 25.10.2016 10:08, Maxence Sartiaux wrote:
> > > I need to migrate a old 2 node cluster to a proxmox cluster with
> > > a
> > > replicated gluster storage between those two (and a third
> > > arbitrer
> > > node).
> > > 
> > > Id like to create a volume with a single node, migrate the data
> > > on
> > > this volume from the old server and then reinstall the second
> > > server
> > > and add the second brick to the volume.
> > > 
> >  
> > If you have only 2 nodes in total, you may lower replica to replica
> > 1, reinstall another node, re-add bricks to form replica 2 and
> > repeat these steps for another node. No downtime and data migration
> > by hands needed. 
> 
> Just want to let you guys know that Kevin had bad experience doing
> add-bricks when VMs are online. We are still not able to recreate
> this bug. So I would advise you to do an offline migration for now.
>  
> > > I found no informations about creating a replicated volume with a
> > > single node, is it possible ?
> > > 
> >  
> > Yes, with "force" option, if you really need that.
> > 
> > > Also is it possible to add a third arbitrary node to an existing
> > > 2
> > > replica volume ?
> > > 
> >  
> > Yes, starting from v3.8.
> > 
> > Regards,
> >   Oleksandr
> > 
> > ___
> > Gluster-users mailing list
> > Gluster-users@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> > 
> 
> 
> ___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] autogen.sh still needed in 3.8.5

[Yannick Perret]

Hello,
some times ago it was reported that 'configure' failed (at least on some 
distro) on 3.8.1 from official .tgz (see 
https://www.gluster.org/pipermail/gluster-users.old/2016-August/027835.html 
for corresponding discussion).


The solution is simple: calling ./autogen.sh before solved to problem.
Checking today for 8.3.5 I get the same "problem", and the same solution 
works fine.


The only note (as far as I can see) in 'INSTALL' file is:
If you have cloned from git, run ./autogen.sh

This should be corrected, either by puting "correct" ./config.guess and 
./config.sub files in the tgz or by updating the INSTALL documentation.


Regards,
--
Y.




smime.p7s
Description: Signature cryptographique S/MIME
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] New style community meetings - No more status updates

[Atin Mukherjee]

Its a +1 from me, however this model will be only successful if we
diligently provide component updates.

On Tuesday 25 October 2016, Kaushal M  wrote:

> On Fri, Oct 21, 2016 at 11:46 AM, Kaushal M  > wrote:
> > On Thu, Oct 20, 2016 at 8:09 PM, Amye Scavarda  > wrote:
> >>
> >>
> >> On Thu, Oct 20, 2016 at 7:06 AM, Kaushal M  > wrote:
> >>>
> >>> Hi All,
> >>>
> >>> Our weekly community meetings have become mainly one hour of status
> >>> updates. This just drains the life out of the meeting, and doesn't
> >>> encourage new attendees to speak up.
> >>>
> >>> Let's try and change this. For the next meeting lets try skipping
> >>> updates all together and instead just dive into the 'Open floor' part
> >>> of the meeting.
> >>>
> >>> Let's have the updates to the regular topics be provided by the
> >>> regular owners before the meeting. This could either be through
> >>> sending out emails to the mailing lists, or updates entered into the
> >>> meeting etherpad[1]. As the host, I'll make sure to link to these
> >>> updates when the meeting begins, and in the meeting minutes. People
> >>> can view these updates later in their own time. People who need to
> >>> provide updates on AIs, just update the etherpad[1]. It will be
> >>> visible from there.
> >>>
> >>> Now let's move why I addressed this mail to this large and specific
> >>> set of people. The people who have been directly addressed are the
> >>> owners of the regular topics. You all are expected, before the next
> >>> meeting, to either,
> >>>  - Send out an update on the status for the topic you are responsible
> >>> for to the mailing lists, and then link to it on the the etherpad
> >>>  - or, provide you updates directly in the etherpad.
> >>> Please make sure you do this without fail.
> >>> If you do have anything to discuss, add it to the "Open floor" section.
> >>> Also, if I've missed out anyone in the addressed list, please make
> >>> sure they get this message too.
> >>>
> >>> Anyone else who wants to share their updates, add it to the 'Other
> >>> updates' section.
> >>>
> >>> Everyone else, go ahead and add anything you want to ask to the "Open
> >>> floor" section. Ensure to have your name with the topic you add
> >>> (etherpad colours are not reliable), and attend the meeting next week.
> >>> When your topic comes up, you'll have the floor.
> >>>
> >>> I hope that this new format helps make our meetings more colourful and
> >>> lively.
> >>>
> >>> As always, our community meetings will be held every Wednesday at
> >>> 1200UTC in #gluster-meeting on Freenode.
> >>> See you all there.
> >>>
> >>> ~kaushal
> >>>
> >>> [1]: https://public.pad.fsfe.org/p/gluster-community-meetings
> >>
> >>
> >> I really like this idea and am all in favor of color + liveliness.
> >> Let's give this new format three weeks or so, and we'll review around
> >> November 9th to see if we like this experiment.
> >> Fair?
> >> -- amye
> >
> > Sounds good to me.
> >
>
> Okay. We have one more day to the meeting, but I've yet to see any
> updates from all of you.
> Please ensure that you do this before the meeting tomorrow.
>
> >>
> >> --
> >> Amye Scavarda | a...@redhat.com  | Gluster Community Lead
> ___
> Gluster-devel mailing list
> gluster-de...@gluster.org 
> http://www.gluster.org/mailman/listinfo/gluster-devel
>


-- 
--Atin
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Epel Repo Link not accessible

[Kaushal M]

On Mon, Oct 24, 2016 at 7:59 PM, aparna  wrote:
> Hi All,
>
> Just wondering if someone can help me. I was trying to access the below link
> :
>
> Link:
> http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo
>
> But didn't find anything. Looking forward for your response.

The repository on download.gluster.org has been discontinued and the
CentOS Storage SIG [1] is the official replacement.
You should use the Storage SIG packages from now on.

[1] https://wiki.centos.org/SpecialInterestGroup/Storage

>
> Thanks
> Aparna
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] New style community meetings - No more status updates

[Kaushal M]

On Fri, Oct 21, 2016 at 11:46 AM, Kaushal M  wrote:
> On Thu, Oct 20, 2016 at 8:09 PM, Amye Scavarda  wrote:
>>
>>
>> On Thu, Oct 20, 2016 at 7:06 AM, Kaushal M  wrote:
>>>
>>> Hi All,
>>>
>>> Our weekly community meetings have become mainly one hour of status
>>> updates. This just drains the life out of the meeting, and doesn't
>>> encourage new attendees to speak up.
>>>
>>> Let's try and change this. For the next meeting lets try skipping
>>> updates all together and instead just dive into the 'Open floor' part
>>> of the meeting.
>>>
>>> Let's have the updates to the regular topics be provided by the
>>> regular owners before the meeting. This could either be through
>>> sending out emails to the mailing lists, or updates entered into the
>>> meeting etherpad[1]. As the host, I'll make sure to link to these
>>> updates when the meeting begins, and in the meeting minutes. People
>>> can view these updates later in their own time. People who need to
>>> provide updates on AIs, just update the etherpad[1]. It will be
>>> visible from there.
>>>
>>> Now let's move why I addressed this mail to this large and specific
>>> set of people. The people who have been directly addressed are the
>>> owners of the regular topics. You all are expected, before the next
>>> meeting, to either,
>>>  - Send out an update on the status for the topic you are responsible
>>> for to the mailing lists, and then link to it on the the etherpad
>>>  - or, provide you updates directly in the etherpad.
>>> Please make sure you do this without fail.
>>> If you do have anything to discuss, add it to the "Open floor" section.
>>> Also, if I've missed out anyone in the addressed list, please make
>>> sure they get this message too.
>>>
>>> Anyone else who wants to share their updates, add it to the 'Other
>>> updates' section.
>>>
>>> Everyone else, go ahead and add anything you want to ask to the "Open
>>> floor" section. Ensure to have your name with the topic you add
>>> (etherpad colours are not reliable), and attend the meeting next week.
>>> When your topic comes up, you'll have the floor.
>>>
>>> I hope that this new format helps make our meetings more colourful and
>>> lively.
>>>
>>> As always, our community meetings will be held every Wednesday at
>>> 1200UTC in #gluster-meeting on Freenode.
>>> See you all there.
>>>
>>> ~kaushal
>>>
>>> [1]: https://public.pad.fsfe.org/p/gluster-community-meetings
>>
>>
>> I really like this idea and am all in favor of color + liveliness.
>> Let's give this new format three weeks or so, and we'll review around
>> November 9th to see if we like this experiment.
>> Fair?
>> -- amye
>
> Sounds good to me.
>

Okay. We have one more day to the meeting, but I've yet to see any
updates from all of you.
Please ensure that you do this before the meeting tomorrow.

>>
>> --
>> Amye Scavarda | a...@redhat.com | Gluster Community Lead
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Epel Repo Link not accessible

[aparna]

Hi All,

Just wondering if someone can help me. I was trying to access the below 
link :


Link: 
http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/glusterfs-epel.repo


But didn't find anything. Looking forward for your response.

Thanks
Aparna
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] [Gluster-devel] New commands for supporting add/remove brick and rebalance on tiered volume

[Hari Gowtham]

I forgot to mention that with the first approach we need a separate tier 
add brick parser. If we add these changes to the existing add-brick parser, 
then the major changes are :
The word count for normal add-brick and tier-add-brick are totally different.
As the word count messes up, we need to put tier-add-brick parsing into a 
separate 
function now itself. 

We can support multiple tier even with the second approach. this doesn't need 
much 
changes in parsing. the existing one can be used with minor tweaks. and can be 
rewritten 
any time to separate it from add-brick parsing.

else if (!strcmp(words[3], "add-hot-brick")) {
 ret = do_cli_cmd_volume_add_hotbr_tier (state, word,
 words, wordcount-1);
 goto out;

This is the only part that will need change. We need to add similar 
function so that in that function we can set the dict to mention whether its 
hot or cold, or in this case the tier-id. 

The tier-id wont need a huge change in the cli part as it is already designed 
similarly 
to add hot or cold brick. Based on this dict value we are making changes to the 
volfile.

Having it as per the second aproach we can add arguments later if a specific 
function 
needs a specific argument.

- Original Message -
> From: "Dan Lambright" 
> To: "Hari Gowtham" 
> Cc: "Atin Mukherjee" , "gluster-users" 
> , "gluster-devel"
> 
> Sent: Saturday, October 22, 2016 10:57:21 PM
> Subject: Re: [Gluster-devel] New commands for supporting add/remove brick and 
> rebalance on tiered volume
> 
> 
> 
> - Original Message -
> > From: "Hari Gowtham" 
> > To: "Atin Mukherjee" 
> > Cc: "gluster-users" , "gluster-devel"
> > 
> > Sent: Friday, October 21, 2016 3:52:34 AM
> > Subject: Re: [Gluster-devel] New commands for supporting add/remove brick
> > and rebalance on tiered volume
> > 
> > Hi,
> > 
> > Currently there are two suggested options for the syntax of add/remove
> > brick:
> > 
> > 1) gluster v tier  add-brick [replica ] [tier-type
> > ]  ...
> > 
> > this syntax shows that its a add-brick operation on a tiered volume through
> > a
> > argument
> > instead of distinguishing using the command. The separation of tier-type is
> > done through
> > parsing. When it comes to the parsing of these [replica ] [tier-type
> > ] ,
> >  we need to parse between the tier-type, replica count and bricks. All
> >  these
> >  three variable
> >  make it complicated to parse and get the replica count, tier-type and
> >  brick.
> > 
> > currently the parsing is like:
> > w = str_getunamb (words[3], opwords_cl);
> > if (!w) {
> > type = GF_CLUSTER_TYPE_NONE;
> > .
> > .
> > } else if ((strcmp (w, "replica")) == 0) {
> > type = GF_CLUSTER_TYPE_REPLICATE;
> > .
> > .
> > }
> > } else if ((strcmp (w, "stripe")) == 0) {
> > type = GF_CLUSTER_TYPE_STRIPE;
> > .
> > .
> > } else {
> > .
> > .
> > }
> > 
> > we can use the same for replica as long as replica comes before tier-type
> > on
> > the syntax.
> > and add the parsing for tier-type using words[4] instead of words[3] and
> > repeat the same.
> > If its a plain distribute then we will be getting tier-type on words[3]. so
> > we have to parse
> > it again by checking on the wordcount. the word count influences the
> > parsing
> > to a great extent.
> > Having the tier-type after replica is looking bit off as tier-type is more
> > important here.
> > So we can have tier-type before replica count. This has to be maintained
> > thorughtout.
> > And a separate parsing can make this work. Both these will influence the
> > brick_index used
> > for parsing the brick making the switch using the word_count bit unclear.
> > This can be done but will add a lot of complications on code.
> > 
> > 2) gluster v tier  add-hot-brick/add-cold-brick [replica ]
> >  ...
> > In this syntax, we remove the tier-type from parsing and mention the type
> > on
> > the command.
> > The parsing remains the same as add-brick parsing. as differentiate between
> > the hot and cold
> > brick is done by the command
> > 
> > if (!strcmp(words[1], "detach-tier")) {
> > ret = do_cli_cmd_volume_detach_tier (state, word,
> >  words, wordcount);
> > goto out;
> > 
> > } else if (!strcmp(words[1], "attach-tier")) {
> > ret = do_cli_cmd_volume_attach_tier (state, word,
> >  words, wordcount);
> > goto out;
> > } 

Re: [Gluster-users] Volume with only one node

[Pranith Kumar Karampuri]

On Tue, Oct 25, 2016 at 2:07 PM, Oleksandr Natalenko <
oleksa...@natalenko.name> wrote:

> Hello.
>
> 25.10.2016 10:08, Maxence Sartiaux wrote:
>
>> I need to migrate a old 2 node cluster to a proxmox cluster with a
>> replicated gluster storage between those two (and a third arbitrer
>> node).
>>
>> Id like to create a volume with a single node, migrate the data on
>> this volume from the old server and then reinstall the second server
>> and add the second brick to the volume.
>>
>
> If you have only 2 nodes in total, you may lower replica to replica 1,
> reinstall another node, re-add bricks to form replica 2 and repeat these
> steps for another node. No downtime and data migration by hands needed.


Just want to let you guys know that Kevin had bad experience doing
add-bricks when VMs are online. We are still not able to recreate this bug.
So I would advise you to do an offline migration for now.


>
> I found no informations about creating a replicated volume with a
>> single node, is it possible ?
>>
>
> Yes, with "force" option, if you really need that.
>
> Also is it possible to add a third arbitrary node to an existing 2
>> replica volume ?
>>
>
> Yes, starting from v3.8.
>
> Regards,
>   Oleksandr
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>



-- 
Pranith
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume with only one node

[Oleksandr Natalenko]

Hello.

25.10.2016 10:08, Maxence Sartiaux wrote:

I need to migrate a old 2 node cluster to a proxmox cluster with a
replicated gluster storage between those two (and a third arbitrer
node).

Id like to create a volume with a single node, migrate the data on
this volume from the old server and then reinstall the second server
and add the second brick to the volume.


If you have only 2 nodes in total, you may lower replica to replica 1, 
reinstall another node, re-add bricks to form replica 2 and repeat these 
steps for another node. No downtime and data migration by hands needed.



I found no informations about creating a replicated volume with a
single node, is it possible ?


Yes, with "force" option, if you really need that.


Also is it possible to add a third arbitrary node to an existing 2
replica volume ?


Yes, starting from v3.8.

Regards,
  Oleksandr
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Volume with only one node

[Kaushal M]

On Tue, Oct 25, 2016 at 1:38 PM, Maxence Sartiaux  wrote:
> Hello,
>
> I need to migrate a old 2 node cluster to a proxmox cluster with a
> replicated gluster storage between those two (and a third arbitrer node).
>
> Id like to create a volume with a single node, migrate the data on this
> volume from the old server and then reinstall the second server and add the
> second brick to the volume.
>
> I found no informations about creating a replicated volume with a single
> node, is it possible ?

This is possible.
You first create a single brick volume on the first server. Note that
this is not a replica volume right now.
# gluster volume create  :

After migration and re-installing the 2nd server, you add the new
server to the GlusterFS trusted pool.
# gluster peer probe 

And add a replica brick the volume you created earlier,
# gluster volume add-brick  replica 2 :


>
> Also is it possible to add a third arbitrary node to an existing 2 replica
> volume ?

I'm not sure this is possible yet. Krutika or Pranith can answer this.

>
> Thank you.
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Volume with only one node

[Maxence Sartiaux]

Hello, 

I need to migrate a old 2 node cluster to a proxmox cluster with a replicated 
gluster storage between those two (and a third arbitrer node). 

Id like to create a volume with a single node, migrate the data on this volume 
from the old server and then reinstall the second server and add the second 
brick to the volume. 

I found no informations about creating a replicated volume with a single node, 
is it possible ? 

Also is it possible to add a third arbitrary node to an existing 2 replica 
volume ? 

Thank you. 
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] strange memory consumption with libgfapi

[Oleksandr Natalenko]

Hello.

25.10.2016 09:11, Pavel Cernohorsky wrote:

Unfortunately it is not
possible to use valgrind properly, because libgfapi seems to leak just
by initializing and deinitializing (tested with different code).


Use Valgrind with Massif tool. That would definitely help.
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] strange memory consumption with libgfapi

[Pavel Cernohorsky]

Hello,

I was experimenting with libgfapi a bit and found something which I 
cannot explain.


I wanted to simulate a behavior of long running system, so I created a 
simple program which reads a file from Gluster volume, saves it under a 
new name, deletes the original file and prints out the memory 
statistics. And again, reads the file saved in the last round, saves a 
new one, … But it seems that such a program just consumes memory 
indefinitely.


The full source code of the program is attached, or available on 
http://pastebin.com/9emyDH5N. It is not a production ready code, just an 
experiment, but Gluster usage should be correct there.


My volume consists of 2 bricks, no replication, default options, Gluster 
server and client have both the same version 3.8.4-1.fc24. The file I am 
using for experiments is ~70 bytes. Program started on 70MB RSS and the 
last time I checked it was 250MB. Unfortunately it is not possible to 
use valgrind properly, because libgfapi seems to leak just by 
initializing and deinitializing (tested with different code).


Is there some call in the library to free the memory that I am not 
using? Some settings of the volume?


Thanks for your suggestions, kind regards,
Pavel

// g++ --std=c++14 -ggdb3 -lgfapi glfs_new_files.cpp

#include 
#include 
#include 
#include 
#include 
#include 

#include 

/// simple representation of the used memory
struct MemUsage
{
  double vsz = 0.0;
  double rss = 0.0;
};

/// Returns memory usage data.
MemUsage getMemUsage()
{
  // dummy vars for leading entries in stat that we don't care about
  std::string pid, comm, state, ppid, pgrp, session, tty_nr;
  std::string tpgid, flags, minflt, cminflt, majflt, cmajflt;
  std::string utime, stime, cutime, cstime, priority, nice;
  std::string O, itrealvalue, starttime;

  // the two fields we want
  unsigned long vsize;
  long rss;

  std::ifstream statStream("/proc/self/stat", std::ios_base::in);
  statStream >> pid >> comm >> state >> ppid >> pgrp >> session >> tty_nr
 >> tpgid >> flags >> minflt >> cminflt >> majflt >> cmajflt
 >> utime >> stime >> cutime >> cstime >> priority >> nice
 >> O >> itrealvalue >> starttime >> vsize >> rss; // don't care about the rest
  statStream.close();

  MemUsage result;
  result.vsz = vsize / 1024.0;
  long pageSizeKB = sysconf(_SC_PAGE_SIZE) / 1024; // in case x86-64 is configured to use 2MB pages
  result.rss = rss * pageSizeKB;
  return result;
}

/// Takes the file name of the original file and continuously replicates it with the "_n" where n is
/// increasing number.
/// Also prints memory statistics after each round.
void glusterReplicateFile(const std::string& initialFileName)
{
  glfs_t* gluster = glfs_new("test_volume");
  if (!gluster)
throw std::runtime_error("Unable to create virtual mount object");

  int error = glfs_set_volfile_server(gluster, "tcp", "10.10.52.92", 24007);
  if (error)
throw std::runtime_error("Unable to set volumefile server: " + std::string(strerror(errno)));

  error = glfs_set_logging(gluster, "/tmp/gluster_test_logging", 127);
  if (error)
throw std::runtime_error("Unable to set logging: " + std::string(strerror(errno)));

  error = glfs_init(gluster);
  if (error)
throw std::runtime_error("Error while initializing gluster: " + std::string(strerror(errno)));

  bool initialFileRead = true;
  uint64_t round = 0;
  std::string lastFileName = initialFileName;
  std::vector fileData;
  double lastRss = 0.0;
  while (++round)
  {
// read out the original file
glfs_fd_t* glusterFile = glfs_open(gluster, lastFileName.c_str(), O_RDONLY);
if (!glusterFile)
throw std::runtime_error("Error while opening the original file: " + std::string(strerror(errno)));
struct stat s;
error = glfs_fstat(glusterFile, );
if (error)
  throw std::runtime_error("Error while stating the original file: " + std::string(strerror(errno)));
fileData.resize(s.st_size);
ssize_t bytesRead = glfs_pread(glusterFile, fileData.data(), fileData.size(), 0, 0);
if (bytesRead != static_cast(fileData.size()))
  throw std::runtime_error("Error while reading the original file: " + std::string(strerror(errno)));
uint64_t fileSum = 0;
for (unsigned char value: fileData)
  fileSum += value;
error = glfs_close(glusterFile);
if (error)
  throw std::runtime_error("Error while closing the original file: " + std::string(strerror(errno)));

// create the new file
std::ostringstream newFileName;
newFileName << initialFileName << "_" << round;
glusterFile = glfs_creat(gluster, newFileName.str().c_str(), O_WRONLY, 0644);
if (!glusterFile)
throw std::runtime_error("Error while opening the new file: " + std::string(strerror(errno)));
ssize_t bytesWritten = glfs_pwrite(glusterFile, fileData.data(), fileData.size(), 0, O_APPEND);
if (bytesWritten != static_cast(fileData.size()))
  throw std::runtime_error("Error while 

Re: [Gluster-users] failed to find key 'child_up' in the options

[Avra Sengupta]

Hi,

The issue Niels is mentioning was present in 3.8.0, which I believe is 
the version you have installed. It was fixed in 3.8.1. Also, he is right 
about the fact that the issue made mounting completely non-functional, 
and there was no delay. To know the issue further could you please 
answer the following questions about your setup.


1. You have two servers, one running 3.7.11, and the other 3.8.0 right? 
Are you mounting the client on one of these servers, if yes then which 
one and whch server's IP are you using in the mount point.


2. Also can you describe your volume's configuration. Is it a replicate 
or dist-replicate volume. This will help us confirm that, the client 
translator is getting the child-up from the node running 3.8.0, and 
hence is operational after some delay.


3. Can you also confirm, if after the delay, you are able to see data 
being written to the server running 3.7.11.


Regards,
Avra

On 10/21/2016 04:13 PM, Niels de Vos wrote:

On Fri, Oct 21, 2016 at 12:14:15PM +0200, Josep Manel Andrés wrote:

Hi again,
I have two servers SLES 12 SP 0 with

Information for package glusterfs:
--
Repository: GlusterFS
Name: glusterfs
Version: 3.7.11-101.1
Arch: x86_64
Vendor: obs://build.opensuse.org/home:kkeithleatredhat


and now I decided to upgrade one server to SP1 and install

Information for package glusterfs:
--
Repository: GlusterFS-3.8 (SLE_12_SP1)
Name: glusterfs
Version: 3.8.0-100.1
Arch: x86_64
Vendor: obs://build.opensuse.org/home:kkeithleatredhat


but what happens now is that when trying to mount a volume with 3.8 it takes
so long...over 6 or 7 seconds, and here is the moment where gets stacked in
the logs:

[2016-10-21 10:05:18.285837] I [MSGID: 108005]
[afr-common.c:4137:afr_notify] 0-volume1-replicate-0: Subvolume
'volume1-client-1' came back up; going online.
[2016-10-21 10:05:18.285872] I [MSGID: 114035]
[client-handshake.c:201:client_set_lk_version_cbk] 0-volume1-client-1:
Server lk version = 1
[2016-10-21 10:05:18.286023] W [MSGID: 114007]
[client-handshake.c:1176:client_setvolume_cbk] 0-volume1-client-0: failed to
find key 'child_up' in the options



[2016-10-21 10:05:29.273913] I [fuse-bridge.c:5241:fuse_graph_setup] 0-fuse:
switched to graph 0
[2016-10-21 10:05:29.274151] I [fuse-bridge.c:4153:fuse_init]
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel
7.22
[2016-10-21 10:05:29.275426] I [MSGID: 108031]
[afr-common.c:1908:afr_local_discovery_cbk] 0-volume1-replicate-0: selecting
local read_child volume1-client-1



Any idea about what happens? Should I upgrade de other server?

This looks related to https://bugzilla.redhat.com/1350326 . It has been
addressed in glusterfs-3.8.0, so I do not know if the delay in mounting
is expected. Without the fix mounting seemed to have been completely
non-functional. Avra should be able to explain the details a little
more.

Niels


___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users