Re: [Gluster-users] [Gluster-devel] A question of GlusterFS dentries!

2016-11-02 Thread Raghavendra G 
On Wed, Nov 2, 2016 at 9:38 AM, Raghavendra Gowdappa 
wrote:

>
>
> - Original Message -
> > From: "Keiviw" 
> > To: gluster-de...@gluster.org
> > Sent: Tuesday, November 1, 2016 12:41:02 PM
> > Subject: [Gluster-devel] A question of GlusterFS dentries!
> >
> > Hi,
> > In GlusterFS distributed volumes, listing a non-empty directory was slow.
> > Then I read the dht codes and found the reasons. But I was confused that
> > GlusterFS dht travesed all the bricks(in the volume) sequentially,why not
> > use multi-thread to read dentries from multiple bricks simultaneously.
> > That's a question that's always puzzled me, Couly you please tell me
> > something about this???
>
> readdir across subvols is sequential mostly because we have to support
> rewinddir(3).


Sorry. seekdir(3) is the more relevant function here. Since rewinddir
resets the dir stream to beginning, its not much of a difficulty to support
rewinddir with parallel readdirs across subvols.


> We need to maintain the mapping of offset and dentry across multiple
> invocations of readdir. In other words if someone did a rewinddir to an
> offset corresponding to earlier dentry, subsequent readdirs should return
> same set of dentries what the earlier invocation of readdir returned. For
> example, in an hypothetical scenario, readdir returned following dentries:
>
> 1. a, off=10
> 2. b, off=2
> 3. c, off=5
> 4. d, off=15
> 5. e, off=17
> 6. f, off=13
>
> Now if we did rewinddir to off 5 and issue readdir again we should get
> following dentries:
> (c, off=5), (d, off=15), (e, off=17), (f, off=13)
>
> Within a subvol backend filesystem provides rewinddir guarantee for the
> dentries present on that subvol. However, across subvols it is the
> responsibility of DHT to provide the above guarantee. Which means we
> should've some well defined order in which we send readdir calls (Note that
> order is not well defined if we do a parallel readdir across all subvols).
> So, DHT has sequential readdir which is a well defined order of reading
> dentries.
>
> To give an example if we have another subvol - subvol2 - (in addiction to
> the subvol above - say subvol1) with following listing:
> 1. g, off=16
> 2. h, off=20
> 3. i, off=3
> 4. j, off=19
>
> With parallel readdir we can have many ordering like - (a, b, g, h, i, c,
> d, e, f, j), (g, h, a, b, c, i, j, d, e, f) etc. Now if we do (with readdir
> done parallely):
>
> 1. A complete listing of the directory (which can be any one of 10P1 = 10
> ways - I hope math is correct here).
> 2. Do rewinddir (20)
>
> We cannot predict what are the set of dentries that come _after_ offset
> 20. However, if we do a readdir sequentially across subvols there is only
> one directory listing i.e, (a, b, c, d, e, f, g, h, i, j). So, its easier
> to support rewinddir.
>
> If there is no POSIX requirement for rewinddir support, I think a parallel
> readdir can easily be implemented (which improves performance too). But
> unfortunately rewinddir is still a POSIX requirement. This also opens up
> another possibility of a "no-rewinddir-support" option in DHT, which if
> enabled results in parallel readdirs across subvols. What I am not sure is
> how many users still use rewinddir? If there is a critical mass which wants
> performance with a tradeoff of no rewinddir support this can be a good
> feature.
>
> +gluster-users to get an opinion on this.
>
> regards,
> Raghavendra
>
> >
> >
> >
> >
> >
> >
> > ___
> > Gluster-devel mailing list
> > gluster-de...@gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> ___
> Gluster-devel mailing list
> gluster-de...@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] 3.8.5 gfapi problem

2016-11-02 Thread Dmitry Melekhov 

Hello!


We got problems after upgrade drom 3.8.4 to 3.8.5, namely we got

block I/O error in device 'drive-ide0-0-0': Input/output error (5)

Really qemu complains about no boot device.


I created bug report for it
https://bugzilla.redhat.com/show_bug.cgi?id=1390521
but there is no attention to it.

Downgrading back to 3.8.4 solved problem, but

Thank you!

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

[Gluster-users] Weekly community meeting - Nov 02, 2016

2016-11-02 Thread Kaushal M 
Hi all,

This week we discussed 3 topics (again!).

1. The FSFE etherpad service (where we host the meeting agenda and a
lot other gluster docs) will be shutdown, so we need to find an
alternative. We decided to make use of etherpads as temporary tools of
live collaboration, and using github wikis to permanently record the
document. Our first task is to identify and record known/live
etherpads. Nigel and I will start working on this, with our first
target being the community meeting agenda.

2. Nigel has created a centos-ci job for Glusto, which is nearly ready for use.

3. We continued discussion on 'Recognizing contributors', and we
couldn't arrive at a conclusion again. This discussion will continue
on the mailing lists.

I've pasted the meeting minutes below for easy reference. This will
also be available from a github wiki soon.

I'll be hosting next weeks meeting. See you all next week, same time &
same place.

~kaushal

## Logs

- Minutes: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-11-02/weekly_community_meeting_nov_2,_2016.2016-11-02-12.00.html
- Minutes (text):
https://meetbot.fedoraproject.org/gluster-meeting/2016-11-02/weekly_community_meeting_nov_2,_2016.2016-11-02-12.00.txt
- Log: 
https://meetbot.fedoraproject.org/gluster-meeting/2016-11-02/weekly_community_meeting_nov_2,_2016.2016-11-02-12.00.log.html

## Topics of Discussion

### Host for next week (DON'T FORGET)

- Next weeks meeting moderator: kshlm
- kkeithley will host the Nov 16 meeting

### Open Floor

- Recognising community contributors(Manikandan Selvaganesh)
 - https://glusterfs.biterg.io/app/kibana#/dashboard/Overview
 - [nigelb] Are we clear on what is being asked for? What is the
problem we are trying to solve
 - People want to be rewarded for their contributions
 - As a reward for their work make them maintainers
 - Need to measure quality, not quantity.
   - Super tough
 - nigelb will be taking discussion to mailing list
- Sending up EasyFix bugs(assigned with a owner) on mailing list so
that new contributors can get started(May be?) (Manikandan
Selvaganesh)
 - Skipped again
- Glusto Job on Centos CI
 - https://ci.centos.org/job/gluster_glusto
   - Job setup, needs tweaking to get it 100% functional.
 - Leverage learaning from 3.9 test day into some tests.
- FSFE pad service is about to be decomissioned. Need to find an alternative.
 - https://wiki.fsfe.org/Teams/System-Hackers/Decommissioning-Pad
 - Pads will turn read-only on Dec 1. And will be deleted Feb 1.
 - Options
   - beta.etherpad.org
   - https://pad.riseup.net/
   - https://github.com/ether/etherpad-lite/wiki/Sites-that-run-Etherpad-Lite
 - Etherpad for live editing documents + Github wiki for permanent records.
 - First find and preserve existing pads. Pick alternative later.

## Updates

### GlusterFS 4.0

- GD2 (kshlm)
 - Volgen-2.0 design and prototype started. More info at
https://www.gluster.org/pipermail/gluster-devel/2016-October/051297.html
- Multiplexing
 - https://www.gluster.org/pipermail/gluster-devel/2016-November/051364.html

### GlusterFS 3.9

- _None_

### GlusterFS 3.8

- _None_


### GlusterFS 3.7

- Release planned for later today. Got delayed because of holidays in India.

### GlusterFS 3.6

- _None_

### Other initiatives and related projects

 Infrastructure

- Ongoing work on getting Glusto jobs on Centos CI
- Network issue between Centos CI and review.gluster.org

 NFS Ganesha

- ganesha 2.4.1 and (lib)ntirpc-1.4.3) were released. packages in
Fedora Updates-Testing (soon Updates) and Ubuntu Launchpad. Other
distributions soon.
- now working on 2.5, emphasis on performance and memory consum

 Samba

- Regression caused due to https://review.gluster.org/#/c/15332/ is
being tracked by upstream bug
https://bugzilla.samba.org/show_bug.cgi?id=12404 so that the fix from
master gets backported to release branches.

 Heketi/Containers

- We have a new repo under Gluster org in Github for anything and
everything related to Gluster as storage in containers. Refer to
https://github.com/gluster/container-storage

 Testing/Glusto

- https://ci.centos.org/job/gluster_glusto

### Last weeks AIs
- rastar_afk/ndevos/jdarcy to improve cleanup to control the processes
that test starts.
 - _No updates_
- atinm to poke 3.9 release mgrs to finish and release
 - _No updates_
- obnox to starting discussion of Samba memory solutions
 - _No updates_
- jdarcy to discuss dbench smoke test failures on email
 - _No updates_

### Other Updates

- amye sent out the monthly newsletter
 - https://www.gluster.org/pipermail/gluster-users/2016-November/028920.html
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Shared Heal Times

2016-11-02 Thread Lindsay Mathieson 

On 2/11/2016 7:24 PM, David Gossage wrote:

What are final settings you ended up with after reverting?


Volume Name: datastore4
Type: Replicate
Volume ID: 0ba131ef-311d-4bb1-be46-596e83b2f6ce
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: vnb.proxmox.softlog:/tank/vmdata/datastore4
Brick2: vng.proxmox.softlog:/tank/vmdata/datastore4
Brick3: vna.proxmox.softlog:/tank/vmdata/datastore4
Options Reconfigured:
performance.readdir-ahead: on
cluster.data-self-heal: on
features.shard: on
cluster.quorum-type: auto
cluster.server-quorum-type: server
nfs.disable: on
nfs.addr-namelookup: off
nfs.enable-ino32: off
performance.strict-write-ordering: off
performance.stat-prefetch: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
cluster.eager-lock: enable
network.remote-dio: enable
features.shard-block-size: 64MB
cluster.granular-entry-heal: yes
cluster.locking-scheme: granular



--
Lindsay Mathieson

___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

Re: [Gluster-users] Shared Heal Times

2016-11-02 Thread David Gossage 
On Tue, Nov 1, 2016 at 9:59 PM, Lindsay Mathieson <
lindsay.mathie...@gmail.com> wrote:

> Just an update - after resetting all the heal "optimisations" :) I set, in
> general heals are much faster and back to normal. I've done several rolling
> upgrades with the servers since, rebooting each one in turn. Usually around
> 300 64MB shards will need healing after each boot. Its spends about 2-3
> minutes doing some fairly intensive CPU, then another 10 minutes to
> complete the heal. All up around 15 minutes per server. I'm more than
> satisfied with that.
>
>
> So no real problem other than PEBKAC.
>
>
> Moral of the story - as always, tuning settings for optimisation almost
> never works.
>

What are final settings you ended up with after reverting?


>
>
>
> thanks,
>
> --
> Lindsay Mathieson
>
> ___
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
___
Gluster-users mailing list
Gluster-users@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users