Re: [Gluster-users] Gluster and LVM
On Mon, Apr 8, 2019, 21:47 Strahil wrote: > Correct me if I'm wrong but thin LVM is needed for creation of snapshots. > Yes, you need thin provisioned logical volumes for gluster snapshots. Actually, gluster snapshots are lvm snapshots under the hood. > I am a new gluster user , but I don't see any LVM issues so far. > Neither me > Best Regards, > Strahil Nikolov > On Apr 8, 2019 21:15, Alex K wrote: > > I use gluster on top of lvm for several years without any issues. > > On Mon, Apr 8, 2019, 10:43 Felix Kölzow wrote: > > Thank you very much for your response. > > I fully agree that using LVM has great advantages. Maybe there is a > misunderstanding, > > but I really got the recommendation to not use (normal) LVM in combination > with gluster to > > increase the volume. *Maybe someone in the community has some good or bad > experience* > > *using LVM and gluster in combination.* So please let me know :) > > > One of the arguments for things like Gluster and Ceph is that you can many > storage nodes that operate in parallel so that the ideal is a very large > number of small drive arrays over a small number of very large drive > arrays. > > I also agree we that. In our case, we actually plan to get Redhat Gluster > Storage Support and an increase of > > storage nodes would mean an increase of support costs while the same > amount of storage volume is available. > > So we are looking for a reasonable compromise. > > Felix > On 03.04.19 17:12, Alvin Starr wrote: > > As a general rule I always suggest using LVM. > I have had LVM save my career a few times. > I believe that if you wish to use Gluster snapshots then the underlying > system needs to be a thinly provisioned LVM volume. > > Adding storage space to an LVM is easy and all modern file-systems support > online growing so it is easy to grow a file-system. > > If you have directory trees that are very deep and wide then you may want > to put a bit of thought into how you configure your Gluster installation. > We have a volume with about 50M files and something like an xfs dump or > rsync of the underlying filesystem will take close to a day but copying the > data over Gluster takes weeks. > This is a problem with all clustered file systems because there is extra > locking and co-ordination required for file operations. > > Also you need to realize that the performance of something like the > powervault is limited to the speed of the connection to your server. > So that a single SAS link is limited to 6Gb(for example) and so is your > disk array but most internal raid controllers will support the number of > ports * 6Gb. > This means that a computer with 12 drives in the front will access disk > faster than a system with a 12 drive disk array attached by a few SAS > links. > > One of the arguments for things like Gluster and Ceph is that you can many > storage nodes that operate in parallel so that the ideal is a very large > number of small drive arrays over a small number of very large drive > arrays. > > > On 4/3/19 10:20 AM, kbh-admin wrote: > > Hello Gluster-Community, > > > we consider to build several Gluster-servers and have a question > regarding lvm and Glusterfs. > > > Scenario 1: Snapshots > > Of course, taking snapshots is a good capability and we want to use lvm > for that. > > > Scenaraio 2: Increase Gluster volume > > We want to increase the Gluster volume by adding hdd's and/or by adding > > dell powervaults later. We got the recommendation to set up a new Gluster > volume > > for the powervaults and don't use lvm in that case (lvresize ) . > > > What would you suggest and how do you manage both lvm and Glusterfs > together? > > > Thanks in advance. > > > Felix > > ___ > Gluster-users mailing list > > > ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster and LVM
I use gluster on top of lvm for several years without any issues. On Mon, Apr 8, 2019, 10:43 Felix Kölzow wrote: > Thank you very much for your response. > > I fully agree that using LVM has great advantages. Maybe there is a > misunderstanding, > > but I really got the recommendation to not use (normal) LVM in combination > with gluster to > > increase the volume. *Maybe someone in the community has some good or bad > experience* > > *using LVM and gluster in combination.* So please let me know :) > > > One of the arguments for things like Gluster and Ceph is that you can many > storage nodes that operate in parallel so that the ideal is a very large > number of small drive arrays over a small number of very large drive > arrays. > > I also agree we that. In our case, we actually plan to get Redhat Gluster > Storage Support and an increase of > > storage nodes would mean an increase of support costs while the same > amount of storage volume is available. > > So we are looking for a reasonable compromise. > > Felix > On 03.04.19 17:12, Alvin Starr wrote: > > As a general rule I always suggest using LVM. > I have had LVM save my career a few times. > I believe that if you wish to use Gluster snapshots then the underlying > system needs to be a thinly provisioned LVM volume. > > Adding storage space to an LVM is easy and all modern file-systems support > online growing so it is easy to grow a file-system. > > If you have directory trees that are very deep and wide then you may want > to put a bit of thought into how you configure your Gluster installation. > We have a volume with about 50M files and something like an xfs dump or > rsync of the underlying filesystem will take close to a day but copying the > data over Gluster takes weeks. > This is a problem with all clustered file systems because there is extra > locking and co-ordination required for file operations. > > Also you need to realize that the performance of something like the > powervault is limited to the speed of the connection to your server. > So that a single SAS link is limited to 6Gb(for example) and so is your > disk array but most internal raid controllers will support the number of > ports * 6Gb. > This means that a computer with 12 drives in the front will access disk > faster than a system with a 12 drive disk array attached by a few SAS > links. > > One of the arguments for things like Gluster and Ceph is that you can many > storage nodes that operate in parallel so that the ideal is a very large > number of small drive arrays over a small number of very large drive > arrays. > > > On 4/3/19 10:20 AM, kbh-admin wrote: > > Hello Gluster-Community, > > > we consider to build several Gluster-servers and have a question > regarding lvm and Glusterfs. > > > Scenario 1: Snapshots > > Of course, taking snapshots is a good capability and we want to use lvm > for that. > > > Scenaraio 2: Increase Gluster volume > > We want to increase the Gluster volume by adding hdd's and/or by adding > > dell powervaults later. We got the recommendation to set up a new Gluster > volume > > for the powervaults and don't use lvm in that case (lvresize ) . > > > What would you suggest and how do you manage both lvm and Glusterfs > together? > > > Thanks in advance. > > > Felix > > ___ > Gluster-users mailing list > Gluster-users@Gluster.org > https://lists.Gluster.org/mailman/listinfo/Gluster-users > > > ___ > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster and LVM
Hi Felix, On 03/04/2019 15:20, kbh-admin wrote: Hello Gluster-Community, we consider to build several gluster-servers and have a question regarding lvm and glusterfs. Scenario 1: Snapshots Of course, taking snapshots is a good capability and we want to use lvm for that. Please keep in mind, not just LVM, but LVM's "thin volumes" Scenaraio 2: Increase gluster volume We want to increase the gluster volume by adding hdd's and/or by adding dell powervaults later. We got the recommendation to set up a new gluster volume for the powervaults and don't use lvm in that case (lvresize ) . What would you suggest and how do you manage both lvm and glusterfs together? Thanks in advance. If you use LVM, it is quite simple just to increase the volume. When you add new HDDs, create new logical volumes on them, make the file system, and add it as another brick to your gluster volume (gluster add-brick volumename replica N ip.add.re.ss:/new-brick ... etc...). -- Kind regards, Boris Zhmurov mailto: b...@kernelpanic.ru ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster and LVM
Thank you very much for your response. I fully agree that using LVM has great advantages. Maybe there is a misunderstanding, but I really got the recommendation to not use (normal) LVM in combination with gluster to increase the volume. *Maybe someone in the community has some good or bad experience* *using LVM and gluster in combination.* So please let me know :) One of the arguments for things like Gluster and Ceph is that you can many storage nodes that operate in parallel so that the ideal is a very large number of small drive arrays over a small number of very large drive arrays. I also agree we that. In our case, we actually plan to get Redhat Gluster Storage Support and an increase of storage nodes would mean an increase of support costs while the same amount of storage volume is available. So we are looking for a reasonable compromise. Felix On 03.04.19 17:12, Alvin Starr wrote: As a general rule I always suggest using LVM. I have had LVM save my career a few times. I believe that if you wish to use Gluster snapshots then the underlying system needs to be a thinly provisioned LVM volume. Adding storage space to an LVM is easy and all modern file-systems support online growing so it is easy to grow a file-system. If you have directory trees that are very deep and wide then you may want to put a bit of thought into how you configure your Gluster installation. We have a volume with about 50M files and something like an xfs dump or rsync of the underlying filesystem will take close to a day but copying the data over Gluster takes weeks. This is a problem with all clustered file systems because there is extra locking and co-ordination required for file operations. Also you need to realize that the performance of something like the powervault is limited to the speed of the connection to your server. So that a single SAS link is limited to 6Gb(for example) and so is your disk array but most internal raid controllers will support the number of ports * 6Gb. This means that a computer with 12 drives in the front will access disk faster than a system with a 12 drive disk array attached by a few SAS links. One of the arguments for things like Gluster and Ceph is that you can many storage nodes that operate in parallel so that the ideal is a very large number of small drive arrays over a small number of very large drive arrays. On 4/3/19 10:20 AM, kbh-admin wrote: Hello Gluster-Community, we consider to build several Gluster-servers and have a question regarding lvm and Glusterfs. Scenario 1: Snapshots Of course, taking snapshots is a good capability and we want to use lvm for that. Scenaraio 2: Increase Gluster volume We want to increase the Gluster volume by adding hdd's and/or by adding dell powervaults later. We got the recommendation to set up a new Gluster volume for the powervaults and don't use lvm in that case (lvresize ) . What would you suggest and how do you manage both lvm and Glusterfs together? Thanks in advance. Felix ___ Gluster-users mailing list Gluster-users@Gluster.org https://lists.Gluster.org/mailman/listinfo/Gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] Gluster and LVM
03.04.2019 18:20, kbh-admin пишет: Hello Gluster-Community, we consider to build several gluster-servers and have a question regarding lvm and glusterfs. Scenario 1: Snapshots Of course, taking snapshots is a good capability and we want to use lvm for that. Scenaraio 2: Increase gluster volume We want to increase the gluster volume by adding hdd's and/or by adding dell powervaults later. We got the recommendation to set up a new gluster volume for the powervaults and don't use lvm in that case (lvresize ) . What would you suggest and how do you manage both lvm and glusterfs together? If you already have storage why you need gluster? Just use it :-) Thanks in advance. Felix ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
[Gluster-users] Gluster and LVM
Hello Gluster-Community, we consider to build several gluster-servers and have a question regarding lvm and glusterfs. Scenario 1: Snapshots Of course, taking snapshots is a good capability and we want to use lvm for that. Scenaraio 2: Increase gluster volume We want to increase the gluster volume by adding hdd's and/or by adding dell powervaults later. We got the recommendation to set up a new gluster volume for the powervaults and don't use lvm in that case (lvresize ) . What would you suggest and how do you manage both lvm and glusterfs together? Thanks in advance. Felix ___ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
