Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, Jan 22, 2016 at 11:02 AM, Ed Heron wrote: > I'm still running CentOS 5 with Xen. > > We recently replaced a virtual host system board with an Intel > S1400FP4, so the host went from a 4 core Xeon with 32G RAM to a 6 core > Xeon with 48G RAM, max 96G. The drives are SSD. > > I was recently asked to move an InterBase server from Windows 7 to > Windows Server. The database is 30G. > > I'm speculating that if I put the database on a 35G virtual disk and > mirror it to a 35G RAM disk, the speed of database access might improve. > > I use local LVM for my virtual disks with DRBD on top to mirror the > disk to a backup server. > > If I change grub.conf to increase RAM disk size and increase host RAM, > I could create a 35G RAM disk. > > I'd modify rc.local to add > pvcreate /dev/ramdisk > vgextend vg /dev/ramdisk > lvconvert -m 1 --corelog vg/lv_database /dev/ramdisk > > Even with lv_database being 35G, it doesn't take long to activate the > mirror. > > I haven't decided where to put the commands to turn off the lvm > mirror. > lvconvert -m 0 vg/lv_database > vgreduce vg /dev/ramdisk > pvremove /dev/ramdisk > > I haven't put this in real world use, yet. > > On it's face, this might speed up database access. Would we expect it > to speed up database access in real world use? > > Should I document the process so others could know how to do this? I > realize new documentation for CentOS 5 virtualization would be > considered obsolete before I wrote it but I'm expecting to test CentOS 7 > virtualization in the next few months and, when I am comfortable, I'd > upgrade my 18 virtual hosts. I would update the documentation, at that > time, as well. > > I may not understand enough to understand what you are doing, you want to actively mirror this with LVM or? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, 2016-01-22 at 14:56 -0600, NightLightHosts Admin wrote: > On Fri, Jan 22, 2016 at 11:02 AM, Ed Heron wrote: > > I'm still running CentOS 5 with Xen. > > > > We recently replaced a virtual host system board with an Intel > > S1400FP4, so the host went from a 4 core Xeon with 32G RAM to a 6 core > > Xeon with 48G RAM, max 96G. The drives are SSD. > > > > I was recently asked to move an InterBase server from Windows 7 to > > Windows Server. The database is 30G. > > > > I'm speculating that if I put the database on a 35G virtual disk and > > mirror it to a 35G RAM disk, the speed of database access might improve. > > > > I use local LVM for my virtual disks with DRBD on top to mirror the > > disk to a backup server. > > > > If I change grub.conf to increase RAM disk size and increase host RAM, > > I could create a 35G RAM disk. > > > > I'd modify rc.local to add > > pvcreate /dev/ramdisk > > vgextend vg /dev/ramdisk > > lvconvert -m 1 --corelog vg/lv_database /dev/ramdisk > > > > Even with lv_database being 35G, it doesn't take long to activate the > > mirror. > > > > I haven't decided where to put the commands to turn off the lvm > > mirror. > > lvconvert -m 0 vg/lv_database > > vgreduce vg /dev/ramdisk > > pvremove /dev/ramdisk > > > > I haven't put this in real world use, yet. > > > > On it's face, this might speed up database access. Would we expect it > > to speed up database access in real world use? > > > > Should I document the process so others could know how to do this? I > > realize new documentation for CentOS 5 virtualization would be > > considered obsolete before I wrote it but I'm expecting to test CentOS 7 > > virtualization in the next few months and, when I am comfortable, I'd > > upgrade my 18 virtual hosts. I would update the documentation, at that > > time, as well. > > > I may not understand enough to understand what you are doing, you want > to actively mirror this with LVM or? Yes, in a test environment, I am mirroring a Logical Volume with a RAM disk to increase the perceived speed of the disk. I'm expecting to convert a live guest to this type of setup, this weekend. I was asking 2 questions. 1. Should I expect a significant increase in speed in a real world environment? With enough RAM, a good caching system will eventually do a similar function. This is almost like pre-loading a cache. 2. Should I document the process for others? I'm using CentOS 5 now, which is on it's way out, but I would update the documentation to include CentOS 7 when I upgrade my servers. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
> Ed Heron wrote: > Yes, in a test environment, I am mirroring a Logical Volume with a RAM > disk to increase the perceived speed of the disk. I'm expecting to > convert a live guest to this type of setup, this weekend. > > I was asking 2 questions. > 1. Should I expect a significant increase in speed in a real world > environment? With enough RAM, a good caching system will eventually do > a similar function. This is almost like pre-loading a cache. > 2. Should I document the process for others? I'm using CentOS 5 now, > which is on it's way out, but I would update the documentation to > include CentOS 7 when I upgrade my servers. > See this is where I was confused. Would not the LVM mirror have to sync all the time with the disk anyways? Is there something about LVM mirroring that can handle disks of different speeds? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, 2016-01-22 at 16:17 -0600, NightLightHosts Admin wrote: > > Ed Heron wrote: > > Yes, in a test environment, I am mirroring a Logical Volume with a RAM > > disk to increase the perceived speed of the disk. I'm expecting to > > convert a live guest to this type of setup, this weekend. > > > > I was asking 2 questions. > > 1. Should I expect a significant increase in speed in a real world > > environment? With enough RAM, a good caching system will eventually do > > a similar function. This is almost like pre-loading a cache. > > 2. Should I document the process for others? I'm using CentOS 5 > > now, > > which is on it's way out, but I would update the documentation to > > include CentOS 7 when I upgrade my servers. > > > > See this is where I was confused. Would not the LVM mirror have to > sync all the time with the disk anyways? Yes, but it isn't that simple. One copy of the mirror would be on a physical disk. The other copy of the mirror would be on RAM disk. Since data in RAM doesn't generally survive reboot, the RAM piece would need to be turned off before shutdown and created on startup. > Is there something about LVM mirroring that can handle disks of > different speeds? With newer LVM, there appears to be some settings that might help with that a bit. With this older verion, I'd be hoping that the next available disk would handle each request. If the physical disk takes longer to deal with the writes, the RAM disk might be the one that is available most of the time. I'd much prefer a method of pre-filling a 35G cache but I saw a reference to creating a disk mirror in RAM and decided to explore it. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, Jan 22, 2016 at 4:54 PM, Ed Heron wrote: > Yes, but it isn't that simple. One copy of the mirror would be on a > physical disk. The other copy of the mirror would be on RAM disk. > Since data in RAM doesn't generally survive reboot, the RAM piece would > need to be turned off before shutdown and created on startup. > >> Is there something about LVM mirroring that can handle disks of >> different speeds? > > With newer LVM, there appears to be some settings that might help with > that a bit. With this older verion, I'd be hoping that the next > available disk would handle each request. If the physical disk takes > longer to deal with the writes, the RAM disk might be the one that is > available most of the time. > > I'd much prefer a method of pre-filling a 35G cache but I saw a > reference to creating a disk mirror in RAM and decided to explore it. > Can you post the results of your test when you get it working? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, 2016-01-22 at 16:59 -0600, NightLightHosts Admin wrote: > On Fri, Jan 22, 2016 at 4:54 PM, Ed Heron wrote: > > Yes, but it isn't that simple. One copy of the mirror would be on a > > physical disk. The other copy of the mirror would be on RAM disk. > > Since data in RAM doesn't generally survive reboot, the RAM piece would > > need to be turned off before shutdown and created on startup. > > > >> Is there something about LVM mirroring that can handle disks of > >> different speeds? > > > > With newer LVM, there appears to be some settings that might help with > > that a bit. With this older verion, I'd be hoping that the next > > available disk would handle each request. If the physical disk takes > > longer to deal with the writes, the RAM disk might be the one that is > > available most of the time. > > > > I'd much prefer a method of pre-filling a 35G cache but I saw a > > reference to creating a disk mirror in RAM and decided to explore it. > > > > Can you post the results of your test when you get it working? Absolutely, I'll share my real world results. I'm happy that I'm not the only person interested in the technique. I'm a little disappointed somebody isn't telling me there is a much simpler method of putting my database in RAM. The technique is only useful in a situation where the server has gobs of RAM so it might only apply to a small subset of users but it might speed up database access. And since it is being done by the virtual host, the guest doesn't need to know anything about it. This keeps guest complexity down. Also, I don't have as much Windows knowledge as I have Linux knowledge so it was easier for me to implement under Linux. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
Ed Heron wrote: > Absolutely, I'll share my real world results. I'm happy that I'm not > the only person interested in the technique. I'm a little disappointed > somebody isn't telling me there is a much simpler method of putting my > database in RAM. The technique is only useful in a situation where the > server has gobs of RAM so it might only apply to a small subset of users > but it might speed up database access. And since it is being done by > the virtual host, the guest doesn't need to know anything about it. > This keeps guest complexity down. Also, I don't have as much Windows > knowledge as I have Linux knowledge so it was easier for me to implement > under Linux. See, This is where I get confused again, which type of database is it? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Fri, 2016-01-22 at 17:39 -0600, NightLightHosts Admin wrote: > Ed Heron wrote: > > Absolutely, I'll share my real world results. I'm happy that I'm not > > the only person interested in the technique. I'm a little disappointed > > somebody isn't telling me there is a much simpler method of putting my > > database in RAM. The technique is only useful in a situation where the > > server has gobs of RAM so it might only apply to a small subset of users > > but it might speed up database access. And since it is being done by > > the virtual host, the guest doesn't need to know anything about it. > > This keeps guest complexity down. Also, I don't have as much Windows > > knowledge as I have Linux knowledge so it was easier for me to implement > > under Linux. > > > See, > > This is where I get confused again, which type of database is it? It is a Customer Relationship Management database running under InterBase on Microsoft Windows Server. However, because the database server is a virtual machine, it doesn't matter. The technique could be useful for speeding up any disk-centric activity. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On 01/22/2016 11:02 AM, Ed Heron wrote: I'm still running CentOS 5 with Xen. We recently replaced a virtual host system board with an Intel S1400FP4, so the host went from a 4 core Xeon with 32G RAM to a 6 core Xeon with 48G RAM, max 96G. The drives are SSD. I was recently asked to move an InterBase server from Windows 7 to Windows Server. The database is 30G. I'm speculating that if I put the database on a 35G virtual disk and mirror it to a 35G RAM disk, the speed of database access might improve. If that were running under Linux rather than Windows I'd suggest just giving that extra 35GB to its kernel and letting its normal caching keep everything in RAM. Whether Windows (7 or Server) would be clever enough to do that is another question. Of course you could just let the Linux host do the caching, but that runs the risk of other VMs or host activity displacing some of that cache and affecting the performance of your database VM. -- Bob Nichols "NOSPAM" is really part of my email address. Do NOT delete it. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] LVM mirror database to ramdisk
On Sat, 2016-01-23 at 09:27 -0600, Robert Nichols wrote: > On 01/22/2016 11:02 AM, Ed Heron wrote: > >I'm still running CentOS 5 with Xen. > > > >We recently replaced a virtual host system board with an Intel > > S1400FP4, so the host went from a 4 core Xeon with 32G RAM to a 6 core > > Xeon with 48G RAM, max 96G. The drives are SSD. > > > >I was recently asked to move an InterBase server from Windows 7 to > > Windows Server. The database is 30G. > > > >I'm speculating that if I put the database on a 35G virtual disk and > > mirror it to a 35G RAM disk, the speed of database access might improve. > > If that were running under Linux rather than Windows I'd suggest just > giving that extra 35GB to its kernel and letting its normal caching > keep everything in RAM. Whether Windows (7 or Server) would be clever > enough to do that is another question. Of course you could just let > the Linux host do the caching, but that runs the risk of other VMs > or host activity displacing some of that cache and affecting the > performance of your database VM. > Yes... You've got much of my thought process. The RAM disk mirror pre-loads the database into memory and forces it to stay in RAM. ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
