Re: Resizing LVM issue
Miroslav Skoric a écrit : On 06/22/2014 03:29 PM, Pascal Hambourg wrote: You should not have allocated all the space in the VG but instead should have left some free space for further growing or creating LVs when the need arises. Let's try once again: I have not allocated anything at the time of installation. Yes you have, by accepting the installer's suggestion : The only thing I've done was to accept Debian installer's suggestion to make the OS installation by using the whole hard disk and make the LVM. (In the other words, I let the installer to calculate particular partitions.) I see now some of you telling it should not be done that way, but would not be better to blame the programmers who had made such a 'bad option' within the installer? You as the user have the final choice. You have to decide if the installer's suggestion fits your needs and constraints. The installer doesn't know about them. Secondly, either the installer and/or some online manuals had suggested that the main purpose of LVM was to allow additional reallocating space within the OS's partitions, later if and when needed, from within an already working system. Yes, but LVs usually contain filesystems, and as you have seen, online shrinking of a mounted filesystem is often difficult or impossible. So it is better to avoid this kind of situation. If you can grow the PV (e.g. by adding a new disk) when you need to grow a LV, then it is fine to allocate all the space of the initial PV. However if you cannot grow the PV, then it is better to leave some of the PV space initially unallocated and to grow LVs from that free space when needed. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53b478b8.9000...@plouf.fr.eu.org
Re: Resizing LVM issue
On 06/22/2014 03:29 PM, Pascal Hambourg wrote: Miroslav Skoric a écrit : 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? If you are in such a situation, then you missed one of the goals of LVM. You should not have allocated all the space in the VG but instead should have left some free space for further growing or creating LVs when the need arises. Let's try once again: I have not allocated anything at the time of installation. The only thing I've done was to accept Debian installer's suggestion to make the OS installation by using the whole hard disk and make the LVM. (In the other words, I let the installer to calculate particular partitions.) I see now some of you telling it should not be done that way, but would not be better to blame the programmers who had made such a 'bad option' within the installer? Secondly, either the installer and/or some online manuals had suggested that the main purpose of LVM was to allow additional reallocating space within the OS's partitions, later if and when needed, from within an already working system. If that's not the case, I'd suggest the programmer community to invent a better solution. Anyway, as I said earlier, I managed to restore the space to the original status, then I reallocated things in the proper order. So, in a couple of days it will be a month since I fixed my wrong attempt to reallocate the space, and the machine is not complaining ever since :0 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53b030f...@eunet.rs
Re: Resizing LVM issue
Miroslav Skoric a écrit : 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? If you are in such a situation, then you missed one of the goals of LVM. You should not have allocated all the space in the VG but instead should have left some free space for further growing or creating LVs when the need arises. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53a6da43.40...@plouf.fr.eu.org
Re: Resizing LVM issue
Bob Proulx a écrit : There are many stories of this from people doing the same thing on the net. It seems that the code for expanding the file system is used often and optimized to run fast but that the code for shrinking it is not used very often and therefore has severe inefficiencies. But if you wait long enough, more than a week in my case, then it will finish successfully. Regardless of any optimization, shrinking a filesystem is much more difficult that expanding it. It requires to move all the used blocks which are allocated beyond the new size. Moving blocks on the same disk is a rather slow operation. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/53a6db85.8010...@plouf.fr.eu.org
Re: Resizing LVM issue
On Jun 15, 2014, at 12:34 PM, Bob Proulx b...@proulx.com wrote: In my case I had read the documentation. I had resized smaller partitions successfully. I had no idea it would take more than a week of 24x7 runtime before completing. If I had I would have done it differently. Which is why I am noting it here as the topic came up. To forewarn others. If I had only known then what I know now I would have copied it off and then back after resizing. Experience is sometimes the scars left behind after having done things poorly the first time. The optimum strategy is probably to make a full backup, then start the resize. If the resize looks like it's going to take too long, you can always stop it, write-off the mess left behind by the incomplete resize, re-partition, and restore from the backup. If the resize finishes in a reasonable amount of time, you go ahead and re-partition and don't have to do any restores. Worst case scenario -- you've lost the time spent in the incomplete resize. Best case scenario, you've lost the time spent in doing the backup, but you've gained the time you would have spent in doing the restore. Since you can't predict hardware/power failures with 100% certainty, doing the backup before you start is a good idea anyway, whether or not you actually need it in the end. Rick -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/577bd307-6bf3-4809-abab-7d5435554...@pobox.com
Re: Resizing LVM issue
On Sun, Jun 15, 2014 at 12:00:12AM +0200, Miroslav Skoric wrote: (Btw, the app apt-on-CD recently started to ask for more space in /tmp. After resizing, that app seems to be happy :-) tal% apt-cache search apt-on-CD tal% Third party? -- If you're not careful, the newspapers will have you hating the people who are being oppressed, and loving the people who are doing the oppressing. --- Malcolm X -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140615160319.GF29225@tal
Re: Resizing LVM issue
Miroslav Skoric wrote: 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? No need to re-install. Brute force works. I would use a second disk large enough to hold everything. Copy off the old, repartition, then copy back to the smaller sized partition. 2. Wouldn't be nice if resizing routines/commands/programs/... show calculated time they would need for such an operation, so a user could decide whether to continue or cancel? Yes. Of course. But it must be possible to estimate this. Sometimes the only way to know is to actually do the work and it isn't known until then. And someone must actually do the work of coding it up. In the case of the ext2 resize2fs the problem is mainly due to some inefficiency in the implemented algorithm. The expansion is well used and quite fast. But the shrink code is only rarely used. The shrink code is not optimized and hasn't had much attention. If someone were to get into that code base and review it I am confident they would find some type of nested loop causing it to operate in nasty exponential time that could be entirely avoided but is currently implemented in a most brute force and inefficient way. For example anyone who has ever implemented an AVL tree will know that supporting adding elements is quite easy. But supporting deleting elements is quite a bit more work. Many things are asymmetrical that way. It is the 80%-20% rule. 80% of the work takes 20% of the time. The remaining 20% of the work takes 80% of the time. In my case I had read the documentation. I had resized smaller partitions successfully. I had no idea it would take more than a week of 24x7 runtime before completing. If I had I would have done it differently. Which is why I am noting it here as the topic came up. To forewarn others. If I had only known then what I know now I would have copied it off and then back after resizing. Experience is sometimes the scars left behind after having done things poorly the first time. Bob signature.asc Description: Digital signature
Re: Resizing LVM issue
On Mon, 16 Jun 2014 04:03:19 +1200 Chris Bannister cbannis...@slingshot.co.nz wrote: On Sun, Jun 15, 2014 at 12:00:12AM +0200, Miroslav Skoric wrote: (Btw, the app apt-on-CD recently started to ask for more space in /tmp. After resizing, that app seems to be happy :-) tal% apt-cache search apt-on-CD tal% Third party? No, it seems to belong to main archive. $ apt-cache search apt on cd | grep ^apt apt - commandline package manager aptdaemon - transaction based package management service aptoncd - Installation disc creator for packages downloaded via APT Reco -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140616005239.ae8dac92d5a1661f543dc...@gmail.com
Re: Resizing LVM issue
On 06/05/2014 11:29 AM, Jochen Spieker wrote: Miroslav Skoric: On 06/01/2014 11:36 PM, Jochen Spieker wrote: If you don't have a backup you can try to resize the LV again to its original size and hope for the best. Thanks for suggestions. Yep, I managed to return back to the original size at first. Then I resized it properly (incl. leaving few gigs as unused space). e2fsck did spent a while to recheck each partition but seems that everything is in order now. We'll see in days to come ... Nice! It is still possible that some of your data was overwritten but if the fsck didn't find anything troubling you are probably safe now. Next todo: implement a useful backup strategy. :) J. Just to let you know that after some ten days after 2nd resizing everything is still in order (no complaints from fsck or else). From the lesson learned: The proper order of commands should be carefully performed; In that case, resizing the LVM is a good option until the installation process improves itself in a way it automatically format new partitions to be better balanced. (I mean, if I remember properly, during the initial system installation some years ago ... it was 6.0.1a that I upgraded to 7.5 since ... it offered to setup the LVM partitions automatically. So I accepted its recommendation.) But recently I realized that I needed more space in /tmp and found that I had more than enough free space in /home ... that was the reason for resizing. (Btw, the app apt-on-CD recently started to ask for more space in /tmp. After resizing, that app seems to be happy :-) M. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/539cc5ec.30...@eunet.rs
Re: Resizing LVM issue
On 06/05/2014 11:04 AM, Bob Proulx wrote: Richard Hector wrote: I prefer not to get in the situation where I have to shrink a filesystem though - xfs doesn't support it anyway. Agreed. Even better is to avoid it. Small ext{3,4} file systems shrink acceptably well. But larger ext{3,4} file systems can take a very long time to shrink. I once made the mistake of trying to shrink a 600G filesystem. The operation was eventually successful. But it took 10 days to complete! And once I started it there was no other option than to let it complete. Two questions: 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? 2. Wouldn't be nice if resizing routines/commands/programs/... show calculated time they would need for such an operation, so a user could decide whether to continue or cancel? M. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/539cc14a.3010...@eunet.rs
Re: Resizing LVM issue
On 06/05/2014 08:42 AM, Richard Hector wrote: If I have to shrink a filesystem, I tend to shrink it to something smaller than my eventual goal, then shrink the LV to the goal, then resize2fs again without specifying the size, so it grows to fit. I prefer not to get in the situation where I have to shrink a filesystem though - xfs doesn't support it anyway. Richard Thanks for suggestions. Well I would not shrink the filesystem (actually a part of it) if I did not need more space on /tmp (as a part of the LVM). Anyway, after this experience, may I suggest to LVM programmers to think about some software routines that would enable users to recompose (resize, shrink, whatever ...) their LVM from within a mounted system, in a way that after the next reboot, the LVM and FS automatically recomposes itself - so to avoid common mistakes. M. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/539cbfa4.7060...@eunet.rs
Re: Resizing LVM issue
On Sat, 14 Jun 2014 23:40:26 +0200 Miroslav Skoric sko...@eunet.rs wrote: 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? http://www.yourhowto.net/increase-tmp-partition-size-linux/ However, adding some GB of RAM would be better (to extend tmpfs). 2. Wouldn't be nice if resizing routines/commands/programs/... show calculated time they would need for such an operation, so a user could decide whether to continue or cancel? This would be only be empiric (or take a huge algorithm and a lot of CPU/RAM to correctly compute). -- ju the Lord does not touch many people nowadays fab priests take care of this for him signature.asc Description: PGP signature
Re: Resizing LVM issue
On 6/14/2014 4:33 PM, Miroslav Skoric wrote: ...may I suggest to LVM programmers to think about some software routines that would enable users to recompose (resize, shrink, whatever ...) their LVM from within a mounted system, in a way that after the next reboot, the LVM and FS automatically recomposes itself - so to avoid common mistakes. This is not possible. A filesystem must be shrunk before the underlying storage device. If you shrink the LV first then portions of the filesystem will now map to non-existent sectors. If files exist in those sectors they will be lost. Same goes for filesystem metadata. It is possible to add sectors to a device under a mounted filesystem because the filesystem has no knowledge of them, and is not mapping them. The same is not true of removing sectors under a mounted filesystem, for the reason above. Cheers, Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/539cd28a.20...@hardwarefreak.com
Re: Resizing LVM issue
Miroslav Skoric: Two questions: 1. What would you do if you need more space in /tmp and you know you have some spare space in /home or else, but do not want to reinstall? If it's only temporarily, I would probably do a bind-mount. Just create ~/tmp-tmp, as root cp -a /tmp ~/tmp-tmp/, mount -o bind ~/tmp-tmp/tmp /tmp. (Sorry for the many tmps. :)) But I never had that issue in several years running sid on a laptop with 4GB RAM and /tmp as tmpfs. 2. Wouldn't be nice if resizing routines/commands/programs/... show calculated time they would need for such an operation, so a user could decide whether to continue or cancel? They would do that if there was a way to know in advance. I don't think it's possible to do more than a wild guess which I assume could still be off by a factor of two or more. J. -- When standing at the top of beachy head I find the rocks below very attractive. [Agree] [Disagree] http://www.slowlydownward.com/NODATA/data_enter2.html signature.asc Description: Digital signature
Re: Resizing LVM issue
On 05/06/14 10:17, Miroslav Skoric wrote: On 06/01/2014 11:03 PM, emmanuel segura wrote: i think the correct steps are: resize2fs /dev/mapper/localhost-home -2G lvresize --size -2G /dev/mapper/localhost-home Thank you. I tried with the first command but it did not work (it returned an error). Pasting the error into your email is almost always beneficial :-) However, I don't think resize2fs can do relative sizes like that - you need to calculate the new size yourself, and specify it, without the minus sign. If I have to shrink a filesystem, I tend to shrink it to something smaller than my eventual goal, then shrink the LV to the goal, then resize2fs again without specifying the size, so it grows to fit. I prefer not to get in the situation where I have to shrink a filesystem though - xfs doesn't support it anyway. Richard -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5390113a.1050...@walnut.gen.nz
Re: Resizing LVM issue
Richard Hector wrote: I prefer not to get in the situation where I have to shrink a filesystem though - xfs doesn't support it anyway. Agreed. Even better is to avoid it. Small ext{3,4} file systems shrink acceptably well. But larger ext{3,4} file systems can take a very long time to shrink. I once made the mistake of trying to shrink a 600G filesystem. The operation was eventually successful. But it took 10 days to complete! And once I started it there was no other option than to let it complete. There are many stories of this from people doing the same thing on the net. It seems that the code for expanding the file system is used often and optimized to run fast but that the code for shrinking it is not used very often and therefore has severe inefficiencies. But if you wait long enough, more than a week in my case, then it will finish successfully. If I ever need to shrink an ext{3,4} file system again I will not shrink it. I will instead create a new file system of the desired size and then copy from the old to the new. That is reliable and will complete in much less time than shrinking an existing file system. Bob signature.asc Description: Digital signature
Re: Resizing LVM issue
Miroslav Skoric: On 06/01/2014 11:36 PM, Jochen Spieker wrote: If you don't have a backup you can try to resize the LV again to its original size and hope for the best. Thanks for suggestions. Yep, I managed to return back to the original size at first. Then I resized it properly (incl. leaving few gigs as unused space). e2fsck did spent a while to recheck each partition but seems that everything is in order now. We'll see in days to come ... Nice! It is still possible that some of your data was overwritten but if the fsck didn't find anything troubling you are probably safe now. Next todo: implement a useful backup strategy. :) J. -- I frequently find myself at the top of the stairs with absolutely nothing happening in my brain. [Agree] [Disagree] http://www.slowlydownward.com/NODATA/data_enter2.html signature.asc Description: Digital signature
Re: Resizing LVM issue
On 06/01/2014 11:03 PM, emmanuel segura wrote: i think the correct steps are: resize2fs /dev/mapper/localhost-home -2G lvresize --size -2G /dev/mapper/localhost-home Thank you. I tried with the first command but it did not work (it returned an error). However later I managed to resize the system to its original state, and from there to make resizing properly. M. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/538f9af6.4070...@eunet.rs
Re: Resizing LVM issue
On 06/01/2014 11:36 PM, Jochen Spieker wrote: If you don't have a backup you can try to resize the LV again to its original size and hope for the best. BTW, I found it to be good practice to initially use less than 100% of available space on my PVs for the LVs. That way I can grow filesystems that are too small easily when I need that space. Thanks for suggestions. Yep, I managed to return back to the original size at first. Then I resized it properly (incl. leaving few gigs as unused space). e2fsck did spent a while to recheck each partition but seems that everything is in order now. We'll see in days to come ... M. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/538f9b0f.4020...@eunet.rs
Re: Resizing LVM issue
from man resize2fs If you wish to shrink an ext2 partition, first use resize2fs to shrink the size of filesystem. Then you may use fdisk(8) to shrink the size of the partition. When shrinking the size of the partition, make sure you do not make it smaller than the new size of the ext2 filesystem! i think the correct steps are: resize2fs /dev/mapper/localhost-home -2G lvresize --size -2G /dev/mapper/localhost-home 2014-06-01 22:00 GMT+02:00 Miroslav Skoric sko...@eunet.rs: Hi, I have encrypted LVM on one of my Wheezy machines, and recently noticed that /tmp space was too low for one application (In fact it was about 350 MB and I wanted it to be around 2.5 GB). So I tried to make /tmp space bigger while I was mounted and online, but vgdisplay reported no free space for that action (something like that): sys@localhost:~$ sudo vgdisplay --- Volume group --- VG Name localhost System ID Formatlvm2 Metadata Areas1 Metadata Sequence No 9 VG Access read/write VG Status resizable MAX LV0 Cur LV6 Open LV 6 Max PV0 Cur PV1 Act PV1 VG Size 297.85 GiB PE Size 4.00 MiB Total PE 76249 Alloc PE / Size 76249 / 297.85 GiB Free PE / Size 0 / 0 VG UUID fbCaw1-u3SN-2HCy- Then I decided to shrink /home for some 2 gig and to add to /tmp : sys@localhost:~$ sudo lvresize --size -2G /dev/mapper/localhost-home sys@localhost:~$ sudo lvresize --size +2G /dev/mapper/localhost-tmp According to df, it did so: sys@localhost:~$ df Filesystem 1K-blocks Used Available Use% Mounted on rootfs329233 219069 93166 71% / udev 102400 10240 0% /dev tmpfs 304560 756303804 1% /run /dev/mapper/localhost-root329233 219069 93166 71% / tmpfs 51200 5120 0% /run/lock tmpfs 609100 80609020 1% /run/shm /dev/sda1 23319131650189100 15% /boot /dev/mapper/localhost-home 289312040 11966292 262649508 5% /home /dev/mapper/localhost-tmp240831511231 2273129 1% /tmp /dev/mapper/localhost-usr8647944 5745732 2462916 70% /usr /dev/mapper/localhost-var2882592 916600 1819560 34% /var sys@localhost:~$ It seems that /dev/mapper/localhost-tmp was about 2.4 GB so I wanted to resize newly changed filesystems: sys@localhost:~$ sudo resize2fs /dev/mapper/localhost-home resize2fs 1.42.5 (29-Jul-2012) Filesystem at /dev/mapper/localhost-home is mounted on /home; on-line resizing required resize2fs: On-line shrinking not supported Similar output was with e2fsck: sys@localhost:~$ sudo e2fsck -p /dev/mapper/localhost-home /dev/mapper/localhost-home is mounted. e2fsck: Cannot continue, aborting. sys@localhost:~$ Obviously I should not have done that while being mounted (or did not know the proper syntax), however it did not complain with resize2fs /dev/mapper/localhost-tmp But after the next reboot, it stopped when tried to perform Checking file systems: /dev/mapper/localhost-home: The filesystem size (according to the superblock) is 73481216 blocks The physical size of the device is 72956928 blocks Either the superblock or the partition table is likely to be corrupt! /dev/mapper/localhost-home: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. (i.e., without -a or -p options) Anyway, the other segments of the filesystem were clean, so by CONTROL-D it was possible to terminate the shell, so to resume system boot. My question is how to solve that inconsistency issue now? At first I tried with dumpe2fs in searching for backup superblocks, then with e2fsck -b one_of_those_backup_superblocks_from_the_list, but without resolution. I mean the inconsistency is not fixed. Probably I do not use e2fsck properly even when /home is not mounted. So the machine still keeps stopping during the boot at filesystem check, so I have to continue booting by pressing CONTROL-D. Any suggestion? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/538b8663.8050...@eunet.rs -- esta es mi vida e me la vivo hasta que dios quiera
Re: Resizing LVM issue
Miroslav Skoric: sys@localhost:~$ sudo lvresize --size -2G /dev/mapper/localhost-home … sys@localhost:~$ sudo resize2fs /dev/mapper/localhost-home You did it the wrong way round. You have to shrink from top to bottom: first the filesystem, then the LV (and then possibly the physical volume followed by the partition). It is hard to know whether your system already overwrote data previously in /home after you cut a part off of it and gave that to /tmp/. If that had happened to me, I would restore from backup. If you don't have a backup you can try to resize the LV again to its original size and hope for the best. BTW, I found it to be good practice to initially use less than 100% of available space on my PVs for the LVs. That way I can grow filesystems that are too small easily when I need that space. J. -- I am very intolerant with other drivers. [Agree] [Disagree] http://www.slowlydownward.com/NODATA/data_enter2.html signature.asc Description: Digital signature