Re: Installing bullseye into previously existing encrypted disk with buster
On 16/02/2022 05:26, Nitebirdz wrote: On Sat, Feb 12, 2022 at 09:36:45AM +1100, David wrote: I'm not really paying attention to the latest capabilites that the installer might have, or to what any other distros are doing, but when I have attempted this in the past it appeared to me that the Debian installer does not directly support installing a fresh installation into a previously created LUKS encrypted volume. However it is certainly "possible" with some complicated tricks, and if you are prepared to risk accidentally destroying the whole encrypted volume if you make a mistake. That's what happened to me the first time I tried it. But I have adequate backups and alternative machines, so that didn't bother me. It is possible to trick the installer into opening the existing encrypted volume. Then (with numerous fiddly steps and using great caution not to make a mistake) the installer can then install into a new partition inside that, in the usual way. However the installation it creates will be broken and likely not bootable. Because we have tricked the installer beyond what it understands, it makes many mistakes. There will be problems with grub, with the cryptsetup configuration, and with the initramfs. That all then needs to be fixed by rebooting into an alternative environment that has cryptsetup tools available. Maybe the installer rescue system is capable of doing that, but I'm not sure because ... Indeed. I did some further searching (it's not an easy thing to search for), and ended up finding the following document: https://consolematt.wordpress.com/2013/06/19/reinstalling-debian-on-existing-lukslvm-partition/ I tested it on a VM inside QEMU, and it worked. So, basically, once we reach the point where we detect the hard drive, we need to drop to the shell, install additional software into the installation environment, and then run the commands to configure the already existing volume group and logical volumes. After that, we can return to the installer, and partman will see everything. We can then configure the proper mounts, and go on with the rest of the standard installation steps. However, as you explained, the installation is not bootable, it fails to recognize the encrypted volume group, and it just drops to the initramfs prompt. However, from there, we can run the commands documented in that blog entry, and it all works. While not disagreeing that the process is messy, I don't think it's quite as difficult in 2022 as it was in 2013. I used these two links to read up on it: https://www.blakehartshorn.com/installing-debian-on-existing-encrypted-lvm/ https://linuxconfig.org/how-to-install-debian-on-an-existing-luks-container and FWIW summarized my experience here: https://forums.bunsenlabs.org/viewtopic.php?pid=118486#p118486 It wasn't all that hard to re-use one of the encrypted partitions and keep the others. (Though I made the mistake of overwriting the previous boot partition, so the old installation would have needed a new /boot created in order to boot. I didn't bother because the partition was still accessible from the new system.) So, confirmed, it can be done. Maybe some day Debian Installer will cope with existing LUKS containers. -- John
Re: Installing bullseye into previously existing encrypted disk with buster
On Sat, Feb 12, 2022 at 09:36:45AM +1100, David wrote: > On Sat, 12 Feb 2022 at 02:54, Nitebirdz wrote: > > On Thu, Feb 10, 2022 at 04:37:55PM -0500, Dan Ritter wrote: > > > Nitebirdz wrote: > > > > > I currently have a laptop running buster on an encrypted disk that boots > > > > via EFI. The filesystems look like this: > > > > > > > > /dev/mapper/tangier--vg-root/ > > > > /dev/mapper/tangier--vg-home/home > > > > /dev/sda1 /boot/efi > > > > /dev/sda2 /boot > > > > > > > > I know I can easily upgrade to bullseye from the running system. > > > > However, > > > > what I usually do when it's time to upgrade Debian on a laptop is to > > > > start > > > > from a clean slate. It's my chance to clean up and remove old cruft > > > > (well, > > > > with the exception of my own home partition, of course). So, instead of > > > > upgrading, I just install the new version of Debian. > > > > > > > > Now, my problem is that, whenever I launch the installer, it wants to > > > > partition the disk. Is there a way to tell the installer to leave the > > > > existing partitioning scheme alone? Also, I'd need the installer to > > > > leave > > > > the home partition alone, and format and install over the other > > > > partitions. Is this possible? If so, how? I've been trying different > > > > approaches, and I don't seem to be able to find the way to do it. > > > > > > Yes. Tell the installer you want to partition the disks > > > manually, and then select each one and assign it to the role > > > that you want. For /home, either don't assign it or make sure > > > that you mark it as "leave the contents alone". > > > > Thanks. But it doesn't appear to work. The disk partitioning tool > > only shows the actual partitions, but no trace of the already existing > > encrypted volumes. See the screenshot attached. > > > > I'm testing this using QEMU. No matter what entry I select on that > > screen, it wants me to continue partitioning, and ends up destroying the > > previous setup. I cannot see a way to just get it to notice the already > > existing layout. That does work for more simple setups, but not for > > encrypted volumes, it seems. > > Hi Nitebirdz, > > For people quickly scanning through a lot of messages that they aren't > heavily interested in, I suspect it was easy to overlook the crucial > word "encrypted" in your first message. I know I didn't notice that > until your second message, which used that word a few more times. > Yep. I totally understand. It happens to me all the time. :) > I'm not really paying attention to the latest capabilites that the > installer might have, or to what any other distros are doing, but when > I have attempted this in the past it appeared to me that the Debian > installer does not directly support installing a fresh installation > into a previously created LUKS encrypted volume. > > However it is certainly "possible" with some complicated tricks, and > if you are prepared to risk accidentally destroying the whole > encrypted volume if you make a mistake. That's what happened to me the > first time I tried it. But I have adequate backups and alternative > machines, so that didn't bother me. > > It is possible to trick the installer into opening the existing > encrypted volume. Then (with numerous fiddly steps and using great > caution not to make a mistake) the installer can then install into a > new partition inside that, in the usual way. > > However the installation it creates will be broken and likely not > bootable. Because we have tricked the installer beyond what it > understands, it makes many mistakes. There will be problems with grub, > with the cryptsetup configuration, and with the initramfs. That all > then needs to be fixed by rebooting into an alternative environment > that has cryptsetup tools available. Maybe the installer rescue system > is capable of doing that, but I'm not sure because ... > Indeed. I did some further searching (it's not an easy thing to search for), and ended up finding the following document: https://consolematt.wordpress.com/2013/06/19/reinstalling-debian-on-existing-lukslvm-partition/ I tested it on a VM inside QEMU, and it worked. So, basically, once we reach the point where we detect the hard drive, we need to drop to the shell, install additional software into the installation environment, and then run the commands to configure the already existing volume group and logical volumes. After that, we can return to the installer, and partman will see everything. We can then configure the proper mounts, and go on with the rest of the standard installation steps. However, as you explained, the installation is not bootable, it fails to recognize the encrypted volume group, and it just drops to the initramfs prompt. However, from there, we can run the commands documented in that blog entry, and it all works. -- Nitebirdz
Re: Installing bullseye into previously existing encrypted disk with buster
On Sat, 12 Feb 2022 at 02:54, Nitebirdz wrote: > On Thu, Feb 10, 2022 at 04:37:55PM -0500, Dan Ritter wrote: > > Nitebirdz wrote: > > > I currently have a laptop running buster on an encrypted disk that boots > > > via EFI. The filesystems look like this: > > > > > > /dev/mapper/tangier--vg-root/ > > > /dev/mapper/tangier--vg-home/home > > > /dev/sda1 /boot/efi > > > /dev/sda2 /boot > > > > > > I know I can easily upgrade to bullseye from the running system. However, > > > what I usually do when it's time to upgrade Debian on a laptop is to start > > > from a clean slate. It's my chance to clean up and remove old cruft (well, > > > with the exception of my own home partition, of course). So, instead of > > > upgrading, I just install the new version of Debian. > > > > > > Now, my problem is that, whenever I launch the installer, it wants to > > > partition the disk. Is there a way to tell the installer to leave the > > > existing partitioning scheme alone? Also, I'd need the installer to leave > > > the home partition alone, and format and install over the other > > > partitions. Is this possible? If so, how? I've been trying different > > > approaches, and I don't seem to be able to find the way to do it. > > > > Yes. Tell the installer you want to partition the disks > > manually, and then select each one and assign it to the role > > that you want. For /home, either don't assign it or make sure > > that you mark it as "leave the contents alone". > > Thanks. But it doesn't appear to work. The disk partitioning tool > only shows the actual partitions, but no trace of the already existing > encrypted volumes. See the screenshot attached. > > I'm testing this using QEMU. No matter what entry I select on that > screen, it wants me to continue partitioning, and ends up destroying the > previous setup. I cannot see a way to just get it to notice the already > existing layout. That does work for more simple setups, but not for > encrypted volumes, it seems. Hi Nitebirdz, For people quickly scanning through a lot of messages that they aren't heavily interested in, I suspect it was easy to overlook the crucial word "encrypted" in your first message. I know I didn't notice that until your second message, which used that word a few more times. I'm not really paying attention to the latest capabilites that the installer might have, or to what any other distros are doing, but when I have attempted this in the past it appeared to me that the Debian installer does not directly support installing a fresh installation into a previously created LUKS encrypted volume. However it is certainly "possible" with some complicated tricks, and if you are prepared to risk accidentally destroying the whole encrypted volume if you make a mistake. That's what happened to me the first time I tried it. But I have adequate backups and alternative machines, so that didn't bother me. It is possible to trick the installer into opening the existing encrypted volume. Then (with numerous fiddly steps and using great caution not to make a mistake) the installer can then install into a new partition inside that, in the usual way. However the installation it creates will be broken and likely not bootable. Because we have tricked the installer beyond what it understands, it makes many mistakes. There will be problems with grub, with the cryptsetup configuration, and with the initramfs. That all then needs to be fixed by rebooting into an alternative environment that has cryptsetup tools available. Maybe the installer rescue system is capable of doing that, but I'm not sure because ... The way I currently manage my machines (which are single-user, not servers) is that for convenience I always have a minimal bootable linux rescue system including the cryptsetup tools available on the disk in a small partition outside the encrypted volume, and I use this for that purpose (I also run the installer from there). But I expect it could be done with any other alternative boot method. So, being totally unaware of your level of skill and interest, I want to strike a balance in the information that I'm giving you here ... If you have adequate backups and are interested in learning more about how your system works, this could be a good project for doing that. Your desired end goal is possible. I spent time doing it and I'm happy I did that, because now all my stuff is configured and encrypted the way I want it, and that feels good. On the other hand, my reaching that goal involved developing the skills required. I don't have time to write out a detailed recipe for you on how to do it, and I don't remember all the detailed steps without some effort which I don't have time for. I can only offer occasional hints like this message. I can't guarantee that any hints given will be correct or will not cause catastropic data loss. There's probably no shortage of people
Re: Installing bullseye into previously existing encrypted disk with buster
On Thu, Feb 10, 2022 at 04:37:55PM -0500, Dan Ritter wrote: > Nitebirdz wrote: > > I currently have a laptop running buster on an encrypted disk that boots > > via EFI. The filesystems look like this: > > > > /dev/mapper/tangier--vg-root/ > > /dev/mapper/tangier--vg-home/home > > /dev/sda1 /boot/efi > > /dev/sda2 /boot > > > > I know I can easily upgrade to bullseye from the running system. However, > > what I usually do when it's time to upgrade Debian on a laptop is to start > > from a clean slate. It's my chance to clean up and remove old cruft (well, > > with the exception of my own home partition, of course). So, instead of > > upgrading, I just install the new version of Debian. > > > > Now, my problem is that, whenever I launch the installer, it wants to > > partition the disk. Is there a way to tell the installer to leave the > > existing partitioning scheme alone? Also, I'd need the installer to leave > > the home partition alone, and format and install over the other > > partitions. Is this possible? If so, how? I've been trying different > > approaches, and I don't seem to be able to find the way to do it. > > Yes. Tell the installer you want to partition the disks > manually, and then select each one and assign it to the role > that you want. For /home, either don't assign it or make sure > that you mark it as "leave the contents alone". > Thanks. But it doesn't appear to work. The disk partitioning tool only shows the actual partitions, but no trace of the already existing encrypted volumes. See the screenshot attached. I'm testing this using QEMU. No matter what entry I select on that screen, it wants me to continue partitioning, and ends up destroying the previous setup. I cannot see a way to just get it to notice the already existing layout. That does work for more simple setups, but not for encrypted volumes, it seems. > But if you're going to do that, why not try a nice in-place > upgrade first? I bet you'll be happy with it, and if not, you've > lost maybe an hour over what you were going to do anyway. > Yes, I've done Debian upgrades before, and they are quite reliable. As mentioned in my original message, though, I take this as an opportunity to "clean up house", and remove all the cruft that I have been accumulating for the last couple of of years. That's the reason why, when a new Debian release comes out, I prefer to reinstall (well, on this particular system; I usually do standard upgrades on others). -- Nitebirdz
Re: Installing bullseye into previously existing encrypted disk with buster
Nitebirdz wrote: > I currently have a laptop running buster on an encrypted disk that boots > via EFI. The filesystems look like this: > > /dev/mapper/tangier--vg-root/ > /dev/mapper/tangier--vg-home/home > /dev/sda1 /boot/efi > /dev/sda2 /boot > > I know I can easily upgrade to bullseye from the running system. However, > what I usually do when it's time to upgrade Debian on a laptop is to start > from a clean slate. It's my chance to clean up and remove old cruft (well, > with the exception of my own home partition, of course). So, instead of > upgrading, I just install the new version of Debian. > > Now, my problem is that, whenever I launch the installer, it wants to > partition the disk. Is there a way to tell the installer to leave the > existing partitioning scheme alone? Also, I'd need the installer to leave > the home partition alone, and format and install over the other > partitions. Is this possible? If so, how? I've been trying different > approaches, and I don't seem to be able to find the way to do it. Yes. Tell the installer you want to partition the disks manually, and then select each one and assign it to the role that you want. For /home, either don't assign it or make sure that you mark it as "leave the contents alone". But if you're going to do that, why not try a nice in-place upgrade first? I bet you'll be happy with it, and if not, you've lost maybe an hour over what you were going to do anyway. -dsr-
Installing bullseye into previously existing encrypted disk with buster
Hi, I currently have a laptop running buster on an encrypted disk that boots via EFI. The filesystems look like this: /dev/mapper/tangier--vg-root/ /dev/mapper/tangier--vg-home/home /dev/sda1 /boot/efi /dev/sda2 /boot I know I can easily upgrade to bullseye from the running system. However, what I usually do when it's time to upgrade Debian on a laptop is to start from a clean slate. It's my chance to clean up and remove old cruft (well, with the exception of my own home partition, of course). So, instead of upgrading, I just install the new version of Debian. Now, my problem is that, whenever I launch the installer, it wants to partition the disk. Is there a way to tell the installer to leave the existing partitioning scheme alone? Also, I'd need the installer to leave the home partition alone, and format and install over the other partitions. Is this possible? If so, how? I've been trying different approaches, and I don't seem to be able to find the way to do it. Thanks. -- Nitebirdz
