Re: Proposal: drop "Test installation media" from live media
On Wed, Dec 16, 2020 at 2:28 PM Chris Murphy wrote: > > On Tue, Dec 15, 2020 at 12:14 PM Brian C. Lane wrote: > > > > On Mon, Dec 14, 2020 at 11:03:03PM -0700, Chris Murphy wrote: > > > Right. The two I've previously suggested: btrfs seed and dm-verity. > > > Every read is verified, the user can't opt out, and they are more > > > performant than checkisomd5. Upon detecting error, both emit EIO which > > > is handled at the application level, i.e. stop the installation and > > > notify the user. > > > > Those would require significant changes to how live works though. Simple > > is better. > > I agree simpler is better. Btrfs is a drop-in replacement for ext4, > assuming no other changes. Like I said, Will Woods had this working a > long time ago. This is not my original idea. I've tried it and it > works fine, and we get integrity checking for data and metadata for > free.[1] I'm not sure what significant change you think is required. > https://www.redhat.com/archives/anaconda-devel-list/2011-March/msg00263.html I don't know where the implied changes below would happen, or even if they'd be accepted. But I did successfully create two Btrfs based images that require no other change to bits in the image itself (same initramfs, kernel, identical files in the Fedora created ISO, etc). I tested with and without the seed feature flag enabled. Everything is assembled correctly, and works as expected. Optical, USB stick media, UEFI and BIOS. A. btrfs+squashfs All I did here was create a new btrfs rootfs, copied the ext4 rootfs from a rawhide image, then mksquashfs to nest it. Then xorriso to make an ISO. No change to boot parameters. I guess it's lorax, pungi, koji that would need a switch to make such a thing, and do mkfs.btrfs instead of mkfs.ext4. B. plain btrfs In this case I used native btrfs compression, zstd-force:11, and then skipped mksquashfs. I also added rd.live.overlay.overlayfs when booting the ISO. This is a variation on the plain squashfs option. I guess new code to skip the mksquashfs step could be significant, I'm not sure. The difficulty with this ISO image, no matter the file system, is that it's an ISO 9660 image first and foremost. And thus read-only. I'll followup on this aspect of my findings in the "Pre-made live image with actual functioning persistent storage?" thread. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/17/20 10:04 AM, Marius Schwarz wrote: Am 17.12.20 um 14:35 schrieb Stephen John Smoogen: Right, but it's not automatic, and requires an existing known-good system, which is the actual 'root of trust' here. This cannot be assumed about a flash drive, which is why the automatic image check is hard. Speaking from Security pov, it's not hard, it's impossible. The attacker can sign everything with it's own cert and putting that into the image itself. Way easier is it to remove the check for a valid sig and always return "true" is asked for a match, as any root kit will do. In a secure boot environment, the root of trust is the motherboard firmware, which has the keys of the next boot step. In Fedora land, this next step is the shim, which was signed by Microsoft because their key is on practically all existing hardware. As I said, the shim would have to be smart enough to securely (TLS) retrieve the signature, and check the boot image. https://docs.fedoraproject.org/en-US/Fedora/18/html-single/UEFI_Secure_Boot_Guide/index.html#sect-UEFI_Secure_Boot_Guide-Implementation_of_UEFI_Secure_Boot-Keys In this scenario, the attacker cannot fake the key, because both UEFI and the shim will stop the boot if their measurements fail to check out, I don't know if the UEFI/shim combo can be enhanced to do those things, though... ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
Am 17.12.20 um 14:35 schrieb Stephen John Smoogen: Right, but it's not automatic, and requires an existing known-good system, which is the actual 'root of trust' here. This cannot be assumed about a flash drive, which is why the automatic image check is hard. Speaking from Security pov, it's not hard, it's impossible. The attacker can sign everything with it's own cert and putting that into the image itself. Way easier is it to remove the check for a valid sig and always return "true" is asked for a match, as any root kit will do. best regards, Marius ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Thu, 17 Dec 2020 at 00:29, przemek klosowski via devel < devel@lists.fedoraproject.org> wrote: > > On 12/16/20 5:38 PM, Kevin Fenzi wrote: > > On Wed, Dec 16, 2020 at 04:28:49PM -0500, przemek klosowski via devel wrote: > > I was trying to make a point that we don't have a way to check the initial > image: it could be altered to falsely claim to be signed by fedoraproject. > > well, we do: https://getfedora.org/security/ > > Right, but it's not automatic, and requires an existing known-good system, > which is the actual 'root of trust' here. This cannot be assumed about a > flash drive, which is why the automatic image check is hard. > > I guess it would involve a secure boot into a fedora signed shim that > >- retrieves the checksum over a verified TLS network connection >- checks the image against that. > > This problem comes up every couple of years and people start trying to work out how to make it so it is 'trustable'. After about 8 to 12 countermeasures piled on because you found a hole in the last layer.. you end up with something which might prove it but is as fragile as a house of cards (which blows over when someone comes up with the 13th whole in the chain). The problem with a chain is it is only as strong as its weakest link.. as soon as you fix that one.. it is only as strong as its next weakest link which turns out to be as weak as the first one. -- Stephen J Smoogen. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Wed, Dec 16, 2020 at 10:29 PM przemek klosowski via devel wrote: > > > On 12/16/20 5:38 PM, Kevin Fenzi wrote: > > On Wed, Dec 16, 2020 at 04:28:49PM -0500, przemek klosowski via devel wrote: > > I was trying to make a point that we don't have a way to check the initial > image: it could be altered to falsely claim to be signed by fedoraproject. > > well, we do: https://getfedora.org/security/ > > Right, but it's not automatic, and requires an existing known-good system, > which is the actual 'root of trust' here. This cannot be assumed about a > flash drive, which is why the automatic image check is hard. > > I guess it would involve a secure boot into a fedora signed shim that > > retrieves the checksum over a verified TLS network connection > checks the image against that. Pretty sure Fedora Media Writer does this if it downloads the ISO file. You can point to an already downloaded ISO file on your computer, and in that case it can't do the check. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/16/20 5:38 PM, Kevin Fenzi wrote: On Wed, Dec 16, 2020 at 04:28:49PM -0500, przemek klosowski via devel wrote: I was trying to make a point that we don't have a way to check the initial image: it could be altered to falsely claim to be signed by fedoraproject. well, we do: https://getfedora.org/security/ Right, but it's not automatic, and requires an existing known-good system, which is the actual 'root of trust' here. This cannot be assumed about a flash drive, which is why the automatic image check is hard. I guess it would involve a secure boot into a fedora signed shim that * retrieves the checksum over a verified TLS network connection * checks the image against that. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Wed, Dec 16, 2020 at 04:28:49PM -0500, przemek klosowski via devel wrote: > > On 12/16/20 2:23 PM, Kevin Fenzi wrote: > > Yeah, there has to be an anchor for your trust. Right now that is "I > > trust the certificate authority that issued fedoraproject.org's cert". > > I was trying to make a point that we don't have a way to check the initial > image: it could be altered to falsely claim to be signed by fedoraproject. well, we do: https://getfedora.org/security/ but again you are in the end anchoring your trust on the ssl cert authoriy here. kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Tue, Dec 15, 2020 at 12:14 PM Brian C. Lane wrote: > > On Mon, Dec 14, 2020 at 11:03:03PM -0700, Chris Murphy wrote: > > Right. The two I've previously suggested: btrfs seed and dm-verity. > > Every read is verified, the user can't opt out, and they are more > > performant than checkisomd5. Upon detecting error, both emit EIO which > > is handled at the application level, i.e. stop the installation and > > notify the user. > > Those would require significant changes to how live works though. Simple > is better. I agree simpler is better. Btrfs is a drop-in replacement for ext4, assuming no other changes. Like I said, Will Woods had this working a long time ago. This is not my original idea. I've tried it and it works fine, and we get integrity checking for data and metadata for free.[1] I'm not sure what significant change you think is required. https://www.redhat.com/archives/anaconda-devel-list/2011-March/msg00263.html New functionality certainly would require significant changes. For example: Matt and Máirín Duffy et al, have been discussing a try-it-out-and-maybe-install-later workflow. This proposes making the USB stick media persistent by default. https://lists.fedoraproject.org/archives/list/desk...@lists.fedoraproject.org/thread/OJYOUDZVPGUILFEXJWOYNTABNWPWO4ZO/ This suggests two new use cases: - Persistence by default for USB stick media - An easy way to migrate the USB stick environment to a permanent installation The live ISO tells three conflicting truths about itself: ISO 9660, MBR, and GPT each tell different stories. To add persistence, we need to add a partition. Per the UEFI spec, this MBR with two partition entries means the GPT is invalid. Possibly the preferred tool is systemd-repart, which only works on GPT, but if it's strictly spec complying it won't see the GPT as valid. The alternative is to get Fedora Media Writer capable of doing surgery on these two partition maps. That sounds significant and I don't really want to go down that rabbit hole if it's avoidable. Maybe the USB stick image needs to be a new dedicated image. Drop the ISO 9660 portion, and just make it a simple spec complying GPT, and then once written to a USB stick, we can setup the additional partition with write persistence from the initramfs. Making the rootfs Btrfs has other advantages for the "try before you (don't) buy (because it's free!)" workflow. It will behave like a real installation, rather than layering two different file systems. Btrfs features are available to this user from the start, rather than getting different experiences. Enabling compression by default will improve throughput and reduce latency. Even fast reading USB sticks write slowly. Data integrity for everything, metadata and data. And the option to do either a clean install from this stick media, or easily migrate their entire persistent modifications to a permanent installation. Somewhat simplified: btrfs device add /dev/laptop / btrfs device remove /dev/stick / That implies mkfs, resize, and kicks off replication of block groups from the USB stick to the laptop drive. All apps (app removals, additions, updates), customizations, user files, subvolumes, snapshots - one command. We still need all the bootloader stuff Anaconda does, the option to clean install or replicate is just a matter of picking between two fs UUIDs or labels. And yeah all of that is significant. But that's new functionality. Not because it's Btrfs. Whether it's something better suited for Ignition than Anaconda, will need to be explored. >If squashfs has integrity checking it would be perfect :) It > looks like zstd has support for checksums but it doesn't look like > that's supported in any of the tools, or the kernel squashfs module. > > Another possibility is for lmc to add a sha256sum of the rootfs image > that can be checked by dracut when booting, or anaconda before > installing. Doesn't this turn back the clock to 2005? The point of computing md5 fragments was to fail sooner. https://www.redhat.com/archives/anaconda-devel-list/2005-March/msg00053.html [1] by default btrfs uses 4 bytes crc32c per 4KiB. md5sum on ~20 fragments, recording only 3 characters from each md5sum to create the ISO's md5sum. This surely exposes the check to collisions, but maybe someone would like to compute this. A 4 byte crc32c per 4KiB has a chance of collisions, but on a file system less than 1T, the probability is low. But if that is considered risky these days, the image can use one of the other hashes available. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/16/20 2:23 PM, Kevin Fenzi wrote: Yeah, there has to be an anchor for your trust. Right now that is "I trust the certificate authority that issued fedoraproject.org's cert". I was trying to make a point that we don't have a way to check the initial image: it could be altered to falsely claim to be signed by fedoraproject. The image check, in other words, defends against random corruption, and is helpless against targeted hacking. I thought it would be good to have a method to assure that it is indeed the same image that fedoracorp.org originally signed. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Tue, Dec 15, 2020 at 01:24:58AM -0500, przemek klosowski via devel wrote: > ...snip... > > I see the need to self-validate against known-good images/repos, either by > checking online, or by leveraging the secure boot, somehow, Unfortunately I > can't think of a foolproof and transparent way of doing it. As it is, I > always try to google the key IDs/fingerprints and make sure that they > correspond to legit package signing keys, but it's all so manual. Yeah, there has to be an anchor for your trust. Right now that is "I trust the certificate authority that issued fedoraproject.org's cert". There has been discussion about doing something with the secure boot chain of trust, which would shift it to "I trust the people who made my pc". It's not a easy problem. Just look at bug 998. ;( kevin signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 11:03:03PM -0700, Chris Murphy wrote: > Right. The two I've previously suggested: btrfs seed and dm-verity. > Every read is verified, the user can't opt out, and they are more > performant than checkisomd5. Upon detecting error, both emit EIO which > is handled at the application level, i.e. stop the installation and > notify the user. Those would require significant changes to how live works though. Simple is better. If squashfs has integrity checking it would be perfect :) It looks like zstd has support for checksums but it doesn't look like that's supported in any of the tools, or the kernel squashfs module. Another possibility is for lmc to add a sha256sum of the rootfs image that can be checked by dracut when booting, or anaconda before installing. Brian -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Tue, Dec 15, 2020 at 8:36 AM Matthew Miller wrote: > > On Mon, Dec 14, 2020 at 11:35:12PM -0700, Chris Murphy wrote: > > But also, we're not using unsquashfs for boot or installation. The > > squashfs image is loop mounted and treated as a random access file > > system. Decompression of blocks is on demand. > > Well, maybe we should? It makes a pretty fast test. :) It'll still boot as a random access device on loop. There's no place to unsquash it at this point. This change for Fedora 34 originally planned to use unsquashfs instead of rsync, but it's now optional. I'm not sure whether it will happen. https://fedoraproject.org/wiki/Changes/OptimizeSquashFSOnDVDByRemovingEXT4FilesystemImageLayer https://github.com/rhinstaller/anaconda/pull/2292 -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 11:35:12PM -0700, Chris Murphy wrote: > Squashfs doesn't have error detection for its metadata or the data > contained in it. I'm not sure why you're having such a high success > rate. Whether lossy or lossless compression algorithms in images, my > experience it is only sometimes results in an error, often we just get > artifacts. i.e. a bit flip turns into multiple wrong bytes of image > data, sometimes an entire row of pixels gets obliterated (thousands). > It just depends on what gets hit. But maybe lzma, which is what xz is > based on and what's used in squashfs images currently, could be > particularly susceptible to bit flips translating into something > detectable. It seems so. I mean, I can do more than a thousand tests if it helps, but that was enough to convince _me_. > But also, we're not using unsquashfs for boot or installation. The > squashfs image is loop mounted and treated as a random access file > system. Decompression of blocks is on demand. So I guess the next thing is: what's the error handling when the kernel hits an uncompress error when reading a compressed squashfs on the fly? It'd be kind of exciting if it logs an error we could actually watch for and pop up a message saying "so, yeah, your USB stick is bad" -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 11:35:12PM -0700, Chris Murphy wrote: > But also, we're not using unsquashfs for boot or installation. The > squashfs image is loop mounted and treated as a random access file > system. Decompression of blocks is on demand. Well, maybe we should? It makes a pretty fast test. :) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 9:20 PM Matthew Miller wrote: > > On Mon, Dec 14, 2020 at 03:32:57PM -0800, Brian C. Lane wrote: > > The problem I see with dropping it is that without it you do not know if > > there are errors in the packages you are installing. With non-live > > installs you can depend on rpm to detect that, but not with live since > > we're just copying the files over. > > With the live media, though, it's a squashfs that gets uncompressed, right? > > I just did a very rudimentary test of copying the Fedora Workstation 33 > squashfs.img and injecting a single random bit flip 1000 times, then running > unsquashfs. Of these 1000 tests, the result was a failure to unpack (with an > error code) every single time. Squashfs doesn't have error detection for its metadata or the data contained in it. I'm not sure why you're having such a high success rate. Whether lossy or lossless compression algorithms in images, my experience it is only sometimes results in an error, often we just get artifacts. i.e. a bit flip turns into multiple wrong bytes of image data, sometimes an entire row of pixels gets obliterated (thousands). It just depends on what gets hit. But maybe lzma, which is what xz is based on and what's used in squashfs images currently, could be particularly susceptible to bit flips translating into something detectable. But also, we're not using unsquashfs for boot or installation. The squashfs image is loop mounted and treated as a random access file system. Decompression of blocks is on demand. > I'm sure this isn't a cryptographically-secure verification, but it seems > like a decent enough practical one. Am I missing something? It seems like > the case of the live cd booting successfully but then installing a corrupted > system is astronomically unlikely. If there is random corruption of typical size on a USB stick, will it manifest in some way that stops the installation? I wouldn't bet on it unless it's based one a design that's intended to detect such incidental corruptions. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/11/20 1:07 PM, Matthew Miller wrote: Right now, when you start Fedora live media to install Workstation or KDE or etc., you get an ugly text prompt which defaults to doing a media test ... the most likely failure modes are like this: 1) Doesn't even write properly. 2) Doesn't boot after you created it. 3) Fails hard and it's definitely done 4) Random transient errors 5) I got this image from the internet, and who knows what is in it. It is an ongoing problem in the Windows world: searches for apps often lead to third party sites which add adware (and sometimes malware) to the installers. Of course the media test does not protect against this type of abuse---fake sites could modify the test as well as the image. Therefore, I actually agree with changing the default---but it sure would be nice if there was an option to check it, preferably more reliable than the current method. It always bugged me that in general, RPM nicely protects the system integrity by signing/verifying packages but 'qui custodiet ipsos custodes': the repo keys are implicitly accepted, both during the installation and afterwards, when additional repo package signing keys are brought in. This is especially relevant today, with the news about the Russians backdooring the supply chain of an important application (SolarWinds) that was then widely installed and exploited. I see the need to self-validate against known-good images/repos, either by checking online, or by leveraging the secure boot, somehow, Unfortunately I can't think of a foolproof and transparent way of doing it. As it is, I always try to google the key IDs/fingerprints and make sure that they correspond to legit package signing keys, but it's all so manual. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 4:33 PM Brian C. Lane wrote: > > On Sat, Dec 12, 2020 at 01:08:47PM -0700, Chris Murphy wrote: > > On Sat, Dec 12, 2020 at 10:43 AM Matthew Miller > > wrote: > > > > > > On Sat, Dec 12, 2020 at 08:19:18AM -0500, Mauricio Tavares wrote: > > > > > I gave those reasons in my initial message? Have you experienced a > > > > > specific case where bad USB media caused a corrupt install? > > > > If by corrupt install you mean the fedora install froze after > > > > USB booted and before it told the install was successful, yes. Like > > > > early this year (so I guess it is too far in the past to be relevant > > > > to this discussion). Interestingly enough I then dd'ed ubuntu -- just > > > > needed to test hardware so any distro with the latest kernel would do > > > > -- in it because I thought it was a hardware incompatibility issue and > > > > it worked well enough to finish my tests. > > > > > > To me, the install freezing is not a lot different from the media check > > > freezing. We can tell people: if that happens, it's probably a bad USB > > > stick. The case that would be really concerning to me if bad media caused > > > an > > > install to appear to work but then not boot. > > > > That could happen, but seems unlikely because anything corrupt that > > affects the as-installed boot, would also affect the live boot. But > > does it result in a crash or does it just result in weird behavior > > that looks like a bug? > > > > I don't know what percent of the bytes of the ISO image is read when > > booted. 25%? That leaves a chunk of programs and other content in an > > installation that could have this corruption, and transferred > > undetected to the target system. And how does it manifest? It's a rare > > upon rare event, and I expect it will be assumed to be a bug, and > > maybe we luck out and it goes away following a software update. > > > > I've suggested dropping the media check a few times over the years, > > but always in the context of replacing it with something better, > > faster, stronger. :P While the risk of dropping the media check is > > probably low, it's not zero risk. > > The problem I see with dropping it is that without it you do not know if > there are errors in the packages you are installing. With non-live > installs you can depend on rpm to detect that, but not with live since > we're just copying the files over. > > Anaconda may be able to add a check to the install process. If you run > rpm -Va on the newly installed system everything should match. But it > doesn't. Ends up there are a number of files with mode differences, and > even some with MD5 mismatches. But maybe anaconda touched them and they > can be filtered from the results? > > Without some other way to verify the installation I'm against dropping > the check from live. Not having it will result in some errors slipping > through and someone, either end users or developers, pulling out their > hair trying to debug it. Right. The two I've previously suggested: btrfs seed and dm-verity. Every read is verified, the user can't opt out, and they are more performant than checkisomd5. Upon detecting error, both emit EIO which is handled at the application level, i.e. stop the installation and notify the user. dm-verity has the advantage of optionally incorporating Reed-Solomon error correction, so that in the typical case the error is fixed seamlessly. Btrfs has the advantage of being simpler to implement. I think Will Woods had it hacked up in the Anaconda ecosystem many moons ago. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Mon, Dec 14, 2020 at 03:32:57PM -0800, Brian C. Lane wrote: > The problem I see with dropping it is that without it you do not know if > there are errors in the packages you are installing. With non-live > installs you can depend on rpm to detect that, but not with live since > we're just copying the files over. With the live media, though, it's a squashfs that gets uncompressed, right? I just did a very rudimentary test of copying the Fedora Workstation 33 squashfs.img and injecting a single random bit flip 1000 times, then running unsquashfs. Of these 1000 tests, the result was a failure to unpack (with an error code) every single time. I'm sure this isn't a cryptographically-secure verification, but it seems like a decent enough practical one. Am I missing something? It seems like the case of the live cd booting successfully but then installing a corrupted system is astronomically unlikely. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 01:08:47PM -0700, Chris Murphy wrote: > On Sat, Dec 12, 2020 at 10:43 AM Matthew Miller > wrote: > > > > On Sat, Dec 12, 2020 at 08:19:18AM -0500, Mauricio Tavares wrote: > > > > I gave those reasons in my initial message? Have you experienced a > > > > specific case where bad USB media caused a corrupt install? > > > If by corrupt install you mean the fedora install froze after > > > USB booted and before it told the install was successful, yes. Like > > > early this year (so I guess it is too far in the past to be relevant > > > to this discussion). Interestingly enough I then dd'ed ubuntu -- just > > > needed to test hardware so any distro with the latest kernel would do > > > -- in it because I thought it was a hardware incompatibility issue and > > > it worked well enough to finish my tests. > > > > To me, the install freezing is not a lot different from the media check > > freezing. We can tell people: if that happens, it's probably a bad USB > > stick. The case that would be really concerning to me if bad media caused an > > install to appear to work but then not boot. > > That could happen, but seems unlikely because anything corrupt that > affects the as-installed boot, would also affect the live boot. But > does it result in a crash or does it just result in weird behavior > that looks like a bug? > > I don't know what percent of the bytes of the ISO image is read when > booted. 25%? That leaves a chunk of programs and other content in an > installation that could have this corruption, and transferred > undetected to the target system. And how does it manifest? It's a rare > upon rare event, and I expect it will be assumed to be a bug, and > maybe we luck out and it goes away following a software update. > > I've suggested dropping the media check a few times over the years, > but always in the context of replacing it with something better, > faster, stronger. :P While the risk of dropping the media check is > probably low, it's not zero risk. The problem I see with dropping it is that without it you do not know if there are errors in the packages you are installing. With non-live installs you can depend on rpm to detect that, but not with live since we're just copying the files over. Anaconda may be able to add a check to the install process. If you run rpm -Va on the newly installed system everything should match. But it doesn't. Ends up there are a number of files with mode differences, and even some with MD5 mismatches. But maybe anaconda touched them and they can be filtered from the results? Without some other way to verify the installation I'm against dropping the check from live. Not having it will result in some errors slipping through and someone, either end users or developers, pulling out their hair trying to debug it. Brian -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 06:45:26PM -0500, Matthew Miller wrote: > On Fri, Dec 11, 2020 at 06:21:50PM -0500, Neal Gompa wrote: > > > In the interest of me being lazy more successfully, can I glom my "change > > > the menu" proposal onto this new Change you are spearheading? :) :) :) > > Oi, oi! I have enough changes for this cycle. If you want to spearhead > > this, I can help, though. :) > > I just saw that the strings are all hard-coded in livecd-creator, and while > I know how to Python enough to do something about it, it'd be even more > awesome if someone in the top, say, 5 contributors > https://github.com/livecd-tools/livecd-tools/graphs/contributors would help, > yes. :) livemedia-creator makes the live images. Config files are here: https://github.com/weldr/lorax/tree/master/share/templates.d/99-generic/live/config_files I'm not opposed to switching to grub for everything, but we need to make sure it fits all the unknown use cases :) Brian -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
Am 13.12.20 um 21:08 schrieb Chris Murphy: This happens on macOS for the HFS+ volume on the stick and GNOME Shell mounts the large FAT partition. Poof, the media checker now fails. (using cinnamon) Which renders the media test useless. The best argument to remove it entirely. The thing with the Surface is, that it doesn't get normal firmware updates, since Linux is running on it, never again ;) I don't know it it will boot when switching to f33. Best regards, Marius ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sun, Dec 13, 2020 at 1:08 PM Chris Murphy wrote: > > Something is writing to the media before the media checker. before the media checker ^runs the check. The media checker doesn't write to the media... -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sun, Dec 13, 2020 at 10:57 AM Marius Schwarz wrote: > > Am 12.12.20 um 19:47 schrieb Chris Murphy: > > > >> Ack. Over the last year, none of the media tests worked proper, but any > >> usb stick and/or CD, it failed on, worked. > > Could you elaborate on this? I've not heard of this coming at all in > > QA. There is a testcase for this. It's certainly expected to work. > > > > ATM i only have one stick laying around.. let's find out if an actual > image gets passed: > > - F32 1.6 Diskimage in Boxen : works > - F33 Beta 1.3 in Boxen fails at 85.7% > - F33 Beta 1.3 in a Surface Pro 4 fails at 4.8% (boots correctly and > works as tested ) > - F33 1.2 in Boxen works > - F33 1.2 in Surface Pro 4 works, but doesn't boot on it self at anytime* > Something is writing to the media before the media checker. If any of the file systems embedded in the install media mount read-write, the file system is immediately modified. And now the entire stick will fail media checks. This can happen just by plugging the stick in and an automounter thinking it needs to mount the stick. This happens on macOS for the HFS+ volume on the stick and GNOME Shell mounts the large FAT partition. Poof, the media checker now fails. > *) This was reported by me to the list, got discussed and died silently > as a firmware bug. > > The F33 images boots only, when the stick gets inserted, when the device > is in bios. It the stick is inside bevor the start , i.e. on a reboot > into bios or a cold start, it does not boot at all. Device resets back > into bios. F31 boots as it should be. That does sound like a firmware bug. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
Am 12.12.20 um 19:47 schrieb Chris Murphy: Ack. Over the last year, none of the media tests worked proper, but any usb stick and/or CD, it failed on, worked. Could you elaborate on this? I've not heard of this coming at all in QA. There is a testcase for this. It's certainly expected to work. ATM i only have one stick laying around.. let's find out if an actual image gets passed: - F32 1.6 Diskimage in Boxen : works - F33 Beta 1.3 in Boxen fails at 85.7% - F33 Beta 1.3 in a Surface Pro 4 fails at 4.8% (boots correctly and works as tested ) - F33 1.2 in Boxen works - F33 1.2 in Surface Pro 4 works, but doesn't boot on it self at anytime* *) This was reported by me to the list, got discussed and died silently as a firmware bug. The F33 images boots only, when the stick gets inserted, when the device is in bios. It the stick is inside bevor the start , i.e. on a reboot into bios or a cold start, it does not boot at all. Device resets back into bios. F31 boots as it should be. best regards, Marius ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 10:43 AM Matthew Miller wrote: > > On Sat, Dec 12, 2020 at 08:19:18AM -0500, Mauricio Tavares wrote: > > > I gave those reasons in my initial message? Have you experienced a > > > specific case where bad USB media caused a corrupt install? > > If by corrupt install you mean the fedora install froze after > > USB booted and before it told the install was successful, yes. Like > > early this year (so I guess it is too far in the past to be relevant > > to this discussion). Interestingly enough I then dd'ed ubuntu -- just > > needed to test hardware so any distro with the latest kernel would do > > -- in it because I thought it was a hardware incompatibility issue and > > it worked well enough to finish my tests. > > To me, the install freezing is not a lot different from the media check > freezing. We can tell people: if that happens, it's probably a bad USB > stick. The case that would be really concerning to me if bad media caused an > install to appear to work but then not boot. That could happen, but seems unlikely because anything corrupt that affects the as-installed boot, would also affect the live boot. But does it result in a crash or does it just result in weird behavior that looks like a bug? I don't know what percent of the bytes of the ISO image is read when booted. 25%? That leaves a chunk of programs and other content in an installation that could have this corruption, and transferred undetected to the target system. And how does it manifest? It's a rare upon rare event, and I expect it will be assumed to be a bug, and maybe we luck out and it goes away following a software update. I've suggested dropping the media check a few times over the years, but always in the context of replacing it with something better, faster, stronger. :P While the risk of dropping the media check is probably low, it's not zero risk. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 4:59 AM Marius Schwarz wrote: > > Am 11.12.20 um 19:07 schrieb Matthew Miller: > > > > The test media option doesn't actually help with any of these except maybe > > making #3 happen slightly sooner. With #4, it actually means that in some > > cases you'd be fine just doing the install and the test fails. > > > > Let's just go ahead and get people started faster. > > > > Ack. Over the last year, none of the media tests worked proper, but any > usb stick and/or CD, it failed on, worked. Could you elaborate on this? I've not heard of this coming at all in QA. There is a testcase for this. It's certainly expected to work. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 08:19:18AM -0500, Mauricio Tavares wrote: > > I gave those reasons in my initial message? Have you experienced a > > specific case where bad USB media caused a corrupt install? > If by corrupt install you mean the fedora install froze after > USB booted and before it told the install was successful, yes. Like > early this year (so I guess it is too far in the past to be relevant > to this discussion). Interestingly enough I then dd'ed ubuntu -- just > needed to test hardware so any distro with the latest kernel would do > -- in it because I thought it was a hardware incompatibility issue and > it worked well enough to finish my tests. To me, the install freezing is not a lot different from the media check freezing. We can tell people: if that happens, it's probably a bad USB stick. The case that would be really concerning to me if bad media caused an install to appear to work but then not boot. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 5:20 PM Matthew Miller wrote: > > On Sat, Dec 12, 2020 at 12:15:07AM +0200, Nikolay Nikolov wrote: > > there's even less reason to skip it. Which really begs the question, > > why do we even assume the media test is only useful for DVD and not > > for USB flash? > > I gave those reasons in my initial message? Have you experienced a > specific case where bad USB media caused a corrupt install? > If by corrupt install you mean the fedora install froze after USB booted and before it told the install was successful, yes. Like early this year (so I guess it is too far in the past to be relevant to this discussion). Interestingly enough I then dd'ed ubuntu -- just needed to test hardware so any distro with the latest kernel would do -- in it because I thought it was a hardware incompatibility issue and it worked well enough to finish my tests. If you mean install was deemed successful but then it would not work, no. > > -- > Matthew Miller > > Fedora Project Leader > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
Am 11.12.20 um 19:07 schrieb Matthew Miller: The test media option doesn't actually help with any of these except maybe making #3 happen slightly sooner. With #4, it actually means that in some cases you'd be fine just doing the install and the test fails. Let's just go ahead and get people started faster. Ack. Over the last year, none of the media tests worked proper, but any usb stick and/or CD, it failed on, worked. But, I'm pretty sure, you can't speed things up, as you need to be able to boot in Basic video mode. So you need a switch and the testmedia option in in this exact switch menu :) best regards, Marius Schwarz ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 11:08 AM Matthew Miller wrote: > > Right now, when you start Fedora live media to install Workstation or KDE or > etc., you get an ugly text prompt which defaults to doing a media test > (although it's not actually even clear from the highlighting that that's the > default). > > I think since burning spinning optical media is no longer the normal way to > do this, we should drop this and just go straight to booting (unless of > course a key is hit to stop things and enter boot parameters). > > In my experience with USB sticks (and i've probably made over 500 of 'em in > the last few years), the most likely failure modes are like this: > > 1) Doesn't even write properly. > 2) Doesn't boot after you created it. > 3) Fails hard and it's definitely done > 4) Random transient errors It varies quite a bit. Transient or persistent corruption seem fairly common with USB sticks. In particular before failure. I currently have a manufacturer supplied warranty replacement that's returning persistent corruption for a couple of sectors (so far). There are no errors returned by the device at all, it's supremely confident it's handing me back my data. But Btrfs is detecting this corruption. All of the failures I've experienced, the drive goes read-only. Without error. Write a bunch of stuff, no error, read it back, no error but it's the old data. > The test media option doesn't actually help with any of these except maybe > making #3 happen slightly sooner. With #4, it actually means that in some > cases you'd be fine just doing the install and the test fails. I have no data on how often problems are detected. I suspect it does sometimes happen, because I've had it catch corruption. > Let's just go ahead and get people started faster. If so I think it should be the most minimal change, make the non-test menu entry the default. Leave the layout the same. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 06:21:50PM -0500, Neal Gompa wrote: > > In the interest of me being lazy more successfully, can I glom my "change > > the menu" proposal onto this new Change you are spearheading? :) :) :) > Oi, oi! I have enough changes for this cycle. If you want to spearhead > this, I can help, though. :) I just saw that the strings are all hard-coded in livecd-creator, and while I know how to Python enough to do something about it, it'd be even more awesome if someone in the top, say, 5 contributors https://github.com/livecd-tools/livecd-tools/graphs/contributors would help, yes. :) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 3:09 PM Matthew Miller wrote: > > On Fri, Dec 11, 2020 at 02:22:28PM -0500, Neal Gompa wrote: > > > I'm not horribly opposed. I just don't want scope creep to mean we can't > > > make a pretty easy change. > > Well, we have to change it in both places anyway. :) > > Dropping isolinux just means we have one less menu to maintain. > > In the interest of me being lazy more successfully, can I glom my "change > the menu" proposal onto this new Change you are spearheading? :) :) :) > Oi, oi! I have enough changes for this cycle. If you want to spearhead this, I can help, though. :) -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 12:40:15AM +0200, Nikolay Nikolov wrote: > So, how can I experience a corrupt install, due to media failure, > since I specifically run the media check to ensure this doesn't > happen, before I attempt an install? Just because I haven't > experienced it, since I always run the test, doesn't mean it won't > happen when people skip the media test. What I'm saying is: yes, people might experience errors, but in the case they do, it's almost always very obvious. So we're adding extra steps and confusion for most people who won't experience errors, and not adding much value to people who do. But I'm happy with the proposal to move the check to troubleshooting. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/12/20 12:20 AM, Matthew Miller wrote: On Sat, Dec 12, 2020 at 12:15:07AM +0200, Nikolay Nikolov wrote: there's even less reason to skip it. Which really begs the question, why do we even assume the media test is only useful for DVD and not for USB flash? I gave those reasons in my initial message? Have you experienced a specific case where bad USB media caused a corrupt install? No, but I have the habit to always run the media check, before attempting install, so it's unlikely that I've encountered an error like that, since I don't attempt to install from media that fails the test. I only skip the media test in two cases: a) installing in a VM b) installing from a media I've already verified (e.g. when installing on several computers from the same USB flash or DVD, I only verify it the first time for USB, and for DVD, I use the "verify" option in the DVD burning software, so I skip the option even during the first install) So, how can I experience a corrupt install, due to media failure, since I specifically run the media check to ensure this doesn't happen, before I attempt an install? Just because I haven't experienced it, since I always run the test, doesn't mean it won't happen when people skip the media test. Nikolay ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Sat, Dec 12, 2020 at 12:15:07AM +0200, Nikolay Nikolov wrote: > there's even less reason to skip it. Which really begs the question, > why do we even assume the media test is only useful for DVD and not > for USB flash? I gave those reasons in my initial message? Have you experienced a specific case where bad USB media caused a corrupt install? -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 12/11/20 8:55 PM, Vitaly Zaitsev via devel wrote: On 11.12.2020 19:07, Matthew Miller wrote: I think since burning spinning optical media is no longer the normal way to do this, we should drop this and just go straight to booting (unless of course a key is hit to stop things and enter boot parameters). USB flash drive can be broken too. I suggest moving it to the Troubleshooting sub-menu instead of completely removing it. I agree. I like to run the test even from USB flash. In fact, strange as it may sound, I've found flash drives to be less reliable in that tools that people use to dump the image to a flash drive don't always do the right thing. DVD burner software always writes the image correctly to a DVD-R or DVD+R disc and usually has an option to verify the burn, which I always use. USB writers often contain bugs or don't verify the written data, so I don't trust any install that haven't booted and run the media self test, regardless of whether it's USB flash or DVD (it's mostly USB flash these days, but I still use DVD for certain systems, where I have trouble booting from USB, due to BIOS bugs). And if I skip the test, it's usually for optical media, not for USB, where I trust the USB writer tools less. And if an install fails, due to failure in the optical media, I usually get useful feedback from the drive (characteristic sound), which makes it clear that the failure is due to unreliable disc or drive, while on USB you usually get nothing, except broken data. Also, the media test is actually faster on USB flash, so there's even less reason to skip it. Which really begs the question, why do we even assume the media test is only useful for DVD and not for USB flash? Nikolay ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 02:22:28PM -0500, Neal Gompa wrote: > > I'm not horribly opposed. I just don't want scope creep to mean we can't > > make a pretty easy change. > Well, we have to change it in both places anyway. :) > Dropping isolinux just means we have one less menu to maintain. In the interest of me being lazy more successfully, can I glom my "change the menu" proposal onto this new Change you are spearheading? :) :) :) -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 11.12.2020 20:18, Neal Gompa wrote: This would probably be easier if we dropped isolinux and used grub2 for BIOS ISO boot just like we do for UEFI ISO boot. +1 for this. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 2:21 PM Matthew Miller wrote: > > On Fri, Dec 11, 2020 at 02:18:38PM -0500, Neal Gompa wrote: > > This would probably be easier if we dropped isolinux and used grub2 > > for BIOS ISO boot just like we do for UEFI ISO boot. > > I'm not horribly opposed. I just don't want scope creep to mean we can't > make a pretty easy change. > Well, we have to change it in both places anyway. :) Dropping isolinux just means we have one less menu to maintain. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 02:18:38PM -0500, Neal Gompa wrote: > This would probably be easier if we dropped isolinux and used grub2 > for BIOS ISO boot just like we do for UEFI ISO boot. I'm not horribly opposed. I just don't want scope creep to mean we can't make a pretty easy change. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 2:06 PM Matthew Miller wrote: > > On Fri, Dec 11, 2020 at 10:37:01AM -0800, Adam Williamson wrote: > > > Let's just go ahead and get people started faster. > > > > Could we maybe just bump it down to the 'troubleshooting' menu or > > whatever it's labelled there and have it not be the default, rather > > than remove it entirely? > > Sure, that seems okay too. In doing that, I suggest we change from: > > _S_tart Fedora-Worksation-Live 33 > Test this _m_edia & start Fedora-Workstation-Live 33 > > Troubleshooting > > to > > _1_. Start Fedora Workstation Live 33 > _2_. Troubleshooting options... > > ...where previously S and m are a slightly different color, which is Secret > DOS Application UI Code for "if you wanted, you could press this letter > instead of using the arrow keys" -- let's use numbers instead, which both > make it more obvious that this is menu of options and are logical easy > things to press. > > The troubleshooting menu currently has help text which appears for some of > the options. We could add some friendly text like: > > "Press Enter to start a live environment which you can experiment with > without making any permanent changes, or use to easily install Fedora > Workstation to your hard drive." > > I'd also like to reduce the timeout from 60 seconds to 15 or something, but > I won't fight anyone on that. > This would probably be easier if we dropped isolinux and used grub2 for BIOS ISO boot just like we do for UEFI ISO boot. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 10:37:01AM -0800, Adam Williamson wrote: > > Let's just go ahead and get people started faster. > > Could we maybe just bump it down to the 'troubleshooting' menu or > whatever it's labelled there and have it not be the default, rather > than remove it entirely? Sure, that seems okay too. In doing that, I suggest we change from: _S_tart Fedora-Worksation-Live 33 Test this _m_edia & start Fedora-Workstation-Live 33 Troubleshooting to _1_. Start Fedora Workstation Live 33 _2_. Troubleshooting options... ...where previously S and m are a slightly different color, which is Secret DOS Application UI Code for "if you wanted, you could press this letter instead of using the arrow keys" -- let's use numbers instead, which both make it more obvious that this is menu of options and are logical easy things to press. The troubleshooting menu currently has help text which appears for some of the options. We could add some friendly text like: "Press Enter to start a live environment which you can experiment with without making any permanent changes, or use to easily install Fedora Workstation to your hard drive." I'd also like to reduce the timeout from 60 seconds to 15 or something, but I won't fight anyone on that. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On 11.12.2020 19:07, Matthew Miller wrote: I think since burning spinning optical media is no longer the normal way to do this, we should drop this and just go straight to booting (unless of course a key is hit to stop things and enter boot parameters). USB flash drive can be broken too. I suggest moving it to the Troubleshooting sub-menu instead of completely removing it. -- Sincerely, Vitaly Zaitsev (vit...@easycoding.org) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, 2020-12-11 at 13:07 -0500, Matthew Miller wrote: > Right now, when you start Fedora live media to install Workstation or KDE or > etc., you get an ugly text prompt which defaults to doing a media test > (although it's not actually even clear from the highlighting that that's the > default). > > I think since burning spinning optical media is no longer the normal way to > do this, we should drop this and just go straight to booting (unless of > course a key is hit to stop things and enter boot parameters). > > In my experience with USB sticks (and i've probably made over 500 of 'em in > the last few years), the most likely failure modes are like this: > > 1) Doesn't even write properly. > 2) Doesn't boot after you created it. > 3) Fails hard and it's definitely done > 4) Random transient errors > > The test media option doesn't actually help with any of these except maybe > making #3 happen slightly sooner. With #4, it actually means that in some > cases you'd be fine just doing the install and the test fails. > > Let's just go ahead and get people started faster. Could we maybe just bump it down to the 'troubleshooting' menu or whatever it's labelled there and have it not be the default, rather than remove it entirely? -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 01:12:05PM -0500, Mauricio Tavares wrote: > I thought you could either bypass or cancel the installation media test. You can, but: 1) it's the default 2) skipping it isn't clearly obvious, especially if you've never used a pre-VGA DOS appliation (*) 3) just having the text mode thing show up signals complexity * I mean, seriously! -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Proposal: drop "Test installation media" from live media
On Fri, Dec 11, 2020 at 1:08 PM Matthew Miller wrote: > > Right now, when you start Fedora live media to install Workstation or KDE or > etc., you get an ugly text prompt which defaults to doing a media test > (although it's not actually even clear from the highlighting that that's the > default). > > I think since burning spinning optical media is no longer the normal way to > do this, we should drop this and just go straight to booting (unless of > course a key is hit to stop things and enter boot parameters). > > In my experience with USB sticks (and i've probably made over 500 of 'em in > the last few years), the most likely failure modes are like this: > > 1) Doesn't even write properly. > 2) Doesn't boot after you created it. > 3) Fails hard and it's definitely done > 4) Random transient errors > > The test media option doesn't actually help with any of these except maybe > making #3 happen slightly sooner. With #4, it actually means that in some > cases you'd be fine just doing the install and the test fails. > > Let's just go ahead and get people started faster. > I thought you could either bypass or cancel the installation media test. > -- > Matthew Miller > > Fedora Project Leader > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Proposal: drop "Test installation media" from live media
Right now, when you start Fedora live media to install Workstation or KDE or etc., you get an ugly text prompt which defaults to doing a media test (although it's not actually even clear from the highlighting that that's the default). I think since burning spinning optical media is no longer the normal way to do this, we should drop this and just go straight to booting (unless of course a key is hit to stop things and enter boot parameters). In my experience with USB sticks (and i've probably made over 500 of 'em in the last few years), the most likely failure modes are like this: 1) Doesn't even write properly. 2) Doesn't boot after you created it. 3) Fails hard and it's definitely done 4) Random transient errors The test media option doesn't actually help with any of these except maybe making #3 happen slightly sooner. With #4, it actually means that in some cases you'd be fine just doing the install and the test fails. Let's just go ahead and get people started faster. -- Matthew Miller Fedora Project Leader ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
