So, it seems we potentially have several different issues, and should discuss them in separate bug reports:
* How to make the documentation clearer about --profiles vs. --auto-profiles * For some reason --locale and --keyboard didn't work for you. * What references should there be between the simple-cdd README and the debian-handbook? * More references in the simple-cdd README to relevent chapters in the debian-installer. * others? I've touched on them each below, but it would be best to break them up into separate issues soon so they can be tracked individually. On 2020-04-29, Buck wrote: >> --profiles selects what profiles are included on the generated images, >> essentially copying the files from ./profiles/ and/or >> /usr/share/simple-cdd/profiles/ files onto the installer media for the >> relevent profile, and determines which packages are available on the >> install media. >> >> --auto-profiles effectively preseeds which profiles to install when you >> boot the installer, if you do not use --auto-profiles, it will ask you >> which profiles you want to install when you boot the installer media >> (e.g. USB, DVD, CD, etc.). > > Interesting. I do not understand what it means "to install when you > boot the installer" vs "include on the generated images" here. > They both sound like exactly the same thing: profiles that exist on a > built installer image. Are you distinguishing between the image, and > the installer? What is the distinction? The image is an installer > image, no? Because from your explanation we have: > Using --profiles: preseeds which profiles to install when you boot the > installer NOT using --auto-profiles: does NOT preseed which profiles > to install when you boot the installer Using --profiles does not select which profiles to install, it selects which provides are *available* to be installed. Using --auto-profiles preseeds which profiles to install. > But there's an omission by contrapositive here. The questions are > what *does* each one do, and how do those things differ? Because, > notice that NOT using a giraffe also does NOT preseed which profiles > to install. Well... --profiles merely determines which simple-cdd profiles are available and present on the generated installer media (CD, DVD, USB stick, etc). While --auto-profiles answers (a.k.a. preseeds) the question that simple-cdd would otherwise ask at run-time (e.g. when you boot the generated installer media). If you use --profiles without --auto-profiles, it should ask you which profiles you want installed while you are running the installer. > I am not trying to be rude, just to point out the source > of the confusion. I think you know enough to infer something that I > do not, and I am asking you to state that inference explicitly so I > understand these two similar flags. They are maybe similar in name, but not in function, but my ability to explain further is exhausted. Hopefully the above finally makes sense to you. If you need more detail, how they work is in the code itself. >> As explained in the simple-cdd README > > You keep coming back to this. Should this be taken to mean that the > README in /usr/share/simple-cdd/README is the authoritative text on > simple-cdd? Yes. >> Presumably, you've encoutered a warning from simple-cdd, as it cannot >> verify that it's a valid preseeding file (the password configuration is >> tested with a different database that requires root access, and >> simple-cdd should not be run as root). > > No, I received no simple-cdd warnings when running `$ build-simple-cdd > --profiles custom` where "custom" is the file you are commenting on. No > warnings at all. Maybe that issue is fixed, then. Or maybe something is broken. I'll have to check myself at this point. >>> # Individual additional packages to install >>> d-i pkgsel/include string build-essential git w3m secure-delete rsync nmap >>> screen tmux sudo pwgen net-tools dnstools > >> You need these packages to be available in one of your profiles, or it >> will require network access... maybe that's ok for you... > > It is, but please explain what you mean by "available in one of your > profiles" unless you mean this section of the README: > > ``` > Create a profile named NAME: > > $ mkdir profiles > $ for p in list-of-packages-you-want-installed ; do echo $p >> > profiles/NAME.packages ; done > ``` Yes, exactly that. > In that case, I understand what you're trying to say, although it took > work to bridge the gap. It sounds, then, that NAME.packages requires > the network when the image is created, so it can have the packages > ready for the installer. Yes, that's largely the point of simple-cdd; It generates an installer media with the packages you want available on the installer media, with debconf preseeding for the installation process itself, as well as preseeding for the packages that you might install. It's intended that you could complete the install from the media without network access, with a reduced set of questions asked during the install process (including zero questions asked for a fully automated install). > But if you include the packages in the NAME.preseed file, then the > installer itself will have to have internet access to download the > packages. Is that correct? It's not clear from the README or > https://www.debian.org/doc/manuals/debian-handbook/sect.automated-installation.en.html > but it makes sense if this is correct. I just now realized that the debian-handbook != the debian-installer manual. I had no idea the debian-handbook actually covers simple-cdd at all(or if I did, had long since forgotten). It looks like it's just repeating what's in the simple-cdd README, more-or-less. Surprise to me! >>> --custom.udebs-- >>> # the udeb needed for simple-cdd >>> simple-cdd-profiles >> >>This is redundant with the default profile, but shouldn't hurt anything. > > Okay, thank you.. The README has no information about what the udeb > is. It is *mentioned* twice, but not explained. It's the debian-installer package (a.k.a. .udeb) that has all the simple-cdd specific code that hooks into debian-installer. >>> My primary source was >>> https://www.debian.org/doc/manuals/debian-handbook/sect.automated-installation.en.html >> >>Ok, that's the underpinnings of how debian-installer works. While the >>documentation is largely relevent for simple-cdd, as simple-cdd >>extensively uses the hooks provided in debian-installer, you're not >>going to find documentation specific to simple-cdd there. > > Okay, that's good to know. Do you know who I may contact to add to > that webpage a link to the proper/authoritative simple-cdd doc? I believe it is the same as the "debian-handbook" package in Debian, which lists the maintainer: https://tracker.debian.org/pkg/debian-handbook You can also find information using: $ apt show debian-handbook > Or a reference if you answer above is that the best reference is > /usr/share/simple-cdd/README Documentation for simple-cdd should be in the simple-cdd README. The wiki page should mirror it. Briefly touching on it in the debian-handbook makes sense and could be updated, sure. >> Apparently, you missed the --locale configuration option mentioned in >> the simple-cdd README: > > No but I was working on other docs that I assumed were authoritative, > at least regarding the NAME.preseed file. Neither is wrong, per se, both are correct, but there are caveats that are apparently difficult to explain... https://www.debian.org/releases/stable/amd64/apbs01.en.html "For initrd preseeding this is right at the start of the installation, before the first question is even asked. Preseeding from the kernel command line happens just after. It is thus possible to override configuration set in the initrd by editing the kernel command line (either in the bootloader configuration or manually at boot time for bootloaders that allow it). For file preseeding this is after the CD or CD image has been loaded. For network preseeding it is only after the network has been configured." Simple-cdd uses file preseeding. The --locale and --keyboard options instruct simple-cdd to generate kernel command line arguments by adjusting the bootloader configuration, so that the preseeding happens very early on. >> One of the challenges with debian-installer documentation is it is >> extremely flexible, and involves a lot of moving parts that may change >> at any time, which makes it very difficult to produce comprehensive >> documentation. > > Understood but (a) the alternative is either i. conversations like > these or ii. people who would benefit from the software not being able > to use it and (b) yeah, it's hard. That's life. I go through > developers like toothpicks because I always insist on good docs. > Anyway in this case, the answer might be simple. Just refer the user > to /usr/share/simple-cdd/README *if* that is being maintained with the > most accurate information. The README is the best documentation for simple-cdd that I'm aware of. The --help output also has information, some of which should maybe move into the README. Haven't checked the man page, but could also use review. > 20+ years of experience with Debian tells me that Debian README files > are rarely a source of useful, recent information. In 16 years of maintaining simple-cdd, little has needed changing in the documentation. It largely works the same as the first versions. How it works "under the hood" has changed, sure. Of course, documentation can always be better... >>> 1. This should almost definitely go into >>> https://www.debian.org/doc/manuals/debian-handbook/sect.automated-installation.en.html >>> section 12.3.3.1. >> >> That is debian-installer documentation. If anything specific to >> simple-cdd needs documentation, it belongs in the simple-cdd >> documentation. For avoidance of doubt, simple-cdd profiles are specific >> to simple-cdd. Oops, yeah, debian-installer manual != debian-handbook. I still think the handbook should only briefly document simple-cdd. > To beat the dead horse, I reiterate two points: > > 1 - Precisely what is "the simple-cdd documentation" referring to > here? The /usr/share/simple-cdd/README file? Yes. What a poor horse. > 2 - Whatever the answer is to #1, that answer to your question (ie - the > reference or link) is what should be at > https://www.debian.org/doc/manuals/debian-handbook/sect.automated-installation.en.html > section 12.3.3.1. > > This is off-topic, but if it prevents similar confusion I hope it is worth > repeating. Seems to be something to ask of the debian-handbook maintainers. >> If you want a fully automated install for SOMEPROFILE, you'll have to >> specify *both* --profiles SOMEPROFILE and --auto-profiles SOMEPROFILE. > > Okay I will. When I understand --profiles vs --auto-profiles (question > above), I may also know why. Let's hope. >>>> [non-interactive installer] should be possible, but often time all the >>>> exact questions asked may >>>> be hard to track down. >>> >>> Can you point me to a doc that has a complete list? >> >>A *mostly* complete list could be found by downloading the approximately >>50000+ packages in Debian and extracting the questions from the >>/var/lib/dpkg/info/*.templates files. Hopefully that illustrates the >>scope of what you're asking for? > > Yes, thank you, but it does not answer the question. There are > well-known solutions to documentation in situations like these. Among > the most common would be: > - if the format of the options is programmatic, then the formula(e) > used to create an option, so users could derive the option they want The format of the questions is programmatic, and there is some structure to the answers (booleans, lists, strings, etc.), but it's largely up to individual packages exactly what those answers actually mean, how they're implemented, etc. > - if the format is not progammatic but custom to each package, then > there really should be a standard location in each of the 50000+ > packages where the non-predictable option is listed, and then the > simple-cdd docs would say: "look in `docs/installer-option` of the > package you want to automate, to find the correct syntax." (And then > a package that does not have that file would receive the bug report, > not simple-cdd.) The maintainer scripts for each individual package and their debconf templates are where to look. > There may be more I do not understand this well enough yet to make > more specific suggestions. But "it's too difficult to document" is > never a solution. There are limits as to what's appropriate to document for a particular project. I wouldn't expect the "hello" program to document the fundamentals of computer science, or the working of electricity, or atomic theory, or the origins of the universe... even though all of those are, ostensibly, all relevent. >> Try an install, note what questions are still asked, and from the >> resulting installed system that asked too many questions, run >> "debconf-get-selections" and "debconf-get-selections --installer" and >> look for the questions you want to preseed, add them to your preseed >> file, try again, etc. > > Great idea! Are you aware that the README does not mention > "debconf-get-selections" once? So it would take someone who does this > every day to think of this excellent solution. Thank you for sharing > this. I recommend adding it to the README, maybe the HowTo. This could perhaps be briefly touched on in the simple-cdd README, and referencing relevent chapters of the debian-installer manual. >>> I would phrase it like this: >>> >>> "When reading profiles, simple-cdd gives precedence to profiles >>> defined in ./profiles/ and then in /usr/share/simple-cdd/profiles." >> >>Thanks for the suggestion. Where *exactly* do you think this should be >>specified? > > That depends on your answer above to what the authoritative doc is. > Is it the README? Then the README is the answer to your question. Is > it https://wiki.debian.org/Simple-CDD/Howto? Both? (Also these > should probably reference each other, and be referenced to from > https://www.debian.org/doc/manuals/debian-handbook/sect.automated-installation.en.html > section 12.3.3.1.) Yes, the README should be the authorative source for simple-cdd. The Howto should be synced with it from time to time. I don't think it belongs in the debian-installer manual, and apparently the debian-handbook should also be updated from time to time as needed. >> As explained in the simple-cdd README, these cannot be effectively >> preseeded using a simple-cdd profile, as preseeding is loaded after the >> questions are asked. > > > Maybe it's stated, but not explained. I still do not understand this > interplay, or the use of the word "preseed." It seems that > NAME.preseed is the file that decides what questions to ask, right? > So preseeding is the process of avoiding being asked things. But then > it's not because "preseeding is loaded after the questions are asked. Well, preseeding is a list of answers to questions that might be asked. If you preseed the answers before the question is asked, the question will not be asked, and it will use the value you've preseeded. If you load the preseeding after the question has already been asked, then at best, it doesn't magically travel through time an un-ask the question... at best, it does nothing, at worst, it might trigger bugs. >> Use --locale=en_CH > > I did this. Specifically, I modified custom.preseed like this: > > ``` > # Preseeding only locale sets language, country and locale. > #d-i debian-installer/locale string en_US > > # The values can also be preseeded individually for greater flexibility. > #d-i debian-installer/locale string en_US.UTF-8 > # Optionally specify additional locales to be generated. > d-i localechooser/supported-locales multiselect en_GB.UTF-8, de_DE.UTF-8 > > ``` > > Then I deleted tmp/ and images/ and I reran like so: > > > build-simple-cdd --profiles custom --auto-profiles custom --locale=en_CH > --keyboard=us > > > Then I waited 12 hours, and used expensive bandwidth, and I got a generated > ISO. > > And again, the ISO asked me to "Configure locales" like always. Did I > misunderstand something you told me to do, or is there a bug? > I mean, either way, there is a "bug," maybe in the code maybe in the doc. > But you know what I mean. Can't know for sure, but sounds like it *might* be a bug in the code. If you could re-try with "simple-cdd --locale=en_CH --keyboard=us" with an empty ./profiles directory and empty ./images directory (you can probably leave ./tmp, since it sounds like you don't have much bandwidth), that would be great. I'll also try to reproduce the problem at some point locally. live well, vagrant
signature.asc
Description: PGP signature
