Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
On 2020-04-30, Vagrant Cascadian wrote: > On 2020-04-29, Buck wrote: >> One difference is you wrote "simple-cdd --locale=en_CH --keyboard=us" >> but I used "build-simple-cdd --locale=en_CH --keyboard=us" and I think >> that was just a typo right? Anyway I also did it your way and the >> result was the same. > > It should make no difference, simple-cdd and build-simple-cdd are > symlinks to the same file. > > >> So it's looking like a bug. I will do more testing if you want, including a >> new images/ directory. > > It sounds like I need to test it for myself at this point and see if I > can reproduce the issue(s), so maybe best to wait until I get that > chance. It may take me a week or so. Sorry it took so long to follow up on this. So, when I run "simple-cdd --locale=en_CH --keyboard=us" I get a working installer image that asks which locale to use, presenting text suggesting what the problem is: There is no locale defined for the combination of language and country you have selected. ... When I build an image with "simple-cdd --locale=en_US --keyboard=us" it skips the question prompting for language, as expected. So, you need to provide values to debian-installer that are supported by debian-installer: https://www.debian.org/releases/stable/amd64/apbs04.en.html#preseed-l10n From the looks of it, locales present in /etc/locale.gen should be supported in general. I do need to explore using the auto-install/enable=true a.k.a. auto=true parameter in simple-cdd, or moving simple-cdd to initrd preseeding, both of which would allow preseeding country/language/keyboard settings through a preseed file, and possibly allow arbitrary selection of country and language. live well, vagrant signature.asc Description: PGP signature
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
On 2020-04-29, Buck wrote: >> The maintainer scripts for each individual package and their debconf >> templates are where to look. > > I recommend putting this in the docs. I'll consider that, or at least referencing the relevent information found in other existing documentation. >> I wouldn't expect the "hello" program to document the fundamentals of >> computer science, or the working of electricity, or atomic theory, or >> the origins of the universe... even though all of those are, ostensibly, >> all relevent. > > I would, in the way that docs are supposed to do that. I think you > don't know a lot about documentation theory. Given your response, clearly I do not. > You do all of that by saying "This project requires a computer running > Debian," and if people aren't sure they have that then they research > Debian, and computers, and their dependencies, etc. That's just the > way all documenation is always written. Anything else is just > scribbles. There are always, of course, some unstated assumptions in any documentation. The meaningful question is which assumptions are a reasonable starting point, in my opinion... and opinions on this will surely vary. > Anyway you effectively do that in yours, so it's not material here. > But it is similar to how the debian-installer doc should reference the > HowTo, which should reference the README as the authoritative doc. The debian-installer documentation should not reference simple-cdd documentation, though the debian-handbook you earlier referenced probably should. > Otherwise we're all just clicking around desperate for anything that > says "simple-cdd" on it. I do take issue with your previous assertions that READMEs aren't generally useful in Debian. I typically first look at the --help output of a program, the manpage, and then whatever I can find in /usr/share/doc/PACKAGE... and I daresay I find this to be quite productive approach. >>> Great idea! Are you aware that the README does not mention >>> "debconf-get-selections" once? So it would take someone who does this >>> every day to think of this excellent solution. Thank you for sharing >>> this. I recommend adding it to the README, maybe the HowTo.. >> >> This could perhaps be briefly touched on in the simple-cdd README, and >> referencing relevent chapters of the debian-installer manual. > > It probably belongs in the same section that the first item does about > debconf templates. In a section about how to create a useful preseed > file. Thanks. >> If you load the preseeding after the question has already been asked, >> then at best, it doesn't magically travel through time an un-ask the >> question... at best, it does nothing, at worst, it might trigger bugs. > > That makes sense but it does not relate to anything clear about how > NAME.preseed works, how --preseed works, and how --auto-preseed works. > Please, I've asked this every way that is possible. Please document > this interplay. There are no --preseed or --auto-preseed options for simple-cdd; I presume you're referring to --profiles and --auto-profiles. I've tried to explain over and over again how they behave, and I'm still unsure what would be documentation that would work for you on this point, so I am apparently every bit at a loss as you are, unfortunately. Let's come back to that once I've had a chance to troubleshoot the locale/country/keyboard selection issues; expected behavior is a bit difficult to demonstrate if there are bugs affecting exactly that behavior. >> an't know for sure, but sounds like it *might* be a bug in the code. >> >> If you could re-try with "simple-cdd --locale=en_CH --keyboard=us" with >> an empty ./profiles directory and empty ./images directory (you can >> probably leave ./tmp, since it sounds like you don't have much >> bandwidth), that would be great. > > I did this, moved profiles/ and deleted images/. The new image was > generated in seconds which makes me wonder if I was really doing what > you want. Thanks for testing. It shouldn't take long when you have all or most of the packages already in the local mirror under ./tmp, so it does not surprise me that the build went quickly. > Anyway, the result is the same. I am asked what installer to use, I > choose regular "Install" and The bootloader prompting you (e.g. where you had to choose "Install") is handled by something else entirely. The "test" profile included as part of simple-cdd sets the BOOT_TIMEOUT debian-cd option in order to handle this. > my first question is what language to use. Ok, I'll attempt to reproduce the issue locally and debug it. > One difference is you wrote "simple-cdd --locale=en_CH --keyboard=us" > but I used "build-simple-cdd --locale=en_CH --keyboard=us" and I think > that was just a typo right? Anyway I also did it your way and the > result was the same. It should make no difference, simple-cdd and build-simple-cdd are symlinks to the same file. > So it's looking like a
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
Package: simple-cdd Version: 0.6.5 Followup-For: Bug #958277 > The maintainer scripts for each individual package and their debconf > templates are where to look. I recommend putting this in the docs. > I wouldn't expect the "hello" program to document the fundamentals of > computer science, or the working of electricity, or atomic theory, or > the origins of the universe... even though all of those are, ostensibly, > all relevent. I would, in the way that docs are supposed to do that. I think you don't know a lot about documentation theory. You do all of that by saying "This project requires a computer running Debian," and if people aren't sure they have that then they research Debian, and computers, and their dependencies, etc. That's just the way all documenation is always written. Anything else is just scribbles. Anyway you effectively do that in yours, so it's not material here. But it is similar to how the debian-installer doc should reference the HowTo, which should reference the README as the authoritative doc. Otherwise we're all just clicking around desperate for anything that says "simple-cdd" on it. >> Great idea! Are you aware that the README does not mention >> "debconf-get-selections" once? So it would take someone who does this >> every day to think of this excellent solution. Thank you for sharing >> this. I recommend adding it to the README, maybe the HowTo.. > > This could perhaps be briefly touched on in the simple-cdd README, and > referencing relevent chapters of the debian-installer manual. It probably belongs in the same section that the first item does about debconf templates. In a section about how to create a useful preseed file. > Yes, the README should be the authorative source for simple-cdd. Okay great. Then the HowTo and the debian-installer manual should reference that. > If you load the preseeding after the question has already been asked, > then at best, it doesn't magically travel through time an un-ask the > question... at best, it does nothing, at worst, it might trigger bugs. That makes sense but it does not relate to anything clear about how NAME.preseed works, how --preseed works, and how --auto-preseed works. Please, I've asked this every way that is possible. Please document this interplay. > an't know for sure, but sounds like it *might* be a bug in the code. > > If you could re-try with "simple-cdd --locale=en_CH --keyboard=us" with > an empty ./profiles directory and empty ./images directory (you can > probably leave ./tmp, since it sounds like you don't have much > bandwidth), that would be great. I did this, moved profiles/ and deleted images/. The new image was generated in seconds which makes me wonder if I was really doing what you want. Anyway, the result is the same. I am asked what installer to use, I choose regular "Install" and my first question is what language to use. One difference is you wrote "simple-cdd --locale=en_CH --keyboard=us" but I used "build-simple-cdd --locale=en_CH --keyboard=us" and I think that was just a typo right? Anyway I also did it your way and the result was the same. So it's looking like a bug. I will do more testing if you want, including a new images/ directory. > it sounds like you don't have much > bandwidth I wouldn't say that. But building these CDs every time I want to test a new config option is taking a toll, sure.
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
Could you please include a subject in your emails? You seem to be replying to the messages and deleting the subject every time. On 2020-04-28, Buck wrote: >> Examples of usage of each of what? > > I was referring to this statement of yours: "profiles" defines which > profiles are included in simple-cdd, "auto-profiles" selects which > profiles to use at run-time without asking the user." > I'm not sure why it didn't make it through. I am still learning this > cool but unusual bug reporting system. It's just email, basically, so many of the same common ettiquette applies here. > Anyway, that statement made no sense to me, even after reading lots > and lots of docs and help files. Please provide examples of each. I'll try again... --profiles selects what profiles are included on the generated images, essentially copying the files from ./profiles/ and/or /usr/share/simple-cdd/profiles/ files onto the installer media for the relevent profile, and determines which packages are available on the install media. --auto-profiles effectively preseeds which profiles to install when you boot the installer, if you do not use --auto-profiles, it will ask you which profiles you want to install when you boot the installer media (e.g. USB, DVD, CD, etc.). >> Yes, please include the contents of these files, otherwise it's >> essentially impossible for me to understand what you're doing or what's > Okay: > > --custom.preseed-- > Contents of the preconfiguration file (for jessie) > ### Localization > # Preseeding only locale sets language, country and locale. > #d-i debian-installer/locale string en_US > > # The values can also be preseeded individually for greater flexibility. > d-i debian-installer/language string en > d-i debian-installer/country string CH > #d-i debian-installer/locale string en_US.UTF-8 As explained in the simple-cdd README, these cannot be effectively preseeded using a simple-cdd profile, as preseeding is loaded after the questions are asked. Use --locale=en_CH > # Keyboard selection. > d-i keyboard-configuration/xkb-keymap select us > # d-i keyboard-configuration/toggle select No toggling Same. Use --keyboard=us Both --locale and --keyboard configure the boot prompt to pass specific preseeded debconf values, which are preseeded very early in the boot process. > # Root password, either in clear text > d-i passwd/root-password password wootaler > d-i passwd/root-password-again password wootaler > # or encrypted using an MD5 hash. > #d-i passwd/root-password-crypted password [MD5 hash] Presumably, you've encoutered a warning from simple-cdd, as it cannot verify that it's a valid preseeding file (the password configuration is tested with a different database that requires root access, and simple-cdd should not be run as root). > # Alternatively, you may specify a disk to partition. If the system has only > # one disk the installer will default to using that, but otherwise the device > # name must be given in traditional, non-devfs format (so e.g. /dev/sda > # and not e.g. /dev/discs/disc0/disc). > # For example, to use the first SCSI/SATA hard disk: > #d-i partman-auto/disk string /dev/sda If you want automatic partitioning, you'll need to uncomment this, or it will ask you... And you need to make sure you're passing it a valid device, which can get very tricky as devices may be enumerated in a non-deterministic way. In qemu, it might be /dev/vda (libvirt) or /dev/sda (scsi, ide, usb), for example. If you have USB sticks, the USB stick might be /dev/sda, or the installed disk might be /dev/sda. With simple systems that you can control, usually it'll be /dev/sda, but no guarantees. > # Individual additional packages to install > d-i pkgsel/include string build-essential git w3m secure-delete rsync nmap > screen tmux sudo pwgen net-tools dnstools You need these packages to be available in one of your profiles, or it will require network access... maybe that's ok for you... > # Due notably to potential USB sticks, the location of the MBR can not be > # determined safely in general, so this needs to be specified: > #d-i grub-installer/bootdev string /dev/sda > # To install to the first device (assuming it is not a USB stick): > #d-i grub-installer/bootdev string default You need to specify this or it will ask you. And again, it needs to match the disks you actually have available, and pick the correct ones. > --custom.packages-- > (blank) You probably want the packages you mentioned in your preseeding file: d-i pkgsel/include string build-essential git w3m secure-delete rsync nmap screen tmux sudo pwgen net-tools dnstools > --custom.postinst-- > (contains a custom BASH script with private data) You taunt me and limit my ability to help you. :P > --custom.description-- > (contains a single-line string with private data describing the project) You taunt me and limit my ability to help you. :P > --custom.udebs-- > # the udeb needed for simple-cdd >
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
On 2020-04-28, Buck wrote: > This is not clear. It sounds like a riddle. Please provide examples of > usage of each. What is not clear? What is a riddle? Examples of usage of each of what? I will admit, I am feeling the same way about these bug reports from this side of the computer screen. :P >>> It would be helpful if you could provide your proposed profile in >>> more detail. > > I will, now that I know you want this. But first in this message and > your next reply, I want to eliminate usage errors as the problem. > Then we can move into the details of the profile. Yes, it could have saved a *lot* of back and forth to provide those up front. It is much easier to explain specific details that demonstrate specific issues and concepts than speak about this sort of thing in general... >> If you want to override parts of the "default" profile, you also need >> ~/my-simple-cdd/profiles/default.* as well. The "default" profile is >> always included, and for questions asked very early in debian-installer, >> the only way to preseed some questions using simple-cdd. > > Wait wait whoah whoah. The docs I've been reading for weeks don't > sound like this at all. I'm going to explain what I've been lead to > believe. You will hopefully tell me *specifically* how I am wrong, > and suggest correct usage instead. What docs are you referring to? From the README: Default Profile the profile named "default" is special, because it always gets installed. modify the profile/default.* files with care, as simple-cdd relies upon the default profile working in certain ways ... > My objective is to build a QEMU image of an installer that requires > zero (ZERO) user interaction, and can actually be run headless, > because it knows the answers ahead of time. Then I want this VM to > automatically run a script after Debian is install, and this script > pulls from a private Git repo, and runs another script from that repo. > That's the objective. It's certainly possible, as long as all the packages you install support debconf preseeding correctly. The "test" profile attempts to automate everything in this way, and might be a good example of the kinds of things you may need for a fully automated install. It's actually used as a profile to test the other profiles with minimal interaction, and is meant to simply be added to the profiles to automate testing a profile: simple-cdd --profiles x-basic,ltsp,router,test --auto-profiles test,router Should include all four profiles (plus the default profile) on the installer image, and then automatically select the test and router profiles without interaction. simple-cdd --profiles x-basic,ltsp,router,test Should generate an image with the same profiles as above, but prompt for which you want to use at installation time (obviously not your desired use-case, but I include it for the clarification of the difference between --profiles and --auto-profiles). > I've been lead to believe that to accomplish this, I need a > custom.preseed file that answers all of the important questions. And > I need a custom.postinst that is a BASH script. Not necessarily bash, most likely /bin/sh, possibly whatever #! you put in your script. Though it's been a long time since I've looked, it might specifically require #!/bin/sh... The .postinst scripts are really just for quick hacks that can't be handled with preseeding. > And these must be the only files read by build-simple-cdd, and they go > in profiles/ under the directory I run build-simple-cdd from (which is > ~/my-simple-cdd). ... > I've been lead to believe that if I put custom.preseed and > custom.postinst into a profiles/ directory under ~/my-simple-cdd, and > I run `build-simple-cdd --profiles custom` from within > ~/my-simple-cdd, then the custom profile will be read. The > implication from the docs is that *only* the custom profile will be > read if I am using `--profiles custom`. I don't understand where you get this impression from. > But when I run with `--profiles custom`, I get an image, which I run > in QEMU and I see a normal default Debian installer that asks me if I > want graphical vs text, etc. and The default profile attempts to answer many debconf questions, but may need updates to reflect new questions in debian-installer or installed packages. It hasn't been tested by me since the release of buster... > then asks me to choose a language and I know I (think I) put that in > my custom.preseed file. Language selection is addressed in the README in the "Language and Country Selection" heading. It's one of those special configurations. There's also some debian-installer documentation the briefly touches on what questions can be answered with which methods of preseeding: https://www.debian.org/releases/stable/amd64/apbs01.en.html There are a lot of other things documented in the adjacent chapters. It's pretty much what simple-cdd is designed to facilitate, as a thin layer on top of
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
On 2020-04-27, Buck wrote: > Yes sorry. The command: > > `~/my-simple-cdd$ build-simple-cdd` > >> You may also want to use "--auto-profiles NAME" additionally > > Why? The documentation is not clear how --auto-profiles vs --profiles works. "profiles" defines which profiles are included in simple-cdd, "auto-profiles" selects which profiles to use at run-time without asking the user. >> It would be helpful if you could provide your proposed profile in >> more detail. > >> Ideally, the exact commands you ran and the exact state of >> the directory you're running them from. > > The command is above. The state is that ~/my-simple-cdd/ has a > profiles/ directory. profiles/ directory contains custom.description, > custom.packages, custom.postinst, custom.udebs, custom.preseed. Do > you want the details from each of these? Yes, please include the contents of these files, otherwise it's essentially impossible for me to understand what you're doing or what's going wrong. If you want to override parts of the "default" profile, you also need ~/my-simple-cdd/profiles/default.* as well. The "default" profile is always included, and for questions asked very early in debian-installer, the only way to preseed some questions using simple-cdd. That said, not including or aggressively overriding contents in the default profile is possible to break how simple-cdd works, so be selective in what you override. In general, I've tried to make the default profile not too intrusive, so it shouldn't need to be overridden in most cases... >>* What exactly did you do (or not do) that was effective (or >> ineffective)? > > I have been changing Debian versions, thinking the problem is bugs in > simple-cdd, tasksel, or reprepro. Different erros come up with > different versions. All are being reported. > >> If you want to override the built-in profiles, you need to create >> replacement files (e.g. profiles/default.preseed, >> profiles/default.*), though I would generally recommend providing >> additional profiles rather than overriding the default profiles. > > I did the latter of these, added profiles/custom.X files. However > when I do this, I get this current error where the CD is created and > it looks like a normal Debian installer that asks quesitons that I > believe are answered in the custom.preseed file. The custom.preseed file may not be loaded till later in the process; it depends on which questions you're thinking are answered by it. There is a question during the install that asks which simple-cdd profiles to load; any pquestions asked before that obviously can't be preseeded. > I also tried using a default-custom (as in, a template for X.preseed > that I found online) and this also build a default Debian install CD. > >>* What was the outcome of this action? >> >> New error: $default_desktop not defined > > The error for this bug is not an error message, it is a working Debian > installer CD that behaves different from I expect, becaues it behaves > like a default Debian installer. > > But what is reporting error > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=958276 > ($default_desktop) is the output of build-simple-cdd. Where do you expect $default_desktop to be defined? Again, including the actual files you're trying to use would make it a *lot* easier for me to see what you're doing, or even reproduce whatever issue you're having if it is indeed a bug. >> It will read the profiles from profiles in your working directory, > > This does not happen, but I am happy that you say this is what should > happen. That tells me that there is a bug that is preventing my > ~/my-simple-cdd/profiles/custom.X files being read. It is also > possible I have a bad config but I don't think so because I tried a > default custom config. Are you running from ~ or running from ~/my-simple-cdd ? > Can you suggest a config I should use to test, that will create a > Debian installer that is different from default to verify that this > preseed is being read? you can just specify one of the profiles in /usr/share/simple-cdd/profiles, preferably in a directory that does not contain any "profiles" directory: simple-cdd --profiles router,x-basic That should create an installer image that prompts weather or not to load the router and x-basic profiles during the install, and then loads the profiles you've asked for, or if you select none of them, a minimal install just using the default profile. If that's not working, then there's definitely a bug in simple-cdd. live well, vagrant signature.asc Description: PGP signature
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
On 2020-04-20, Buck wrote: >* What led up to the situation? > > Create my-simple-cdd/profiles/NAME.x files that specify answers > to installer questions > > Run build-simple-cdd --profiles NAME Did you run this command from within the "my-simple-cdd" directory? You may also want to use "--auto-profiles NAME" additionally, if you want the preseeding for NAME be applied in all cases. It would be helpful if you could provide your proposed profile in more detail. Ideally, the exact commands you ran and the exact state of the directory you're running them from. >* What exactly did you do (or not do) that was effective (or > ineffective)? > > Removed /usr/share/simple-cdd/profiles. Removing /usr/share/simple-cdd/profiles provided by the package is not a supported mode of operation (in general, removing arbitrary files in any package is generally unsupported and likely will cause bugs). If you want to override the built-in profiles, you need to create replacement files (e.g. profiles/default.preseed, profiles/default.*), though I would generally recommend providing additional profiles rather than overriding the default profiles. >* What was the outcome of this action? > > New error: $default_desktop not defined What is reporting this error? >* What outcome did you expect instead? > > I expected that removing /usr/share/simple-cdd/profiles would > force build-simple-cdd to read the profile files in > ~/my-simple-cdd/profiles/NAME.x It will read the profiles from profiles in your working directory, overriding any matching profiles in /usr/share/simple-cdd/profiles. There is no need to break the packages installed files to get it to do this. live well, vagrant signature.asc Description: PGP signature
Bug#958277: simple-cdd: Resulting Install CD asks for questions answered in the profiles/NAME.x files
Package: simple-cdd Version: 0.6.5 Severity: normal Dear Maintainer, *** Reporter, please consider answering these questions, where appropriate *** * What led up to the situation? Create my-simple-cdd/profiles/NAME.x files that specify answers to installer questions Run build-simple-cdd --profiles NAME * What exactly did you do (or not do) that was effective (or ineffective)? Removed /usr/share/simple-cdd/profiles. * What was the outcome of this action? New error: $default_desktop not defined * What outcome did you expect instead? I expected that removing /usr/share/simple-cdd/profiles would force build-simple-cdd to read the profile files in ~/my-simple-cdd/profiles/NAME.x -- System Information: Debian Release: 9.12 APT prefers oldstable APT policy: (500, 'oldstable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.107-1.pvops.qubes.x86_64 (SMP w/2 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages simple-cdd depends on: ii dctrl-tools 2.24-2+b1 ii debian-cd 3.1.20 ii lsb-release 9.20161125 ii python3 3.5.3-1 ii python3-simple-cdd 0.6.5 ii reprepro5.1.1-1 ii rsync 3.1.2-1+deb9u2 ii wget1.18-5+deb9u3 Versions of packages simple-cdd recommends: ii dose-distcheck 5.0.1-8+deb9u1 Versions of packages simple-cdd suggests: ii qemu-kvm 1:2.8+dfsg-6+deb9u9 -- no debconf information
