Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
On 3/19/20 8:17 PM, PGNet Dev wrote: > On 3/19/20 8:04 PM, Eli Schwartz wrote: >> On 3/19/20 10:02 PM, Dimitri John Ledkov wrote: >>> I hope you do understand that it >>> is intended for all of the Ubuntu installer & upgrade integrations to keep >>> valid configs compatible. For anyone similarly interested, it looks like Ubu 18 *LTS* will NOT be getting a fast fix for 'this' Posix support for a bit, if at all, as it remains a ... "highly unusual configuration". https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868138/comments/8 "So given that 8.13 is affected - which was released a year ago, and it's a highly unusual configuration, this is not in need of an urgent hotfix." ::facepalm:: Eli, your admonition to consider a more 'stable' option moves to the front of the line. Migrations to Grub + Arch & OpenSUSE spinning up today. Thx for the comments folks. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
On 3/19/20 8:04 PM, Eli Schwartz wrote: > On 3/19/20 10:02 PM, Dimitri John Ledkov wrote: >> I am sorry which email are the "quoted" phrases from? not from an email. directly from the bug I referenced, prior to coming here: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868138/comments/4 , subsequent to chats in #irc >> Because that's nothing that I have said. sure. that's clear, and certainly not assumed! >> I hope you do understand that it >> is intended for all of the Ubuntu installer & upgrade integrations to keep >> valid configs compatible. That's the assumption -- or at least the hope -- for all the mainstream distros. And Grub's certainly about as consistent/portable/reliable/cross-everything as it gets. And appears to include aforementioned Posix script compliance for the config script. Ubu dev's proclaimation as otherwise is what caused the ... surprise ... & the questions here. Bottom line ... Grub continues to work exactly reliably/predictably, as it should. Ubu's install should be fixed, not hand-waved away for specious reasons. That can be dealt with best in the bug itself. thx. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
On 3/19/20 10:02 PM, Dimitri John Ledkov wrote: > I am sorry which email are the "quoted" phrases from? > > Because that's nothing that I have said. I hope you do understand that it > is intended for all of the Ubuntu installer & upgrade integrations to keep > valid configs compatible. And in general, distros do not intentially break > their users for no reason. I simply stated what are the most typical and > the most tested customisation and integration point for grub configs on > Ubuntu, without making any statements about supportability of the configs > that you have observed getting malformed. It is the position of the Ubuntu developer who responded to the OP's launchpad.net bug which motivated this cross-post to the grub-devel list. Direct link to the quoted phrases: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868138/comments/4 -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
I am sorry which email are the "quoted" phrases from? Because that's nothing that I have said. I hope you do understand that it is intended for all of the Ubuntu installer & upgrade integrations to keep valid configs compatible. And in general, distros do not intentially break their users for no reason. I simply stated what are the most typical and the most tested customisation and integration point for grub configs on Ubuntu, without making any statements about supportability of the configs that you have observed getting malformed. On Fri, 20 Mar 2020, 01:08 PGNet Dev, wrote: > hi > > On 3/19/20 5:57 PM, Dimitri John Ledkov wrote: > > > > What is the bug number there? > > > > > https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868138 > > > > > In general, we advise to customize via grub.d drop-in files instead of > modifying etc/default/grub file itself. > > > > Sure, that's fine as advice. Fwiw, it's different for different distros. > > > > Declarations that using posix-compliant continuations is "highly unusual > abuse of POSIX shell semantics" is a different matter. > > > > Changing/breaking behavior in managing valid/working grub configs, and > rationalizing it by "I do not see a need to support it" > > is ... unhelpful. > > > > Particularly if Grub project/devs say that its usage *is* OK & compliant. > Which is why I asked _here_; and, iiuc, seems that it *is*. > > > > Grub's stability, reproducibility and portability are critical. It all > works great, particularly when the excellent docs are followed. > > Happy to take ongoing conversation to the bug. > > > > ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
hi, On 3/19/20 6:41 PM, Eli Schwartz wrote: > In general, other distros avoid parsing and rewriting configs because > this should be the user's responsibility. :p > > The OP should consider that maybe such distros are more stable than Ubuntu? I do. Where I directly control them, they're in use. It includes my own spins ... All use Grub. "Happily". ;-) Ubu's a not-infrequent reality nonetheless. > The OP's question is > > "Does the grub project consider /etc/default/grub as a shell-compatible > file which is sourced, to be the PUBLIC INTERFACE of this configuration > file?" yep. well restated. > The conclusion is simple enough: if you are going to automatically > rewrite this grub file, i'd thought that non-mangling of it would be a reasonable position; that, apparently, is not a mandate, iiuc. tbh, i'm not clear _why_ it's being written at all. i agree with the "this should be the user's responsibility", and would add "only the user's prerogative". but that's me. well, maybe not JUST me ... > you must do so using a parser engine that > implements POSIX shell and knows how to properly tokenize and rewrite > the full range of language features permitted by POSIX shell. > > Doing otherwise is YOLO, rash, buggy, and prone to breakage as > demonstrated by the OP's bug report. > > If a downstream project making use of grub, then goes ahead and calls > the use of backslash escapes for the very simple purpose of line > continuation across multiple lines (of all the things one might do in a > POSIX sh formatted configuration file, this is pretty tame), an issue of > "highly unusual abuse of POSIX shell semantics", this is pretty bad I > have to say. +1 > Just my $0.02 as a distro person (not a grub dev). heh. cheap at twice the price! ;-) thx. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
On 3/19/20 8:57 PM, Dimitri John Ledkov wrote: > Have you opened a launchpad bug report against the grub2 package with both > configs before and after? What is the bug number there? > > In general, we do parse and rewrite configs using debconf which is Perl / C > / Shell processing using tools external to grub. In general, we advise to > customize via grub.d drop-in files instead of modifying etc/default/grub > file itself. > > This is a bit out of scope for the upstream mailing list, as debconf > integration is downstream specific on Ubuntu/Debian systems. In general, other distros avoid parsing and rewriting configs because this should be the user's responsibility. :p The OP should consider that maybe such distros are more stable than Ubuntu? ... Anyway, it seems clear-cut to me, as per the question which is NOT about what should be done with debconf. The OP's question is very simple: "Does the grub project consider /etc/default/grub as a shell-compatible file which is sourced, to be the PUBLIC INTERFACE of this configuration file?" It seems clear-cut to me. The quoted manual section explicitly documents that it is POSIX shell format, then calls out the idea of KEY=VALUE assignments and declares this to be *merely* "normally", thus highlighting the possibility that it isn't exclusively so. Compare this to, for example, the os-release(5) man page which states the following format for /etc/os-release: """ The basic file format of os-release is a newline-separated list of environment-like shell-compatible variable assignments. It is possible to source the configuration from shell scripts, however, beyond mere variable assignments, no shell features are supported (this means variable expansion is explicitly not supported), allowing applications to read the file without implementing a shell compatible execution engine. """ Note how os-release(5) explicitly goes to some lengths to declare the need for parsing this file in C, that it is *not* valid to simply use shell code, and then continues on to enumerate which subset of shell code it supports. The conclusion is simple enough: if you are going to automatically rewrite this grub file, you must do so using a parser engine that implements POSIX shell and knows how to properly tokenize and rewrite the full range of language features permitted by POSIX shell. Doing otherwise is YOLO, rash, buggy, and prone to breakage as demonstrated by the OP's bug report. If a downstream project making use of grub, then goes ahead and calls the use of backslash escapes for the very simple purpose of line continuation across multiple lines (of all the things one might do in a POSIX sh formatted configuration file, this is pretty tame), an issue of "highly unusual abuse of POSIX shell semantics", this is pretty bad I have to say. Also, it's victim blaming. Just my $0.02 as a distro person (not a grub dev). -- Eli Schwartz Arch Linux Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
hi On 3/19/20 5:57 PM, Dimitri John Ledkov wrote: > What is the bug number there? https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1868138 > In general, we advise to customize via grub.d drop-in files instead of > modifying etc/default/grub file itself. Sure, that's fine as advice. Fwiw, it's different for different distros. Declarations that using posix-compliant continuations is "highly unusual abuse of POSIX shell semantics" is a different matter. Changing/breaking behavior in managing valid/working grub configs, and rationalizing it by "I do not see a need to support it" is ... unhelpful. Particularly if Grub project/devs say that its usage *is* OK & compliant. Which is why I asked _here_; and, iiuc, seems that it *is*. Grub's stability, reproducibility and portability are critical. It all works great, particularly when the excellent docs are followed. Happy to take ongoing conversation to the bug. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
Have you opened a launchpad bug report against the grub2 package with both configs before and after? What is the bug number there? In general, we do parse and rewrite configs using debconf which is Perl / C / Shell processing using tools external to grub. In general, we advise to customize via grub.d drop-in files instead of modifying etc/default/grub file itself. This is a bit out of scope for the upstream mailing list, as debconf integration is downstream specific on Ubuntu/Debian systems. On Thu, 19 Mar 2020, 21:20 PGNet Dev, wrote: > a recent grub package update, in ubuntu 18LTS, is breaking > /etc/default/grub by mangling/overwriting users' entries, in the specific > case of using continuation lines in the file/config. and, subsequently, the > upgrade process on these servers. > > as stated clearly at > > > https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html#Simple-configuration-handling > > "... > The file /etc/default/grub controls the operation of > grub-mkconfig. It is sourced by a shell script, and so must be valid POSIX > shell input; normally, it will just be a sequence of ‘KEY=value’ lines, but > if the value contains spaces or other special characters then it must be > quoted > ..." > > per POSIX > > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_02_01 > > "... > Escape Character (Backslash) > > A that is not quoted shall preserve the literal value > of the following character, with the exception of a . If a > follows the , the shell shall interpret this as line > continuation. The and shall be removed before > splitting the input into tokens. Since the escaped is removed > entirely from the input and is not replaced by any white space, it cannot > serve as a token separator. > > ..." > > all that's to say that continuation lines appear to be supported in > posix-shell-compliant /etc/default/grub > > they've been in use here, on 100s of servers, in grub configs for ages. > > with grub 2.04 on other, non Ubu OS, it continues to work fine. > > re: the use of continuation lines, ubu dev states that > > "Regardless of what the POSIX spec says, this is highly unusual > abuse of POSIX shell semantics, and I do not see a need to support it" > > it'd be useful/helpful to get clear on what the 'official' grub project > support for use of posix-shell-compliant continuation lines is ... > > are they 'supported' as valid use in /etc/default/grub, or not? > > also useful to know/understand whether any grub update can/should mangle a > user's /etc/default/grub. allowed? expected? > > thx! > > > > ___ > Grub-devel mailing list > Grub-devel@gnu.org > https://lists.gnu.org/mailman/listinfo/grub-devel > ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
On Thu, Mar 19, 2020 at 5:19 PM PGNet Dev wrote: > > a recent grub package update, in ubuntu 18LTS, is breaking /etc/default/grub > by mangling/overwriting users' entries, in the specific case of using > continuation lines in the file/config. and, subsequently, the upgrade process > on these servers. This sounds like an issue you should hash out with your distro maintainer. How Ubuntu manages package upgrades is not within the scope of this mailing list. > it'd be useful/helpful to get clear on what the 'official' grub project > support for use of posix-shell-compliant continuation lines is ... > are they 'supported' as valid use in /etc/default/grub, or not? The manual section you quoted seems pretty clear to me: it must be valid POSIX shell input. Backslash-escaped newlines are valid POSIX shell input. Technically speaking, /etc/default/grub is sourced by grub-mkconfig, which is executed by /bin/sh. So long as /bin/sh is able to correctly process both grub-mkconfig and /etc/default/grub, you probably wont have a problem. If /bin/sh happens to be bash, you could probably get away with putting bash-specific code in /etc/default/grub if you really wanted to. What your distro chooses to support is really up to them, however. > also useful to know/understand whether any grub update can/should mangle a > user's /etc/default/grub. allowed? expected? I think this is not something the grub developers care about; work it out with your distro maintainer. ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
are posix-shell-compliant continuation lines valid/supported, or not, in /etc/default/grub ?
a recent grub package update, in ubuntu 18LTS, is breaking /etc/default/grub by mangling/overwriting users' entries, in the specific case of using continuation lines in the file/config. and, subsequently, the upgrade process on these servers. as stated clearly at https://www.gnu.org/software/grub/manual/grub/html_node/Simple-configuration.html#Simple-configuration-handling "... The file /etc/default/grub controls the operation of grub-mkconfig. It is sourced by a shell script, and so must be valid POSIX shell input; normally, it will just be a sequence of ‘KEY=value’ lines, but if the value contains spaces or other special characters then it must be quoted ..." per POSIX https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_02_01 "... Escape Character (Backslash) A that is not quoted shall preserve the literal value of the following character, with the exception of a . If a follows the , the shell shall interpret this as line continuation. The and shall be removed before splitting the input into tokens. Since the escaped is removed entirely from the input and is not replaced by any white space, it cannot serve as a token separator. ..." all that's to say that continuation lines appear to be supported in posix-shell-compliant /etc/default/grub they've been in use here, on 100s of servers, in grub configs for ages. with grub 2.04 on other, non Ubu OS, it continues to work fine. re: the use of continuation lines, ubu dev states that "Regardless of what the POSIX spec says, this is highly unusual abuse of POSIX shell semantics, and I do not see a need to support it" it'd be useful/helpful to get clear on what the 'official' grub project support for use of posix-shell-compliant continuation lines is ... are they 'supported' as valid use in /etc/default/grub, or not? also useful to know/understand whether any grub update can/should mangle a user's /etc/default/grub. allowed? expected? thx! ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
