Re: Layout of ‘define-configuration’ records
Hi Simon, zimoun writes: > Hi Maxim, > > On Mon, 21 Nov 2022 at 16:00, Maxim Cournoyer > wrote: > >> That sounds very appropriate indeed. I guess we could send >> announcements on API breaking changes to both places. I suppose not >> many people are registered to 'info-guix' (I wasn't myself until >> recently ^^'). > > Well, more is better here, no? :-) > > >> I guess that's a question of how disruptive the API change is, but it'd >> be convenient if it was 2 weeks to match the time the change might >> appear on guix-patches un-reviewed. > > Well, the process for API change could be: > > 1. + submit to guix-patches, > + in the same time, announce to guix-devel; so people not subscribed to > guix-patches can chime. > 2. + after 2 weeks, or consensus, merge the change, > + in the same time, announce to guix-devel, info-guix and --news. > > There is no extra burden and it smooths the change for many users, IMHO. > > WDYT? This sounds reasonable to me, with the addition that the "core" team members should be added as CC when submitting the API changing patch(es) (etc/teams.scm cc core). -- Thanks, Maxim
Re: Layout of ‘define-configuration’ records
Hi Ludovic, Ludovic Courtès writes: > Hi, > > Maxim Cournoyer skribis: > >> Ludovic Courtès writes: [...] >> Ah! Thanks for pointing my silly mistake. Then the argument would >> become... if it's good for define-configuration, it should have been >> good for define-record-type* the same (why the discrepancy?). > > ‘define-record-type*’ is generic; there’s no reason for it to add a > ‘location’ field. > >> After your new documentation in place to guide users to DTRT with >> regards to matching records, if you think %location should be the first >> field, then we should make it so in both instances, perhaps? > > ‘%location’ only appears in ‘define-configuration’; what did you mean by > “both instances”? Hmm, that's right. Nevermind, I thought the later had a %location "special" field too. >> Oops! Another point to add to our future coding style guidelines :-). > > In the end, I guess the lesson is that, indeed, not all the design > choices and rationales are properly documented. That’ll always be the > case to a large extent though, so changes “close to the core” require > more careful review and discussion to fully understand the implications > of the change—it might look innocuous but have broader implications than > expected. Agreed. I'll try to make better use of the etc/teams.scm script in the future to ping the right people for such changes. -- Thanks, Maxim
Re: Layout of ‘define-configuration’ records
Hi, Maxim Cournoyer skribis: > Ludovic Courtès writes: [...] One last thing: placing ‘%location’ first can let us implement: (define (configuration-location config) (struct-ref config 0)) >>> >>> Would this have worked? >>> >>> scheme@(gnu services mcron)> (define config (mcron-configuration)) >>> scheme@(gnu services mcron)> (struct-ref 0 config) >> >> You got the order wrong. :-) > > Ah! Thanks for pointing my silly mistake. Then the argument would > become... if it's good for define-configuration, it should have been > good for define-record-type* the same (why the discrepancy?). ‘define-record-type*’ is generic; there’s no reason for it to add a ‘location’ field. > After your new documentation in place to guide users to DTRT with > regards to matching records, if you think %location should be the first > field, then we should make it so in both instances, perhaps? ‘%location’ only appears in ‘define-configuration’; what did you mean by “both instances”? > Oops! Another point to add to our future coding style guidelines :-). In the end, I guess the lesson is that, indeed, not all the design choices and rationales are properly documented. That’ll always be the case to a large extent though, so changes “close to the core” require more careful review and discussion to fully understand the implications of the change—it might look innocuous but have broader implications than expected. Thanks, Ludo’.
Re: Layout of ‘define-configuration’ records
Hi Maxim, On Mon, 21 Nov 2022 at 16:00, Maxim Cournoyer wrote: > That sounds very appropriate indeed. I guess we could send > announcements on API breaking changes to both places. I suppose not > many people are registered to 'info-guix' (I wasn't myself until > recently ^^'). Well, more is better here, no? :-) > I guess that's a question of how disruptive the API change is, but it'd > be convenient if it was 2 weeks to match the time the change might > appear on guix-patches un-reviewed. Well, the process for API change could be: 1. + submit to guix-patches, + in the same time, announce to guix-devel; so people not subscribed to guix-patches can chime. 2. + after 2 weeks, or consensus, merge the change, + in the same time, announce to guix-devel, info-guix and --news. There is no extra burden and it smooths the change for many users, IMHO. WDYT? Cheers, simon
Re: Layout of ‘define-configuration’ records
Hi, Ludovic Courtès writes: > Hi Maxim, > > Maxim Cournoyer skribis: > >> Ludovic Courtès writes: > > [...] > + (%location #,(id #'stem #'stem #'-location) + (default (and=> (current-source-location) + source-properties->location)) + (innate))) >>> >>> Moving the field last is problematic as we’ve seen for any user that >>> uses ‘match’ on records—something that’s not recommended but still used >>> a lot. >> >> Yep. I had that on mind when I made the change, though I still missed a >> few occurrences. > > [...] > >> I wanted match on define-configuration'd fields to be >> backward-compatible with fields migrated from define-record-type*, so >> that they such change can be made without worrying breakages. > > That had the opposite effect: it introduced breakage precisely because > existing uses of ‘match’ on records need to be verified manually, one by > one. Yes. C.f. "I missed a few occurrences" above :-). > That led me to improve ‘match-record’, to recommend it in the manual, > and do “convert” some uses of ‘match’ to ‘match-record’: > > https://issues.guix.gnu.org/59390 > > That’s a good outcome, but I’d prefer not feeling this kind of pressure. Neat (not the under pressure part)! Perhaps srfi-worthy? >> I initially got tricked by that discrepancy and it took me quite some >> time hunting down a cryptic backtrace when authoring the new mcron >> configuration records. > > I see. However, this is a wide-ranging change, so I think this should > have been discussed separately from the mcron changes. I find it > important to take time for review and discussion for such changes. > >>> One last thing: placing ‘%location’ first can let us implement: >>> >>> (define (configuration-location config) >>> (struct-ref config 0)) >> >> Would this have worked? >> >> scheme@(gnu services mcron)> (define config (mcron-configuration)) >> scheme@(gnu services mcron)> (struct-ref 0 config) > > You got the order wrong. :-) Ah! Thanks for pointing my silly mistake. Then the argument would become... if it's good for define-configuration, it should have been good for define-record-type* the same (why the discrepancy?). After your new documentation in place to guide users to DTRT with regards to matching records, if you think %location should be the first field, then we should make it so in both instances, perhaps? >> All in all, I think that's a rather small change that got our internal >> implementation of both type of records in sync between >> define-configuration and define-record-type*, that should pave the way >> for migrating more of the later to the former without risking breaking >> something, going forward. > > Fundamentally, the layout of record types should not be visible to > users. That it is visible via ‘match’ is the problem, and the solution > is not to tweak record type layout but instead tp make sure ‘match’ uses > on records vanish. > > I hope that makes sense! Yes, and I agree. >> scheme@(gnu services mcron)> ,use (oop goops) > > Speaking of which: there was a conscious decision to not use GOOPS in > Guix from day one. Perhaps some day we’ll want to collectively question > that, but let’s make sure we don’t add that dependency on a whim. > > For example: > > class-of -> struct-vtable > class-name -> record-type-name > > See commit 50c17ddd9e2983d71c125d89b422fd20fca476e1 for an example. Oops! Another point to add to our future coding style guidelines :-). Thanks for showing the simple workaround to goops. -- Thanks, Maxim
Re: Layout of ‘define-configuration’ records
Hi Katherine, Katherine Cox-Buday writes: > Maxim Cournoyer writes: > >> Apologies for causing grief! > > No worries at all, Maxim! The good you do far outweighs any grief, and > even if that weren't the case, we're only human :) > >> I'm taking yours and Ludovic's feedback into account for the future >> and will reach out to guix-devel with heads-up when introducing >> breaking changes to Guix APIs. > > This seems like the logical place to me, but it's also a bit busy for > announcements! I'm sure I'm not alone, but sometimes I'm very busy and > distracted and I need a very loud signal to let me know I need to take > action :) That's why I suggested info-guix. Its description reads: > > #+begin_quote > Subscribe to the info-guix low-traffic mailing list to receive important > announcements sent by the project maintainers (in English). > #+end_quote That sounds very appropriate indeed. I guess we could send announcements on API breaking changes to both places. I suppose not many people are registered to 'info-guix' (I wasn't myself until recently ^^'). > Would it make sense to publish there? I don't think we break APIs very often? > > Also, do we need any kind of formalization around lead-time for these > announcements? I guess that's a question of how disruptive the API change is, but it'd be convenient if it was 2 weeks to match the time the change might appear on guix-patches un-reviewed. I'll try to do this in the future (better yet, try to not affect APIs at all); if it works well for everybody, we could formalize that in our contributing section. -- Thanks, Maxim
Re: Layout of ‘define-configuration’ records
Hi, Ludovic Courtès writes: > Maxim Cournoyer skribis: > >> A side-note, it seems that Ludovic has been >> working toward eliminating the use of match patterns matching the fields >> directly, instead encouraging the use of 'match-record', see >> https://issues.guix.gnu.org/59390. I haven't checked if this is >> compatible with define-configuration records though. > > It is: ‘define-configuration’ builds on ‘define-record-type*’, which > builds on SRFI-9, which builds on Guile “records”, which builds on Guile > “structs”. :-) Excellent! Thanks for having confirmed that. -- Maxim
Re: Layout of ‘define-configuration’ records
Maxim Cournoyer writes: > Apologies for causing grief! No worries at all, Maxim! The good you do far outweighs any grief, and even if that weren't the case, we're only human :) > I'm taking yours and Ludovic's feedback into account for the future > and will reach out to guix-devel with heads-up when introducing > breaking changes to Guix APIs. This seems like the logical place to me, but it's also a bit busy for announcements! I'm sure I'm not alone, but sometimes I'm very busy and distracted and I need a very loud signal to let me know I need to take action :) That's why I suggested info-guix. Its description reads: #+begin_quote Subscribe to the info-guix low-traffic mailing list to receive important announcements sent by the project maintainers (in English). #+end_quote Would it make sense to publish there? I don't think we break APIs very often? Also, do we need any kind of formalization around lead-time for these announcements? > A side-note, it seems that Ludovic has been working toward eliminating > the use of match patterns matching the fields directly, instead > encouraging the use of 'match-record', see > https://issues.guix.gnu.org/59390. I haven't checked if this is > compatible with define-configuration records though. Thanks, I'll have a look! -- Katherine
Re: Layout of ‘define-configuration’ records
Maxim Cournoyer skribis: > A side-note, it seems that Ludovic has been > working toward eliminating the use of match patterns matching the fields > directly, instead encouraging the use of 'match-record', see > https://issues.guix.gnu.org/59390. I haven't checked if this is > compatible with define-configuration records though. It is: ‘define-configuration’ builds on ‘define-record-type*’, which builds on SRFI-9, which builds on Guile “records”, which builds on Guile “structs”. :-) --8<---cut here---start->8--- scheme@(guile-user)> ,m (gnu home services desktop) scheme@(gnu home services desktop)> (home-redshift-configuration) $1 = #< redshift: # location-provider: geoclue2 adjustment-method: randr daytime-temperature: 6500 nighttime-temperature: 4500 daytime-brightness: %unset-marker% nighttime-brightness: %unset-marker% latitude: %unset-marker% longitude: %unset-marker% dawn-time: %unset-marker% dusk-time: %unset-marker% extra-content: "" %location: #f> scheme@(gnu home services desktop)> (match-record $1 (adjustment-method nighttime-temperature) ... (list nighttime-temperature adjustment-method)) $2 = (4500 randr) --8<---cut here---end--->8--- Ludo’.
Re: Layout of ‘define-configuration’ records
Hi Maxim, Maxim Cournoyer skribis: > Ludovic Courtès writes: [...] >>> + (%location #,(id #'stem #'stem #'-location) >>> + (default (and=> (current-source-location) >>> + source-properties->location)) >>> + (innate))) >> >> Moving the field last is problematic as we’ve seen for any user that >> uses ‘match’ on records—something that’s not recommended but still used >> a lot. > > Yep. I had that on mind when I made the change, though I still missed a > few occurrences. [...] > I wanted match on define-configuration'd fields to be > backward-compatible with fields migrated from define-record-type*, so > that they such change can be made without worrying breakages. That had the opposite effect: it introduced breakage precisely because existing uses of ‘match’ on records need to be verified manually, one by one. That led me to improve ‘match-record’, to recommend it in the manual, and do “convert” some uses of ‘match’ to ‘match-record’: https://issues.guix.gnu.org/59390 That’s a good outcome, but I’d prefer not feeling this kind of pressure. > I initially got tricked by that discrepancy and it took me quite some > time hunting down a cryptic backtrace when authoring the new mcron > configuration records. I see. However, this is a wide-ranging change, so I think this should have been discussed separately from the mcron changes. I find it important to take time for review and discussion for such changes. >> One last thing: placing ‘%location’ first can let us implement: >> >> (define (configuration-location config) >> (struct-ref config 0)) > > Would this have worked? > > scheme@(gnu services mcron)> (define config (mcron-configuration)) > scheme@(gnu services mcron)> (struct-ref 0 config) You got the order wrong. :-) > All in all, I think that's a rather small change that got our internal > implementation of both type of records in sync between > define-configuration and define-record-type*, that should pave the way > for migrating more of the later to the former without risking breaking > something, going forward. Fundamentally, the layout of record types should not be visible to users. That it is visible via ‘match’ is the problem, and the solution is not to tweak record type layout but instead tp make sure ‘match’ uses on records vanish. I hope that makes sense! > scheme@(gnu services mcron)> ,use (oop goops) Speaking of which: there was a conscious decision to not use GOOPS in Guix from day one. Perhaps some day we’ll want to collectively question that, but let’s make sure we don’t add that dependency on a whim. For example: class-of -> struct-vtable class-name -> record-type-name See commit 50c17ddd9e2983d71c125d89b422fd20fca476e1 for an example. Thanks, Ludo’.
Re: Layout of ‘define-configuration’ records
Hi Katherine, Katherine Cox-Buday writes: > Maxim Cournoyer writes: > >>> Moving the field last is problematic as we’ve seen for any user that >>> uses ‘match’ on records—something that’s not recommended but still >>> used a lot. > > Some feedback I hope is useful: > > I'm one such newbie, and had to stumble my way into discovering why some > of my services suddenly broke. All I had to go on was with "invalid > G-expression input" at a location in an `operating-system` declaration. > > Initially, because of the reference to gexps, I thought something had > changed with `local-file` which I am using to deploy source code off of > a local branch. Through trial and error and reading git logs, I > discovered this was not the case, and saw that `define-configuration` > had changed. From there I was able to discover the problem. > >> Yep. I had that on mind when I made the change, though I still missed >> a few occurrences. > > I came looking for an announcement of this change somewhere on a mailing > list (info-guix maybe?) or the Guix blog but couldn't find anything. I > understand the focus is on in-repo code, but could we consider > announcing changes which might affect channels and such? I don't always > have time to stay up to date with the changes to the project :( Apologies for causing grief! Ironically, the change was motivated by going through such pain myself when attempting to migrate the mcron-configuration from a define-record-type* to a define-configuration produced record, that's why I wanted to standardize their layout. I'm taking yours and Ludovic's feedback into account for the future and will reach out to guix-devel with heads-up when introducing breaking changes to Guix APIs. A side-note, it seems that Ludovic has been working toward eliminating the use of match patterns matching the fields directly, instead encouraging the use of 'match-record', see https://issues.guix.gnu.org/59390. I haven't checked if this is compatible with define-configuration records though. -- Thanks, Maxim
Re: Layout of ‘define-configuration’ records
Maxim Cournoyer writes: >> Moving the field last is problematic as we’ve seen for any user that >> uses ‘match’ on records—something that’s not recommended but still >> used a lot. Some feedback I hope is useful: I'm one such newbie, and had to stumble my way into discovering why some of my services suddenly broke. All I had to go on was with "invalid G-expression input" at a location in an `operating-system` declaration. Initially, because of the reference to gexps, I thought something had changed with `local-file` which I am using to deploy source code off of a local branch. Through trial and error and reading git logs, I discovered this was not the case, and saw that `define-configuration` had changed. From there I was able to discover the problem. > Yep. I had that on mind when I made the change, though I still missed > a few occurrences. I came looking for an announcement of this change somewhere on a mailing list (info-guix maybe?) or the Guix blog but couldn't find anything. I understand the focus is on in-repo code, but could we consider announcing changes which might affect channels and such? I don't always have time to stay up to date with the changes to the project :( -- Katherine
Re: Layout of ‘define-configuration’ records
Hi Ludo, Ludovic Courtès writes: > Hi, > > Maxim Cournoyer skribis: > >> This is so that the first field of the generated record matches the first one >> declared, which makes 'define-configuration' record API compatible with >> define-record-type* ones. >> >> * gnu/services/configuration.scm (define-configuration-helper): Move the >> %location field below the ones declared by the user. >> * gnu/services/monitoring.scm (zabbix-front-end-config): Adjust match pattern >> accordingly. > > [...] > >> +++ b/gnu/services/configuration.scm >> @@ -242,17 +242,17 @@ (define-record-type* #,(id #'stem #'< #'stem #'>) >> stem >> #,(id #'stem #'make- #'stem) >> #,(id #'stem #'stem #'?) >> - (%location #,(id #'stem #'stem #'-location) >> - (default (and=> (current-source-location) >> - source-properties->location)) >> - (innate)) >> #,@(map (lambda (name getter def) >> #`(#,name #,getter (default #,def) >> (sanitize >> #,(id #'stem #'validate- #'stem #'- >> name >> #'(field ...) >> #'(field-getter ...) >> - #'(field-default ...))) >> + #'(field-default ...)) >> + (%location #,(id #'stem #'stem #'-location) >> + (default (and=> (current-source-location) >> + source-properties->location)) >> + (innate))) > > Moving the field last is problematic as we’ve seen for any user that > uses ‘match’ on records—something that’s not recommended but still used > a lot. Yep. I had that on mind when I made the change, though I still missed a few occurrences. >> (define (zabbix-front-end-config config) >>(match-record config >> -(%location db-host db-port db-name db-user db-password db-secret-file >> - zabbix-host zabbix-port) >> +(db-host db-port db-name db-user db-password db-secret-file >> + zabbix-host zabbix-port %location) > > This change has no effect because ‘match-record’ matches fields by name, > precisely to avoid problems when changing the layout of record types. Good catch; I got confused there, although my confusion luckily had no side effect :-). > I’m not sure what was meant by “compatible” in the commit log; how > fields are laid out is something user code should not be aware of. I wanted match on define-configuration'd fields to be backward-compatible with fields migrated from define-record-type*, so that they such change can be made without worrying breakages. It seems good for consistency that both our define-record-type* and define-configuration fields share a similar internal layout, at least until all the old-fashion (ice-9 match) record matching usages are rewritten to use something like 'match-record'. I initially got tricked by that discrepancy and it took me quite some time hunting down a cryptic backtrace when authoring the new mcron configuration records. > The only thing to keep in mind is that records must not be matched with > ‘match’ because then the code silently breaks when the record type > layout is changed. This is why ‘match-record’ was introduced: > > https://issues.guix.gnu.org/28960#4 > > One last thing: placing ‘%location’ first can let us implement: > > (define (configuration-location config) > (struct-ref config 0)) Would this have worked? --8<---cut here---start->8--- scheme@(gnu services mcron)> (define config (mcron-configuration)) scheme@(gnu services mcron)> (struct-ref 0 config) ice-9/boot-9.scm:1685:16: In procedure raise-exception: In procedure struct-ref: Wrong type argument in position 1 (expecting struct): 0 Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. scheme@(gnu services mcron) [1]> ,q scheme@(gnu services mcron)> ,use (oop goops) scheme@(gnu services mcron)> (class-of config) $5 = #< <> 7fe20fd83e00> --8<---cut here---end--->8--- All in all, I think that's a rather small change that got our internal implementation of both type of records in sync between define-configuration and define-record-type*, that should pave the way for migrating more of the later to the former without risking breaking something, going forward. Alternatively, if we have a good reason to not to go with this, I think we should abstract all the internal-implementation dependent code from our code base (e.g., the match patterns directly matching on field slots). What do you think? -- Thanks, Maxim
