Re: series-agnostic charm URLs
Casey is the one to talk to. I'm not sure if he was just trying to be conservative, but he did intentionally not want to pun the types. I think he wanted to be clear that things passed to *these* API calls must be fully formed, while ones passed to *those* could be expanded. We could do what you say, but it starts meaning every API that takes a URL has to start worrying about expanding it late. Since it is no longer "charm.URL" but really "charm.MaybeURL". John =:-> On Wed, Jul 23, 2014 at 2:35 PM, roger peppe wrote: > In the charm store and the bundle specification, we use both fully > specified charm URLs (with series) and partially specified charm URLs > (without series and/or version). > > Currently in the charm package, the charm.URL type represents the > former, the charn.Reference type represents the former; but there is > no type that > can represent both forms. > > We want to store charm URLs in mongo-db that are agnostic whether > the series is specified or not. For example, in a bundle, a service > is free to specify a series in the charm name or not. > > I'd like to suggest that we remove the Reference type and use the URL > type throughout, allowing it to have an unspecified series > where the string form does not specify a series. > > This means that the URL type would be an exact reflection of the string > form of a charm URL. > > We would add a ParseFullURL method which is like ParseURL > currently - it would check that the series is fully specified. All > occurrences of charm.ParseURL in juju-core would be changed to use > charm.ParseFullURL. All occurrences of ParseReference would change to > use ParseURL. > > This would change the semantics of URL unmarshaling, > but the Juju API does not currently rely on that. > There is at least one place (params.ServiceDeploy) where > we would now be able to use a *charm.URL rather than > a string. > > The changes required in juju-core are largely mechanical, > and generally make things simpler. > > Does this sound reasonable? > > cheers, > rog. > > -- > Juju-dev mailing list > Juju-dev@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Wed, Jul 23, 2014 at 7:35 AM, roger peppe wrote: > We want to store charm URLs in mongo-db that are agnostic whether > the series is specified or not. For example, in a bundle, a service > is free to specify a series in the charm name or not. That sounds slightly surprising. How do we plan to define what the bundle actually means? While having one or two types to represent the concept may be argued back and forth, there's an underlying concept that is important: one form is a loose wildcard that has to be resolved depending on context before being useful, and was originally designed to be used in command lines and the such, while the other is a more formal specification (must have a schema, must have series). Accepting the loosely defined form in a bundle seems surprising, even if it just means not having a series, given that deploying the bundle would hopefully be somewhat deterministic in terms of which distributions are being used. > I'd like to suggest that we remove the Reference type and use the URL > type throughout, allowing it to have an unspecified series > where the string form does not specify a series. > > This means that the URL type would be an exact reflection of the string > form of a charm URL. As noted above, a Reference may not have a schema as well, so this suggestion seems to imply that "foo" becomes a valid URL. Maybe having just URL could be made cleaner, though. This should be judged based on a more detailed proposal. gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Wed, 23 Jul 2014, Gustavo Niemeyer wrote: > On Wed, Jul 23, 2014 at 7:35 AM, roger peppe wrote: > > We want to store charm URLs in mongo-db that are agnostic whether > > the series is specified or not. For example, in a bundle, a service > > is free to specify a series in the charm name or not. > > That sounds slightly surprising. How do we plan to define what the > bundle actually means? > > While having one or two types to represent the concept may be argued > back and forth, there's an underlying concept that is important: one > form is a loose wildcard that has to be resolved depending on context > before being useful, and was originally designed to be used in command > lines and the such, while the other is a more formal specification > (must have a schema, must have series). Accepting the loosely defined > form in a bundle seems surprising, even if it just means not having a > series, given that deploying the bundle would hopefully be somewhat > deterministic in terms of which distributions are being used. This is driven by requirements from ecosystem and users where bundles define a 'solution'. A mongodb-cluster bundle doesn't need to be updated every time a new revision comes out, or even if a new series comes out. It is a usable solution regardless. Bundles can be as specific as they wish to be, however requiring them to define charms specifically reduces their reusability and causes us to be less flexible. We also have to worry about historical usage as we've always supported the vague behaviour and many of the current of bundles take advantage of it. -- Rick Harding -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On 23 July 2014 11:51, John Meinel wrote: > Casey is the one to talk to. I'm not sure if he was just trying to be > conservative, but he did intentionally not want to pun the types. I think he > wanted to be clear that things passed to *these* API calls must be fully > formed, while ones passed to *those* could be expanded. > We could do what you say, but it starts meaning every API that takes a URL > has to start worrying about expanding it late. > Since it is no longer "charm.URL" but really "charm.MaybeURL". Luckily there are no APIs that currently take a URL type as an argument. The only ones that do (ServiceDeploy, for example) currently use a string, because they allow a partially specified URL, so can't use charm.URL. Note that not even charm.URL is itself currently necessarily fully specified - the revision can be omitted. On 23 July 2014 13:01, Gustavo Niemeyer wrote: > On Wed, Jul 23, 2014 at 7:35 AM, roger peppe wrote: >> We want to store charm URLs in mongo-db that are agnostic whether >> the series is specified or not. For example, in a bundle, a service >> is free to specify a series in the charm name or not. > > That sounds slightly surprising. How do we plan to define what the > bundle actually means? The charm URL in a bundle means exactly what it would mean if you typed it in a juju deploy command. That is, it is dependent on the charms available at bundle deploy time. An unspecified revision uses the latest available revision; an unspecified series uses the latest LTS series that's available for the charm. > As noted above, a Reference may not have a schema as well, so this > suggestion seems to imply that "foo" becomes a valid URL. Yes, both ParseURL and ParseReference both already allow the schema to be omitted, defaulting to "cs:" if so. > Maybe having just URL could be made cleaner, though. This should > be judged based on a more detailed proposal. I do believe having just URL would be significantly cleaner. What area would you like to see more detail on? cheers, rog. -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Wed, Jul 23, 2014 at 9:13 AM, Richard Harding wrote: > This is driven by requirements from ecosystem and users where bundles > define a 'solution'. A mongodb-cluster bundle doesn't need to be updated > every time a new revision comes out, or even if a new series comes out. It > is a usable solution regardless. Bundles can be as specific as they wish to > be, however requiring them to define charms specifically reduces their > reusability and causes us to be less flexible. When you design a system there's always a tension between what people need and what they think they need. Speaking of a different area close to our hearts, programming languages such as Perl evolved with the author hearing user requests.. developers, even fairly experienced ones, tend to want to pack as much power on as few key strokes as possible, and a language that has a very high rate of meaning per key stroke is often deemed as an expressive and powerful programming language. That feeling presumes that there is a high cost in typing a bit more, but as time passes we're learning that the semantic load has a more relevant cost on itself, and simpler but consistent primitives often yield better results. Going back to bundles, not having to update a bundle when a new, entirely different, release of Ubuntu comes out, is of course much more expressive, and people love expression, but carries with it a relevant semantic load. It also means neither we nor anybody else has any idea about what people actually get when they deploy a bundle, and whether the bundle will even work tomorrow once a new major upgrade is pushed to the repository. Our focus should not be to encourage that, but to help people express what they mean clearly and easily. If they want a new release of the bundle with a slightly different meaning, that should be trivial, but it should not be trivial to express lack of clarity. > We also have to worry about historical usage as we've always supported the > vague behaviour and many of the current of bundles take advantage of it. Yes, bundles were very organically developed. But I won't re-raise that rant. gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Wed, Jul 23, 2014 at 9:59 AM, roger peppe wrote: > The charm URL in a bundle means exactly what it would mean if > you typed it in a juju deploy command. That is, it is dependent > on the charms available at bundle deploy time. I would fix that instead. > I do believe having just URL would be significantly cleaner. > What area would you like to see more detail on? The code review, but it doesn't have to be me judging it. gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Thu, Jul 24, 2014 at 12:59 AM, roger peppe wrote: > On 23 July 2014 11:51, John Meinel wrote: > > Casey is the one to talk to. I'm not sure if he was just trying to be > > conservative, but he did intentionally not want to pun the types. I > think he > > wanted to be clear that things passed to *these* API calls must be fully > > formed, while ones passed to *those* could be expanded. > > We could do what you say, but it starts meaning every API that takes a > URL > > has to start worrying about expanding it late. > > Since it is no longer "charm.URL" but really "charm.MaybeURL". > > Luckily there are no APIs that currently take a URL type as an > argument. The only ones that do (ServiceDeploy, for example) > currently use a string, because they allow a partially specified > URL, so can't use charm.URL. > WTF? We have charm.InferURL *and* charm.ParseReference for the cases where sloppy input is reasonable -- charm.ParseURL should *parse a URL*, and fail if not given a string that represents a URL. And ServiceDeploy should *definitely* not accept sloppy input: I'm certain we had exactly this argument back in the early API days, and looking at the code STM to imply that a reasonable attempt was made to follow my instructions -- ie ServiceDeploy will fail if the charm revision is not specified, as it *should* if the schema or series or anything else is left unspecified. Anyway, someone changed some code and introduced a bug that got missed in review; it happens. However, someone now needs to go into the charm package and add some tests to ParseURL to ensure it errors as it should when schema, or series, is omitted. Note that not even charm.URL is itself currently necessarily fully > specified - the revision can be omitted. > It's a valid charm URL with or without a revision. It is *not* a valid charm URL without a schema, series, or name. These definitions have not changed in years. On 23 July 2014 13:01, Gustavo Niemeyer wrote: > > On Wed, Jul 23, 2014 at 7:35 AM, roger peppe wrote: > >> We want to store charm URLs in mongo-db that are agnostic whether > >> the series is specified or not. For example, in a bundle, a service > >> is free to specify a series in the charm name or not. > > > > That sounds slightly surprising. How do we plan to define what the > > bundle actually means? > > The charm URL in a bundle means exactly what it would mean if > you typed it in a juju deploy command. That is, it is dependent > on the charms available at bundle deploy time. > An unspecified revision uses the latest available revision; > an unspecified series uses the latest LTS series that's > available for the charm. > This makes me shudder -- why would we want to encourage distribution of a format encoding vague guesses as to validated best practice, rather than encoding *actual* validated best practice? `juju deploy foo` is great for demos and exploration, but I'd expect any serious user to value reproducibility enough to specify the actual charm precisely; and for them to expect that the bundles they use would do the same. > As noted above, a Reference may not have a schema as well, so this > > suggestion seems to imply that "foo" becomes a valid URL. > > Yes, both ParseURL and ParseReference both already allow > the schema to be omitted, defaulting to "cs:" if so. > > > Maybe having just URL could be made cleaner, though. This should > > be judged based on a more detailed proposal. > > I do believe having just URL would be significantly cleaner. > What area would you like to see more detail on? > Fuzziness over the meaning of URL appears to have already broken ParseURL, and hence ServiceDeploy; let's not double down on that. Even if the bundle format's sloppiness is already entrenched, that's no reason for juju-core to give up and just start accepting whatever -- the clients can damn well rewrite sloppy bundles before using them with juju, just as they should, and do (or at least *did*... who knows what's happened recently since validation got broken) rewrite sloppy charm URLs before passing them to juju. Cheers William -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Wed, 23 Jul 2014, Gustavo Niemeyer wrote: > Going back to bundles, not having to update a bundle when a new, > entirely different, release of Ubuntu comes out, is of course much > more expressive, and people love expression, but carries with it a > relevant semantic load. It also means neither we nor anybody else has > any idea about what people actually get when they deploy a bundle, and > whether the bundle will even work tomorrow once a new major upgrade is > pushed to the repository. Our focus should not be to encourage that, > but to help people express what they mean clearly and easily. If they > want a new release of the bundle with a slightly different meaning, > that should be trivial, but it should not be trivial to express lack > of clarity. I think it's important to look at this from the viewpoint of the effort required to have the non-specific charm definition in a bundle. - If the user builds their bundle from the Juju GUI all charms are completely qualified. Series, name, and version are at a firm definition and the bundle is completely safe to redeploy and is repeatable. - If the user exports an existing live environment, the above is true as well. Everything is firmly qualified. - If the user hand constructs a bundles.yaml file they are able to type whatever they desire into that `charm:` field. I would argue that it's quite reasonable for the user to be able to copy/paste a command line target and expect it to work just as their `juju deploy` cli did. - If the user were to take a bundles.yaml from the GUI and proceeds to edit it to remove the series specifier, they're expressing their desire explicitly and it's something we know stakeholders require (ecosystem team). Give this, the vast majority of bundles are fully qualified. In the JAAS scenaraio, where users are building bundles through the webui and they're stored directly, we can encourage/enforce qualified names by controlling the UI interaction. However, we want to maintain the escape hatch for the stakeholders that require it. If we want to remove this support then we need to involve them in a process to come together, agree on the decision, and update our existing tools to not permit it. > > We also have to worry about historical usage as we've always supported the > > vague behaviour and many of the current of bundles take advantage of it. > > Yes, bundles were very organically developed. But I won't re-raise that rant. Ok, I'll let history be history. However, if you have any feedback on bundles I'd love to have a chance to hear your thoughts as our team is working on moving bundles and their definition to the charmstore and will bring up moving to being a first class juju-core entity in Nuremburg. -- Rick Harding -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: series-agnostic charm URLs
On Fri, Jul 25, 2014 at 10:01 AM, Richard Harding wrote: (...) > Give this, the vast majority of bundles are fully qualified. In the JAAS > scenaraio, where users are building bundles through the webui and they're > stored directly, we can encourage/enforce qualified names by controlling > the UI interaction. Sounds great, and as one more reason to be strict. > Ok, I'll let history be history. However, if you have any feedback on > bundles I'd love to have a chance to hear your thoughts as our team is > working on moving bundles and their definition to the charmstore and will > bring up moving to being a first class juju-core entity in Nuremburg. Sure, let's talk live there. gustavo @ http://niemeyer.net -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
