Yeah, this is my thinking too. Regards,
Rob... > On 27 Jul 2017, at 16:20, Carlos Santana <[email protected]> wrote: > > Being thinking more about this, and having a core set of packages included > by default is very useful and I think critical for open source integration > and continues integration in the open. > Also keeping large actions away from the system for folks that just want to > get started and kick the tires using the embedded packages > > I think we could have a mid way alternative: > > What about if we create new kinds every time we want to pickup an update > for new language update but at the minor (y) level (version x.y.z). > Also include a very small set of core packages, for example request for > nodejs very stable and popular package > > We create a new kind to pick up a new minor update and at that time will > pickup the latest version of the packages. > For patch updates, at least for security updates this are done at the patch > level, will update the current kind to update the patch level of the > language. > > So I think this aligns with what Rob mentioned about trying to keep kinds > versions at the minor level. > > Taking nodejs as an example: > we have nodejs:6 (this is set to 6.9) > let's create a new kind for node nodejs:6.11 > This will include latest version of 6.11 and latest versions of packages at > the time of creating of the new kind nodejs:6.11 > If there are nodejs patch updates to pickup for 6.11 we update without > creating a new kind will be kept at nodejs:6.11 with the packages pinned to > the versions that they shipped. > I think nodejs moves a bit faster than the other languages so we could > pickup a new minor nodejs:6.12 or 6.14 when ever the community thinks > there is a minor update from nodejs worth the update to create a new kind. > > There might be a gap, I think it's OK for those corner cases that an user > wants to use the latest nodejs version but use old version of the packages, > or use the latest version of the package but an old version of nodejs. I > think for this cases user would need to pinned and include the exact > version of the modules to include and that's OK, I think this will be a > normal thing for people going into production and pinning and controlling > dependencies. > > I think it's hard to come up with silver bullet for all languages, as every > language updates at a different speed, and dependencies are more slowest > pace than others. > > > > > > > >> On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <[email protected]> wrote: >> >> One thought I have being debating is that the open source project doesn't >> include any packages going forward. >> The project will have an image that is small and common denominator that >> the operators deploying decide which packages to add on top. >> >> For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017), >> >> Will keep the version of language nodejs 8 up to date to the latest as >> official releases come out from nodejs >> Operators like IBM and Adobe, RH, others, can base their custom images for >> their offerings on the same base image, adding packages that they want base >> on their offering. >> >> Same for swift new kind swift:4, and so on. >> >> This will be a general policy, as every language has their peculiarity and >> probably we need our runtimes SMEs to talk about the risk on updating or >> not updating per language. >> >> dealing with user level packages is always a pickle for example today we >> have nodejs:6 and we have a set of users that benefit from updating the >> packages to the latest latest, and could have users that they depend on a >> current version of the package and updating the package on them could cause >> problems. If I had my way I would say thought luck for using the embedded >> package and not packaging or doing a bundle build, but serverless is on >> baby years of adoption and I don't want to be thought yet :-) >> >> So for nodejs8 we say thought luck no embedded packages, but will do >> something to make it super easy will have the system help you build/compile >> your action and include your dependencies, these system actions that allow >> users to provide only their code/git-url, and package.json/package.swift, >> and these actions will compile/npm install etc.. and install your action >> including the declared dependencies. >> >> >> >> >> >> On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <[email protected]> >> wrote: >> >>> Hi Rob, >>> >>> Re “I wonder how many deployments roll their own action containers?” >>> >>> I spoke too general (implying what most OW deployments would want). What >>> I can say is that our team wishes to be in control of the libraries that go >>> into the action containers, that’s why we need to roll our own. >>> >>> Agree with the approach you have outlined on patch and minor releases. >>> >>> The topic of deprecation IMHO is the same discussion as when to upgrade >>> language runtimes in action containers: at least for us the deprecation >>> policy is something that we need to be in control of (which is a given as >>> we roll our own containers). >>> +1 to the deprecation tooling you hint at regarding not creating actions >>> for deprecated containers (this is needed regardless of any deprecation >>> policies). This is not covered by the issues you linked to, is it? >>> >>> Cheers >>> Michael >>> >>> >>> >>>> On 14/07/17 19:08, "Rob Allen" <[email protected]> wrote: >>>> >>>> Hi Michael, all, >>>> >>>> I wonder how many deployments roll their own action containers? I >>> suspect not many. >>>> >>>> I agree with you both that it would be nice to keep packages (modules) >>> up to date on a regular basis within a kind makes the most sense and a >>> regular schedule would be nice, especially if there's a documented way to >>> do the update for each kind. >>>> >>>> I think we should also keep the language of the kind up to date at the >>> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 - >>> unless there are language kinds that break semver at the patch level? (In >>> which case my head may explode!) >>>> >>>> For minor releases, it's a little more wooly at the moment at least. For >>> NodeJS, [PR 2078 [1] simply updates the `node6` kind from 6.9.1 to 6.11.0 >>> without creating a new new "kind". However, for Swift, [PR] [2120] gave us >>> a new `swift:3.1.1` kind rather than updating the `swift3` kind which runs >>> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, so >>> maybe it should have been called `swift:3.1`? >>>> >>>> My thoughts on this is that we should create new kinds for minor >>> language releases on the principle of least surprise for the container >>> users. >>>> >>>> I would like to fairly aggressively deprecate old ones to stop new >>> actions being created for out-of-date language versions, but as deprecating >>> a kind prevents invoking any actions for that kind that's not possible >>> unless some work is done to allow invoking actions on a deprecated kind, >>> but not creating or updating them would be necessary for that to be a >>> possibility. >>>> >>>> I think GitHub Issue [319] [3] is relevant in terms of the tooling >>> around this. >>>> >>>> >>>> Regards, >>>> >>>> Rob... >>>> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2078 >>>> [2]: https://github.com/apache/incubator-openwhisk/pull/2120 >>>> [3]: https://github.com/apache/incubator-openwhisk/issues/319 >>>> >>>> >>>>> On 14 Jul 2017, at 16:27, Michael Marth <[email protected]> >>> wrote: >>>>> >>>>> Rob, all, >>>>> >>>>> I largely echo what James wrote below (except moving this discussion >>> off-list :) ) >>>>> >>>>> Incidentally, we had a very similar discussion within my team recently >>> on how to handle this topic for our OW deployment. The bottom line is that >>> the trade-off between stability and being up-to-date can only be decided in >>> a given business context of the OW deployment. >>>>> We (as a project) do not have that context and cannot really make the >>> decision, IMHO. In that light the only reasonable policy in my view is to >>> strive to keep everything updated as best as we can, like monthly or every >>> second month, etc. >>>>> If we miss a beat (i.e. don’t immediately update to latest release) I >>> don’t think that’d be critical. Like James said, most OW providers will >>> have their own runtimes anyway and would (should) take the latest if they >>> need it. >>>>> >>>>> Cheers >>>>> Michael >>>>> >>>>> >>>>>> On 13/07/17 11:02, "James Thomas" <[email protected]> wrote: >>>>>> >>>>>> Good ideas Rob. I had a similar issue when looking at the Swift >>> runtime >>>>>> recently. >>>>>> >>> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E >>>>>> >>>>>> Would it be possible to have a scheduled upgrade process for installed >>>>>> modules? Once every four, six or eight weeks? If OpenWhisk did start >>> to >>>>>> produce "releases", it could tie in with that. >>>>>> >>>>>> I'd guess that most people using the built-in packages are more >>> kicking the >>>>>> tires than building production apps. Once you start being a >>> production app, >>>>>> you'll want to explicitly bundle and control your app dependencies. >>> I'd +1 >>>>>> on being more aggressive with upgrading module versions. >>>>>> >>>>>> I'd like to have a Github issue to follow for this, I find it easier >>> than >>>>>> the mailing list. >>>>>> >>>>>>> On 13 July 2017 at 09:33, Rob Allen <[email protected]> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> On the PHP PR, @rr [commented] [1]: >>>>>>> >>>>>>>> The built in packages are convenient - less zip files for the >>> initial >>>>>>> ramp up. But it creates a maintenance issue: when do you pick up >>> updates to >>>>>>> the packages (minor/patch level only?) and not break existing >>> actions using >>>>>>> the "kind". That is: is the kind itself semantically versioned? >>>>>>> >>>>>>> This applies to all kinds and so probably should be discussed project >>>>>>> level and ideally we should document how this is handled. >>>>>>> >>>>>>> There are two things here: >>>>>>> >>>>>>> 1. The language runtimes release patch updates for minor versions. >>> e.g. >>>>>>> PHP `7.1.7` will become `7.1.8` next month with a small number of >>> bug fixes >>>>>>> including crashers and possibly security fixes. >>>>>>> >>>>>>> 2. Each kind bindles a number of packages via the language's standard >>>>>>> package management system: Swift Package Manager for Swift, NPM for >>> NodeJs, >>>>>>> etc. The projects that produce these packages update them with new >>> versions >>>>>>> minor and patch versions. >>>>>>> >>>>>>> The tension is obviously between keeping updated for fixes vs the >>> risk of >>>>>>> breaks due to a project's inability to keep BC between patch >>> versions. e.g. >>>>>>> the NodeJS kind comes with the `async v2.1.4` package. However >>> `v2.1.5` of >>>>>>> that package fixes a stack overflow issue. Should our actions have >>> that >>>>>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, >>> but the >>>>>>> latest is `v3.6.0` which is needed for non-experimental API Gateway >>> support… >>>>>>> >>>>>>> Some questions: >>>>>>> >>>>>>> 1. Should we update the language runtime for a kind for a patch level >>>>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the >>> latest >>>>>>> `6.9.5`? >>>>>>> 2. Should we ever update the language runtime for a kind for a minor >>> level >>>>>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the >>> latest >>>>>>> `6.11.1`? >>>>>>> 3. Should we ever update the packages in a kind to the latest patch >>> level >>>>>>> or minor level? >>>>>>> 4. What's our policy when a security issue is published for a >>> language or >>>>>>> a package that we ship in a non-deprecated kind? >>>>>>> >>>>>>> Whatever the answers are, I think we should document them clearly >>>>>>> somewhere. >>>>>>> >>>>>>> >>>>>>> Also, I've started this conversation as a mailing list topic as it's >>> a >>>>>>> "policy" thing. Given my previous comments on mailing lists, should >>> I also >>>>>>> create a GitHub issue prefixed with "Discussion" to provide more >>> visibility >>>>>>> in order to garner wider community input? >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Rob... >>>>>>> >>>>>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415# >>>>>>> issuecomment-314716101 <https://github.com/apache/ >>>>>>> incubator-openwhisk/pull/2415#issuecomment-314716101> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> James Thomas >>>> >>>> -- >>>> Development thoughts at http://akrabat.com >>>> Daily Jotter for macOS at http://dailyjotter.com >>>> >>> >>
