That sounds good. I've been thinking about reviewing the existing packages and trying to propose a new list based upon real-world usage and relevance. Looking at the current Node.js list: https://github.com/apache/incubator-openwhisk/blob/master/core/nodejs6Action/Dockerfile#L33
It has packages like "express" amongst others which I can't see a use for? On 27 July 2017 at 15:20, Carlos Santana <[email protected]> wrote: > Being thinking more about this, and having a core set of packages included > by default is very useful and I think critical for open source integration > and continues integration in the open. > Also keeping large actions away from the system for folks that just want to > get started and kick the tires using the embedded packages > > I think we could have a mid way alternative: > > What about if we create new kinds every time we want to pickup an update > for new language update but at the minor (y) level (version x.y.z). > Also include a very small set of core packages, for example request for > nodejs very stable and popular package > > We create a new kind to pick up a new minor update and at that time will > pickup the latest version of the packages. > For patch updates, at least for security updates this are done at the patch > level, will update the current kind to update the patch level of the > language. > > So I think this aligns with what Rob mentioned about trying to keep kinds > versions at the minor level. > > Taking nodejs as an example: > we have nodejs:6 (this is set to 6.9) > let's create a new kind for node nodejs:6.11 > This will include latest version of 6.11 and latest versions of packages at > the time of creating of the new kind nodejs:6.11 > If there are nodejs patch updates to pickup for 6.11 we update without > creating a new kind will be kept at nodejs:6.11 with the packages pinned to > the versions that they shipped. > I think nodejs moves a bit faster than the other languages so we could > pickup a new minor nodejs:6.12 or 6.14 when ever the community thinks > there is a minor update from nodejs worth the update to create a new kind. > > There might be a gap, I think it's OK for those corner cases that an user > wants to use the latest nodejs version but use old version of the packages, > or use the latest version of the package but an old version of nodejs. I > think for this cases user would need to pinned and include the exact > version of the modules to include and that's OK, I think this will be a > normal thing for people going into production and pinning and controlling > dependencies. > > I think it's hard to come up with silver bullet for all languages, as every > language updates at a different speed, and dependencies are more slowest > pace than others. > > > > > > > > On Wed, Jul 26, 2017 at 8:30 PM Carlos Santana <[email protected]> > wrote: > > > One thought I have being debating is that the open source project doesn't > > include any packages going forward. > > The project will have an image that is small and common denominator that > > the operators deploying decide which packages to add on top. > > > > For example for nodejs 8, we create the kind nodejs:8 (at LTS Oct/2017), > > > > Will keep the version of language nodejs 8 up to date to the latest as > > official releases come out from nodejs > > Operators like IBM and Adobe, RH, others, can base their custom images > for > > their offerings on the same base image, adding packages that they want > base > > on their offering. > > > > Same for swift new kind swift:4, and so on. > > > > This will be a general policy, as every language has their peculiarity > and > > probably we need our runtimes SMEs to talk about the risk on updating or > > not updating per language. > > > > dealing with user level packages is always a pickle for example today we > > have nodejs:6 and we have a set of users that benefit from updating the > > packages to the latest latest, and could have users that they depend on a > > current version of the package and updating the package on them could > cause > > problems. If I had my way I would say thought luck for using the embedded > > package and not packaging or doing a bundle build, but serverless is on > > baby years of adoption and I don't want to be thought yet :-) > > > > So for nodejs8 we say thought luck no embedded packages, but will do > > something to make it super easy will have the system help you > build/compile > > your action and include your dependencies, these system actions that > allow > > users to provide only their code/git-url, and package.json/package.swift, > > and these actions will compile/npm install etc.. and install your action > > including the declared dependencies. > > > > > > > > > > > > On Tue, Jul 18, 2017 at 4:08 AM Michael Marth <[email protected]> > > wrote: > > > >> Hi Rob, > >> > >> Re “I wonder how many deployments roll their own action containers?” > >> > >> I spoke too general (implying what most OW deployments would want). What > >> I can say is that our team wishes to be in control of the libraries > that go > >> into the action containers, that’s why we need to roll our own. > >> > >> Agree with the approach you have outlined on patch and minor releases. > >> > >> The topic of deprecation IMHO is the same discussion as when to upgrade > >> language runtimes in action containers: at least for us the deprecation > >> policy is something that we need to be in control of (which is a given > as > >> we roll our own containers). > >> +1 to the deprecation tooling you hint at regarding not creating actions > >> for deprecated containers (this is needed regardless of any deprecation > >> policies). This is not covered by the issues you linked to, is it? > >> > >> Cheers > >> Michael > >> > >> > >> > >> On 14/07/17 19:08, "Rob Allen" <[email protected]> wrote: > >> > >> >Hi Michael, all, > >> > > >> >I wonder how many deployments roll their own action containers? I > >> suspect not many. > >> > > >> >I agree with you both that it would be nice to keep packages (modules) > >> up to date on a regular basis within a kind makes the most sense and a > >> regular schedule would be nice, especially if there's a documented way > to > >> do the update for each kind. > >> > > >> >I think we should also keep the language of the kind up to date at the > >> *patch* level on the same cadence too. e.g. Node 6.11.0 to Node 6.11.1 - > >> unless there are language kinds that break semver at the patch level? > (In > >> which case my head may explode!) > >> > > >> >For minor releases, it's a little more wooly at the moment at least. > For > >> NodeJS, [PR 2078 [1] simply updates the `node6` kind from 6.9.1 to > 6.11.0 > >> without creating a new new "kind". However, for Swift, [PR] [2120] gave > us > >> a new `swift:3.1.1` kind rather than updating the `swift3` kind which > runs > >> Swift 3.0. I'm not sure that Swift 3.1.2 will need its own kind though, > so > >> maybe it should have been called `swift:3.1`? > >> > > >> >My thoughts on this is that we should create new kinds for minor > >> language releases on the principle of least surprise for the container > >> users. > >> > > >> >I would like to fairly aggressively deprecate old ones to stop new > >> actions being created for out-of-date language versions, but as > deprecating > >> a kind prevents invoking any actions for that kind that's not possible > >> unless some work is done to allow invoking actions on a deprecated kind, > >> but not creating or updating them would be necessary for that to be a > >> possibility. > >> > > >> >I think GitHub Issue [319] [3] is relevant in terms of the tooling > >> around this. > >> > > >> > > >> >Regards, > >> > > >> >Rob... > >> > > >> >[1]: https://github.com/apache/incubator-openwhisk/pull/2078 > >> >[2]: https://github.com/apache/incubator-openwhisk/pull/2120 > >> >[3]: https://github.com/apache/incubator-openwhisk/issues/319 > >> > > >> > > >> >> On 14 Jul 2017, at 16:27, Michael Marth <[email protected]> > >> wrote: > >> >> > >> >> Rob, all, > >> >> > >> >> I largely echo what James wrote below (except moving this discussion > >> off-list :) ) > >> >> > >> >> Incidentally, we had a very similar discussion within my team > recently > >> on how to handle this topic for our OW deployment. The bottom line is > that > >> the trade-off between stability and being up-to-date can only be > decided in > >> a given business context of the OW deployment. > >> >> We (as a project) do not have that context and cannot really make the > >> decision, IMHO. In that light the only reasonable policy in my view is > to > >> strive to keep everything updated as best as we can, like monthly or > every > >> second month, etc. > >> >> If we miss a beat (i.e. don’t immediately update to latest release) I > >> don’t think that’d be critical. Like James said, most OW providers will > >> have their own runtimes anyway and would (should) take the latest if > they > >> need it. > >> >> > >> >> Cheers > >> >> Michael > >> >> > >> >> > >> >> On 13/07/17 11:02, "James Thomas" <[email protected]> wrote: > >> >> > >> >>> Good ideas Rob. I had a similar issue when looking at the Swift > >> runtime > >> >>> recently. > >> >>> > >> https://lists.apache.org/thread.html/fa01baca7d97d08c855abfc69fc17a > 23e038115fcfc4f2a31d650fa1@%3Cdev.openwhisk.apache.org%3E > >> >>> > >> >>> Would it be possible to have a scheduled upgrade process for > installed > >> >>> modules? Once every four, six or eight weeks? If OpenWhisk did start > >> to > >> >>> produce "releases", it could tie in with that. > >> >>> > >> >>> I'd guess that most people using the built-in packages are more > >> kicking the > >> >>> tires than building production apps. Once you start being a > >> production app, > >> >>> you'll want to explicitly bundle and control your app dependencies. > >> I'd +1 > >> >>> on being more aggressive with upgrading module versions. > >> >>> > >> >>> I'd like to have a Github issue to follow for this, I find it easier > >> than > >> >>> the mailing list. > >> >>> > >> >>> On 13 July 2017 at 09:33, Rob Allen <[email protected]> wrote: > >> >>> > >> >>>> Hi all, > >> >>>> > >> >>>> On the PHP PR, @rr [commented] [1]: > >> >>>> > >> >>>>> The built in packages are convenient - less zip files for the > >> initial > >> >>>> ramp up. But it creates a maintenance issue: when do you pick up > >> updates to > >> >>>> the packages (minor/patch level only?) and not break existing > >> actions using > >> >>>> the "kind". That is: is the kind itself semantically versioned? > >> >>>> > >> >>>> This applies to all kinds and so probably should be discussed > project > >> >>>> level and ideally we should document how this is handled. > >> >>>> > >> >>>> There are two things here: > >> >>>> > >> >>>> 1. The language runtimes release patch updates for minor versions. > >> e.g. > >> >>>> PHP `7.1.7` will become `7.1.8` next month with a small number of > >> bug fixes > >> >>>> including crashers and possibly security fixes. > >> >>>> > >> >>>> 2. Each kind bindles a number of packages via the language's > standard > >> >>>> package management system: Swift Package Manager for Swift, NPM for > >> NodeJs, > >> >>>> etc. The projects that produce these packages update them with new > >> versions > >> >>>> minor and patch versions. > >> >>>> > >> >>>> The tension is obviously between keeping updated for fixes vs the > >> risk of > >> >>>> breaks due to a project's inability to keep BC between patch > >> versions. e.g. > >> >>>> the NodeJS kind comes with the `async v2.1.4` package. However > >> `v2.1.5` of > >> >>>> that package fixes a stack overflow issue. Should our actions have > >> that > >> >>>> fix? Closer to home, the NodeJS kind ships with `OpenWhisk v3.3.2`, > >> but the > >> >>>> latest is `v3.6.0` which is needed for non-experimental API Gateway > >> support… > >> >>>> > >> >>>> Some questions: > >> >>>> > >> >>>> 1. Should we update the language runtime for a kind for a patch > level > >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the > >> latest > >> >>>> `6.9.5`? > >> >>>> 2. Should we ever update the language runtime for a kind for a > minor > >> level > >> >>>> change (e.g. update the current NodeJS:6 kind from `6.9.1` to the > >> latest > >> >>>> `6.11.1`? > >> >>>> 3. Should we ever update the packages in a kind to the latest patch > >> level > >> >>>> or minor level? > >> >>>> 4. What's our policy when a security issue is published for a > >> language or > >> >>>> a package that we ship in a non-deprecated kind? > >> >>>> > >> >>>> Whatever the answers are, I think we should document them clearly > >> >>>> somewhere. > >> >>>> > >> >>>> > >> >>>> Also, I've started this conversation as a mailing list topic as > it's > >> a > >> >>>> "policy" thing. Given my previous comments on mailing lists, should > >> I also > >> >>>> create a GitHub issue prefixed with "Discussion" to provide more > >> visibility > >> >>>> in order to garner wider community input? > >> >>>> > >> >>>> > >> >>>> Regards, > >> >>>> > >> >>>> Rob... > >> >>>> > >> >>>> [1]: https://github.com/apache/incubator-openwhisk/pull/2415# > >> >>>> issuecomment-314716101 <https://github.com/apache/ > >> >>>> incubator-openwhisk/pull/2415#issuecomment-314716101> > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Regards, > >> >>> James Thomas > >> > > >> >-- > >> >Development thoughts at http://akrabat.com > >> >Daily Jotter for macOS at http://dailyjotter.com > >> > > >> > > > -- Regards, James Thomas
