When I say infrastructure I am by no means talking about the Apache Infra *people* who are asked to do an awful lot of work and they do it well. They are asked to create and maintain the equivalent of Github, Gmail, Google Forums, and other tools. It might be time for Apache to re-think this but it is a bit off subject for this thread so I’ll save it for infrastructure@. Unfortunately we can’t wait for something to come out of that discussion to decide what we do for our first Apache release.
If we remove the question of how much work it is to create a new Apache git repo what would we do? PIO (Tappingstone), Salesforce, and ActionML chose a template per repo and users who make simple mods follow this model for the reasons I listed below. Let’s not forget we’ll be talking about this again when we get to SDKs, maybe containers, and microservice refactoring as well. All of which seem natural to think of as separate repos. Let’s start the discussion with what is easier for maintainers and users? What fits current best practice better? What method fits the practical needs for releasing code best? What method scales and fits future needs better? What method will encourage 3rd party extensions like templates and sdks? What is easiest to document or automate? If we decide to have a monolithic repo, let's do it because it’s the best thing for the project. What do others think? On Aug 14, 2016, at 11:02 AM, Andrew Purtell <andrew.purt...@gmail.com> wrote: Interesting idea to try separate repos for landing to-be-donated templates. On GitHub repositories are cheap and, though not necessarily at this moment at Apache, easy to provision. If you think separate repo and lifecycle is the best way to manage templates then we PPMC plus mentors should make that happen here. I would think if the project comes to have a set of core templates used for such things as integration testing and good out of the box experience, these will in fact be coupled with changes to framework and having both in the same repo would seem like a fine idea. Just a thought. Not sure how useful it is, FWIW Anyway, rather than rant at infrastructure (or suppress one (smile)) I'd encourage you to engage in a proactive manner. Everything self service we have at the foundation today has stemmed from a pain point and constructive proposal. So if you'd like to have a repo per template and anticipate frequent repo provisioning requests to be a problem, start a discussion about self service repo provisioning (or whatever you think will work) on infrastructure@. Podlings are meant to stretch and grow the foundation as much as assimilate IP and governance practice. > On Aug 14, 2016, at 8:30 AM, Pat Ferrel <p...@occamsmachete.com> wrote: > > Important things unresolved for release: > > 1) What to do with Apache templates—my suggestion is separate repos for the 3 > reasons below. > 2) We need a Gallery page listing at least a couple converted tested > templates—couldn’t find this in the site but maybe I missed it. The UR is now > converted and tested as are the examples. > 3) AML fork healed, should be pushed this afternoon. > 4) the rest of the release magic > > PIO is rather moot without templates but if they are on a separate release > schedule like the doc site (which also needs work) we could release without 1 > & 2. I’d favor that rather than releasing with templates in a templates/ > directory. > > Anything else? How are people feeling about release? > > > On Aug 12, 2016, at 10:18 AM, Pat Ferrel <p...@occamsmachete.com> wrote: > > Independent of Apache I’d suggest each template have their own git repo as > they do now. Is this practically possible in ASF (another example of my > infrastructure rant but leave that for now). The semantics of this make > sense. The requirements for all apache or non-apache templates seem to be: > > 1) They need to be allowed to have separate release cycles. > 2) There should be only one way to install templates apache or non-apache > 3) This method should support tags and branches, and separate PRs > > Practically speaking for independent ones (the Universal Recommender will > remain independent until the issues above are resolved) the templates *will* > be a separate repo and `git pull` *will* be the method of download. To me > this implies the Apache ones should use the same pattern. > > The separate issue of rules for inclusion in Apache should include a test > requirement IMO. But inclusion in the Gallery seems a looser attachment to > the project—just one persons opinion > > > On Aug 11, 2016, at 3:42 PM, Chan Lee <chanlee...@gmail.com > <mailto:chanlee...@gmail.com>> wrote: > > I've begun working on migrating official PredictionIO templates into the main > apache repository, and there are a few points I'd like to bring to attention. > > 1) Template code will be merged under templates/ directory, and all commit > history and authorship will be preserved. > > 2) Organization namespace will change in the template code from > "io.prediction" to "org.apache.predictionio". > > 3) In order to download an engine template, users could (1) git clone the > entire repo and copy the files in templates/<some-template>, or (2) use > something like svn to download only the necessary subdirectory. I'd like to > ask what everyone thinks should be the recommended approach. The > documentation should be updated accordingly (instead of `pio template get`). > > 4) I'd like to ask if there are any ActionML/outside templates that should be > included or updated. For instance, > template-scala-parallel-universal-recommendation in PredictionIO repo seems > outdate compared to one in actionml repo. > > > Thanks, > Chan > > > > On Mon, Aug 8, 2016 at 9:16 AM, Marcin Ziemiński <ziem...@gmail.com > <mailto:ziem...@gmail.com>> wrote: > I think that this is a very good idea. Obviously it would require > configuring more concurrent builds to finish in reasonable time, what > should not be a problem. Besides, it would make the official templates be > consistent with every next release of PredictionIO and what is very > important - it would provide many different tests for the repository. > Having working templates of different types could give us more material to > test not only the reliability of PredictionIO, but also its performance > with every change. It would require adding different types of tests and > tools to gather runtime statistics, but this is something I would like to > take a look at in the future. > > niedz., 7.08.2016 o 10:21 użytkownik Donald Szeto <don...@apache.org > <mailto:don...@apache.org>> > napisał: > >> We can also make these templates part of the integration test that Chan and >> Marcin just submitted ( >> https://github.com/apache/incubator-predictionio/pull/267 >> <https://github.com/apache/incubator-predictionio/pull/267>). That way we >> can >> make commitment to included templates be part of every committer's effort. >> Just my 2c. >> >>> On Sun, Aug 7, 2016 at 9:47 AM, Pat Ferrel <p...@occamsmachete.com >>> <mailto:p...@occamsmachete.com>> wrote: >>> >>> If this sound ok, I propose the process be: >>> 1) inclusion in the gallery is a PR to the yaml gallery file. This is >>> reviewable by all but takes only one committer to approve and merge. >>> 2) submission for inclusion in the project can be a PR to the new >>> templates directory as Simon suggests. But this may require a more formal >>> vote, and come with some commitment for support and possibly >> consideration >>> of the author for committer status. >>> >>> Someone who knows Apache rules better than me may know the type of vote >> to >>> have for #2, maybe this is only informal review and single committer >>> acceptance too as long as the committer is willing to support the >> template. >>> >>> As far as the Salesforce templates marked official, I’d hope that most >>> would be donated. According to the proposed rules all we need is a PR for >>> each. And again a big thanks to Salesforce! >>> >>> >>> On Aug 4, 2016, at 2:55 PM, Pat Ferrel <p...@occamsmachete.com >>> <mailto:p...@occamsmachete.com>> wrote: >>> >>> Actually this is mostly a fine idea but I think the bigger question is >> how >>> do we treat templates in general. >>> >>> IMO the maintaining author can decide to contribute them or not and the >>> committers can decide to accept or not. For example in the case of the >> UR I >>> may decide to support and maintain it outside of Apache while some from >>> Salesforce might be submitted as donations. Donation comes at some >>> obligation. There are lots of examples in Mahout of algorithms whose >>> authors no longer support them. These are slowly being deprecated and >>> removed so I’d like to see a method that avoids this issue. >>> >>> If we allow maintainers to submit templates to the Gallery with a link to >>> code as well as support then donating the template code to Apache is >>> independent of acceptance to the Gallery. Any template donated to Apache >>> should come with some commitment by the author to support it there >>> indefinitely and perhaps even acceptance as a PIO committer—and if they >>> disappear or don’t support the template it is easy to deprecate/remove. >>> >>> That means if Salesforce wants to donate the templates it >> maintains—great. >>> Personally I'd would vote for acceptance of most of them. Then the >> support >>> link can be to the Apache user list or instructions for joining it. If a >>> maintainer wants to stay out of the Apache repo and support separately >> then >>> this is possible also. With all templates treated equally—official or >>> not—and there is a route to becoming official for non-official ones. >>> >>> BTW it might be good to remove the “official” designation in favor of >>> “Apache supported” or the PredictionIO logo or something like that. We >>> really need to encourage template development even if they are not code >>> contributions. >>> >>> >>> On Aug 3, 2016, at 3:32 PM, Simon Chan <si...@salesforce.com >>> <mailto:si...@salesforce.com>> wrote: >>> >>> Hey guys, >>> >>> I just want to start the discussion about moving previously "official" >>> PredictionIO engine templates into Apache. >>> >>> Official templates are those marked "Official" here: >>> http://templates.prediction.io/ <http://templates.prediction.io/> >>> >>> In order to move them to ASF Git, would the best way be creating >>> sub-directory under PredictionIO repo? >>> >>> Simon
