On 12 August 2015 at 18:38, Marco Ceppi <marco.ce...@canonical.com> wrote:
> I can see how this helps for discoverability for those already pretty > intimate with the code base, but a lot of the key complaints we've gotten > around new authors getting started is there is just an overwhelming amount > of code. If contrib just became charmutils or some other big umbrella I'd be > better with that, though I still don't think it solves the problems of > versioning. If the BIg Data charmers want to publish a backwards > incompatible change how does that get broadcast from a contrib directory? I think the documentation at https://pythonhosted.org/charmhelpers/ is key here. To aid new authors, we need a reference guide of all the helpful tools. Even if charm-helpers is split up, I think that we still need some way of generating this reference as a cohesive whole. If not, we are back where we started with the original problem charm-helpers was created to solve - a pile of libraries that interoperated poorly and developed in isolation, often wastefully reimplementing each others work. I personally don't see any advantages in splitting charm-helpers into two separate branches. I think it will increase the maintenance burden and raise the barriers to entry. Splitting it even further, into core, extras, contrib_a, contrib_b, etc. even more so. From my experience, this approach just spreads the maintainers too thin, with some components badly maintained and others totally unmaintained. I do agree that backwards compatible changes are an issue. Components certainly can be distributed as separately versioned wheels, so charms depending on bigdata 1.x don't explode when bigdata 2.0 is released. This is not tied to the branch structure used. One tree, many trees, nested trees, all can do it. I don't think the problem is that there is too much code or too many features. I think a problem is we have a bad and ill-defined structure. We have core. And we have contrib, which started as a place to hold the mess of pre-charmhelpers packages we were trying to integrate but has grown in scope. And we have other stuff, which is generally useful but not hard-core enough to be considered core. And we have stuff in core that really isn't core. And if you are disagreeing with me at this point, it is likely because people define 'core' subjectively, and why I consider it ill-defined. I think another problem is that code lacks ownership. I'm all for pulling out any code no people or teams are prepared to own. Someone needs to take responsibility for reviews, ensuring that their standards and are met or exceeded and each landing improves both the component and the project as a whole. And owners get to express their opinions about style and keep the component cohesive. So how about we say that everything in charmhelpers will go, except for things people are prepared to claim ownership over. I'll claim my bits, you can claim core.hookenv and core.host, hopefully someone else will claim contrib.charmsupport.nrpe... and when we see what we have we can try and define the package heirarchy and namespaces we need. in contrib, I'm actively using contrib.network.ufw and contrib.charmsupport.nrpe. I also had a brief liaison with contrib.unison, abusing it to authorize ssh access between units rather than for its intended purpose. And I might need to use contrib.ssl shortly. peer_storage has been superseded by Juju leadership. >> And your test harnesses are already setup for you. It is easy to add >> new cool stuff, and its tests. And those tests get run under both >> Python 2 and Python 3 with the same versions of dependencies and your >> cool stuff is much more likely to work with other peoples cool stuff. >> I know for a fact if things get spit out they will have less rigorous >> testing, and we will end up with tech debt Py2 only modules for >> example, or conflicting dependencies. > > If we provide similar scaffolding as we have now for those projects I don't > see why they wouldn't continue this way, if people don't take pride in their > projects they either don't get used or get forked and made better Its extra maintenance burden. Improvements in the scaffolding in one project don't automatically flow to the other projects. And dependencies - if we are writing code we want to coexist, we want to run with the same set of dependencies so we pick up incompatibilities. >> Could we use simplestreams for this? Or whatever the mechanism is that >> Juju uses to install the correct version of the juju tools tarball >> when provisioning a unit? > > This is a great idea, I'll look into this more since it's a common > convention already with Juju. I think Juju growing the notion of > charm-helpers as a first class citizen and the charm either declaring what > helpers it needs (language/version) or something similar could be really > useful. There's also been talk of using juju resources for this when that > feature goes live, which would be a simplestream for embedded objects in the > charm. Just guessing here, but it may even be possible to not involve Juju. A small bootstrap may be able to discover and access the resources the same way that juju does the tools, then we don't need to wait for juju feature work. We just need to write the botstrap and be able to publish the tarballs (and document how to private clouds publish the tarballs). >> Slimming down. I've bitched before about too much stuff being in >> charmhelpers.core, and 'helped' by landing my own stuff I don't think >> should be core in the parent package. I tend to think >> charmhelpers.core should provide wrappers around the juju hook >> environment, matching the documented juju hook environment behavior, >> and required low level primitives you need for writing charms. It is >> the foundation for the rest of charm-helpers, which can provide higher >> level and friendlier interfaces. I believe that these higher level >> tools should be Pythonic. So your cli wrappers can publish >> charmhelpers.core for people insane or uneducated enough to write Bash >> charms, and the trivial charmhelpers.core would be trivially ported to >> other high level languages. But porting the Python decorators provided >> by charmhelpers.coordinator or the Pythonic datastructures provided by >> charmhelpers.context would not be helpful. Nor would porting >> charmhelpers.core.templating, since it is currently Jinja2 specific >> (and why it should be charmhelpers.templating). > > I agree with the first half of this statement, there is too much in core and > core should be basically hookenv. I think, however, that most of the other > top level modules you've talked about should just be their own packages. > I'll address your bash observations in the next few chunks. I believe they should be integrated into an umbrella project, such as charm-helpers. If not they will all wither and die. I don't see any of them being able to support a community. There is no point me releasing cool stuff of interest to other charmers as a separate branch or package, as they will never discover it. And if I did, bug fixes or suggestions will be rare, because it is perceived to be my project and not their project. For the Pythonic side, I'm mainly thinking of charmhelpers.context which I have up for review and have used extensively in the PostgreSQL charm rewrite. I got sick of writing relation_set/relation_get etc. everywhere, so wrote a higher level API that exposes a big chunk of the hook environment as dictionary like objects. I personally think it makes charm code much clearer, and if people agree then I think it should be in charmhelpers and charmers encouraged to use it over the low level API provided by hookenv and Juju. But if this was a 3rd party library, it would never gain traction and I doubt I would get any feedback on the design at all. I think it is helpful and interesting, but not enough for people to go out of their way do discover and use. And it is certainly not the sort of thing to port to other languages - you could do something similar, but the design would necessarily be different. -- Stuart Bishop <stuart.bis...@canonical.com> -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
