On April 11, 2016 at 4:08:51 AM, R.I.Pienaar (r...@devco.net) wrote:
----- Original Message ----- > From: "Thomas Gelf" <tho...@gelf.net> > To: "puppet-dev" <puppet-dev@googlegroups.com> > Sent: Monday, 11 April, 2016 03:30:58 > Subject: [Puppet-dev] Re: The Future of Puppet [Was: Deprecation logs] > Am 11.04.2016 um 03:01 schrieb Rob Nelson: >> Of course there's lock in. You can't click a button and go from Puppet >> OSS to Chef OSS; [...] >> Is their concern about being able to contribute to it or even fork? I >> suspect that's what most lock in concerns are really based on. > > It's about going from Puppet OSS to PE, forth and and back. No problem > with loosing the GUI or special add-ons. But the core functionality of > your CM tool, that's what they want to be free software. > Eric asked so here it is, this is my feedback with a open source user hat on. Echoing much what was said. I hope others send in their story. Thanks for the replies, the level of detail and obvious care and emotion that goes into these mails have been good to see and I am sure valuable to everyone. I hoped my mail would spark a wider conversation hopefully involving some Puppet people as well as some of the community - even those who don’t often engage and it looks like we’re off to a good start. While my earlier mail was most certainly squarely to the one side of the coin as pointed out there is another. My earlier mail shows how a company in Puppets position might think, I do not know if I am wrong or not as I have no insights into the internal convos but my gut feel is I am not far off. But I specifically wanted to state that side first because it’s important to understand as it frames expectations one should hold. The other side of the coin for me is that of a open source user and I think the outlook is particularly dire. Excuse the following wall of text but we have to talk about how we feel and what the current reality means to us as a class of user. Open Source users often see the existence of some open source tool as a implied contract that it will always be so, and their adoption of it is based on that perception. This is encoded in the very licences we hold deer and when that inevitably turns out not to be realistic business model the companies involved will do what they can to survive. Inevitably this ends up breaking the ‘implied contract’ the open source user rightly or wrongly felt they had - cue much anguish and resentment. The company will then go on to explain themselves and make commitments - we will keep the core product open but innovate on paid for enterprise features like RBAC and very featurefull GUIs. We will not work to exclude other makers of tools and your home grown tooling, we will make the extension points open and APIs open for others to innovate on. I don’t have the actual quote to hand as I am sat on a plane but this seems to me more or less the story we were told when PE became a thing and that seemed good and a model worth supporting. After all what would be open source is not unlike most other open source and we are not averse to applying some duct tape and glue to make things work. Fast forward to now and the situation is not like this at all. Today we have the puppetserver product touted as the only way forward and not only does it have PE only APIs its core operability features are PE only too. Performance metrics and atomic deploys. No longer is it a matter of if you put in the effort and build the tooling you can use the Open Source product but we will sell world class tooling. We’re now in a world where the ability to monitor the product and gain the insight you need to scale it requires you to adopt a per node based payment model. Being able to monitor a service only when you pay for it. I am not sure I've seen a worse model for the 'free' users. Now tools like Foreman cannot compete because they cannot monitor the master and they cannot have atomic deploys. Who knows whats future PE only APIs will come? I'd bet things around file serving, catalog life cycle management etc. Now similarly open source users cannot compete because they cannot gain the insights they need to scale this java blackbox, even if they invest the time to learn the JVM, the software in the JVM is actively resisting their efforts of gaining operational insight. Their own tooling cannot compete against PE because some APIs are PE only. Personally as a open source user I use ‘apply’ - I decided during my P4 migration to look again at using a master - puppetserver - and was all set to move machines onto it that weekend when the metrics blog post came out. The puppetserver VM got immediately deleted. This is not a future direction that is cooperative or long term tenable for me. If you’re a open source user you need to look very carefully at the messaging and feature set and you can really only come to one conclusion and that is that to Puppet you’re just not a priority. Less than a priority you’re effectively being locked out and worked against and lack the ability to make a well monitored CM system if you adopt open source puppet. I hope the metrics situation will change - there has been other PE-first features and they came to Open Source so its not unprecedented but in this case it will pay to wait and see since if these features are not coming to Open Source then the whole is unusable. It’s a pendulum that swings, whats the right answer and right balance is not black and white. It’s a fiendishly difficult business problem and you have to commend Puppet for even bothering with Open source at all this point so I try and see the good intentions towards the open source user base. And while I am quite negative in this mail I do not think they are maliciously working towards excluding us. Hopefully Puppet realise in these cases the pendulum swung too far and will correct the course. Either way, the signal to Open Source users is that you need to really deeply consider your options post Puppet 3. It’s undeniably a high risk situation. It might well be the most significant decision you make in the next 5 years of your infrastructure and of your career. On the points of migrating to P4 from earlier Puppet. Its a major problem. Not only did the entire system change - in every respect, every component has had a major rework and rethink from 2.6 days but the language have also in effect become a new language. Not surprising for a 10 year old product. Yes there is some semblance of backward compatibility on the language level. But there is no doubt that New Puppet is exactly that. It’s a entirely new product with a backwards translation layer. Ask yourself does just updating to P4 and running your existing code achieve much? No it doesn’t it means you’re still running an outdated code. But getting to the point where you’ve updated Puppet to 4 and are now on a new set of software but still not rewrote your code to New Puppet is a colossal task since the world we live in is so resistant to upfront testing and rollback. And then you’ve still achieved more or less nothing because now you also have to consider basically rewriting your code to take advantage of the new features. Worse before you can even begin to look at refactoring your code you need to learn an entire new Puppet language and set of technologies and their environments like the JVM, new deployment locations, new packaging models new everything - worse ones that resist introspection. All while there is scant information on new patterns, no new style guides, the supporting system like rspec-puppet doesn’t support P4. Documentation is lacking or embryonic and so forth. rspec-puppet especially is a bad situation since many who have invested in a tested infra will loose the value. Ditto for puppet strings etc. I hope my blog posts and upcoming talks at conferences help those who do want to take on this task, but it’s a task much larger than that. Meanwhile other tools like mcollective are effectively abandonware with no viable alternative. Feedback I’ve had on some of the modern Puppet code I’ve put out is that it does not look like Puppet at all. It’s a new thing, literally your foundations of your 10 story building needs replacing if you wish to move forward. In the world IT admins live in you’re asking to replace the foundations while keeping the building standing, a tall order at best. (see https://github.com/ripienaar/puppet-classifier for modern puppet code) It’s truly a perfect storm of a fuckup when looked at from a Ops perspective as an open source user. So while I made a point of calling out the ‘resistant to change’ user, there are several version of these. You have your people who just won’t change - I have no sympathy for them as was clear. But you also have those who wish to change and who look critically at the P4 vs < P4 and come to the correct conclusion that this is a high risk multi month if not multi year project. They correctly deduce that Puppet is now so big that you need a dedicated team managing the manager. Something many of us just can’t afford or indeed who wants to spend their days doing nothing but manage Puppet? For many just getting Puppet in has been a massive politically charged internal struggle. It’s a constant battle between forward thinking admins, old style admins, developers and management who want to instead see features delivered and not management tools managed. Having just won the battle, one that was in many cases a battle fought out of having a deep respect and affinity for the company, its founder and products, now they find they’re squarely screwed and have to again invest possibly years and again have the fight with everyone the exact same battles and again have to budget for years of disruption. While it’s clear the Puppet won’t slow down its change curve. They correctly conclude that they WANT to change but cannot afford to. They also correctly conclude that the task is so big and Puppet moving so fast that P5 will be around long before they are on P4. And so the cycle will just start again. Its realistic to think the opportunity cost of continuing with Puppet is simply to high a price to pay for what it brings when you’re on <P4 today. Staying is no option as it will be unsupported and we have to consider security updates and such. Change they have to change, rock and a hard place. There is not much more frustrating and insulting than a company who puts you in that position and needless to say if the effort has to be put in anyway they will look elsewhere. I suspect the paying customer base will approach the transition similarly. I have no data on this but would be interesting to know how many old PE users are transitioning to P4 based PE once the older support cycle ends. I am deeply sympathetic to these people and it’s sad to say but where we are today is that they are simply SOL and there is no light at the end of the tunnel when you can’t even have a hope of gaining deep insight into New Puppet without paying. So given a choice to rewrite all my code to P4 I instead came to a hybrid approach: * Move my actual things I run into containers - web sites, bind, smtp servers, etc all in containers. Not puppet managed. * As machines move to CentOS 7, make them P4. * When on P4 only manage with Puppet the low hanging fruit like resolvers, sudo, users, sensu and bacula. Most from the Forge. Why do it this way? Well my environment is a R&D environment so I enjoy looking towards new tech and this is the world I want to innovate in but more than that where my P4 is now it’s doing very little. So little that any competitor in the CM market can step up and replace P4 for me really quickly as it appears more and more likely this is an important consideration. I felt I had no choice but to approach it this way because as a Open Source user the future is not bright at all - as per my previous mail I am just not a priority - and I can’t really blame the company for feeling this way. I’d use PE as I have nothing against it but as this is a R&D network the $3000 licence for my node count I’d need does not seem like a good way to spend my own pocket money. I myself am bout 50/50 through my P4 migration and it’s been a daunting task to say the least - though I am under no time pressures and am taking the time to gain a deep insight into what P4 is, what motivates its existence and to regain the deep understanding I had of < P4 and so forth. I am though a consultant in this space and Puppet is creating a massive customer base for what I do so it’s not like its wasted effort for me, as was pointed out, I am not typical though. Given the amount of money out there for consultants in a world Puppet literally create from nothing - wearing my consultant hat I’d say keep going as you were. And Puppet does really good things to support the consultants in their space via partner programs and such. P4 on it’s own is a brilliant tool brilliantly engineered and the language innovations are really good. The engineering rigour is orders of magnitude better than we had before and it really shows in the end result. I have nothing but positive things to say for P4 and the team that created it. Hi Arri, it’s been a while :) I agree with most of your rant until here. While I think the engineering quality of what Puppet is producing is indeed miles ahead of where it was, the product design itself (including language innovations) are core of the problem to me. They aren’t, as best as I can tell, designed with a particular customer in mind - I certainly can’t find the sweet spot customer for whom PE is the right product. For every quality innovation, there are 2-3 seriously breaking changes that not only indicate a lack of understanding of how Puppet is currently managed in the field (what the ecosystem is doing, what typical deployments look like, etc), but also that are likely to lead the new, inexperienced, middle-of-the-road customers PE used to be targeted at down entirely incorrect paths. The current state of the Classifier and Code Manager are examples of this. >From my own experience, customers are losing confidence that Puppet >understands their world. I certainly am no longer certain that it does. Both >from the rapid iteration of change in order to add functionality that doesn’t >visibly improve their work but mandates frequent refactoring to moves like >launching products without support for operating behind proxies, the >perception that Puppet is a tool by admins for admins is waning pretty >seriously. And most new users, as you note, are embracing the wrong kind of >configuration tools specifically to get around it. -Eric It’s unfortunate P4 is combined with a supporting eco system that while innovative, powerful and potentially game changing is making it irrelevant to me as a class of user since they are more and more approaching things in a way that locks me out of the future I - and many like me - helped create. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/371534503.181383.1460372927683.JavaMail.zimbra%40devco.net. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/etPan.570bc6a4.1788828a.206%40gmail.com. For more options, visit https://groups.google.com/d/optout.
