> On 11 Apr 2016, at 17:45, Eric Shamow <eric.sha...@gmail.com> wrote: > >> On April 11, 2016 at 4:08:51 AM, R.I.Pienaar (r...@devco.net) wrote: >> >> >> ----- Original Message ----- >> > From: "Thomas Gelf" <tho...@gelf.net> >> > To: "puppet-dev" <puppet-dev@googlegroups.com> >> > Sent: Monday, 11 April, 2016 03:30:58 >> > Subject: [Puppet-dev] Re: The Future of Puppet [Was: Deprecation logs] >> >> > Am 11.04.2016 um 03:01 schrieb Rob Nelson: >> >> Of course there's lock in. You can't click a button and go from Puppet >> >> OSS to Chef OSS; [...] >> >> Is their concern about being able to contribute to it or even fork? I >> >> suspect that's what most lock in concerns are really based on. >> > >> > It's about going from Puppet OSS to PE, forth and and back. No problem >> > with loosing the GUI or special add-ons. But the core functionality of >> > your CM tool, that's what they want to be free software. >> > >> >> Eric asked so here it is, this is my feedback with a open source user hat >> on. Echoing much what was said. I hope others send in their story. >> >> Thanks for the replies, the level of detail and obvious care and emotion >> that >> goes into these mails have been good to see and I am sure valuable to >> everyone. >> I hoped my mail would spark a wider conversation hopefully involving some >> Puppet people as well as some of the community - even those who don’t often >> engage and it looks like we’re off to a good start. >> >> While my earlier mail was most certainly squarely to the one side of the >> coin >> as pointed out there is another. My earlier mail shows how a company in >> Puppets position might think, I do not know if I am wrong or not as I have >> no >> insights into the internal convos but my gut feel is I am not far off. But I >> specifically wanted to state that side first because it’s important to >> understand as it frames expectations one should hold. >> >> The other side of the coin for me is that of a open source user and I think >> the >> outlook is particularly dire. Excuse the following wall of text but we have >> to >> talk about how we feel and what the current reality means to us as a class >> of >> user. >> >> Open Source users often see the existence of some open source tool as a >> implied >> contract that it will always be so, and their adoption of it is based on >> that >> perception. This is encoded in the very licences we hold deer and when that >> inevitably turns out not to be realistic business model the companies >> involved >> will do what they can to survive. Inevitably this ends up breaking the >> ‘implied contract’ the open source user rightly or wrongly felt they had - >> cue >> much anguish and resentment. >> >> The company will then go on to explain themselves and make commitments - we >> will keep the core product open but innovate on paid for enterprise features >> like RBAC and very featurefull GUIs. We will not work to exclude other >> makers >> of tools and your home grown tooling, we will make the extension points open >> and APIs open for others to innovate on. >> >> I don’t have the actual quote to hand as I am sat on a plane but this seems >> to >> me more or less the story we were told when PE became a thing and that >> seemed >> good and a model worth supporting. After all what would be open source is >> not >> unlike most other open source and we are not averse to applying some duct >> tape >> and glue to make things work. >> >> Fast forward to now and the situation is not like this at all. Today we have >> the puppetserver product touted as the only way forward and not only does it >> have PE only APIs its core operability features are PE only too. Performance >> metrics and atomic deploys. No longer is it a matter of if you put in the >> effort and build the tooling you can use the Open Source product but we will >> sell world class tooling. We’re now in a world where the ability to monitor >> the product and gain the insight you need to scale it requires you to adopt >> a >> per node based payment model. Being able to monitor a service only when you >> pay for it. I am not sure I've seen a worse model for the 'free' users. >> >> Now tools like Foreman cannot compete because they cannot monitor the master >> and they cannot have atomic deploys. Who knows whats future PE only APIs >> will >> come? I'd bet things around file serving, catalog life cycle management etc. >> >> Now similarly open source users cannot compete because they cannot gain the >> insights they need to scale this java blackbox, even if they invest the time >> to >> learn the JVM, the software in the JVM is actively resisting their efforts >> of >> gaining operational insight. Their own tooling cannot compete against PE >> because some APIs are PE only. >> >> Personally as a open source user I use ‘apply’ - I decided during my P4 >> migration to look again at using a master - puppetserver - and was all set >> to >> move machines onto it that weekend when the metrics blog post came out. The >> puppetserver VM got immediately deleted. This is not a future direction that >> is cooperative or long term tenable for me. >> >> If you’re a open source user you need to look very carefully at the >> messaging >> and feature set and you can really only come to one conclusion and that is >> that >> to Puppet you’re just not a priority. Less than a priority you’re >> effectively >> being locked out and worked against and lack the ability to make a well >> monitored CM system if you adopt open source puppet. >> >> I hope the metrics situation will change - there has been other PE-first >> features and they came to Open Source so its not unprecedented but in this >> case >> it will pay to wait and see since if these features are not coming to Open >> Source then the whole is unusable. >> >> It’s a pendulum that swings, whats the right answer and right balance is not >> black and white. It’s a fiendishly difficult business problem and you have >> to >> commend Puppet for even bothering with Open source at all this point so I >> try >> and see the good intentions towards the open source user base. And while I >> am >> quite negative in this mail I do not think they are maliciously working >> towards >> excluding us. Hopefully Puppet realise in these cases the pendulum swung too >> far and will correct the course. >> >> Either way, the signal to Open Source users is that you need to really >> deeply >> consider your options post Puppet 3. It’s undeniably a high risk situation. >> It >> might well be the most significant decision you make in the next 5 years of >> your infrastructure and of your career. >> >> On the points of migrating to P4 from earlier Puppet. Its a major problem. >> Not only did the entire system change - in every respect, every component >> has >> had a major rework and rethink from 2.6 days but the language have also in >> effect become a new language. Not surprising for a 10 year old product. >> >> Yes there is some semblance of backward compatibility on the language level. >> But there is no doubt that New Puppet is exactly that. It’s a entirely new >> product with a backwards translation layer. Ask yourself does just updating >> to >> P4 and running your existing code achieve much? No it doesn’t it means >> you’re >> still running an outdated code. >> >> But getting to the point where you’ve updated Puppet to 4 and are now on a >> new >> set of software but still not rewrote your code to New Puppet is a colossal >> task since the world we live in is so resistant to upfront testing and >> rollback. And then you’ve still achieved more or less nothing because now >> you >> also have to consider basically rewriting your code to take advantage of the >> new features. >> >> Worse before you can even begin to look at refactoring your code you need to >> learn an entire new Puppet language and set of technologies and their >> environments like the JVM, new deployment locations, new packaging models >> new >> everything - worse ones that resist introspection. All while there is scant >> information on new patterns, no new style guides, the supporting system like >> rspec-puppet doesn’t support P4. Documentation is lacking or embryonic and >> so >> forth. rspec-puppet especially is a bad situation since many who have >> invested >> in a tested infra will loose the value. Ditto for puppet strings etc. >> >> I hope my blog posts and upcoming talks at conferences help those who do >> want >> to take on this task, but it’s a task much larger than that. Meanwhile other >> tools like mcollective are effectively abandonware with no viable >> alternative. >> >> Feedback I’ve had on some of the modern Puppet code I’ve put out is that it >> does not look like Puppet at all. It’s a new thing, literally your >> foundations >> of your 10 story building needs replacing if you wish to move forward. In >> the >> world IT admins live in you’re asking to replace the foundations while >> keeping >> the building standing, a tall order at best. (see >> https://github.com/ripienaar/puppet-classifier for modern puppet code) >> >> It’s truly a perfect storm of a fuckup when looked at from a Ops perspective >> as >> an open source user. >> >> So while I made a point of calling out the ‘resistant to change’ user, there >> are several version of these. >> >> You have your people who just won’t change - I have no sympathy for them as >> was >> clear. >> >> But you also have those who wish to change and who look critically at the P4 >> vs < P4 and come to the correct conclusion that this is a high risk multi >> month >> if not multi year project. They correctly deduce that Puppet is now so big >> that you need a dedicated team managing the manager. Something many of us >> just >> can’t afford or indeed who wants to spend their days doing nothing but >> manage >> Puppet? >> >> For many just getting Puppet in has been a massive politically charged >> internal >> struggle. It’s a constant battle between forward thinking admins, old style >> admins, developers and management who want to instead see features delivered >> and not management tools managed. >> >> Having just won the battle, one that was in many cases a battle fought out >> of >> having a deep respect and affinity for the company, its founder and >> products, >> now they find they’re squarely screwed and have to again invest possibly >> years >> and again have the fight with everyone the exact same battles and again have >> to >> budget for years of disruption. While it’s clear the Puppet won’t slow down >> its >> change curve. >> >> They correctly conclude that they WANT to change but cannot afford to. They >> also correctly conclude that the task is so big and Puppet moving so fast >> that >> P5 will be around long before they are on P4. And so the cycle will just >> start >> again. Its realistic to think the opportunity cost of continuing with Puppet >> is simply to high a price to pay for what it brings when you’re on <P4 >> today. >> >> Staying is no option as it will be unsupported and we have to consider >> security >> updates and such. Change they have to change, rock and a hard place. >> >> There is not much more frustrating and insulting than a company who puts you >> in >> that position and needless to say if the effort has to be put in anyway they >> will look elsewhere. I suspect the paying customer base will approach the >> transition similarly. I have no data on this but would be interesting to >> know >> how many old PE users are transitioning to P4 based PE once the older >> support >> cycle ends. >> >> I am deeply sympathetic to these people and it’s sad to say but where we are >> today is that they are simply SOL and there is no light at the end of the >> tunnel when you can’t even have a hope of gaining deep insight into New >> Puppet >> without paying. >> >> So given a choice to rewrite all my code to P4 I instead came to a hybrid >> approach: >> >> * Move my actual things I run into containers - web sites, bind, smtp >> servers, etc all in containers. Not puppet managed. >> * As machines move to CentOS 7, make them P4. >> * When on P4 only manage with Puppet the low hanging fruit like resolvers, >> sudo, users, sensu and bacula. Most from the Forge. >> >> Why do it this way? Well my environment is a R&D environment so I enjoy >> looking >> towards new tech and this is the world I want to innovate in but more than >> that >> where my P4 is now it’s doing very little. So little that any competitor in >> the CM market can step up and replace P4 for me really quickly as it appears >> more and more likely this is an important consideration. >> >> I felt I had no choice but to approach it this way because as a Open Source >> user the future is not bright at all - as per my previous mail I am just not >> a >> priority - and I can’t really blame the company for feeling this way. >> >> I’d use PE as I have nothing against it but as this is a R&D network the >> $3000 >> licence for my node count I’d need does not seem like a good way to spend my >> own >> pocket money. >> >> I myself am bout 50/50 through my P4 migration and it’s been a daunting task >> to >> say the least - though I am under no time pressures and am taking the time >> to >> gain a deep insight into what P4 is, what motivates its existence and to >> regain >> the deep understanding I had of < P4 and so forth. >> >> I am though a consultant in this space and Puppet is creating a massive >> customer base for what I do so it’s not like its wasted effort for me, as >> was >> pointed out, I am not typical though. Given the amount of money out there >> for >> consultants in a world Puppet literally create from nothing - wearing my >> consultant hat I’d say keep going as you were. And Puppet does really good >> things to support the consultants in their space via partner programs and >> such. >> >> P4 on it’s own is a brilliant tool brilliantly engineered and the language >> innovations are really good. The engineering rigour is orders of magnitude >> better than we had before and it really shows in the end result. I have >> nothing >> but positive things to say for P4 and the team that created it. > > Hi Arri, it’s been a while :) > > I agree with most of your rant until here. While I think the engineering > quality of what Puppet is producing is indeed miles ahead of where it was, > the product design itself (including language innovations) are core of the > problem to me. They aren’t, as best as I can tell, designed with a particular > customer in mind - I certainly can’t find the sweet spot customer for whom PE > is the right product. For every quality innovation, there are 2-3 seriously > breaking changes that not only indicate a lack of understanding of how Puppet > is currently managed in the field (what the ecosystem is doing, what typical > deployments look like, etc), but also that are likely to lead the new, > inexperienced, middle-of-the-road customers PE used to be targeted at down > entirely incorrect paths. > > The current state of the Classifier and Code Manager are examples of this. >
To be clear when I say P4 I do mean puppet and not all the rest of PE :) > From my own experience, customers are losing confidence that Puppet > understands their world. I certainly am no longer certain that it does. Both > from the rapid iteration of change in order to add functionality that doesn’t > visibly improve their work but mandates frequent refactoring to moves like > launching products without support for operating behind proxies, the > perception that Puppet is a tool by admins for admins is waning pretty > seriously. And most new users, as you note, are embracing the wrong kind of > configuration tools specifically to get around it. > > -Eric > > > > > >> >> >> It’s unfortunate P4 is combined with a supporting eco system that while >> innovative, powerful and potentially game changing is making it irrelevant >> to >> me as a class of user since they are more and more approaching things in a >> way >> that locks me out of the future I - and many like me - helped create. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/371534503.181383.1460372927683.JavaMail.zimbra%40devco.net. >> >> For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/167F3655-0AC7-4429-BCA3-BBDC66842A1E%40devco.net. For more options, visit https://groups.google.com/d/optout.
