On Apr 11, 2016, at 4:08 AM, R.I.Pienaar <r...@devco.net> wrote: > ----- Original Message ----- >> From: "Thomas Gelf" <tho...@gelf.net> >> To: "puppet-dev" <puppet-dev@googlegroups.com> >> Sent: Monday, 11 April, 2016 03:30:58 >> Subject: [Puppet-dev] Re: The Future of Puppet [Was: Deprecation logs] > >> Am 11.04.2016 um 03:01 schrieb Rob Nelson: >>> Of course there's lock in. You can't click a button and go from Puppet >>> OSS to Chef OSS; [...] >>> Is their concern about being able to contribute to it or even fork? I >>> suspect that's what most lock in concerns are really based on. >> >> It's about going from Puppet OSS to PE, forth and and back. No problem >> with loosing the GUI or special add-ons. But the core functionality of >> your CM tool, that's what they want to be free software. >> > > Eric asked so here it is, this is my feedback with a open source user hat > on. Echoing much what was said. I hope others send in their story.
Wow. This is, ah, comprehensive. I’m not going to try to respond to all of this, but I’m going to do what I can to reply to the core concerns in a couple of these emails. In any replies you might make, it’s safe to assume that most of the things we’re doing that you don’t like aren’t being done by “Puppet”, they’re being done by me, and/or by someone else on this list who works somewhere in my organization. That is, there is no “Puppet”, there’s just the group of us doing what we can to build a company to automate the world. Not sure if that matters, but it is worth pointing out there are people you can have these conversations with at any time, and those people are easy to find. In particular, I’ve specifically been leading the strategy for deciding how to keep our OSS software healthy and great, our community strong, and our business viable, so a lot of those decisions were either made by me, or made in direct consultation with me. I’m definitely the right person to bring these questions up with, although in so many cases, we’re still learning. It is worth pointing out that I consider OSS and community to be clearly related, but more orthogonal efforts to increase the area of a curve, rather than the same thing. It’s very easy to think “OSS == community”, but there are plenty of fantastic communities around shared identities, shared content, shared experiences, and shared goals; it doesn’t all have to be shared code. In our case, Puppet’s community has always been much more of a user community than a developer community, and we’ve invested more in the user community than the dev community (which I think is both rational and the right decision). As we work hard to figure out how our business will work over the next ten years, it’s worth reiterating that community will always be critical to me. It’s how we know we’re building good software, it’s how you succeed (because people help each other), and in a lot of ways, it’s what makes it worth it. Having users say again and again that Puppet made their lives better is what keeps me at this. Everything I do is in relentless pursuit of giving more people the opportunity to get the benefits that Puppet and related tools have so far given to a small slice of the world. As a last point, today we at Puppet have more people working on OSS and are shipping more OSS than we’ve ever done before, and I’m incredibly proud of that. The fact that the percentage is smaller in no way diminishes the total amount of work we’re doing here. […] > Open Source users often see the existence of some open source tool as a > implied > contract that it will always be so, and their adoption of it is based on that > perception. This is encoded in the very licences we hold deer and when that > inevitably turns out not to be realistic business model the companies involved > will do what they can to survive. Inevitably this ends up breaking the > ‘implied contract’ the open source user rightly or wrongly felt they had - > cue > much anguish and resentment. Not surprisingly, I have a lot of opinions on OSS in the world of software companies, and again, not surprisingly, some of my opinions have shifted dramatically over the last 11 years. It’s not a coincidence that only two or three public-size companies built on OSS have ever reached sustainability (Red Hat, Sourcefire, and…?). We’re trying to do something so fundamentally hard that only a few have ever done it, and we can’t repeat the ways they did it. Everyone else who has tried is either still in process (Cloudera, Hortonworks) or ended badly (MySQL). There is a ton of successful OSS out there, but almost none of it is produced by companies like us, who are building their company around making you successful. Instead, it’s almost all built by companies like Intel, IBM, and HP, who are contributing to a project as part of a larger strategy. It’s a loss leader in some way for some other project. We don’t have that luxury. We have given away the thing you find most valuable about us, and yet we still have to find a way to build a business capable of sustaining the software you rely on, and hopefully actually do much more than that. Certainly I’m not running this company to maintain the status quo. And that gets to my last point. We are still very, very far from the whole market being automated. My goal is above and beyond all else to get the whole market automated. Providing great open source software is a critical part of the goal, but building a large, successful company that my customers love is another big important part of that goal, not because I’m greedy, but because I want to do big things, and big things require big levers. And in terms of implied contracts, the only thing encoded in your OSS contracts is your right to use what’s posted. Encoded in our community is our own standard of software, which is that we work hard to build software people love, and we think its users and the community around it matter a lot. We don’t succeed all the time, but more importantly, for us to even consider keeping that promise, we have to constantly evolve, we have to stop maintaining some projects (which you can then take over, if you’re so inclined, because it’s OSS), and we have to start new ones. In other words, our license entitles you to the code included in that repo. Our community has a statement of how we expect to work, and your engagement in that community gets you far more rights than any license ever word. The reason my team works so hard to make you, and everyone else, happy has nothing to do with OSS. It has everything to do with community. I wish I could say that the community was responsible for Puppet, and so many of the tools around it. Certainly I can say that you, RI, have contributed incredibly important work to Puppet. But in general, the software itself is maintained by people who Puppet pays. > The company will then go on to explain themselves and make commitments - we > will keep the core product open but innovate on paid for enterprise features > like RBAC and very featurefull GUIs. We will not work to exclude other makers > of tools and your home grown tooling, we will make the extension points open > and APIs open for others to innovate on. > > I don’t have the actual quote to hand as I am sat on a plane but this seems to > me more or less the story we were told when PE became a thing and that seemed > good and a model worth supporting. After all what would be open source is not > unlike most other open source and we are not averse to applying some duct tape > and glue to make things work. I’m sure I’ve said some of these things, but I’ve also said plenty of other things. That’s more about helping everyone understand how we’re working today, rather than a long-term commitment for how we’ll always work. I stood up at PuppetConf more than a year ago and said we’d tend to build commercial software in the core and OSS at the edge. Note that’s not “we’ll commercialize stuff in the core”; just, net new projects that I fund will follow this pattern. I’m proud that we wake up smarter every day, that we learn, that the way we work today is unlikely to be the way we work tomorrow. I’m proud we’ve built a lot of great software over the years. I wish I was proud of every decision and every piece of software, but, well, that’s the way of it. > Fast forward to now and the situation is not like this at all. Today we have > the puppetserver product touted as the only way forward and not only does it > have PE only APIs its core operability features are PE only too. Performance > metrics and atomic deploys. No longer is it a matter of if you put in the > effort and build the tooling you can use the Open Source product but we will > sell world class tooling. We’re now in a world where the ability to monitor > the product and gain the insight you need to scale it requires you to adopt a > per node based payment model. Being able to monitor a service only when you > pay for it. I am not sure I've seen a worse model for the 'free' users. Hmm. I read this as, we built a much, much better Puppet server, that’s faster, more stable, more scalable, lighter weight, and much more, and then given it away for free. For zero cost on your part, you have an unquestionably better product in pretty much every way. We’ve also decided to do more work beyond that, but not release all of it for free. That other work was never available in the old system. That other work was never available for free. And no matter how much you might be interested in paying, you couldn’t get those other features with the old system. >From my perspective, everyone wins here: People unwilling or unable to pay for software get a much better product. People who pay for the software get a better product and some additional functionality. We can afford to continue investing in the product and making both the free and commercial bits better every year. > Now tools like Foreman cannot compete because they cannot monitor the master > and they cannot have atomic deploys. Who knows whats future PE only APIs will > come? I'd bet things around file serving, catalog life cycle management etc. Red Hat — which funds Foreman — can easily compete. So far, though, they haven’t contributed any patches, asked for any help, or done any of the work implying they care about it. Heck, they could do what we did, and build from scratch a brand new system that works better, is faster, lighter weight, has support for operational metrics, and then given everything away instead of nearly everything. I know for a fact Red Hat has a ton more developers than we do, they just don’t have any devoted to working on Puppet itself, or any of the core systems in it. > Now similarly open source users cannot compete because they cannot gain the > insights they need to scale this java blackbox, even if they invest the time > to > learn the JVM, the software in the JVM is actively resisting their efforts of > gaining operational insight. Their own tooling cannot compete against PE > because some APIs are PE only. I’m not really sure what you mean by ‘compete’ here - Foreman is run by Red Hat, who is basically a competitor from this perspective, but the users you’re talking about aren’t trying to compete with us, or anyone else, so I don’t know how ‘compete’ fits in. > Personally as a open source user I use ‘apply’ - I decided during my P4 > migration to look again at using a master - puppetserver - and was all set to > move machines onto it that weekend when the metrics blog post came out. The > puppetserver VM got immediately deleted. This is not a future direction that > is cooperative or long term tenable for me. > > If you’re a open source user you need to look very carefully at the messaging > and feature set and you can really only come to one conclusion and that is > that > to Puppet you’re just not a priority. Less than a priority you’re effectively > being locked out and worked against and lack the ability to make a well > monitored CM system if you adopt open source puppet. I’m sorry, but this is just not fair, and I think this perspective harms all of us. If OSS users weren’t a priority, we wouldn’t be actively posting so much new free software. Puppet-server would be a commercial replacement for the puppetmaster, the compiled Facter would be commercial, the compiled Puppet language would be, etc. We wouldn’t have had Henrik spend so much time making the language do what so many always wanted. These are all examples of us spending a ton of money, time, and energy building software that makes your life better for free. I don’t know what else we could do to convince you that this matters to us, other than just literally give away everything we do, but if we did that (which we did for 7 years, so I have a lot of experience with its consequences), you’d have a worse product (because relying on services would innately push us to build a product that needed services work) and we’d probably be out of business. > I hope the metrics situation will change - there has been other PE-first > features and they came to Open Source so its not unprecedented but in this > case > it will pay to wait and see since if these features are not coming to Open > Source then the whole is unusable. Strategically we rarely build commercial features with the plan for them to be free; we generally (but not always) intend on a given capability being free or commercial through its lifetime. I think that’s a much better and clearer picture for everyone — if you’re using all free software, you’re using the latest and greatest, rather than planning for OSS users to only have years-old features. I would be happy to have a conversation about whether we’ve made the right decision on this being OSS or not, but frankly, this is the first I’ve heard this complaint from anyone outside the company. > It’s a pendulum that swings, whats the right answer and right balance is not > black and white. It’s a fiendishly difficult business problem and you have to > commend Puppet for even bothering with Open source at all this point so I try > and see the good intentions towards the open source user base. And while I am > quite negative in this mail I do not think they are maliciously working > towards > excluding us. Hopefully Puppet realise in these cases the pendulum swung too > far and will correct the course. That’s good to hear, at least. Hanlon’s razor. :) > Either way, the signal to Open Source users is that you need to really deeply > consider your options post Puppet 3. It’s undeniably a high risk situation. It > might well be the most significant decision you make in the next 5 years of > your infrastructure and of your career. > > On the points of migrating to P4 from earlier Puppet. Its a major problem. > Not only did the entire system change - in every respect, every component has > had a major rework and rethink from 2.6 days but the language have also in > effect become a new language. Not surprising for a 10 year old product. We’re very much feeling the pain of the upgrades here, but that has nothing to do with OSS or anything else. It has to do with us moving from a very informally designed language that I built, to a much more formally built language that Henrik has formed it into. That, and people really hate to upgrade, especially on-premise software. We’ve recognized the gap here, and will be working hard to help people through that. It’s fair to say that I consider our community’s success at upgrading to this version — or at least, staying on what we consider to be the latest version of the language — to be an existential question. For what it’s worth, one of the long-pole priorities in our business overall is to keep a huge portion of our community on the latest versions. I know we haven’t succeeded at that in the past, but it’s something you’ll see us continue to invest in heavily indefinitely. And that should help all users. […] > There is not much more frustrating and insulting than a company who puts you > in > that position and needless to say if the effort has to be put in anyway they > will look elsewhere. I suspect the paying customer base will approach the > transition similarly. I have no data on this but would be interesting to know > how many old PE users are transitioning to P4 based PE once the older support > cycle ends. For what it’s worth, I would absolutely love to lock in a given piece of tech and just leave it. But then we get accused (rightfully) of abandoning things. We have to walk this tightrope between always being modern but never asking our users to change. I think P4 was comfortably too much change in one go, and we’ve got work to do to recover from that. But a lot that went into P4 is in response to users all over the community asking for them. > I am deeply sympathetic to these people and it’s sad to say but where we are > today is that they are simply SOL and there is no light at the end of the > tunnel when you can’t even have a hope of gaining deep insight into New Puppet > without paying. I’d argue no one could get any insight into the old system regardless of willingness to pay, so again, every member of the community is better off with the new system. […] > I felt I had no choice but to approach it this way because as a Open Source > user the future is not bright at all - as per my previous mail I am just not a > priority - and I can’t really blame the company for feeling this way. I think you’ve pegged yourself incorrectly. You’re a Puppet user, but you use plenty of commercial software (e.g., your phone, which is probably as critical to you as Puppet is). Richard Stallman is the only OSS user I know. Your particular use cases are unlike most of our customers, so there is some dissonance there. I don’t know of anyone who runs as much infrastructure individually as you do, especially not for R&D. You’ve decided that it’s not worth paying for our software in this space, which is (I assume) a rational decision on your part, > I’d use PE as I have nothing against it but as this is a R&D network the $3000 > licence for my node count I’d need does not seem like a good way to spend my > own > pocket money. > > I myself am bout 50/50 through my P4 migration and it’s been a daunting task > to > say the least - though I am under no time pressures and am taking the time to > gain a deep insight into what P4 is, what motivates its existence and to > regain > the deep understanding I had of < P4 and so forth. > > I am though a consultant in this space and Puppet is creating a massive > customer base for what I do so it’s not like its wasted effort for me, as was > pointed out, I am not typical though. Given the amount of money out there for > consultants in a world Puppet literally create from nothing - wearing my > consultant hat I’d say keep going as you were. And Puppet does really good > things to support the consultants in their space via partner programs and > such. It’s great to hear this feedback, thank you. > P4 on it’s own is a brilliant tool brilliantly engineered and the language > innovations are really good. The engineering rigour is orders of magnitude > better than we had before and it really shows in the end result. I have > nothing > but positive things to say for P4 and the team that created it. > > It’s unfortunate P4 is combined with a supporting eco system that while > innovative, powerful and potentially game changing is making it irrelevant to > me as a class of user since they are more and more approaching things in a way > that locks me out of the future I - and many like me - helped create. I completely agree many helped create this world, and you personally were and continue to be a big part of it. — @puppetmasterd | http://about.me/lak | http://puppetlabs.com/ -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/1950ABD9-8B86-4AAE-B998-E7C7B40BA002%40puppet.com. For more options, visit https://groups.google.com/d/optout.
