Would 2.0 be an appropriate juncture to revisit the plugin architecture? I've often gotten bitten by classloader issues in 2nd/3rd party dependencies (e.g: plugin wants newer version of guava than jenkins-core). Shading is a reasonably complicated thing to do for a plugin, and I wonder if something cleverer could be done - without disappearing down a (say) complicated solution like OSGi.
On Fri, Sep 25, 2015 at 5:53 AM, Kohsuke Kawaguchi <[email protected]> wrote: > Jenkins is over 10 years old, and it came quite a long way. I still > remember the first few plugins that I wrote by myself, and now we have > close to 1100 plugins. What's started as a hobby project that had run under > my desk today boasts more than 100K installations driving half a > million-ish build machines. > > We collectively came quite a long way. When I started Jenkins, having a > server building & testing on every commit was a cutting-edge practice. So > are automatically capturing changelogs and analysing test reports. But now, > those are tablestakes, and the frontier of automation has moved further. > Now we are talking about building pipelines & workflows, continuously > deploying to servers, leveraging containers, and/or testing pull requests > before they get merged. I'm going to call this much bigger entangled > automation "Continuous Delivery", to contrast this with more classical > automated build & test executions (aka "Continuous Integration") that we > set out to do. > > The other thing I'd like to point out is that the adoption of Jenkins > continues to grow at the incredible pace of 30% year over year. That is, a > lot of people are starting new with Jenkins, and they are looking for us to > help them get Continuous Delivery done. Therefore, this is a good time to > step back and think about whether we are addressing those current user > demands. > > > For example, despite this advance during the last 10 years and 1000+ > plugins we've created, messaging in our website hasn't changed much since > the first version I wrote on java.net. It spends more space talking about > JNLP and zero mention of Git, pipeline, or Docker. > > The fresh installation of Jenkins is not much better. The CVS support is > available out of the box, but Git isn't. All the cool stuff that the > community has done and its collective best practices still need be learned > by each and every new user. It's bit like we are making everyone assemble > LEGO blocks. That's not doing enough justice to the 30K+ users that will be > joining us in this year. > > So I propose we do Jenkins 2.0 to fix this. > > There are three important goals that I see in Jenkins 2.0. > > 1. We need to claim our rightful place in Continuous Delivery. We have > lots of pieces that address these modern needs, but we are not > communicating this very well. > > 2. We need to revisit out of the box experience, so that Jenkins > itself speaks that story and makes it clear what we are aiming for. Our > software needs to speak for itself and tell people where we are going, so > that the community can better focus efforts on the important parts. > > 3. And we need to do this while keeping what makes Jenkins great in > the first place, which are the ecosystem, the community, and the > extensibility that recognizes that one size does not fit all and let people > do what they want to do. > > > Incrementing the major version sends a clear message to people that we are > moving forward. That's why I think 2.0 is appropriate for this effort. > > > > Now, 2.0 can mean a lot of different things to a lot of people, so let me > outline what I think we should do and we shouldn't do. > > It's very important for me to make sure that my fellow Jenkins developers > understand the motivation and the goal of this proposal, because that > drives much of what we should and shouldn't do. So instead of deep-diving > into technical parts, please take time to try to understand why I am > proposing this. > > We need to contain the scope. 2.0 has to have enough in it to justify the > major version number increase, but it creates a period of pause and > uncertainty, so it cannot keep dragging on for too long. 2.0 cannot be > everything everyone ever wanted. > > We cannot do massively disruptive 2.0, because it ends up splitting the > community. If users perceive that the upgrade to 2.0 is risky, enough of > them will stay behind with 1.x, plugin authors would want to continue > supporting them, which makes 1.x more liveable, which makes the transition > slower. I do not want to end up in Python2/3 situation, and nobody wins. > > That means we cannot be really breaking plugins. We cannot do > s/hudson/jenkins/g in the package names because doing that will break all > the plugins. 2.0 does come with the expectation that it is more disruptive > than usual 1.630 to 1.631 upgrade, so we have some "disruption budget", but > we have to use it really wisely. > > Simiarly, for me it is an absolute requirement that we keep people's > $JENKINS_HOME functioning. A lot of sweat, tear, and blood went into those > right set of plugins and elaborate job configurations. When users upgrade > to 2.0, they need to continue to work, or else Jenkins 2.0 will be Jenkins > in just the name only. > > Therefore, we cannot make massive internal changes. In many ways, it has > to be evolutionary instead of revolutionary, when it comes to the code. > This is not a "let's redo everything from scratch" kind of 2.0. In any > case, I think it's a pitfall to focus too much on internals. We all have a > long list of things we want to fix and the technical debt that we want to > pay down. My cautionary tale here is that of Maven 2 to Maven 3 upgrade. > The developers of the project spent a lot of efforts redoing all the > plumbings. Plexus gave way to Guice, and the dependency resolution engine > got completely rewritten. Then to keep plugins working, more efforts were > spent on building the backward compatibility layer. After something like 18 > months, Maven 3 came out, which did more or less the same thing as far as > users are concerned. I'm sure I'm over-simplifying this, but you get the > point. > > > > So given all that constraints, I think 2.0 should have the following 3 > major pillars: > > - Messaging changes, to make sure people coming into the Continuous > Delivery space will get that Jenkins does what they want. > - Software that backs up our messages. Out of the box experience that > caters to Continuous Delivery needs. > - Targeted internal plumbing changes that enable those goals > > I have some concrete ideas in each of these pillars, and I'll describe > them below. But I also need help from everyone to come up with, discuss, > and decide what other things will advance those pillars. > > Messaging: > > - Domain name. It's kind of a problem that we have "ci" baked into our > domain name jenkins-ci.org. We have acquired http://jenkins.cd/ How > about we change the domain name? I think it sends another clear signal. > > - We need more up-to-date feature list page (like > http://arquillian.org/features/) that talks about things that matter > to the modern users. > > - We need authoritative and curated getting started guide that expands > on the things listed in the features page and help people understand those > features, so that we have clearly marked trails. > > - This is probably out of scope for the initial 2.0 launch, but in the > future we want to redo the plugin listing page as well. This is a > persistent feedback that we hear from users. > > - All the above things call for better infra that can handle this. > Right now we have our web assets are split into Drupal and Wiki, but the > former can be only touched by a few people and the latter is slow and > klunky. I think this is the time to switch to some static site generator, > so that everyone can contribute content through Git and pull requests, just > like how we collaborate on plugins. > > > Out of the Box Experience: > > - This work is already in progress, but we really need some initial > setup wizard. We can use it to install plugins so that new instances come > up more useful from get-go --- things like git, workflow, pipeline as code, > folders, and so on. These plugins together tell the story of how we want > users to use Jenkins. > > - Another work that's already under way is the UX improvement, > specifically the config form re-layout. This is the kind of change that > helps people (literally) see that 2.0 is different. UX in general is > clearly one of the places we should spend our precious disruption budget > for. > > - To reinforce the message that workflow is the future, CloudBees is > going to open-source our workflow stage view plugin that was previously a > part of CloudBees Jenkins Enterprise. > > > Internals: > > - Let's define a policy to remove APIs after they are deprecated. We > have talked about this in FOSDEM, and this could be as easy as "N releases > after deprecation". Feedbacks from users at the San Jose JAM was that > things like this is OK, but we need to help people identify plugins that > will be impacted to give them earlier warnings. > > - As a part of the UX rebump effort, Tom et al has been working on a > brand-new way of doing frontend in Jenkins plugins. His JUC talk has some > materials. Given that user experience is a major theme in 2.0, I think this > internal plumbing change makes sense. > > - Let's use the opportunity to update some of the libraries. I'm > thinking about things like Groovy, which according to the testing done > during Copenhagen Hackathon, should be compatible. This shouldn't include > updates that are known to be compatibility breaking, such as Acegi Security > to Spring Security (which involves package name changes.) > > - Time to bump up the system requirement to Java 8 and Servlet 3.0. > Let's think about what this would enable to users. Again, we talked about > this a bit in FOSDEM. > > > Finally, timeline-wise, my aspirational timeline is as follows, though > obviously this is largely dependent on feedback to the proposal: > > - Announce the proposal publicly and have discussions to nail the > details (Sep-Oct) > - Execution (Oct-Dec) > - Periodic alpha/beta releases to solicit feedbacks from users > - PR activities > - This phase concludes with the release candidate > - Plugin sweep to ensure key plugins are "2.0 ready". This is the > opportunity to find issues (Jan 2016) > - Release (end Jan?) > - Drop 1.x development as soon as possible to focus on 2.x. > > > There are a lot of things I haven't captured, but this email is aleady > getting too long. Looking forward to having more conversations about this > with everyone. > > -- > Kohsuke Kawaguchi > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zMNF-BJnZ%3DwSxYJiXBkExPvNUnaNU4S5Ru6p-PgJmUbw%40mail.gmail.com > <https://groups.google.com/d/msgid/jenkinsci-dev/CAN4CQ4zMNF-BJnZ%3DwSxYJiXBkExPvNUnaNU4S5Ru6p-PgJmUbw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83Rx3Nt2ecARsqez1D%2BQ_wQbTfneSHq5cm9UQhfQ9Am3kA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
