Hey Jay, Framing the problem strictly as developer productivity is not entirely accurate. I agree with you from that standpoint. I'd never want to remove JDK 6 support, since I'm growing to dislike all new things. :)
But the reality is that we're foregoing features that we could take advantage of. For example, YARN 2.7 is slated to have disk I/O isolation CGroup support. Without an upgrade in Samza, this can't be supported (other than taking whatever the NM gives us by default). This is a specific example, but more will pop up as time passes. Another would be Scala 2.11. If we don't build against Scala 2.11, then you actually can't write Samza StreamTasks against it, which some users have asked for. So I think the real question is about timing and messaging (roadmap, anyone?). People need to know if and when, so that they can make decisions. We could push off Scala 2.11 and YARN 2.7 support for a year from now (YARN 2.4 is already 8 months old, and 0.8 just shipped with it). Personally, I'm OK waiting longer than six months. The thing is, I don't think that there's going to be a real forcing function for it for several years, BUT things will get steadily worse in the meantime. The second note that I have is that I think the long term goal should remain to deprecate and remove Scala from Samza's implementation, and in general be extraordinarily cautious about which dependencies we pull into Samza. So, 1 year? I'm OK with that. Cheers, Chris On 12/10/14 11:15 AM, "Jay Kreps" <[email protected]> wrote: >My two cents: > >I think this is usually a trade off of optimizing for the project >developers (who ideally would want to use the newest thing) and the >project >users, where many will be stuck on old versions and can't use it at all if >it doesn't work with their version. For the most part I think the real, >tangible productivity gain for project developers on new versions is not >that high but the gain for users stuck on old versions is really high >(since they can't use it at all). Software that is heavily used (or that >wants to be heavily used) should optimize for the users and take a pretty >conservative stance on compatibility. > >-Jay > >On Wed, Dec 10, 2014 at 8:59 AM, Chris Riccomini < >[email protected]> wrote: > >> Hey all, >> >> When Hadoop drops JDK6 support, we will be forced to upgrade as part of >> our YARN support. If 0.9.0 uses Hadoop 2.7, then our next release will >> require it. They seem to be releasing every 3-4 months, so it's likely >> than in the Feb-Apr '15 time frame, we'll be forced to move to JDK 7 >> anyway. >> >> With this in mind, I think we should move forward with an end-of-March >>JDK >> 6 deprecation date. As Jakob mentioned in >> https://issues.apache.org/jira/browse/SAMZA-455, I think we should also >> put something up on the website to make it abundantly clear what's >> happening. Does that sound workable? >> >> @Jakob/@Zhijie any idea what the schedule looks like for the Hadoop 2.7 >> release? >> >> @Eric sorry to hear versioning issues ended up being unworkable for you >> with Samza. Out of curiosity, was it the JDK support, or the Scala >> versioning that caused problems? >> >> Cheers, >> Chris >> >> On 12/9/14 5:37 PM, "Paul Brown" <[email protected]> wrote: >> >> >JDK6 will have been EOL'd for *two years* come February next year[1]. >> >IMHO, the only reason to still build for older JDKs is as a convenient >> >proxy for Android support. >> > >> >[1] http://www.oracle.com/technetwork/java/eol-135779.html >> > >> >‹ >> >[email protected] | Multifarious, Inc. | http://mult.ifario.us/ >> > >> >On Tue, Dec 9, 2014 at 1:47 PM, Eric Sammer <[email protected]> >> >wrote: >> > >> >> On Tue, Dec 9, 2014 at 1:04 PM, Jakob Homan <[email protected]> >>wrote: >> >> >> >> > Eric had been requesting the JDK6 support on Twitter >> >> > (https://twitter.com/esammer/status/516681461219737600). In >>response, >> >> > SAMZA-455 was opened but not much lobbying was done there. >> >> > >> >> > @Eric, is it still the case that you feel JDK6 is a hard >>requirement? >> >> > You made a strong case in the JIRA. I note that Hadoop is >>currently >> >> > going JDK7 in 2.7. >> >> > >> >> >> >> Thanks for remembering. :) >> >> >> >> At the end of the day, there's some threshold where, if N% of >>projects >> >>drop >> >> support, users are forced to upgrade. When they do, they tend to do >> >> everything on a box (and its cluster) together. Mixed-mode >>deployments >> >> (e.g. Samza @ JDK7, Kafka @ 7, HDFS @ 6) is a recipe for disaster. >>The >> >> short way of saying that is minVersion(commonProjectsUsedTogether) is >> >>the >> >> ideal version to support. If Hadoop and others are dropping support, >> >>it's >> >> probably fine. I think the most important thing is that it's clearly >> >> communicated prior to doing so (insert big discussion about >>deprecation >> >> cycles on "supported platforms"). We weren't able to use Samza as >>part >> >>of >> >> our product as a result of minimum versions. Scala-based projects >>have >> >> historically been an enormous pain in this regard. I don't know how >>many >> >> others will be in that boat. >> >> >> >> >> >> > Would it work for Samza 0.8 to be the last to provide JDK6 support? >> >> > It's likely that Samza 0.9 won't be out for at least a few months, >>and >> >> > Eric had proposed a strawman approach of continuing JDK6 support >>for >> >> > six months (back in early November). So, it's likely that 0.9 >>would >> >> > reasonably closely meet that timeframe and could be the first >> >> > JDK7-only release... >> >> > >> >> >> >> I think that's probably fine. It sounds like 0.8 will have a good >> >>lifecycle >> >> prior to 0.9 taking over, giving folks enough runway to plan for a >>JVM >> >> upgrade. Like I said, when we evaluated Samza, we were blocked on the >> >> dependency. With our timing, it forced us on to other projects, as >>much >> >>as >> >> we really liked and wanted to use Samza. I think there's a big >>divide in >> >> terms of tolerance of supported platforms between building internal >> >>systems >> >> (i.e. SaaS, or in-house) and building "enterprise software" (i.e. >> >>software >> >> you ship to folks). I don't pretend our requirements are indicative >>of >> >>the >> >> majority or important to everyone. I also respect the desire for >>forward >> >> motion in what's supported, and what features are accessible. >> >> >> >> The next discussion is probably around which version of Scala to >>track >> >>for >> >> the Samza community over the next N months. There are some obvious >> >> contentious positions[1] on Java 8 being required there as well. >>That's >> >>an >> >> even tougher nut to crack. Some of the related projects still have >>some >> >> issues running on 8 (ZK, or at least a few months ago when I tried >>it). >> >> >> >> [1] http://scala-lang.org/news/2.12-roadmap >> >> >> >> Thanks all! >> >> >> >> >> >> > -jakob >> >> > >> >> > On Tue, Dec 9, 2014 at 8:39 AM, Chris Riccomini >> >> > <[email protected]> wrote: >> >> > > Hey all, >> >> > > >> >> > > We've reached a bit of an impasse between upgrading to Scala >>2.11: >> >> > > >> >> > > https://issues.apache.org/jira/browse/SAMZA-469 >> >> > > >> >> > > And deprecating JDK 6: >> >> > > >> >> > > https://issues.apache.org/jira/browse/SAMZA-455 >> >> > > >> >> > > It looks as though Scalatra 2.3, which is required for Scala 2.11 >> >> > support, >> >> > > was built using JDK 7. This means that upgrading to Scala 2.11 >> >>forces >> >> us >> >> > > to deprecate JDK 6. It is possible for us to work around this by >> >> > > eliminating the Scalatra dependency, but this would require some >> >>work >> >> in >> >> > > samza-yarn. >> >> > > >> >> > > Thoughts? >> >> > > >> >> > > Cheers, >> >> > > Chris >> >> > > >> >> > > On 11/4/14 6:58 AM, "Martin Kleppmann" <[email protected]> >> wrote: >> >> > > >> >> > >>Hi Tommy, >> >> > >> >> >> > >>There was a discussion about minimum JDK requirements a few >>months >> >>ago, >> >> > >>and at the time, nobody was asking for JDK 6 support, so we >>bumped >> >>the >> >> > >>requirement up to JDK 7. However, in the meantime, there have >>been >> >>some >> >> > >>requests for JDK 6. >> >> > >> >> >> > >>I've tried to summarise the state of the discussion on this >>ticket: >> >> > >>https://issues.apache.org/jira/browse/SAMZA-455 -- please chime >>in >> >> > there. >> >> > >> >> >> > >>Thanks, >> >> > >>Martin >> >> > >> >> >> > >>On 4 Nov 2014, at 13:05, Tommy Becker <[email protected]> wrote: >> >> > >> >> >> > >>> Hey folks, >> >> > >>> I've noticed that the Samza jars from Maven are compiled for >>JDK >> >>7. >> >> I >> >> > >>>don't see anything about a minimum JDK version on the site. We >>are >> >> > >>>currently still on JDK 6 and I'm trying to decide whether we >> >>should go >> >> > >>>ahead and upgrade or whether we can recompile Samza for JDK 6. >> >>What >> >> are >> >> > >>>your thoughts? >> >> > >>> >> >> > >>> -Tommy >> >> > >>> >> >> > >>> >> >> > >>> ________________________________ >> >> > >>> >> >> > >>> This email and any attachments may contain confidential and >> >> privileged >> >> > >>>material for the sole use of the intended recipient. Any review, >> >> > >>>copying, or distribution of this email (or any attachments) by >> >>others >> >> is >> >> > >>>prohibited. If you are not the intended recipient, please >>contact >> >>the >> >> > >>>sender immediately and permanently delete this email and any >> >> > >>>attachments. No employee or agent of TiVo Inc. is authorized to >> >> conclude >> >> > >>>any binding agreement on behalf of TiVo Inc. by email. Binding >> >> > >>>agreements with TiVo Inc. may only be made by a signed written >> >> > agreement. >> >> > >> >> >> > > >> >> > >> >> >> >> >> >> >> >> -- >> >> E. Sammer >> >> CTO - ScalingData >> >> >> >>
