We are starting to test it internally in our cluster. This is a broader hadoop issue. Note, Oracle's last public (free) release of a Java 6 JRE will be towards the end of this year. After that, even security updates will ONLY be available in a Java 7 JRE.
On 4/16/12 10:28 AM, "Dmitriy Ryaboy" <dvrya...@gmail.com> wrote: >Last time I tried to run a Java 7 jar on a java 6 jvm, things blew up. >This suggests to me that before Pig can use Java 7, Hadoop has to be >able to (since hadoop spins out the jvm process that runs the pig >map/reduce jobs). Do you know if Hadoop can run on Java 7? > >D > >On Mon, Apr 16, 2012 at 10:18 AM, Scott Carey <sc...@richrelevance.com> >wrote: >> >> >> On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <g...@apache.org> >> wrote: >> >>>Sure, >>>probably Java7 is more mature than Pig. >>>However as far as I know it is a pain to install on Mac. >> >> Yes, you have to install using the developer preview, but it is >>available >> from the same place all other JRE's from Oracle are. [1] >> >> 7u4 should be out in a few weeks [2], which will include a Mac OSX >>version >> (Unfortunately only for Lion, and not Snow Leopard). >> >> Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7 >> is widely supported on the mac. They will likely wait a couple months >> more than that. However, that still puts JRE 6's end this year. >> >> [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html >> [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012) >> >>>This is just one small example of not mature enough. >> >> Availability on Mac is probably the biggest example in my opinion. What >> else is there? >> >> Most of the risky features in Java 7's JVM are already back-ported to >>Java >> 6. This leaves the new APIs and features as the biggest risks, but most >> of these should reduce risk (try-with-resources, much better file system >> API, better exception handling). This leaves mostly MethodHandle and >> invokevirtual as the new features that are risky -- these are being >> heavily tested and used by others however (e.g. JRuby). >> >>>I am under the impression that Java7 not yet mainstream, correct me if I >>>am >>>wrong. >> >> What is the definition of mainstream in this situation? Why does it >> matter? I am struggling to imagine a concrete definition of mainstream >> that would be useful in this context. >> >> Other than fear of new things and a lack of a corporate backed JVM on >>Mac, >> we have the real issue that there are significant resources required to >> test such a change in Pig and its related execution and development >> environments. However, this issue does not go away -- Java 6 can't be >>the >> only supported flavor forever. Some day in the not too distant future, a >> Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem. >> >> It would be nice if high value features that could take advantage of >>Java >> 7 features started development early enough to be available at about the >> time Java 7 was common in the ecosystem. Waiting until Java 7 is >> mainstream to start such features means adding a 12 month delay into the >> process. >> >>> >>>I am fine with having optional performance improvements enabled by >>>Java7. >>>+1 >>>I am not fine with having a dependency on Java7 for users. -1 >>>I am not comfortable with having a dependency on Java7 for pig >>>developers. >>>I am afraid it will put off new contributors. -0 >> >> How many potential contributors are interested in being able to try new >> technologies, versus those only wanting to use what they currently know? >> >>> >>>Cheers, >>>-- >>>Gianmarco >>> >>> >>> >>>On Fri, Apr 13, 2012 at 01:10, Scott Carey <sc...@richrelevance.com> >>>wrote: >>> >>>> >>>> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" >>>><g...@apache.org> >>>> wrote: >>>> >>>> >My personal opinion is Yes, if there is a compelling reason, but not >>>>now. >>>> >Still not mature enough. >>>> >>>> What is the definition of "mature enough"? >>>> >>>> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the >>>>most >>>> mature. >>>> >>>> Granted, I also think the whole concept of maturity is subjective and >>>>not >>>> conducive to good discussion. >>>> >>>> >>>> What objective characteristics are required of Java 7 to support it? >>>>When >>>> approximately would that likely occur? Subtract 6 months from that (a >>>> typical Pig dev cycle), how soon is that? >>>> >>>> There is a big difference between _requiring_ Java 7 and having an >>>> optional feature built with Java 7 into its own jar that requires a >>>>Java 7 >>>> cluster to use. Some performance features might fall into that >>>>category. >>>> >>>> > >>>> >Cheers, >>>> >-- >>>> >Gianmarco >>>> > >>>> > >>>> > >>>> >On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <jcove...@gmail.com> >>>> >wrote: >>>> > >>>> >> Scott Carey brought Java 7 up in PIG-2643, and I think it's >>>>something we >>>> >> need to think about. When do we want to start taking advantage of >>>>new >>>> >> features that may not exist on Java 6? Do we ever? >>>> >> >>>> >> 2012/4/12 Scott Carey (Commented) (JIRA) <j...@apache.org> >>>> >> >>>> >> > >>>> >> > [ >>>> >> > >>>> >> >>>> >> >>>> >>>>https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira. >>>>pl >>>> >>>>>>ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706 >>>>>>#c >>>>>>om >>>> >>ment-13252706 >>>> >> ] >>>> >> > >>>> >> > Scott Carey commented on PIG-2643: >>>> >> > ---------------------------------- >>>> >> > >>>> >> > Another thought for this sort of thing: >>>> >> > >>>> >> > This might be achievable without bytecode generation and good >>>> >>performance >>>> >> > with Java 7 MethodHandles [1][2]. Of course, that would require >>>>Java >>>> >>7, >>>> >> > but Java 6 support ends later year [3], about the time Pig 0.11 >>>>would >>>> >>be >>>> >> > out anyway. >>>> >> > >>>> >> > >>>> >> > [1] >>>> >> > >>>> >> >>>> >> >>>> >>>>http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle. >>>>ht >>>> >>ml >>>> >> > [2] >>>> >> > >>>> >> >>>> >> >>>> >>>>http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all- >>>>ab >>>> >>out >>>> >> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date >>>> >> > >>>> >> > > Use bytecode generation to make a performance replacement for >>>> >> > InvokeForLong, InvokeForString, etc >>>> >> > > >>>> >> > >>>> >> >>>> >>>>>>--------------------------------------------------------------------- >>>>>>-- >>>>>>-- >>>> >>------------------------ >>>> >> > > >>>> >> > > Key: PIG-2643 >>>> >> > > URL: >>>>https://issues.apache.org/jira/browse/PIG-2643 >>>> >> > > Project: Pig >>>> >> > > Issue Type: Improvement >>>> >> > > Reporter: Jonathan Coveney >>>> >> > > Assignee: Jonathan Coveney >>>> >> > > Priority: Minor >>>> >> > > Labels: codegen >>>> >> > > Fix For: 0.11, 0.10.1 >>>> >> > > >>>> >> > > Attachments: PIG-2643-0.patch >>>> >> > > >>>> >> > > >>>> >> > > This is basically to cut my teeth for much more ambitious code >>>> >> > generation down the line, but I think it could be performance and >>>> >>useful. >>>> >> > > the new syntax is: >>>> >> > > {code}a = load 'thing' as (x:chararray); >>>> >> > > define concat >>>> >>InvokerGenerator('java.lang.String','concat','String'); >>>> >> > > define valueOf >>>> >> InvokerGenerator('java.lang.Integer','valueOf','String'); >>>> >> > > define valueOfRadix >>>> >> > InvokerGenerator('java.lang.Integer','valueOf','String,int'); >>>> >> > > b = foreach a generate x, valueOf(x) as vOf; >>>> >> > > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR; >>>> >> > > d = foreach c generate x, vOf, vOfR, concat(concat(x, >>>> >>(chararray)vOf), >>>> >> > (chararray)vOfR); >>>> >> > > dump d; >>>> >> > > {code} >>>> >> > > There are some differences between this version and Dmitriy's >>>> >> > implementation: >>>> >> > > - it is no longer necessary to declare whether the method is >>>>static >>>> >>or >>>> >> > not. This is gleaned via reflection. >>>> >> > > - as per the above, it is no longer necessary to make the first >>>> >> argument >>>> >> > be the type of the object to invoke the method on. If it is not a >>>> >>static >>>> >> > method, then the type will implicitly be the type you need. So in >>>>the >>>> >> case >>>> >> > of concat, it would need to be passed a tuple of two inputs: one >>>>for >>>> >>the >>>> >> > method to be called against (as it is not static), and then the >>>> >>'string' >>>> >> > that was specified. In the case of valueOf, because it IS static, >>>>then >>>> >> the >>>> >> > 'String' is the only value. >>>> >> > > - The arguments are type sensitive. Integer means the Object >>>> >>Integer, >>>> >> > whereas int (or long, or float, or boolean, etc) refer to the >>>> >>primitive. >>>> >> > This is necessary to properly reflect the arguments. Values >>>>passed >>>>in >>>> >> WILL, >>>> >> > however, be properly unboxed as necessary. >>>> >> > > - The return type will be reflected. >>>> >> > > This uses the ASM API to generate the bytecode, and then a >>>>custom >>>> >> > classloader to load it in. I will add caching of the generated >>>>code >>>> >>based >>>> >> > on the input strings, etc, but I wanted to get eyes and opinions >>>>on >>>> >> this. I >>>> >> > also need to benchmark, but it should be native speed (excluding >>>>a >>>> >>little >>>> >> > startup time to make the bytecode, but ASM is really fast). >>>> >> > > Another nice benefit is that this bypasses the need for the >>>>JDK, >>>> >>though >>>> >> > it adds a dependency on ASM (which is a super tiny dependency). >>>> >> > > Patch incoming. >>>> >> > >>>> >> > -- >>>> >> > This message is automatically generated by JIRA. >>>> >> > If you think it was sent incorrectly, please contact your JIRA >>>> >> > administrators: >>>> >> > >>>> >>>>>>https://issues.apache.org/jira/secure/ContactAdministrators!default.j >>>>>>sp >>>>>>a >>>> >> > For more information on JIRA, see: >>>> >> http://www.atlassian.com/software/jira >>>> >> > >>>> >> > >>>> >> > >>>> >> >>>> >>
