On 4/13/12 1:30 AM, "Gianmarco De Francisci Morales" <[email protected]> wrote:
>Sure, >probably Java7 is more mature than Pig. >However as far as I know it is a pain to install on Mac. Yes, you have to install using the developer preview, but it is available from the same place all other JRE's from Oracle are. [1] 7u4 should be out in a few weeks [2], which will include a Mac OSX version (Unfortunately only for Lion, and not Snow Leopard). Oracle's official stance is that JRE 6 can go EOL 6 months after Java 7 is widely supported on the mac. They will likely wait a couple months more than that. However, that still puts JRE 6's end this year. [1] http://www.oracle.com/technetwork/java/javase/downloads/index.html [2] http://jdk7.java.net/download.html (build b20, as of April 12, 2012) >This is just one small example of not mature enough. Availability on Mac is probably the biggest example in my opinion. What else is there? Most of the risky features in Java 7's JVM are already back-ported to Java 6. This leaves the new APIs and features as the biggest risks, but most of these should reduce risk (try-with-resources, much better file system API, better exception handling). This leaves mostly MethodHandle and invokevirtual as the new features that are risky -- these are being heavily tested and used by others however (e.g. JRuby). >I am under the impression that Java7 not yet mainstream, correct me if I >am >wrong. What is the definition of mainstream in this situation? Why does it matter? I am struggling to imagine a concrete definition of mainstream that would be useful in this context. Other than fear of new things and a lack of a corporate backed JVM on Mac, we have the real issue that there are significant resources required to test such a change in Pig and its related execution and development environments. However, this issue does not go away -- Java 6 can't be the only supported flavor forever. Some day in the not too distant future, a Java 7 JRE/JDK will be the default and preferred JVM in Pig's ecosystem. It would be nice if high value features that could take advantage of Java 7 features started development early enough to be available at about the time Java 7 was common in the ecosystem. Waiting until Java 7 is mainstream to start such features means adding a 12 month delay into the process. > >I am fine with having optional performance improvements enabled by Java7. >+1 >I am not fine with having a dependency on Java7 for users. -1 >I am not comfortable with having a dependency on Java7 for pig developers. >I am afraid it will put off new contributors. -0 How many potential contributors are interested in being able to try new technologies, versus those only wanting to use what they currently know? > >Cheers, >-- >Gianmarco > > > >On Fri, Apr 13, 2012 at 01:10, Scott Carey <[email protected]> >wrote: > >> >> On 4/12/12 12:02 PM, "Gianmarco De Francisci Morales" <[email protected]> >> wrote: >> >> >My personal opinion is Yes, if there is a compelling reason, but not >>now. >> >Still not mature enough. >> >> What is the definition of "mature enough"? >> >> I'd argue that out of Java 7, Hadoop, Avro, and Pig... Java 7 is the >>most >> mature. >> >> Granted, I also think the whole concept of maturity is subjective and >>not >> conducive to good discussion. >> >> >> What objective characteristics are required of Java 7 to support it? >>When >> approximately would that likely occur? Subtract 6 months from that (a >> typical Pig dev cycle), how soon is that? >> >> There is a big difference between _requiring_ Java 7 and having an >> optional feature built with Java 7 into its own jar that requires a >>Java 7 >> cluster to use. Some performance features might fall into that >>category. >> >> > >> >Cheers, >> >-- >> >Gianmarco >> > >> > >> > >> >On Thu, Apr 12, 2012 at 20:55, Jonathan Coveney <[email protected]> >> >wrote: >> > >> >> Scott Carey brought Java 7 up in PIG-2643, and I think it's >>something we >> >> need to think about. When do we want to start taking advantage of new >> >> features that may not exist on Java 6? Do we ever? >> >> >> >> 2012/4/12 Scott Carey (Commented) (JIRA) <[email protected]> >> >> >> >> > >> >> > [ >> >> > >> >> >> >> >> >>https://issues.apache.org/jira/browse/PIG-2643?page=com.atlassian.jira.pl >> >>>>ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252706#c >>>>om >> >>ment-13252706 >> >> ] >> >> > >> >> > Scott Carey commented on PIG-2643: >> >> > ---------------------------------- >> >> > >> >> > Another thought for this sort of thing: >> >> > >> >> > This might be achievable without bytecode generation and good >> >>performance >> >> > with Java 7 MethodHandles [1][2]. Of course, that would require >>Java >> >>7, >> >> > but Java 6 support ends later year [3], about the time Pig 0.11 >>would >> >>be >> >> > out anyway. >> >> > >> >> > >> >> > [1] >> >> > >> >> >> >> >> >>http://docs.oracle.com/javase/7/docs/api/java/lang/invoke/MethodHandle.ht >> >>ml >> >> > [2] >> >> > >> >> >> >> >> >>http://stackoverflow.com/questions/8823793/methodhandle-what-is-it-all-ab >> >>out >> >> > [3] https://blogs.oracle.com/henrik/entry/updated_java_6_eol_date >> >> > >> >> > > Use bytecode generation to make a performance replacement for >> >> > InvokeForLong, InvokeForString, etc >> >> > > >> >> > >> >> >> >>>>----------------------------------------------------------------------- >>>>-- >> >>------------------------ >> >> > > >> >> > > Key: PIG-2643 >> >> > > URL: >>https://issues.apache.org/jira/browse/PIG-2643 >> >> > > Project: Pig >> >> > > Issue Type: Improvement >> >> > > Reporter: Jonathan Coveney >> >> > > Assignee: Jonathan Coveney >> >> > > Priority: Minor >> >> > > Labels: codegen >> >> > > Fix For: 0.11, 0.10.1 >> >> > > >> >> > > Attachments: PIG-2643-0.patch >> >> > > >> >> > > >> >> > > This is basically to cut my teeth for much more ambitious code >> >> > generation down the line, but I think it could be performance and >> >>useful. >> >> > > the new syntax is: >> >> > > {code}a = load 'thing' as (x:chararray); >> >> > > define concat >> >>InvokerGenerator('java.lang.String','concat','String'); >> >> > > define valueOf >> >> InvokerGenerator('java.lang.Integer','valueOf','String'); >> >> > > define valueOfRadix >> >> > InvokerGenerator('java.lang.Integer','valueOf','String,int'); >> >> > > b = foreach a generate x, valueOf(x) as vOf; >> >> > > c = foreach b generate x, vOf, valueOfRadix(x, 16) as vOfR; >> >> > > d = foreach c generate x, vOf, vOfR, concat(concat(x, >> >>(chararray)vOf), >> >> > (chararray)vOfR); >> >> > > dump d; >> >> > > {code} >> >> > > There are some differences between this version and Dmitriy's >> >> > implementation: >> >> > > - it is no longer necessary to declare whether the method is >>static >> >>or >> >> > not. This is gleaned via reflection. >> >> > > - as per the above, it is no longer necessary to make the first >> >> argument >> >> > be the type of the object to invoke the method on. If it is not a >> >>static >> >> > method, then the type will implicitly be the type you need. So in >>the >> >> case >> >> > of concat, it would need to be passed a tuple of two inputs: one >>for >> >>the >> >> > method to be called against (as it is not static), and then the >> >>'string' >> >> > that was specified. In the case of valueOf, because it IS static, >>then >> >> the >> >> > 'String' is the only value. >> >> > > - The arguments are type sensitive. Integer means the Object >> >>Integer, >> >> > whereas int (or long, or float, or boolean, etc) refer to the >> >>primitive. >> >> > This is necessary to properly reflect the arguments. Values passed >>in >> >> WILL, >> >> > however, be properly unboxed as necessary. >> >> > > - The return type will be reflected. >> >> > > This uses the ASM API to generate the bytecode, and then a custom >> >> > classloader to load it in. I will add caching of the generated code >> >>based >> >> > on the input strings, etc, but I wanted to get eyes and opinions on >> >> this. I >> >> > also need to benchmark, but it should be native speed (excluding a >> >>little >> >> > startup time to make the bytecode, but ASM is really fast). >> >> > > Another nice benefit is that this bypasses the need for the JDK, >> >>though >> >> > it adds a dependency on ASM (which is a super tiny dependency). >> >> > > Patch incoming. >> >> > >> >> > -- >> >> > This message is automatically generated by JIRA. >> >> > If you think it was sent incorrectly, please contact your JIRA >> >> > administrators: >> >> > >> >>>>https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp >>>>a >> >> > For more information on JIRA, see: >> >> http://www.atlassian.com/software/jira >> >> > >> >> > >> >> > >> >> >>
