[
https://issues.apache.org/jira/browse/CRUNCH-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090802#comment-15090802
]
David Whiting edited comment on CRUNCH-585 at 1/9/16 10:12 PM:
---------------------------------------------------------------
I think it's possible to set up a profile for java 8 conditional on jdk version
>=1.8, then add the additional crunch-lambda module as part of the profile
definition. I tried this:
{code:xml}
<profiles>
<profile>
<id>java-8</id>
<activation>
<jdk>[1.8,]</jdk>
</activation>
<modules>
<module>crunch-lambda</module>
</modules>
</profile>
</profiles>
{code}
and it seems to compile everything as expected, but I don't have a <=1.7 jdk to
make sure it doesn't try and compile the module on that.
I can add this change to the next patch version (also with some minor fixes and
better testing and documentation).
was (Author: davw):
I think it's possible to set up a profile for java 8 conditional on jdk version
>=1.8, then add the additional crunch-lambda module as part of the profile
definition. I tried this:
<profiles>
<profile>
<id>java-8</id>
<activation>
<jdk>[1.8,]</jdk>
</activation>
<modules>
<module>crunch-lambda</module>
</modules>
</profile>
</profiles>
and it seems to compile everything as expected, but I don't have a <=1.7 jdk to
make sure it doesn't try and compile the module on that.
I can add this change to the next patch version (also with some minor fixes and
better testing and documentation).
> Move Java 8 lambda support into separate module
> ------------------------------------------------
>
> Key: CRUNCH-585
> URL: https://issues.apache.org/jira/browse/CRUNCH-585
> Project: Crunch
> Issue Type: Improvement
> Reporter: David Whiting
> Fix For: 0.14.0
>
> Attachments: 0001-Java-8-lambda-support-for-Apache-Crunch.patch,
> 0001-Java-8-lambda-support-for-Apache-Crunch.patch
>
>
> As discussed on a previous dev list thread, this patch implements a set of
> operations to conveniently use Java 8 lambda expressions and method
> references to construct Crunch pipelines by wrapping the PCollection
> instances into analogous "LCollection" instances which delegate the necessary
> operations, in much the same way the Scrunch wraps the Crunch Core API.
> I'm still not 100% convinced that this is better for the user than the
> existing lambda support via IMapFn and IDoFn PCollection operations, so I'm
> still interested in people's views on this.
> Advantages:
> - Concise self-contained implementation
> - Methods implemented in terms of a very basic subset of PCollection
> operations (useful if we want to scale down the PCollection API at some point)
> - API can be written in terms of the Java 8 library, operating on streams and
> functional interfaces, making in more familiar to a new developer.
> - Retain "type '.' and see what I can do" experience.
> - Really easy to add new operations (just default method on interface)
> Disadvantages:
> - PCollections must be wrapped into LCollections before use.
> - LCollections must be unwrapped into PCollections to access some existing
> operations.
> - Using counters and other contextual data is far more complex.
> Some limitations of this particular patch:
> - Some omissions in API (not sure how much to implement)
> - No Javadocs yet.
> - Very poor tests.
> - Naming is a bit off (eg. reduce() or reduceValues(), get() or underlying())
> I can fix all that, but I wanted to bring the community in at this point to
> get some feedback on both the idea and the implementation as it's quite a big
> patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
