wonook commented on a change in pull request #28: [NEMO-12] Frontend support
for Scala Spark
URL: https://github.com/apache/incubator-nemo/pull/28#discussion_r193324652
##########
File path:
compiler/frontend/spark/src/main/java/edu/snu/nemo/compiler/frontend/spark/core/SparkFrontendUtils.java
##########
@@ -135,4 +150,74 @@ public static Serializer deriveSerializerFrom(final
SparkContext sparkContext) {
return DataCommunicationPatternProperty.Value.OneToOne;
}
}
+
+ /**
+ * Converts a {@link Function1} to a corresponding {@link Function}.
+ *
+ * @param scalaFunction the scala function to convert.
+ * @param <I> the type of input.
+ * @param <O> the type of output.
+ * @return the converted Java function.
+ */
+ public static <I, O> Function<I, O> toJavaFunction(final Function1<I, O>
scalaFunction) {
+ return new Function<I, O>() {
+ @Override
+ public O call(final I v1) throws Exception {
+ return scalaFunction.apply(v1);
+ }
+ };
+ }
+
+ /**
+ * Converts a {@link scala.Function2} to a corresponding {@link
org.apache.spark.api.java.function.Function2}.
+ *
+ * @param scalaFunction the scala function to convert.
+ * @param <I1> the type of first input.
+ * @param <I2> the type of second input.
+ * @param <O> the type of output.
+ * @return the converted Java function.
+ */
+ public static <I1, I2, O> Function2<I1, I2, O> toJavaFunction(final
scala.Function2<I1, I2, O> scalaFunction) {
+ return new Function2<I1, I2, O>() {
+ @Override
+ public O call(final I1 v1, final I2 v2) throws Exception {
+ return scalaFunction.apply(v1, v2);
+ }
+ };
+ }
+
+ /**
+ * Converts a {@link Function1} to a corresponding {@link FlatMapFunction}.
+ *
+ * @param scalaFunction the scala function to convert.
+ * @param <I> the type of input.
+ * @param <O> the type of output.
+ * @return the converted Java function.
+ */
+ public static <I, O> FlatMapFunction<I, O> toFlatMapFunction(final
Function1<I, TraversableOnce<O>> scalaFunction) {
+ return new FlatMapFunction<I, O>() {
+ @Override
+ public Iterator<O> call(final I i) throws Exception {
+ return
JavaConverters.asJavaIteratorConverter(scalaFunction.apply(i).toIterator()).asJava();
+ }
+ };
+ }
+
+ /**
+ * Converts a {@link PairFunction} to a plain map {@link Function}.
+ *
+ * @param pairFunction the pair function to convert.
+ * @param <T> the type of original element.
+ * @param <K> the type of converted key.
+ * @param <V> the type of converted value.
+ * @return the converted map function.
+ */
+ public static <T, K, V> Function<T, Tuple2<K, V>>
pairFunctionToFunction(final PairFunction<T, K, V> pairFunction) {
Review comment:
`pairFunctionToJavaFunction` and `scalaPairFunction`?
I would like to make this bit clear for everyone
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
