juripetersen commented on code in PR #468:
URL: https://github.com/apache/incubator-wayang/pull/468#discussion_r1745450491
##########
guides/develop-with-Wayang.md:
##########
@@ -71,7 +71,7 @@ This tutorial shows users how to import Wayang in their Java
project using the m
```
A sample pom file can be found [here](pom-example.xml).
-# Test WordCount
+# Test org.apache.wayang.examples.WordCount
Review Comment:
Same here, this doesn't really add anything.
##########
bin/wayang-submit:
##########
@@ -120,5 +119,11 @@ do
ARGS="$ARGS \"${arg}\""
done
+WAYANG_CLASSPATH="${WAYANG_CLASSPATH}:${WAYANG_APP_HOME}"
+
+echo $WAYANG_CLASSPATH
Review Comment:
Are these just for debugging? I don't think they are needed as these echos
are only run after submitting a job.
##########
guides/wayang-examples.md:
##########
@@ -51,7 +51,7 @@ public class WordcountJava {
.withPlugin(Java.basicPlugin())
.withPlugin(Spark.basicPlugin());
JavaPlanBuilder planBuilder = new JavaPlanBuilder(wayangContext)
- .withJobName(String.format("WordCount (%s)", inputUrl))
+
.withJobName(String.format("org.apache.wayang.examples.WordCount (%s)",
inputUrl))
Review Comment:
Same here
##########
guides/wayang-examples.md:
##########
@@ -17,13 +17,13 @@
-->
This page contains examples to be executed using Wayang.
-- [WordCount](#wordcount)
+- [org.apache.wayang.examples.WordCount](#wordcount)
* [Java scala-like API](#java-scala-like-api)
* [Scala API](#scala-api)
- [k-means](#k-means)
* [Scala API](#scala-api-1)
-## WordCount
+## org.apache.wayang.examples.WordCount
Review Comment:
Same here
##########
wayang-api/wayang-api-scala-java/README.md:
##########
@@ -87,12 +87,12 @@ import org.apache.wayang.core.api.Configuration
import org.apache.wayang.java.Java
import org.apache.wayang.spark.Spark
-class WordCount {}
+class org.apache.wayang.examples.WordCount {}
Review Comment:
This is an invalid class name in this case.
##########
wayang-api/wayang-api-scala-java/README.md:
##########
@@ -87,12 +87,12 @@ import org.apache.wayang.core.api.Configuration
import org.apache.wayang.java.Java
import org.apache.wayang.spark.Spark
-class WordCount {}
+class org.apache.wayang.examples.WordCount {}
-object WordCount {
+object org.apache.wayang.examples.WordCount {
Review Comment:
This is an invalid object name in this case.
##########
wayang-applications/pom.xml:
##########
@@ -0,0 +1,139 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang</artifactId>
+ <version>0.7.1</version>
+ </parent>
+
+ <artifactId>wayang-applications</artifactId>
+ <packaging>jar</packaging>
+
+ <name>wayang-applications</name>
+ <url>http://maven.apache.org</url>
+
+
+ <properties>
+
<DATA_REPO_001>file://${project.basedir}/wayang-applications/data/case-study/DATA_REPO_001</DATA_REPO_001>
+
<DATA_REPO_002>https://kamir.solidcommunity.net/public/ecolytiq-sustainability-profile</DATA_REPO_002>
+
<DIST_WAYANG_HOME>${project.basedir}/../wayang-assembly/target/apache-wayang-assembly-0.7.1-incubating-dist/wayang-0.7.1</DIST_WAYANG_HOME>
+ <WAYANG_VERSION>0.7.1</WAYANG_VERSION>
+
+ <spark.version>3.5.0</spark.version>
+ <maven.compiler.source>11</maven.compiler.source>
+ <maven.compiler.target>11</maven.compiler.target>
+
+ <!-- We also specify the file encoding of our source files, to avoid a
warning -->
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+
+ <assertj.version>3.17.2</assertj.version>
+ <commons-io.version>2.5</commons-io.version>
+ <guava.version>19.0</guava.version>
+ <hamcrest.version>1.3</hamcrest.version>
+ <jackson.version>2.10.2</jackson.version>
+ <jacoco.version>0.8.5</jacoco.version>
+ <jodatime.version>2.10.6</jodatime.version>
+ <jsonpath.version>2.4.0</jsonpath.version>
+ <junit5.version>5.6.1</junit5.version>
+ <mockito.version>3.5.10</mockito.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-core</artifactId>
+ <version>${WAYANG_VERSION}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-basic</artifactId>
+ <version>${WAYANG_VERSION}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-java</artifactId>
+ <version>${WAYANG_VERSION}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-spark</artifactId>
+ <version>${WAYANG_VERSION}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-api-scala-java</artifactId>
+ <version>${WAYANG_VERSION}</version>
+ <!--scope>system</scope-->
+
<!--systemPath>/Users/kamir/GITHUB.merge/incubator-wayang/wayang-assembly/target/apache-wayang-assembly-0.7.1-incubating-dist/wayang-0.7.1/jars/wayang-api-scala-java-0.7.1.jar</systemPath-->
Review Comment:
This is local to your machine and can be removed.
##########
wayang-applications/src/main/java/org/apache/wayang/applications/App.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.wayang.applications;
+
+public class App
+{
+ public static void main( String[] args )
+ {
+ System.out.println( "Hello Apache Wayang friends!" );
Review Comment:
This class seems to be obsolete
##########
wayang-docs/src/main/resources/index.md:
##########
@@ -172,7 +172,7 @@ This tool will attempt to determine suitable values for the
question marks (`?`)
For some executable examples, have a look at [this
repository](https://github.com/sekruse/rheem-examples).
-### WordCount
+### org.apache.wayang.examples.WordCount
Review Comment:
Same here
##########
wayang-docs/src/main/resources/index.md:
##########
@@ -198,7 +198,7 @@ public class WordcountJava {
.withPlugin(Java.basicPlugin())
.withPlugin(Spark.basicPlugin());
JavaPlanBuilder planBuilder = new JavaPlanBuilder(wayangContext)
- .withJobName(String.format("WordCount (%s)", inputUrl))
+
.withJobName(String.format("org.apache.wayang.examples.WordCount (%s)",
inputUrl))
Review Comment:
Same here
##########
guides/ml-in-Wayang.md:
##########
@@ -28,7 +28,7 @@ public class CustomEstimatableCost implements EstimatableCost
{
* by implementing the interface in this class.
*/
}
-public class WordCount {
+public class org.apache.wayang.examples.WordCount {
Review Comment:
I think you replaced everything in a recursive search, this is not valid.
##########
guides/tutorial.md:
##########
@@ -16,7 +16,7 @@
limitations under the License.
-->
-This tutorial will show users how to run the WordCount example locally with
Wayang.
+This tutorial will show users how to run the
org.apache.wayang.examples.WordCount example locally with Wayang.
Review Comment:
Same here
##########
wayang-applications/bin/run_wordcount_kafka.sh:
##########
@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+export JAVA_HOME=/Users/kamir/.sdkman/candidates/java/current
Review Comment:
This seems to be hardcoded to your machine?
##########
guides/wayang-examples.md:
##########
@@ -17,13 +17,13 @@
-->
This page contains examples to be executed using Wayang.
-- [WordCount](#wordcount)
+- [org.apache.wayang.examples.WordCount](#wordcount)
Review Comment:
Same here.
##########
guides/tutorial.md:
##########
@@ -57,7 +57,7 @@ source ~/.zshrc
# Run the program
-To execute the WordCount example with Apache Wayang, you need to execute your
program with the 'wayang-submit' command:
+To execute the org.apache.wayang.examples.WordCount example with Apache
Wayang, you need to execute your program with the 'wayang-submit' command:
Review Comment:
Same here
##########
wayang-applications/bin/run_wordcount.sh:
##########
@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+export JAVA_HOME=/Users/kamir/.sdkman/candidates/java/current
Review Comment:
This seems to be hardcoded to your machine?
##########
wayang-api/wayang-api-scala-java/README.md:
##########
@@ -87,12 +87,12 @@ import org.apache.wayang.core.api.Configuration
import org.apache.wayang.java.Java
import org.apache.wayang.spark.Spark
-class WordCount {}
+class org.apache.wayang.examples.WordCount {}
-object WordCount {
+object org.apache.wayang.examples.WordCount {
def main(args: Array[String]): Unit = {
- println("WordCount")
+ println("org.apache.wayang.examples.WordCount")
Review Comment:
Same here
##########
guides/wayang-examples.md:
##########
@@ -107,7 +107,7 @@ object WordcountScala {
.withPlugin(Java.basicPlugin)
.withPlugin(Spark.basicPlugin)
val planBuilder = new PlanBuilder(wayangContext)
- .withJobName(s"WordCount ($inputUrl)")
+ .withJobName(s"org.apache.wayang.examples.WordCount ($inputUrl)")
Review Comment:
Same here
##########
wayang-applications/src/main/java/org/apache/wayang/applications/WordCount.java:
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.wayang.applications;
+
+import org.apache.wayang.api.JavaPlanBuilder;
+import org.apache.wayang.basic.data.Tuple2;
+import org.apache.wayang.core.api.Configuration;
+import org.apache.wayang.core.api.WayangContext;
+import
org.apache.wayang.core.optimizer.cardinality.DefaultCardinalityEstimator;
+import org.apache.wayang.java.Java;
+import org.apache.wayang.spark.Spark;
+import java.util.Collection;
+import java.util.Arrays;
+
+public class WordCount {
Review Comment:
This class just seems to be another ordinary WordCount example that we have
in multiple other modules like wayang-benchmark. Why do we need another one?
##########
wayang-applications/src/main/java/org/apache/wayang/applications/OutputSerializer.java:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.wayang.applications;
+
+import org.apache.wayang.core.function.FunctionDescriptor.SerializableFunction;
+import org.apache.wayang.basic.data.Tuple2;
+
+public class OutputSerializer implements SerializableFunction<Tuple2<String,
Integer>, String> {
+
+ @Override
+ public String apply(Tuple2<String, Integer> tuple) {
+ return tuple.getField0() + ": " + tuple.getField1();
+ }
+
+ // Example usage within a main method or similar test environment
+ public static void main(String[] args) {
Review Comment:
A "utility" class like this should probably not contain a main.
##########
wayang-docs/src/main/resources/index.md:
##########
@@ -254,7 +254,7 @@ object WordcountScala {
.withPlugin(Java.basicPlugin)
.withPlugin(Spark.basicPlugin)
val planBuilder = new PlanBuilder(wayangContext)
- .withJobName(s"WordCount ($inputUrl)")
+ .withJobName(s"org.apache.wayang.examples.WordCount ($inputUrl)")
Review Comment:
Same here
##########
wayang-applications/pom.xml:
##########
@@ -0,0 +1,139 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang</artifactId>
+ <version>0.7.1</version>
+ </parent>
+
+ <artifactId>wayang-applications</artifactId>
+ <packaging>jar</packaging>
+
+ <name>wayang-applications</name>
+ <url>http://maven.apache.org</url>
+
+
+ <properties>
+
<DATA_REPO_001>file://${project.basedir}/wayang-applications/data/case-study/DATA_REPO_001</DATA_REPO_001>
+
<DATA_REPO_002>https://kamir.solidcommunity.net/public/ecolytiq-sustainability-profile</DATA_REPO_002>
+
<DIST_WAYANG_HOME>${project.basedir}/../wayang-assembly/target/apache-wayang-assembly-0.7.1-incubating-dist/wayang-0.7.1</DIST_WAYANG_HOME>
+ <WAYANG_VERSION>0.7.1</WAYANG_VERSION>
Review Comment:
In other pom.xml's we use the naming convention that would result in this
property being named `wayang.version`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
