Repository: flink Updated Branches: refs/heads/master 3042b182c -> 9d3f127e4
[FLINK-6195] [build] Move gelly-examples jar from opt to examples The opt directory should be reserved for Flink JARs which users may optionally move to lib to be loaded by the runtime. flink-gelly-examples is a user program so is moved to the examples folder. This closes #3691 Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/568e6428 Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/568e6428 Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/568e6428 Branch: refs/heads/master Commit: 568e64282eaa2671b4e46655e1fa773f9bb1e492 Parents: 3042b18 Author: Greg Hogan <c...@greghogan.com> Authored: Tue Apr 18 09:27:46 2017 -0400 Committer: Greg Hogan <c...@greghogan.com> Committed: Tue Apr 18 09:27:46 2017 -0400 ---------------------------------------------------------------------- docs/dev/libs/gelly/index.md | 26 +++++++++----------- flink-dist/pom.xml | 20 +++++++++------ flink-dist/src/main/assemblies/bin.xml | 8 ++++++ flink-dist/src/main/assemblies/opt.xml | 7 ------ .../java/org/apache/flink/graph/Runner.java | 4 +-- 5 files changed, 35 insertions(+), 30 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/568e6428/docs/dev/libs/gelly/index.md ---------------------------------------------------------------------- diff --git a/docs/dev/libs/gelly/index.md b/docs/dev/libs/gelly/index.md index 40018e8..193ba86 100644 --- a/docs/dev/libs/gelly/index.md +++ b/docs/dev/libs/gelly/index.md @@ -71,24 +71,22 @@ The remaining sections provide a description of available methods and present se Running Gelly Examples ---------------------- -The Gelly library and examples jars are provided in the [Flink distribution](https://flink.apache.org/downloads.html "Apache Flink: Downloads") -in the folder **opt** (for versions older than Flink 1.2 these can be manually downloaded from -[Maven Central](http://search.maven.org/#search|ga|1|flink%20gelly)). - -To run the Gelly examples the **flink-gelly** (for Java) or **flink-gelly-scala** (for Scala) jar must be copied to -Flink's **lib** directory. +The Gelly library jars are provided in the [Flink distribution](https://flink.apache.org/downloads.html "Apache Flink: Downloads") +in the **opt** directory (for versions older than Flink 1.2 these can be manually downloaded from +[Maven Central](http://search.maven.org/#search|ga|1|flink%20gelly)). To run the Gelly examples the **flink-gelly** (for +Java) or **flink-gelly-scala** (for Scala) jar must be copied to Flink's **lib** directory. ~~~bash cp opt/flink-gelly_*.jar lib/ cp opt/flink-gelly-scala_*.jar lib/ ~~~ -Gelly's examples jar includes drivers for each of the library methods. After configuring and starting the cluster, list -the available algorithm classes: +Gelly's examples jar includes drivers for each of the library methods and is provided in the **examples** directory. +After configuring and starting the cluster, list the available algorithm classes: ~~~bash ./bin/start-cluster.sh -./bin/flink run opt/flink-gelly-examples_*.jar +./bin/flink run examples/flink-gelly-examples_*.jar ~~~ The Gelly drivers can generate graph data or read the edge list from a CSV file (each node in a cluster must have access @@ -96,13 +94,13 @@ to the input file). The algorithm description, available inputs and outputs, and algorithm is selected. Print usage for [JaccardIndex](./library_methods.html#jaccard-index): ~~~bash -./bin/flink run opt/flink-gelly-examples_*.jar --algorithm JaccardIndex +./bin/flink run examples/flink-gelly-examples_*.jar --algorithm JaccardIndex ~~~ Display [graph metrics](./library_methods.html#metric) for a million vertex graph: ~~~bash -./bin/flink run opt/flink-gelly-examples_*.jar \ +./bin/flink run examples/flink-gelly-examples_*.jar \ --algorithm GraphMetrics --order directed \ --input RMatGraph --type integer --scale 20 --simplify directed \ --output print @@ -119,17 +117,17 @@ Run a few algorithms and monitor the job progress in Flink's Web UI: ~~~bash wget -O - http://snap.stanford.edu/data/bigdata/communities/com-lj.ungraph.txt.gz | gunzip -c > com-lj.ungraph.txt -./bin/flink run -q opt/flink-gelly-examples_*.jar \ +./bin/flink run -q examples/flink-gelly-examples_*.jar \ --algorithm GraphMetrics --order undirected \ --input CSV --type integer --simplify undirected --input_filename com-lj.ungraph.txt --input_field_delimiter $'\t' \ --output print -./bin/flink run -q opt/flink-gelly-examples_*.jar \ +./bin/flink run -q examples/flink-gelly-examples_*.jar \ --algorithm ClusteringCoefficient --order undirected \ --input CSV --type integer --simplify undirected --input_filename com-lj.ungraph.txt --input_field_delimiter $'\t' \ --output hash -./bin/flink run -q opt/flink-gelly-examples_*.jar \ +./bin/flink run -q examples/flink-gelly-examples_*.jar \ --algorithm JaccardIndex \ --input CSV --type integer --simplify undirected --input_filename com-lj.ungraph.txt --input_field_delimiter $'\t' \ --output hash http://git-wip-us.apache.org/repos/asf/flink/blob/568e6428/flink-dist/pom.xml ---------------------------------------------------------------------- diff --git a/flink-dist/pom.xml b/flink-dist/pom.xml index 0eea0aa..e42aea6 100644 --- a/flink-dist/pom.xml +++ b/flink-dist/pom.xml @@ -156,6 +156,19 @@ under the License. </dependency> <!-- + The following dependencies are packaged in 'examples/' + The scope of these dependencies needs to be 'provided' so that + they are not included into the 'flink-dist' uber jar. + --> + + <dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-gelly-examples_2.10</artifactId> + <version>${project.version}</version> + <scope>provided</scope> + </dependency> + + <!-- The following dependencies are packaged in 'opt/' The scope of these dependencies needs to be 'provided' so that they are not included into the 'flink-dist' uber jar. @@ -222,13 +235,6 @@ under the License. <dependency> <groupId>org.apache.flink</groupId> - <artifactId>flink-gelly-examples_2.10</artifactId> - <version>${project.version}</version> - <scope>provided</scope> - </dependency> - - <dependency> - <groupId>org.apache.flink</groupId> <artifactId>flink-ml_2.10</artifactId> <version>${project.version}</version> <scope>provided</scope> http://git-wip-us.apache.org/repos/asf/flink/blob/568e6428/flink-dist/src/main/assemblies/bin.xml ---------------------------------------------------------------------- diff --git a/flink-dist/src/main/assemblies/bin.xml b/flink-dist/src/main/assemblies/bin.xml index e1ee00e..d3f2005 100644 --- a/flink-dist/src/main/assemblies/bin.xml +++ b/flink-dist/src/main/assemblies/bin.xml @@ -59,6 +59,14 @@ under the License. <outputDirectory>conf</outputDirectory> <fileMode>0644</fileMode> </file> + + <!-- copy jar file of the Gelly examples --> + <file> + <source>../flink-libraries/flink-gelly-examples/target/flink-gelly-examples_2.10-${project.version}.jar</source> + <outputDirectory>examples/</outputDirectory> + <destName>flink-gelly-examples_2.10-${project.version}.jar</destName> + <fileMode>0644</fileMode> + </file> </files> <fileSets> http://git-wip-us.apache.org/repos/asf/flink/blob/568e6428/flink-dist/src/main/assemblies/opt.xml ---------------------------------------------------------------------- diff --git a/flink-dist/src/main/assemblies/opt.xml b/flink-dist/src/main/assemblies/opt.xml index c6dc307..3622ece 100644 --- a/flink-dist/src/main/assemblies/opt.xml +++ b/flink-dist/src/main/assemblies/opt.xml @@ -53,13 +53,6 @@ </file> <file> - <source>../flink-libraries/flink-gelly-examples/target/flink-gelly-examples_2.10-${project.version}.jar</source> - <outputDirectory>opt/</outputDirectory> - <destName>flink-gelly-examples_2.10-${project.version}.jar</destName> - <fileMode>0644</fileMode> - </file> - - <file> <source>../flink-libraries/flink-gelly-scala/target/flink-gelly-scala_2.10-${project.version}-jar-with-dependencies.jar</source> <outputDirectory>opt/</outputDirectory> <destName>flink-gelly-scala_2.10-${project.version}.jar</destName> http://git-wip-us.apache.org/repos/asf/flink/blob/568e6428/flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java ---------------------------------------------------------------------- diff --git a/flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java b/flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java index 0324814..4b6cf42 100644 --- a/flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java +++ b/flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/Runner.java @@ -108,7 +108,7 @@ public class Runner { strBuilder .appendNewLine() - .appendln("Select an algorithm to view usage: flink run opt/flink-gelly-examples_<version>.jar --algorithm <algorithm>") + .appendln("Select an algorithm to view usage: flink run examples/flink-gelly-examples_<version>.jar --algorithm <algorithm>") .appendNewLine() .appendln("Available algorithms:"); @@ -139,7 +139,7 @@ public class Runner { .appendNewLine() .appendln(algorithm.getLongDescription()) .appendNewLine() - .append("usage: flink run opt/flink-gelly-examples_<version>.jar --algorithm ") + .append("usage: flink run examples/flink-gelly-examples_<version>.jar --algorithm ") .append(algorithmName) .append(" [algorithm options] --input <input> [input options] --output <output> [output options]") .appendNewLine()
