Hi Max,
You are right. The uber jar is built in "intsall" or "package" stage and Yarn
test assumes the flink uber jar to pre-exist. We may have to document the steps
so that it will not be missed when someone else tries in the future.
I have also noticed errors with the "CliFrontendAddressConfigurationTest"
(happens when you execute mvn clean install/verify after running Yarn test) and
the reason is because the temporary file
(https://github.com/apache/flink/blob/f2e9c521aa4e0cbc14b71c561b8c5dd46201a45a/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L79)
is not deleted and the test code picks up wrong job manager address. I had to
manually delete the /tmp/.yarn-properties-<USER> file before running the test
code again.
RegardsVijay
On Tuesday, June 28, 2016 1:53 AM, Maximilian Michels <m...@apache.org>
wrote:
Hi Vijay,
Hasn't been failing for me whenever I pulled in the latest master.
Neither when I run "mvn clean verify" nor when I execute the test from
within IntelliJ. That said, there are some unstable tests that fail on
Travis.
>YARNSessionFIFOITCase.setup:71->YarnTestBase.startYARNWithConfig:343 Flink
>uberjar not found
Your test run fails because it can't find the Flink jar. Most likely,
you have not run 'mvn clean install' before running the Yarn tests.
You need to 'package' or 'install' at least flink-yarn in order to
execute flink-yarn-tests.
Cheers,
Max
On Mon, Jun 27, 2016 at 6:07 PM, Vijay Srinivasaraghavan
<vijikar...@yahoo.com.invalid> wrote:
> I just pulled the latest code and ran below commands. Both of them failed and
> it is consistent.
> 1) "mvn clean install"
> Results :
>
> Failed tests:
> LocalFlinkMiniClusterITCase.testLocalFlinkMiniClusterWithMultipleTaskManagers:166
> Thread Thread[initialSeedUniquifierGenerator,5,main] was started by the mini
> cluster, but not shut down
> Tests run: 1269, Failures: 1, Errors: 0, Skipped: 9
> [INFO]
> ------------------------------------------------------------------------[INFO]
> Reactor Summary:[INFO] [INFO] force-shading
> ...................................... SUCCESS [ 1.038 s][INFO] flink
> .............................................. SUCCESS [ 2.097 s][INFO]
> flink-annotations .................................. SUCCESS [ 1.045
> s][INFO] flink-shaded-hadoop ................................ SUCCESS [
> 0.107 s][INFO] flink-shaded-hadoop2 ............................... SUCCESS [
> 5.412 s][INFO] flink-shaded-include-yarn-tests .................... SUCCESS
> [ 6.747 s][INFO] flink-shaded-curator ...............................
> SUCCESS [ 0.101 s][INFO] flink-shaded-curator-recipes
> ....................... SUCCESS [ 0.843 s][INFO] flink-shaded-curator-test
> .......................... SUCCESS [ 0.148 s][INFO] flink-core
> ......................................... SUCCESS [ 35.891 s][INFO]
> flink-java ......................................... SUCCESS [ 14.604
> s][INFO] flink-runtime ...................................... SUCCESS [03:34
> min][INFO] flink-optimizer .................................... SUCCESS [
> 6.584 s][INFO] flink-clients ...................................... SUCCESS [
> 18.831 s][INFO] flink-test-utils ................................... SUCCESS
> [ 4.042 s][INFO] flink-scala ........................................
> SUCCESS [ 21.669 s][INFO] flink-runtime-web
> .................................. SUCCESS [ 11.744 s][INFO]
> flink-streaming-java ............................... SUCCESS [ 53.149
> s][INFO] flink-examples ..................................... SUCCESS [
> 0.826 s][INFO] flink-examples-batch ............................... SUCCESS [
> 9.919 s][INFO] flink-contrib ...................................... SUCCESS
> [ 0.118 s][INFO] flink-statebackend-rocksdb .........................
> SUCCESS [ 7.223 s][INFO] flink-tests
> ........................................ FAILURE [05:27 min][INFO]
> flink-streaming-scala .............................. SKIPPED[INFO]
> flink-streaming-connectors ......................... SKIPPED[INFO]
> flink-connector-flume .............................. SKIPPED[INFO]
> flink-libraries .................................... SKIPPED[INFO]
> flink-table ........................................ SKIPPED[INFO]
> flink-connector-kafka-base ......................... SKIPPED[INFO]
> flink-connector-kafka-0.8 .......................... SKIPPED[INFO]
> flink-connector-kafka-0.9 .......................... SKIPPED[INFO]
> flink-connector-elasticsearch ...................... SKIPPED[INFO]
> flink-connector-elasticsearch2 ..................... SKIPPED[INFO]
> flink-connector-rabbitmq ........................... SKIPPED[INFO]
> flink-connector-twitter ............................ SKIPPED[INFO]
> flink-connector-nifi ............................... SKIPPED[INFO]
> flink-connector-cassandra .......................... SKIPPED[INFO]
> flink-connector-filesystem ......................... SKIPPED[INFO]
> flink-batch-connectors ............................. SKIPPED[INFO] flink-avro
> ......................................... SKIPPED[INFO] flink-jdbc
> ......................................... SKIPPED[INFO]
> flink-hadoop-compatibility ......................... SKIPPED[INFO]
> flink-hbase ........................................ SKIPPED[INFO]
> flink-hcatalog ..................................... SKIPPED[INFO]
> flink-examples-streaming ........................... SKIPPED[INFO]
> flink-gelly ........................................ SKIPPED[INFO]
> flink-gelly-scala .................................. SKIPPED[INFO]
> flink-gelly-examples ............................... SKIPPED[INFO]
> flink-python ....................................... SKIPPED[INFO] flink-ml
> ........................................... SKIPPED[INFO] flink-cep
> .......................................... SKIPPED[INFO] flink-cep-scala
> .................................... SKIPPED[INFO] flink-scala-shell
> .................................. SKIPPED[INFO] flink-quickstart
> ................................... SKIPPED[INFO] flink-quickstart-java
> .............................. SKIPPED[INFO] flink-quickstart-scala
> ............................. SKIPPED[INFO] flink-storm
> ........................................ SKIPPED[INFO] flink-storm-examples
> ............................... SKIPPED[INFO] flink-streaming-contrib
> ............................ SKIPPED[INFO] flink-tweet-inputformat
> ............................ SKIPPED[INFO] flink-operator-stats
> ............................... SKIPPED[INFO] flink-connector-wikiedits
> .......................... SKIPPED[INFO] flink-yarn
> ......................................... SKIPPED[INFO] flink-dist
> ......................................... SKIPPED[INFO] flink-metrics
> ...................................... SKIPPED[INFO] flink-metrics-dropwizard
> ........................... SKIPPED[INFO] flink-metrics-ganglia
> .............................. SKIPPED[INFO] flink-metrics-graphite
> ............................. SKIPPED[INFO] flink-metrics-statsd
> ............................... SKIPPED[INFO] flink-fs-tests
> ..................................... SKIPPED[INFO] flink-java8
> ........................................ SKIPPED[INFO]
> ------------------------------------------------------------------------[INFO]
> BUILD FAILURE[INFO]
> ------------------------------------------------------------------------[INFO]
> Total time: 12:24 min[INFO] Finished at: 2016-06-27T08:19:18-07:00[INFO]
> Final Memory: 147M/4711M[INFO]
> ------------------------------------------------------------------------[ERROR]
> Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (integration-tests) on project flink-tests_2.10: There are test
> failures.[ERROR] [ERROR] Please refer to
> /workspace/git-projects/junk/flink/flink-tests/target/surefire-reports for
> the individual test results.[ERROR] -> [Help 1][ERROR] [ERROR] To see the
> full stack trace of the errors, re-run Maven with the -e switch.[ERROR]
> Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR]
> For more information about the errors and possible solutions, please read the
> following articles:[ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command[ERROR] mvn <goals> -rf :flink-tests_2.10
> 2) mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests
> -Dtest=YARNSessionFIFOITCase#testJavaAPI -DfailIfNoTests=false
> ------------------------------------------------------- T E S T
> S-------------------------------------------------------Running
> org.apache.flink.yarn.YARNSessionFIFOITCase08:28:32,368 INFO
> org.apache.flink.yarn.YarnTestBase - Trying to
> locate uberjar in ..Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 0.366 sec <<< FAILURE! - in
> org.apache.flink.yarn.YARNSessionFIFOITCaseorg.apache.flink.yarn.YARNSessionFIFOITCase
> Time elapsed: 0.366 sec <<< FAILURE!java.lang.AssertionError: Flink
> uberjar not found at org.junit.Assert.fail(Assert.java:88) at
> org.junit.Assert.assertTrue(Assert.java:41) at
> org.junit.Assert.assertNotNull(Assert.java:621) at
> org.apache.flink.yarn.YarnTestBase.startYARNWithConfig(YarnTestBase.java:343)
> at
> org.apache.flink.yarn.YARNSessionFIFOITCase.setup(YARNSessionFIFOITCase.java:71)
>
> Results :
> Failed tests:
> YARNSessionFIFOITCase.setup:71->YarnTestBase.startYARNWithConfig:343 Flink
> uberjar not found
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
> [INFO]
> ------------------------------------------------------------------------[INFO]
> BUILD FAILURE[INFO]
> ------------------------------------------------------------------------[INFO]
> Total time: 23.960 s[INFO] Finished at: 2016-06-27T08:28:32-07:00[INFO]
> Final Memory: 135M/3439M[INFO]
> ------------------------------------------------------------------------[ERROR]
> Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
> (integration-tests) on project flink-yarn-tests_2.10: There are test
> failures.[ERROR] [ERROR] Please refer to
> /workspace/git-projects/junk/flink/flink-yarn-tests/target/surefire-reports
> for the individual test results.[ERROR] -> [Help 1][ERROR] [ERROR] To see the
> full stack trace of the errors, re-run Maven with the -e switch.[ERROR]
> Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR]
> For more information about the errors and possible solutions, please read the
> following articles:[ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>
>
> On Monday, June 27, 2016 7:16 AM, Aljoscha Krettek <aljos...@apache.org>
>wrote:
>
>
> Hi,the first command you mentioned doesn't work because the Yarn tests are
>not executed in the traditional phases and therefore maven will complain. If
>you do it like this it will work (I tried it):
> mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests
> -Dtest=YARNSessionFIFOITCase#testJavaAPI -DfailIfNoTests=false
>
> (notice the added -DfailIfNoTests=false).
> I also ran a "mvn clean install" afterwards and it worked as well. Could you
> try with the extended command from above?
> Cheers,Aljoscha
>
>
> On Mon, 27 Jun 2016 at 14:57 Vijay Srinivasaraghavan <vijikar...@yahoo.com>
> wrote:
>
>
> I am on Ubuntu 16.x, Java OpenJDK 1.8.0_91.
>
> Can you try below commands and see if its working with the latest trunk code.
> mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests
> -Dtest=YARNSessionFIFOITCase#testJavaAPI
>
> mvn clean install
> RegardsVijay
>
> On Monday, June 27, 2016 5:44 AM, Aljoscha Krettek <aljos...@apache.org>
>wrote:
>
>
> Hi,I just ran a "mvn clean verify" and it passed on my machine (latest
>master, OS X El Capitan, Java 1.8.0_40, Maven 3.3.9). What's your environment?
> Cheers,Aljoscha
> On Fri, 24 Jun 2016 at 16:47 Vijay Srinivasaraghavan
> <vijikar...@yahoo.com.invalid> wrote:
>
> I am seeing below failure consistently with the latest trunk code when I run
> "mvn clean verify". Is anyone seeing similar error in your environment?
> Failed tests:
> LocalFlinkMiniClusterITCase.testLocalFlinkMiniClusterWithMultipleTaskManagers:166
> Thread Thread[initialSeedUniquifierGenerator,5,main] was started by the mini
> cluster, but not shut down
> RegardsVijay
>
>
>
>
>
>
