+1 - verified compilation, tests - verified checksum and gpg files - verified sbt templates (g8, quickstart) - run assemblies on local cluster
- I could not execute the nightly-tests.sh though. The tests that were failing most often are: - test_streaming_file_sink.sh - test_streaming_elasticsearch.sh Those are connectors though and it might be only tests flakiness so I think it should not block the release. On 08/08/18 16:36, Chesnay Schepler wrote: > I did not use the tools/list_deps.py script as I wasn't aware that it > existed. > > Even if I were I wouldn't have used it and in fact would advocate for > removing it. > It manually parses and constructs dependency information which is > utterly unnecessary as maven already provides this functionality, with > the added bonus of also accounting for dependencyManagement and > transitive dependencies which we obviously have to take into account. > > I used this one-liner instead: > |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e > ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed| > |'s/:[a-z]*$//g'| || ||sort| |-u > > |which I have documented here: > https://cwiki.apache.org/confluence/display/FLINK/Dependencies > > On 08.08.2018 15:06, Aljoscha Krettek wrote: >> +1 >> >> - verified checksum and gpg files >> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE >> had one unnecessary part removed >> >> Side comment: I'm not sure whether the "Verify that the LICENSE and >> NOTICE file is correct for the binary and source releases" part is >> valid anymore because we only have one LICENSE and NOTICE file. also >> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer >> to the binary distribution and mention all of Flink's Maven >> dependencies as well" can be dropped because we don't have them anymore. >> >> I came to the same conclusion on dependencies. I used >> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's >> probably what Chesnay also did ... :-) >> >>> On 8. Aug 2018, at 14:43, Chesnay Schepler <ches...@apache.org> wrote: >>> >>> +1 >>> >>> - verified source release contains no binaries >>> - verified correct versions in source release >>> - verified compilation, tests and E2E-tests pass (on travis) >>> - verified checksum and gpg files >>> >>> New dependencies (excluding dependencies where we simply depend on a >>> different version now): >>> Apache licensed: >>> io.confluent:common-utils:jar:3.3.1 >>> io.confluent:kafka-schema-registry-client:jar:3.3.1 >>> io.prometheus:simpleclient_pushgateway:jar:0.3.0 >>> various Apache Nifi dependencies >>> various Apache Parquet dependencies >>> various ElasticSearch dependencies >>> CDDL: >>> javax.ws.rs:javax.ws.rs-api:jar:2.1 >>> Bouncycastle (MIT-like): >>> org.bouncycastle:bcpkix-jdk15on:jar:1.59 >>> org.bouncycastle:bcprov-jdk15on:jar:1.59 >>> MIT: >>> org.projectlombok:lombok:jar:1.16.20 >>> >>> On 08.08.2018 13:28, Till Rohrmann wrote: >>>> Thanks for reporting these problems Chesnay. The usage string in >>>> `standalone-job.sh` is out dated and should be updated. The same >>>> applies to >>>> the typo. >>>> >>>> When calling `standalone-job.sh start --job-classname foobar.Job` >>>> please >>>> make sure that the user code jar is contained in the classpath (e.g. >>>> putting the jar in the lib directory). Documenting this behaviour >>>> is part >>>> of the pending issue FLINK-10001. >>>> >>>> We should fix all of these issues. They are, however, no release >>>> blockers. >>>> >>>> Cheers, >>>> Till >>>> >>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler >>>> <ches...@apache.org> wrote: >>>> >>>>> I found some issues with the standalone-job.sh script. >>>>> >>>>> I ran "./bin/standalone-job.sh start" as described by the usage >>>>> string. >>>>> >>>>> 2018-08-08 09:22:34,385 ERROR >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>>> Could not parse command line arguments [--configDir, >>>>> /home/zento/svn/flink-1.6.0/flink-1.6.0/conf]. >>>>> org.apache.flink.runtime.entrypoint.FlinkParseException: >>>>> Failed to >>>>> parse the command line arguments. >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143) >>>>> >>>>> Caused by: org.apache.commons.cli.MissingOptionException: >>>>> Missing >>>>> required option: j >>>>> at >>>>> >>>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199) >>>>> >>>>> at >>>>> >>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130) >>>>> at >>>>> >>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81) >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50) >>>>> >>>>> ... 1 more >>>>> >>>>> The script should fail earlier if no jar is provided, with a better >>>>> error message. >>>>> It is also undocumented, and the usage instructions don't appear >>>>> correct. >>>>> >>>>> Passing a jar with the -j option leads to a ClassNotFoundException: >>>>> "./bin/standalone-job.sh start -j examples/streaming/WordCount.jar" >>>>> >>>>> 2018-08-08 09:26:30,562 ERROR >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>>> Cluster initialization failed. >>>>> java.lang.reflect.UndeclaredThrowableException >>>>> at >>>>> >>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158) >>>>> >>>>> Caused by: org.apache.flink.util.FlinkException: Could not >>>>> load the >>>>> provied entrypoint class. >>>>> at >>>>> >>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232) >>>>> >>>>> at >>>>> >>>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190) >>>>> >>>>> at >>>>> java.security.AccessController.doPrivileged(Native Method) >>>>> at javax.security.auth.Subject.doAs(Subject.java:422) >>>>> at >>>>> >>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836) >>>>> >>>>> ... 3 more >>>>> Caused by: java.lang.ClassNotFoundException: >>>>> examples/streaming/WordCount.jar >>>>> at >>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>> at >>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) >>>>> at >>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> >>>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89) >>>>> >>>>> ... 11 more >>>>> >>>>> So this seems to not work at all, but maybe I'm using it wrong? >>>>> >>>>> (There's also typo in "Could not load the provied entrypoint class") >>>>> >>>>> On 08.08.2018 10:33, Piotr Nowojski wrote: >>>>>> +1 from my side >>>>>> >>>>>> I’ve spent some time playing around with various examples (batching, >>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with >>>>> different configuration options (number of task >>>>> managers/memory/Flip6/credit base flow control/metrics) and >>>>> everything >>>>> looks now fine (after fixing >>>>> https://issues.apache.org/jira/browse/FLINK-9969 < >>>>> https://issues.apache.org/jira/browse/FLINK-9969> ). >>>>>> Piotrek >>>>>> >>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrm...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>> Hi everyone, >>>>>>> Please review and vote on the release candidate #4 for the version >>>>> 1.6.0, >>>>>>> as follows: >>>>>>> [ ] +1, Approve the release >>>>>>> [ ] -1, Do not approve the release (please provide specific >>>>>>> comments) >>>>>>> >>>>>>> >>>>>>> The complete staging area is available for your review, which >>>>>>> includes: >>>>>>> * JIRA release notes [1], >>>>>>> * the official Apache source release and binary convenience >>>>>>> releases to >>>>> be >>>>>>> deployed to dist.apache.org [2], which are signed with the key with >>>>>>> fingerprint 1F302569A96CFFD5 [3], >>>>>>> * all artifacts to be deployed to the Maven Central Repository [4], >>>>>>> * source code tag "release-1.6.0-rc4" [5], >>>>>>> * website pull request listing the new release and adding >>>>>>> announcement >>>>> blog >>>>>>> post [6]. >>>>>>> >>>>>>> Please use this document for coordinating testing efforts: [7] >>>>>>> >>>>>>> The vote will be shortened since we only adde a minor fix on top >>>>>>> of the >>>>> RC >>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by majority >>>>>>> approval, with at least 3 PMC affirmative votes. >>>>>>> >>>>>>> Thanks, >>>>>>> Your friendly Release Manager >>>>>>> >>>>>>> [1] >>>>>>> >>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760 >>>>> >>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/ >>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS >>>>>>> [4] >>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178 >>>>> >>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4 >>>>>>> [6] https://github.com/apache/flink-web/pull/117 >>>>>>> [7] >>>>>>> >>>>> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing >>>>> >>>>>>> Pro-tip: you can create a settings.xml file with these contents: >>>>>>> >>>>>>> <settings> >>>>>>> <activeProfiles> >>>>>>> <activeProfile>flink-1.6.0</activeProfile> >>>>>>> </activeProfiles> >>>>>>> <profiles> >>>>>>> <profile> >>>>>>> <id>flink-1.6.0</id> >>>>>>> <repositories> >>>>>>> <repository> >>>>>>> <id>flink-1.6.0</id> >>>>>>> <url> >>>>>>> >>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/ >>>>>>> >>>>>>> </url> >>>>>>> </repository> >>>>>>> <repository> >>>>>>> <id>archetype</id> >>>>>>> <url> >>>>>>> >>>>>>> https://repository.apache.org/content/repositories/orgapacheflink-1178/ >>>>>>> >>>>>>> </url> >>>>>>> </repository> >>>>>>> </repositories> >>>>>>> </profile> >>>>>>> </profiles> >>>>>>> </settings> >>>>>>> >>>>>>> And reference that in you maven commands via --settings >>>>>>> path/to/settings.xml. This is useful for creating a quickstart >>>>>>> based on >>>>> the >>>>>>> staged release and for building against the staged jars. >> > >
