+1
- successfully run `mvn clean verify` locally
- successfully run end-to-end tests locally (except for SQL Client
end-to-end test)
Found a bug in the class loading of SQL JAR files. This is not a blocker
but a bug that we should fix soon. As an easy workaround user should not
use different Kafka versions as SQL Client dependencies.
Regards,
Timo
Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz:
+1
- verified compilation, tests
- verified checksum and gpg files
- verified sbt templates (g8, quickstart) - run assemblies on local cluster
- I could not execute the nightly-tests.sh though. The tests that were
failing most often are:
- test_streaming_file_sink.sh
- test_streaming_elasticsearch.sh
Those are connectors though and it might be only tests flakiness so I
think it should not block the release.
On 08/08/18 16:36, Chesnay Schepler wrote:
I did not use the tools/list_deps.py script as I wasn't aware that it
existed.
Even if I were I wouldn't have used it and in fact would advocate for
removing it.
It manually parses and constructs dependency information which is
utterly unnecessary as maven already provides this functionality, with
the added bonus of also accounting for dependencyManagement and
transitive dependencies which we obviously have to take into account.
I used this one-liner instead:
|mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e
||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed|
|'s/:[a-z]*$//g'| || ||sort| |-u
|which I have documented here:
https://cwiki.apache.org/confluence/display/FLINK/Dependencies
On 08.08.2018 15:06, Aljoscha Krettek wrote:
+1
- verified checksum and gpg files
- verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE
had one unnecessary part removed
Side comment: I'm not sure whether the "Verify that the LICENSE and
NOTICE file is correct for the binary and source releases" part is
valid anymore because we only have one LICENSE and NOTICE file. also
"The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer
to the binary distribution and mention all of Flink's Maven
dependencies as well" can be dropped because we don't have them anymore.
I came to the same conclusion on dependencies. I used
tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's
probably what Chesnay also did ... :-)
On 8. Aug 2018, at 14:43, Chesnay Schepler <ches...@apache.org> wrote:
+1
- verified source release contains no binaries
- verified correct versions in source release
- verified compilation, tests and E2E-tests pass (on travis)
- verified checksum and gpg files
New dependencies (excluding dependencies where we simply depend on a
different version now):
Apache licensed:
io.confluent:common-utils:jar:3.3.1
io.confluent:kafka-schema-registry-client:jar:3.3.1
io.prometheus:simpleclient_pushgateway:jar:0.3.0
various Apache Nifi dependencies
various Apache Parquet dependencies
various ElasticSearch dependencies
CDDL:
javax.ws.rs:javax.ws.rs-api:jar:2.1
Bouncycastle (MIT-like):
org.bouncycastle:bcpkix-jdk15on:jar:1.59
org.bouncycastle:bcprov-jdk15on:jar:1.59
MIT:
org.projectlombok:lombok:jar:1.16.20
On 08.08.2018 13:28, Till Rohrmann wrote:
Thanks for reporting these problems Chesnay. The usage string in
`standalone-job.sh` is out dated and should be updated. The same
applies to
the typo.
When calling `standalone-job.sh start --job-classname foobar.Job`
please
make sure that the user code jar is contained in the classpath (e.g.
putting the jar in the lib directory). Documenting this behaviour
is part
of the pending issue FLINK-10001.
We should fix all of these issues. They are, however, no release
blockers.
Cheers,
Till
On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler
<ches...@apache.org> wrote:
I found some issues with the standalone-job.sh script.
I ran "./bin/standalone-job.sh start" as described by the usage
string.
2018-08-08 09:22:34,385 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Could not parse command line arguments [--configDir,
/home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
org.apache.flink.runtime.entrypoint.FlinkParseException:
Failed to
parse the command line arguments.
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
Caused by: org.apache.commons.cli.MissingOptionException:
Missing
required option: j
at
org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
at
org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
at
org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
... 1 more
The script should fail earlier if no jar is provided, with a better
error message.
It is also undocumented, and the usage instructions don't appear
correct.
Passing a jar with the -j option leads to a ClassNotFoundException:
"./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"
2018-08-08 09:26:30,562 ERROR
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Cluster initialization failed.
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
Caused by: org.apache.flink.util.FlinkException: Could not
load the
provied entrypoint class.
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
at
org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
at
java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
... 3 more
Caused by: java.lang.ClassNotFoundException:
examples/streaming/WordCount.jar
at
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at
java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
... 11 more
So this seems to not work at all, but maybe I'm using it wrong?
(There's also typo in "Could not load the provied entrypoint class")
On 08.08.2018 10:33, Piotr Nowojski wrote:
+1 from my side
I’ve spent some time playing around with various examples (batching,
streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with
different configuration options (number of task
managers/memory/Flip6/credit base flow control/metrics) and
everything
looks now fine (after fixing
https://issues.apache.org/jira/browse/FLINK-9969 <
https://issues.apache.org/jira/browse/FLINK-9969> ).
Piotrek
On 7 Aug 2018, at 17:17, Till Rohrmann <trohrm...@apache.org>
wrote:
Hi everyone,
Please review and vote on the release candidate #4 for the version
1.6.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific
comments)
The complete staging area is available for your review, which
includes:
* JIRA release notes [1],
* the official Apache source release and binary convenience
releases to
be
deployed to dist.apache.org [2], which are signed with the key with
fingerprint 1F302569A96CFFD5 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "release-1.6.0-rc4" [5],
* website pull request listing the new release and adding
announcement
blog
post [6].
Please use this document for coordinating testing efforts: [7]
The vote will be shortened since we only adde a minor fix on top
of the
RC
3. It will close on Wednesday 6:30pm CET. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
Thanks,
Your friendly Release Manager
[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
[2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4]
https://repository.apache.org/content/repositories/orgapacheflink-1178
[5] https://github.com/apache/flink/tree/release-1.6.0-rc4
[6] https://github.com/apache/flink-web/pull/117
[7]
https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
Pro-tip: you can create a settings.xml file with these contents:
<settings>
<activeProfiles>
<activeProfile>flink-1.6.0</activeProfile>
</activeProfiles>
<profiles>
<profile>
<id>flink-1.6.0</id>
<repositories>
<repository>
<id>flink-1.6.0</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1178/
</url>
</repository>
<repository>
<id>archetype</id>
<url>
https://repository.apache.org/content/repositories/orgapacheflink-1178/
</url>
</repository>
</repositories>
</profile>
</profiles>
</settings>
And reference that in you maven commands via --settings
path/to/settings.xml. This is useful for creating a quickstart
based on
the
staged release and for building against the staged jars.
