Hi,这个日志全是 。。。。有点头大。。。
我刚想到，除了task重启外，还有一种情况是task没有调度成功。
你能通过flink web ui观察到task的状态吗，都是RUNNING吗？
如果一直是schedule，那应该是缺少对应的资源进行调度，需要检查下task manager提供的slot资源以及任务所需的资源。
如果是running、failed、schedule的不断切换，那需要检查task manager的日志，应该有warn。
Bruce <bruceleeof1...@qq.com> 于2020年8月10日周一 下午6:12写道：
> 下面是附件的内容，请问是因为什么导致重启呢？
>
>
> 2阶段提交demo:
>
>
> @Slf4j public class CommonOracleSink extends
> TwoPhaseCommitSinkFunction<LinkedList<Object>,
> CommonOracleSink.ConnectionState, Void> { private
> transient String sinkSQL; public CommonOracleSink() {
> super(new
> KryoSerializer<>(ConnectionState.class, new ExecutionConfig()),
> VoidSerializer.INSTANCE); } @Override
> public void open(Configuration parameters) throws Exception {
> super.open(parameters);
> ParameterTool params = (ParameterTool)
> getRuntimeContext().getExecutionConfig().getGlobalJobParameters();
> sinkSQL = params.getRequired("sinkSQL");
> } @Override protected void
> invoke(ConnectionState connectionState, LinkedList<Object> colList,
> Context context){ try {
> System.err.println("start invoke.......");
> Connection connection =
> connectionState.connection;
> log.info("colList---------------------->",
> JSON.toJSONString(colList));
> TKQueryRunner runner = new TKQueryRunner();
> Object[] params = colList.toArray();
> System.err.println("params
> size----->"+params.length);
> runner.update(connection,sinkSQL,params);
> }catch (Exception e){
> log.error(e.getMessage(),e);
> System.err.println(e.getMessage()); }
> } /** * 获取连接，开启手动提交事物
> * * @return * @throws
> Exception */ @Override
> protected ConnectionState beginTransaction() throws Exception {
> Connection connection = HikariOUtils.getConnection();
> log.info("start beginTransaction......." +
> connection); return new
> ConnectionState(connection); } /**
> * 预提交，这里预提交的逻辑在invoke方法中 *
> * @param connectionState * @throws Exception
> */ @Override protected void
> preCommit(ConnectionState connectionState) throws Exception {
> log.info("start preCommit......." + connectionState);
> } /** *
> 如果invoke方法执行正常，则提交事务 * * @param
> connectionState */ @Override
> protected void commit(ConnectionState connectionState) {
> log.info("start commit......." + connectionState);
> Connection connection =
> connectionState.connection; try {
> connection.commit();
> connection.close();
> } catch (SQLException e) { throw
> new RuntimeException("提交事物异常"); }
> } /** *
> 如果invoke执行异常则回滚事物，下一次的checkpoint操作也不会执行 *
> * @param connectionState */
> @Override protected void abort(ConnectionState
> connectionState) { log.error("start abort
> rollback......." + connectionState); Connection
> connection = connectionState.connection; try {
> connection.rollback();
> connection.close();
> } catch (SQLException e) {
> throw new RuntimeException("回滚事物异常"); }
> } static class ConnectionState {
> private final transient Connection connection;
> ConnectionState(Connection connection) {
> this.connection = connection;
> } } }
>
> jobmanager日志
>
> 2020-08-10 16:37:31,892 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] -
> --------------------------------------------------------------------------------2020-08-10
> 16:37:31,897 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Starting YarnJobClusterEntrypoint (Version: 1.11.1,
> Scala: 2.11, Rev:7eb514a, Date:2020-07-15T07:02:09+02:00)2020-08-10
> 16:37:31,898 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - OS current user: root2020-08-10 16:37:32,295 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Current Hadoop/Kerberos user: root2020-08-10
> 16:37:32,295 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle
> Corporation - 1.8/25.121-b132020-08-10 16:37:32,295 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Maximum heap size: 3166 MiBytes2020-08-10
> 16:37:32,295 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - JAVA_HOME: /home/xxx/app/jdk1.8.0_1212020-08-10
> 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Hadoop version: 2.7.72020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - JVM Options:2020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - -Xmx34628173762020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - -Xms34628173762020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - -XX:MaxMetaspaceSize=2684354562020-08-10
> 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] -
> -Dlog.file=/home/xxx/app/hadoop-2.6.0-cdh5.15.1/logs/userlogs/application_1591335931326_0024/container_1591335931326_0024_01_000001/jobmanager.log2020-08-10
> 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] -
> -Dlog4j.configuration=file:log4j.properties2020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] -
> -Dlog4j.configurationFile=file:log4j.properties2020-08-10 16:37:32,297 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Program Arguments: (none)2020-08-10 16:37:32,297
> INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Classpath:
> :UnifyCompFlink-1.0.jar:lib/flink-csv-1.11.1.jar:lib/flink-json-1.11.1.jar:lib/flink-shaded-hadoop-2-uber-2.6.5-10.0.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.11-1.11.1.jar:lib/flink-table_2.11-1.11.1.jar:lib/log4j-1.2-api-2.12.1.jar:lib/log4j-api-2.12.1.jar:lib/log4j-core-2.12.1.jar:lib/log4j-slf4j-impl-2.12.1.jar:flink-dist_2.11-1.11.1.jar:job.graph:flink-conf.yaml::/home/xxx/app/hadoop-2.6.0-cdh5.15.1/etc/hadoop:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/hadoop-nfs-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/hadoop-common-2.6.0-cdh5.15.1-tests.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/hadoop-common-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/zookeeper-3.4.5-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/logredactor-1.0.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/activation-1.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-codec-1.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/junit-4.11.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jsr305-3.0.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/netty-3.10.5.Final.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/httpclient-4.2.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/avro-1.7.6-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jetty-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-cli-1.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/hadoop-auth-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-el-1.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-net-3.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jersey-json-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jsch-0.1.42.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-lang-2.6.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jettison-1.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/hadoop-annotations-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/gson-2.2.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/guava-11.0.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/paranamer-2.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jersey-server-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/hamcrest-core-1.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-io-2.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jsp-api-2.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jersey-core-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/xmlenc-0.52.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-digester-1.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/servlet-api-2.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/log4j-1.2.17.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/xz-1.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/stax-api-1.0-2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jets3t-0.9.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/httpcore-4.2.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-beanutils-1.9.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-collections-3.2.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/curator-client-2.7.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/curator-framework-2.7.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/common/lib/asm-3.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.15.1-tests.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/hadoop-hdfs-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jetty-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-io-2.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jackson-core-asl-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/htrace-core4-4.0.1-incubating.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/jackson-mapper-asl-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/hdfs/lib/asm-3.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-common-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-registry-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-api-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/hadoop-yarn-client-2.6.0-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/zookeeper-3.4.5-cdh5.15.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/activation-1.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-codec-1.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/javax.inject-1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jersey-client-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jline-2.11.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jetty-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-cli-1.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jackson-jaxrs-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jersey-json-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-lang-2.6.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jettison-1.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/guava-11.0.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-io-2.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jackson-xc-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/guice-3.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/servlet-api-2.5.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/xz-1.0.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jackson-core-asl-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jackson-mapper-asl-1.8.8.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/home/xxx/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/yarn/lib/asm-3.2.jar2020-08-10
> 16:37:32,299 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] -
> --------------------------------------------------------------------------------2020-08-10
> 16:37:32,301 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Registered UNIX signal handlers for [TERM, HUP,
> INT]2020-08-10 16:37:32,306 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - YARN daemon is running as: root Yarn client user
> obtainer: root2020-08-10 16:37:32,311 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.process.size, 4 gb2020-08-10 16:37:32,311 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> internal.jobgraph-path, job.graph2020-08-10 16:37:32,311 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.execution.failover-strategy, region2020-08-10 16:37:32,312 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> high-availability.cluster-id, application_1591335931326_00242020-08-10
> 16:37:32,312 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.rpc.address, localhost2020-08-10 16:37:32,312 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: execution.target,
> yarn-per-job2020-08-10 16:37:32,312 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.memory.process.size, 4 gb2020-08-10 16:37:32,312 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.rpc.port, 61232020-08-10 16:37:32,312 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.savepoint.ignore-unclaimed-state, false2020-08-10 16:37:32,313
> INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.attached, true2020-08-10 16:37:32,313 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> internal.cluster.execution-mode, NORMAL2020-08-10 16:37:32,313 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.shutdown-on-attached-exit, false2020-08-10 16:37:32,313 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: pipeline.jars,
> file:/home/xxx/app/flink-1.11.1/UnifyCompFlink-1.0.jar2020-08-10
> 16:37:32,313 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> parallelism.default, 82020-08-10 16:37:32,313 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.numberOfTaskSlots, 12020-08-10 16:37:32,313 WARN
> org.apache.flink.configuration.GlobalConfiguration
> [] - Error while trying to split key and value in
> configuration file
> /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1591335931326_0024/container_1591335931326_0024_01_000001/flink-conf.yaml:16:
> "pipeline.classpaths: "2020-08-10 16:37:32,314 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> $internal.deployment.config-dir, /home/xxx/app/flink-1.11.1/conf2020-08-10
> 16:37:32,314 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> $internal.yarn.log-config-file,
> /home/xxx/app/flink-1.11.1/conf/log4j.properties2020-08-10 16:37:32,347
> WARN org.apache.flink.configuration.Configuration
> [] - Config uses deprecated
> configuration key 'web.port' instead of proper key
> 'rest.bind-port'2020-08-10 16:37:32,362 INFO
> org.apache.flink.runtime.clusterframework.BootstrapTools
> [] - Setting directories for temporary files to:
> /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1591335931326_00242020-08-10
> 16:37:32,368 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Starting YarnJobClusterEntrypoint.2020-08-10 16:37:32,413
> INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Install default filesystem.2020-08-10 16:37:32,461
> INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Install security context.2020-08-10 16:37:32,520
> INFO org.apache.flink.runtime.security.modules.HadoopModule
> [] - Hadoop user set to root (auth:SIMPLE)2020-08-10
> 16:37:32,529 INFO
> org.apache.flink.runtime.security.modules.JaasModule
> [] - Jaas file will be created as
> /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1591335931326_0024/jaas-1114046375892877617.conf.2020-08-10
> 16:37:32,539 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> [] - Initializing cluster services.2020-08-10 16:37:32,556
> INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils
> [] - Trying to start actor system, external address
> node3:0, bind address 0.0.0.0:0.2020-08-10 16:37:33,191 INFO
> akka.event.slf4j.Slf4jLogger
>
> [] - Slf4jLogger started2020-08-10 16:37:33,218 INFO
> akka.remote.Remoting
>
> [] - Starting remoting2020-08-10 16:37:33,378 INFO
> akka.remote.Remoting
>
> [] - Remoting started; listening on addresses
> :[akka.tcp://flink@node3:40657]2020-08-10 16:37:33,506 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils
> [] - Actor system started at
> akka.tcp://flink@node3:406572020-08-10
> 16:37:33,539 WARN org.apache.flink.configuration.Configuration
> [] - Config uses
> deprecated configuration key 'web.port' instead of proper key
> 'rest.port'2020-08-10 16:37:33,551 INFO
> org.apache.flink.runtime.blob.BlobServer
> [] - Created BLOB server storage
> directory
> /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1591335931326_0024/blobStore-15a573e2-a671-4eb9-975b-b5229cec6bde2020-08-10
> 16:37:33,555 INFO org.apache.flink.runtime.blob.BlobServer
> [] - Started
> BLOB server at 0.0.0.0:34380 - max concurrent requests: 50 - max backlog:
> 10002020-08-10 16:37:33,570 INFO
> org.apache.flink.runtime.metrics.MetricRegistryImpl
> [] - No metrics reporter configured, no metrics will be
> exposed/reported.2020-08-10 16:37:33,574 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils
> [] - Trying to start actor system, external address node3:0,
> bind address 0.0.0.0:0.2020-08-10 16:37:33,591 INFO
> akka.event.slf4j.Slf4jLogger
>
> [] - Slf4jLogger started2020-08-10 16:37:33,597 INFO
> akka.remote.Remoting
>
> [] - Starting remoting2020-08-10 16:37:33,606 INFO
> akka.remote.Remoting
>
> [] - Remoting started; listening on addresses
> :[akka.tcp://flink-metrics@node3:43096]2020-08-10 16:37:33,642 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils
> [] - Actor system started at
> akka.tcp://flink-metrics@node3:430962020-08-10
> 16:37:33,659 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
> [] - Starting RPC endpoint for
> org.apache.flink.runtime.metrics.dump.MetricQueryService at
> akka://flink-metrics/user/rpc/MetricQueryService .2020-08-10 16:37:33,721
> WARN org.apache.flink.configuration.Configuration
> [] - Config uses deprecated
> configuration key 'web.port' instead of proper key
> 'rest.bind-port'2020-08-10 16:37:33,723 INFO
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
> Upload directory
> /tmp/flink-web-d25a365e-b6cb-45e6-bb6e-3068b7412f06/flink-web-upload does
> not exist. 2020-08-10 16:37:33,724 INFO
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
> Created directory
> /tmp/flink-web-d25a365e-b6cb-45e6-bb6e-3068b7412f06/flink-web-upload for
> file uploads.2020-08-10 16:37:33,748 INFO
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
> Starting rest endpoint.2020-08-10 16:37:34,110 INFO
> org.apache.flink.runtime.webmonitor.WebMonitorUtils
> [] - Determined location of main cluster component log
> file:
> /home/xxx/app/hadoop-2.6.0-cdh5.15.1/logs/userlogs/application_1591335931326_0024/container_1591335931326_0024_01_000001/jobmanager.log2020-08-10
> 16:37:34,111 INFO org.apache.flink.runtime.webmonitor.WebMonitorUtils
> [] - Determined location of main cluster
> component stdout file:
> /home/xxx/app/hadoop-2.6.0-cdh5.15.1/logs/userlogs/application_1591335931326_0024/container_1591335931326_0024_01_000001/jobmanager.out2020-08-10
> 16:37:34,309 INFO
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
> Rest endpoint listening at node3:394692020-08-10 16:37:34,311 INFO
> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
> http://node3:39469 was granted leadership with
> leaderSessionID=00000000-0000-0000-0000-0000000000002020-08-10 16:37:34,312
> INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint []
> - Web frontend listening at http://node3:39469.2020-08-10 16:37:34,403
> INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
> [] - Starting RPC endpoint for
> org.apache.flink.yarn.YarnResourceManager at
> akka://flink/user/rpc/resourcemanager_0 .2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.memory.process.size, 4 gb2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> internal.jobgraph-path, job.graph2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.execution.failover-strategy, region2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> high-availability.cluster-id, application_1591335931326_00242020-08-10
> 16:37:34,417 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.rpc.address, localhost2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: execution.target,
> yarn-per-job2020-08-10 16:37:34,417 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.memory.process.size, 4 gb2020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> jobmanager.rpc.port, 61232020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.savepoint.ignore-unclaimed-state, false2020-08-10 16:37:34,418
> INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.attached, true2020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> internal.cluster.execution-mode, NORMAL2020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> execution.shutdown-on-attached-exit, false2020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property: pipeline.jars,
> file:/home/xxx/app/flink-1.11.1/UnifyCompFlink-1.0.jar2020-08-10
> 16:37:34,418 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> parallelism.default, 82020-08-10 16:37:34,418 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> taskmanager.numberOfTaskSlots, 12020-08-10 16:37:34,418 WARN
> org.apache.flink.configuration.GlobalConfiguration
> [] - Error while trying to split key and value in
> configuration file
> /tmp/hadoop-root/nm-local-dir/usercache/root/appcache/application_1591335931326_0024/container_1591335931326_0024_01_000001/flink-conf.yaml:16:
> "pipeline.classpaths: "2020-08-10 16:37:34,419 INFO
> org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> $internal.deployment.config-dir, /home/xxx/app/flink-1.11.1/conf2020-08-10
> 16:37:34,419 INFO org.apache.flink.configuration.GlobalConfiguration
> [] - Loading configuration property:
> $internal.yarn.log-config-file,
> /home/xxx/app/flink-1.11.1/conf/log4j.properties2020-08-10 16:37:34,450
> INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils
> [] - Enabled external resources: []2020-08-10 16:37:34,519 INFO
> org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess
> [] - Start JobDispatcherLeaderProcess.2020-08-10 16:37:34,527 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService
> [] - Starting RPC endpoint for
> org.apache.flink.runtime.dispatcher.MiniDispatcher at
> akka://flink/user/rpc/dispatcher_1 .2020-08-10 16:37:34,572 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService
> [] - Starting RPC endpoint for
> org.apache.flink.runtime.jobmaster.JobMaster at
> akka://flink/user/rpc/jobmanager_2 .2020-08-10 16:37:34,582 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - Initializing job empJOB
> (eb447d27efb8134da40c0c1dd19fffdf).2020-08-10 16:37:34,615 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - Using restart back off time
> strategy
> FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=2147483647,
> backoffTimeMS=1000) for empJOB
> (eb447d27efb8134da40c0c1dd19fffdf).2020-08-10 16:37:34,667 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - Running initialization on master
> for job empJOB (eb447d27efb8134da40c0c1dd19fffdf).2020-08-10 16:37:34,806
> INFO org.apache.flink.runtime.jobmaster.JobMaster
> [] - Successfully ran
> initialization on master in 139 ms.2020-08-10 16:37:34,876 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Recovered 0 containers from
> previous attempts ([]).2020-08-10 16:37:34,877 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Register application master
> response contains scheduler resource types: [MEMORY, CPU].2020-08-10
> 16:37:34,877 INFO org.apache.flink.yarn.YarnResourceManager
> [] -
> Container matching strategy: MATCH_VCORE.2020-08-10 16:37:34,887 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - ResourceManager
> akka.tcp://flink@node3:40657/user/rpc/resourcemanager_0 was granted
> leadership with fencing token 000000000000000000000000000000002020-08-10
> 16:37:34,891 INFO
> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl
> [] - Starting the SlotManager.2020-08-10 16:37:35,466 INFO
> org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology
> [] - Built 1 pipelined regions in 2 ms2020-08-10 16:37:35,483 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - No state backend has been
> configured, using default (Memory / JobManager) MemoryStateBackend (data in
> heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints:
> 'null', asynchronous: TRUE, maxStateSize: 5242880)2020-08-10 16:37:35,503
> INFO org.apache.flink.runtime.jobmaster.JobMaster
> [] - Using failover strategy
> org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@3915bc20
> for empJOB (eb447d27efb8134da40c0c1dd19fffdf).2020-08-10 16:37:35,509 INFO
> org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl
> [] - JobManager runner for job empJOB
> (eb447d27efb8134da40c0c1dd19fffdf) was granted leadership with session id
> 00000000-0000-0000-0000-000000000000 at
> akka.tcp://flink@node3:40657/user/rpc/jobmanager_2.2020-08-10
> 16:37:35,514 INFO org.apache.flink.runtime.jobmaster.JobMaster
> [] - Starting execution of
> job empJOB (eb447d27efb8134da40c0c1dd19fffdf) under job master id
> 00000000000000000000000000000000.2020-08-10 16:37:35,517 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - Starting scheduling with scheduling
> strategy
> [org.apache.flink.runtime.scheduler.strategy.EagerSchedulingStrategy]2020-08-10
> 16:37:35,518 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Job empJOB (eb447d27efb8134da40c0c1dd19fffdf) switched from
> state CREATED to RUNNING.2020-08-10 16:37:35,535 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (1/6)
> (5a6410258857c02ebd1b5ec03a78be4b) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (2/6)
> (299de0d4a8affe02a999edeb84957c41) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (3/6)
> (1b98df27c9019f64835b55fa3de3f363) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (4/6)
> (a7612608772c018d819741ce4d9320bd) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (5/6)
> (b19828c85fc0e92e62f2a7241b610f5b) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Source: HiveTableSource(empno, ename, job, mgr, hiredate, sal,
> comm, deptno) TablePath: flink000.emp, PartitionPruned: false,
> PartitionNums: null -> SinkConversionToRow (6/6)
> (c2178a51eda2db900d3212e4f488d00f) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (1/8)
> (2a8db3a2b4cd65fd7cd3e6bac031a971) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (2/8)
> (7aa8dd779d4ff75e4c985be75a52c427) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (3/8)
> (867c814978ea302537065f51516ed766) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,536 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (4/8)
> (4e186575ab42cc6c1d599ae027bf99b8) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,537 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (5/8)
> (b107b8bfb0a08c5e7937400c43a0f9ff) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,537 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (6/8)
> (28e1f0fa1b9ebed59e4c67b0598864b9) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,537 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (7/8)
> (e27e60ff7dcd5245dfd21b23bbd49985) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,537 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph
> [] - Process -> Sink: Unnamed (8/8)
> (c0f6b9e623c68fd7e9205a8ad686d4e5) switched from CREATED to
> SCHEDULED.2020-08-10 16:37:35,558 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{7c8417d17f555e546cd6427733db6a3b}]2020-08-10 16:37:35,565
> INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{eb7c6ff479c977fdb6312f8e61dfdfba}]2020-08-10 16:37:35,565
> INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{a1d23d7e17b9369e80833de148f54bac}]2020-08-10 16:37:35,566
> INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{2447496efd24d542bce06de1b69ec70d}]2020-08-10 16:37:35,566
> INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{2ab761d21cd4368751f3187f122705fa}]2020-08-10 16:37:35,566
> INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Cannot serve slot request, no ResourceManager connected. Adding
> as pending request
> [SlotRequestId{08a52f113cce64d1309509da5d25a1bb}]2020-08-10 16:37:35,574
> INFO org.apache.flink.runtime.jobmaster.JobMaster
> [] - Connecting to
> ResourceManager
> akka.tcp://flink@node3:40657/user/rpc/resourcemanager_*(00000000000000000000000000000000)2020-08-10
> 16:37:35,579 INFO org.apache.flink.runtime.jobmaster.JobMaster
> [] - Resolved
> ResourceManager address, beginning registration2020-08-10 16:37:35,584 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Registering job manager
> 00000000000000000000000000000...@akka.tcp://flink@node3:40657/user/rpc/jobmanager_2
> for job eb447d27efb8134da40c0c1dd19fffdf.2020-08-10 16:37:35,589 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Registered job manager
> 00000000000000000000000000000...@akka.tcp://flink@node3:40657/user/rpc/jobmanager_2
> for job eb447d27efb8134da40c0c1dd19fffdf.2020-08-10 16:37:35,593 INFO
> org.apache.flink.runtime.jobmaster.JobMaster
> [] - JobManager successfully registered
> at ResourceManager, leader id: 00000000000000000000000000000000.2020-08-10
> 16:37:35,594 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{7c8417d17f555e546cd6427733db6a3b}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,595 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id e490d3208119fe28d97f4f0fe94cab28.2020-08-10 16:37:35,595 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{eb7c6ff479c977fdb6312f8e61dfdfba}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,595 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{a1d23d7e17b9369e80833de148f54bac}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,595 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{2447496efd24d542bce06de1b69ec70d}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,596 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{2ab761d21cd4368751f3187f122705fa}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,596 INFO
> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl
> [] - Requesting new slot
> [SlotRequestId{08a52f113cce64d1309509da5d25a1bb}] and profile
> ResourceProfile{UNKNOWN} from resource manager.2020-08-10 16:37:35,612 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 1.2020-08-10 16:37:35,614 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id b74299e8619de93adec5869d1fa79d73.2020-08-10 16:37:35,615 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 2.2020-08-10 16:37:35,615 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id 046a3dcf1af40e0539f15fcddfbddf77.2020-08-10 16:37:35,615 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 3.2020-08-10 16:37:35,615 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id 90870250ae0f3bef44cbdd675dede57b.2020-08-10 16:37:35,616 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 4.2020-08-10 16:37:35,616 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id f8063a6fc86162712215a92533532b65.2020-08-10 16:37:35,616 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 5.2020-08-10 16:37:35,616 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Request slot with profile
> ResourceProfile{UNKNOWN} for job eb447d27efb8134da40c0c1dd19fffdf with
> allocation id cfe671ec5448d440838f02145cb6267f.2020-08-10 16:37:35,617 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Requesting new TaskExecutor
> container with resource WorkerResourceSpec {cpuCores=1.0,
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemSize=1.340gb
> (1438814063 bytes)}. Number pending workers of this resource is
> 6.2020-08-10 16:37:38,391 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:37:40,933 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Received 1
> containers.2020-08-10 16:37:40,940 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Received 1 containers with
> resource <memory:4096, vCores:1>, 6 pending container
> requests.2020-08-10 16:37:40,953 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - TaskExecutor
> container_1591335931326_0024_01_000003 will be started on node1 with
> TaskExecutorProcessSpec {cpuCores=1.0, frameworkHeapSize=128.000mb
> (134217728 bytes), frameworkOffHeapSize=128.000mb (134217728 bytes),
> taskHeapSize=1.425gb (1530082070 bytes), taskOffHeapSize=0 bytes,
> networkMemSize=343.040mb (359703515 bytes), managedMemorySize=1.340gb
> (1438814063 bytes), jvmMetaspaceSize=256.000mb (268435456 bytes),
> jvmOverheadSize=409.600mb (429496736 bytes)}.2020-08-10 16:37:40,976 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Creating container launch
> context for TaskManagers2020-08-10 16:37:40,978 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Starting
> TaskManagers2020-08-10 16:37:40,995 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Removing container request
> Capability[<memory:4096, vCores:1>]Priority[1].2020-08-10 16:37:40,995
> INFO org.apache.flink.yarn.YarnResourceManager
> [] - Accepted 1 requested
> containers, returned 0 excess containers, 5 pending container requests of
> resource <memory:4096, vCores:1>.2020-08-10 16:37:46,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:37:47,712 INFO
> org.apache.flink.yarn.YarnResourceManager
> [] - Registering TaskManager with
> ResourceID container_1591335931326_0024_01_000003
> (akka.tcp://flink@node1:40857/user/rpc/taskmanager_0)
> at ResourceManager2020-08-10 16:37:54,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:02,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:10,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:18,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:26,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:34,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:42,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:50,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:38:58,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:06,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:14,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:22,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:30,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:38,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:46,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.2020-08-10 16:39:54,389 INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator
> [] - Checkpoint triggering task Source: HiveTableSource(empno, ename,
> job, mgr, hiredate, sal, comm, deptno) TablePath: flink000.emp,
> PartitionPruned: false, PartitionNums: null -> SinkConversionToRow (1/6)
> of job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but
> SCHEDULED instead. Aborting checkpoint.
>
>
>
> job代码
>
> public class PlatJobExecution {
> private volatile ParameterTool parameters;
> public PlatJobExecution(ParameterTool parameters) {
> this.parameters = parameters; }
> public void execute() throws Exception {
>
> //目标数据源: //目标数据表：
> //1.读取数据 kafka /oracle
> 把流注册成一张表【这个过程可以手动完成】 --hive
> //2.执行sql，返回结果
> //3.把结果写入目标数据表 / 写入redis / 写入kafka
> InputStream is =
> ReadKafkaPrint.class.getClassLoader().getResourceAsStream("config.properties");
> ParameterTool parameters2 =
> ParameterTool.fromPropertiesFile(is);
>
>
>
> String targetDatabase =
> parameters.get("sourceDatabase"); String
> executiveSql = parameters.get("executiveSql");
> String sinkSQL = parameters.get("sinkSQL");
> String jobName = parameters.get("jobName");
> Map<String, String> pMap =
> Maps.newHashMap();
> pMap.putAll(parameters2.toMap());
> pMap.put("sinkSQL",sinkSQL);
> parameters2 = ParameterTool.fromMap(pMap);
>
> //1.创建执行环境
> StreamExecutionEnvironment streamEnv =
> StreamExecutionEnvironment.getExecutionEnvironment();
> //全局参数设置
> streamEnv.getConfig().setGlobalJobParameters(parameters2);
> streamEnv.enableCheckpointing(8000,
> CheckpointingMode.EXACTLY_ONCE);//每隔5s进行一次checkpoint
> EnvironmentSettings tableEnvSettings =
> EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build();
> //2.流式的TableEnv
> StreamTableEnvironment tableEnv = StreamTableEnvironment.create(streamEnv,
> tableEnvSettings);
> tableEnv.getConfig().getConfiguration().set(ExecutionCheckpointingOptions.CHECKPOINTING_MODE,
> CheckpointingMode.EXACTLY_ONCE);
> tableEnv.getConfig().getConfiguration().set(ExecutionCheckpointingOptions.CHECKPOINTING_INTERVAL,
> Duration.ofSeconds(8));
> //3.注册HiveCatalog
> String name =
> targetDatabase; String defaultDatabase =
> targetDatabase; String hiveConfDir
> = "/home/xxx/app/flink-1.11.1/jobcfg"; String
> version = "1.1.0";
> HiveCatalog catalog = new HiveCatalog(name,
> defaultDatabase, hiveConfDir, version);
> tableEnv.registerCatalog(name, catalog);
> tableEnv.useCatalog(name);
> //4.流式读取Hive
> tableEnv.getConfig().getConfiguration().setBoolean(TableConfigOptions.TABLE_DYNAMIC_TABLE_OPTIONS_ENABLED,
> true);
> //query Table table
> = tableEnv.sqlQuery(executiveSql);
> // CREATE/INSERT //
> tableEnv.executeSql()
> // tableEnv.toRetractStream(table,
> Row.class).print().setParallelism(1);
>
> SingleOutputStreamOperator<LinkedList<Object>> colList =
> tableEnv.toAppendStream(table, Row.class).process(new ProcessFunction<Row,
> LinkedList<Object>>() {
> @Override public void
> processElement(Row row, Context context,
> Collector<LinkedList<Object>> collector) throws Exception {
> LinkedList<Object>
> linkedList = Lists.newLinkedList();
> for (int i = 0; i < row.getArity(); i++) {
>
> linkedList.add(row.getField(i));
> }
> collector.collect(linkedList);
> } });
> colList.addSink(new CommonOracleSink());
>
> //sink to Oracle
> streamEnv.execute(jobName);
>
> }}
>
>
> 发自我的iPhone
>
>
> ------------------ 原始邮件 ------------------
> 发件人: shizk233 <wangwangdaxian...@gmail.com>
> 发送时间: 2020年8月10日 18:04
> 收件人: user-zh@flink.apache.org <user-zh@flink.apache.org>
> 主题: 回复：读取hive表的数据转化为流；按照jdbc两阶段提交方式写入oracle数据库,没有异常但是checkpoint失败
>
>
>
> hi，附件挂了，没看到饿。可以考虑挂gist放链接出来。
>
> 不过看这个信息，checkpoint的失败是由于任务状态变更导致的（应该是任务重启了，所以running变scheduled了）。
> 建议往任务重启的方向排查一下。
>
> Bruce <bruceleeof1...@qq.com> 于2020年8月10日周一 下午5:01写道：
>
> > 您好，这里有个问题反馈下！
> >
> > 读取hive表的数据转化为流；按照jdbc两阶段提交方式写入oracle数据库，
> > 没有抛任何异常但是checkpoint失败:
>
> > job eb447d27efb8134da40c0c1dd19fffdf is not in state RUNNING but SCHEDULED
> > instead. Aborting checkpoint.
> > 附件
> > 1.flink.log是yarn jobmanager打印的伪日志
> > 2.Job.txt是job的伪代码
> > 3.jdbc两阶段提交的伪代码附件
> > ------------------------------
> > 发自我的iPhone
> >
