hi,all:
??flink-sql cdc
sink?sinkupdate??hive??
??hive??join
??flink-sql??hive??MySQL??flink-sql
hi??Rui Li??
commited??add partition
/user/hive/warehouse/rt_dwd.db/dwd_music_copyright_test/dt=2020-08-19/hour=07/part-b7d8f3c6-f1f3-40d4-a269-1ccf2c9a7720-0-1031
----
??:
sql??
INSERT INTO rt_dwd.dwd_music_copyright_test
SELECT url,md5,utime,title,singer,company,level,
from_unixtime(cast(utime/1000 as int),'-MM-dd')
,from_unixtime(cast(utime/1000 as int),'HH') FROM music_source;
----
??:
hi, Rui Li:
2020-09-04 17:17:10,548 INFO
org.apache.flink.streaming.api.operators.AbstractStreamOperator [] - Partition
{dt=2020-08-22, hour=18} of table
`hive_catalog`.`rt_dwd`.`dwd_music_copyright_test` is ready to
??checkpointtm??info??warn
2020-09-04 17:17:59,520 INFO org.apache.hadoop.io.retry.RetryInvocationHandler
[] - Exception while invoking create of class
ClientNamenodeProtocolTranslatorPB over
uhadoop-op3raf-master2/10.42.52.202:8020 after 14 fail over
??
----
??:
"M
hi,jingsong:
??
https://s1.ax1x.com/2020/09/07/wn1CFg.png
checkpoint??
2020-09-04 17:17:59
org.apache.flink.util.FlinkRuntimeException: Exceeded checkpoint tolerable
failure threshold.
at
hi, all??
1. DataStream APIkafka??DataStream ds1??
2. tableEnvhive catalog??
tableEnv.registerCatalog(catalogName, catalog);
tableEnv.useCatalog(catalogName);
3. ??ds1??table
Table sourcetable =
hi, all??
DataStream APIkafka??DataStream ds1??
tableEnvhive catalog??
tableEnv.registerCatalog(catalogName, catalog);
tableEnv.useCatalog(catalogName);
??ds1??table
Table sourcetable =
??attemptsmin(yarnyarn.resourcemanager.am.max-attempts,flinkyarn.application-attempts)yarn??
Best,
MuChen.
----
??:"jiliang1993"https://blog.csdn.net/cndota
hi
??
Best,
MuChen.
----
??:""https://blog.csdn.net/cndotaci/article/details/106870413
??flinkyarn??2??6??
hi??all??
??https://blog.csdn.net/cndotaci/article/details/106870413??flinkyarn??2??6yarn??
1.
2.
??
??hive
table??dag??hive??hive3??subtask??
https://s1.ax1x.com/2020/06/30/N4qxNq.png
$ grep -v \# sql-client-defaults.yaml |grep -v ^$ catalogs:- name: myhive
type: hive hive-conf-dir: /home/fsql/hive/conf default-database:
default execution: planner: blink type: streaming time-characteristic:
event-time
hi,
yarn-session??bin/yarn-session.sh -jm 1g -tm 4g -s 4 -qu root.flink -nm
fsql-cli 21
sql-clientsql??
kafkahive??joinmysql
Hi,Roc Marshal:
Best,
MuChen.
----
??:"Roc Marshal"http://uhadoop-op3raf-core24:42976
sql-client:
bin/sql-client.sh embedded
hbaseflinksql?
Hi, All:
??flinksqlhbase
hadoop??masterflink??
yarn-session:
bin/yarn-session.sh -jm 1g -tm 4g -s 4 -qu root.flink -nm fsql-cli 21
# ??
,
MuChen
----
??:"Yang Wang"https://imgchr.com/i/NJIn4x
??Flink session cluster
??root.default
??
flink??1.10.0
flink-conf.yaml??
[fsql@10-42-63-116 conf]$ grep -v ^# flink
hi,all:
hadoop??A.
Abin/yarn-session.sh -n 5 -jm 1g -tm 4g -s 4 -qu root.flink -nm
fsql-cli /dev/null 21 yarn-session??
root.flinkfsql-cli.
hi,
yarn-session:bin/yarn-session.sh -n 5 -jm 1g -tm 4g -s 4 -qu root.flink
-nm fsql-cli /dev/null 21
sql-clientsql??
kafkahive??joinmysql
sql??
# --
hi,
yarn-session:bin/yarn-session.sh -n 5 -jm 1g -tm 4g -s 4 -qu root.flink
-nm fsql-cli /dev/null 21
sql-clientsql??
kafkahive??joinmysql
21 matches
Mail list logo
