回复：flink写入mysql数据异常

2023-03-23 文章 孙冬燕 
退订
--
发件人：小昌同学 
发送时间：2023年3月24日(星期五) 13:22
收件人：user-zh 
抄　送：user-zh 
主　题：回复： flink写入mysql数据异常
好滴呀，谢谢您的建议；
https://www.yuque.com/g/echochangtongxue/yxxdbg/iyfqa9fh34i5lssu/collaborator/join?token=KZCQVX5pqH3rmPNP#
 
<https://www.yuque.com/g/echochangtongxue/yxxdbg/iyfqa9fh34i5lssu/collaborator/join?token=KZCQVX5pqH3rmPNP#
 > 邀请你共同编辑文档《Flink SQL写入到mysql的问题》
我创建了一个语雀，我将代码以及问题都写在文档里了，麻烦大佬们帮忙看一下问题呀
| |
小昌同学
|
|
ccc0606fight...@163.com
|
 回复的原邮件 
| 发件人 | Shammon FY |
| 发送日期 | 2023年3月24日 13:08 |
| 收件人 |  |
| 主题 | Re: flink写入mysql数据异常 |
Hi
你可以将问题描述和sql放在一个外部文档，例如google文档，然后将文档连接发在邮件里
Best,
Shammon FY
On Fri, Mar 24, 2023 at 10:58 AM 孙冬燕 
wrote:
退订
--
发件人：小昌同学 
发送时间：2023年3月24日(星期五) 10:57
收件人：user-zh 
抄 送：user-zh 
主 题：回复： flink写入mysql数据异常
您好， 可能是我这边上传附件的方式不对，我场景描述的不够准确；
您看是否方便加一个微信呢【15956076613】，我将文档和截图发您，帮忙看一下；
谢谢大佬的指导
| |
小昌同学
|
|
ccc0606fight...@163.com
|
 回复的原邮件 
| 发件人 | Jane Chan |
| 发送日期 | 2023年3月23日 20:40 |
| 收件人 |  |
| 主题 | Re: flink写入mysql数据异常 |
附件还是没有收到哦.
Flink SQL 支持 INSERT INTO table_identifier (column_identifier1 [,
column_identifier2, ...]) 插入指定列, 具体语法可以参考 [1]
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/insert/#insert-from-select-queries
 
<https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/insert/#insert-from-select-queries
 >
<
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/insert/#insert-from-select-queries
 
<https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/insert/#insert-from-select-queries
 >
On Thu, Mar 23, 2023 at 5:35 PM 小昌同学  wrote:
您好，我刚刚重新上传了附件；是的，Flink
SQL已经支持了Upsert模式，但是这种更新都是行级别的更新，我想要实现仅仅只是变动一行数据中的部分字段。还望大佬指导
小昌同学
ccc0606fight...@163.com
<
https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D>
 
<https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D>
 >
<
https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D>
 
<https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D>
 >
 回复的原邮件 
发件人 Jane Chan 
发送日期 2023年3月23日 15:42
收件人  
主题 Re: flink写入mysql数据异常
Hi,
没有看到附件哦. 回到你的问题, Flink SQL 目前支持以 Upsert 模式写入 MySQL, 前提是 Sink 表的 DDL 声明主键,
并且与数据库中物理表主键保持一致. 可以参考 [1].
[1]
https://github.com/apache/flink-connector-jdbc/blob/main/docs/content.zh/docs/connectors/table/jdbc.md#%E9%94%AE%E5%A4%84%E7%90%86
 
<https://github.com/apache/flink-connector-jdbc/blob/main/docs/content.zh/docs/connectors/table/jdbc.md#%E9%94%AE%E5%A4%84%E7%90%86
 >
<
https://github.com/apache/flink-connector-jdbc/blob/main/docs/content.zh/docs/connectors/table/jdbc.md#%E9%94%AE%E5%A4%84%E7%90%86
 
<https://github.com/apache/flink-connector-jdbc/blob/main/docs/content.zh/docs/connectors/table/jdbc.md#%E9%94%AE%E5%A4%84%E7%90%86
 >
On Thu, Mar 23, 2023 at 2:54 PM 小昌同学  wrote:
大佬，你好，代码上传在附件中了；
就是我想实现flink sql写MySQL时能支持update吗 类似ON DUPLICATE KEY UPDATE 的语法？
小昌同学
ccc0606fight...@163.com
<
https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D
 
<https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D
 >
<
https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D
 
<https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D
 >
 回复的原邮件 
发件人 Jane Chan 
发送日期 2023年3月23日 14:23
收件人  
主题 Re: flink写入mysql数据异

回复：flink写入mysql数据异常

2023-03-23 文章 孙冬燕 
退订
--
发件人：小昌同学 
发送时间：2023年3月24日(星期五) 10:57
收件人：user-zh 
抄　送：user-zh 
主　题：回复： flink写入mysql数据异常
您好， 可能是我这边上传附件的方式不对，我场景描述的不够准确；
您看是否方便加一个微信呢【15956076613】，我将文档和截图发您，帮忙看一下；
谢谢大佬的指导
| |
小昌同学
|
|
ccc0606fight...@163.com
|
 回复的原邮件 
| 发件人 | Jane Chan |
| 发送日期 | 2023年3月23日 20:40 |
| 收件人 |  |
| 主题 | Re: flink写入mysql数据异常 |
附件还是没有收到哦.
Flink SQL 支持 INSERT INTO table_identifier (column_identifier1 [,
column_identifier2, ...]) 插入指定列, 具体语法可以参考 [1]
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/dev/table/sql/insert/#insert-from-select-queries
 

On Thu, Mar 23, 2023 at 5:35 PM 小昌同学  wrote:
您好，我刚刚重新上传了附件；是的，Flink
SQL已经支持了Upsert模式，但是这种更新都是行级别的更新，我想要实现仅仅只是变动一行数据中的部分字段。还望大佬指导
小昌同学
ccc0606fight...@163.com

 

 >
 回复的原邮件 
发件人 Jane Chan 
发送日期 2023年3月23日 15:42
收件人  
主题 Re: flink写入mysql数据异常
Hi,
没有看到附件哦. 回到你的问题, Flink SQL 目前支持以 Upsert 模式写入 MySQL, 前提是 Sink 表的 DDL 声明主键,
并且与数据库中物理表主键保持一致. 可以参考 [1].
[1]
https://github.com/apache/flink-connector-jdbc/blob/main/docs/content.zh/docs/connectors/table/jdbc.md#%E9%94%AE%E5%A4%84%E7%90%86
 

On Thu, Mar 23, 2023 at 2:54 PM 小昌同学  wrote:
大佬，你好，代码上传在附件中了；
就是我想实现flink sql写MySQL时能支持update吗 类似ON DUPLICATE KEY UPDATE 的语法？
小昌同学
ccc0606fight...@163.com
<
https://dashi.163.com/projects/signature-manager/detail/index.html?ftlId=1=%E5%B0%8F%E6%98%8C%E5%90%8C%E5%AD%A6=ccc0606fighting%40163.com=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsmed9b2013afa816e025ae07760d572391.jpg=%5B%22ccc0606fighting%40163.com%22%5D
 

 回复的原邮件 
发件人 Jane Chan 
发送日期 2023年3月23日 14:23
收件人  
主题 Re: flink写入mysql数据异常
可以把完整 SQL 发出来看看
祝好!
Jane
On Thu, Mar 23, 2023 at 1:39 PM 小昌同学  wrote:
使用flink
sql多表关联实时的将数据写入到mysql，mysql中定义了联合主键，查看日志发现为啥相同的数据插入到mysql表中，一条是insert
,另外一条是delete啊，我想实现的是upsert，这样该怎么操作啊
| |
小昌同学
|
|
ccc0606fight...@163.com
|

flink提交jar包时报连接不上BlobServer

2023-02-02 文章 孙冬燕 
2023-02-02 19:21:21,288 WARN 
org.apache.flink.client.deployment.application.DetachedApplicationRunner [] - 
Could not execute application: 
org.apache.flink.client.program.ProgramInvocationException: The main method 
caused an error: Failed to execute job 'Flink Streaming Job'.
 at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:98) 
~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.deployment.application.DetachedApplicationRunner.tryExecuteJobs(DetachedApplicationRunner.java:84)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.deployment.application.DetachedApplicationRunner.run(DetachedApplicationRunner.java:70)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$handleRequest$0(JarRunHandler.java:107)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
 [?:1.8.0_66]
 at java.lang.Thread.run(Thread.java:756) [?:1.8.0_66]
Caused by: org.apache.flink.util.FlinkException: Failed to execute job 'Flink 
Streaming Job'.
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2203)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:206)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:144)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2049)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2027)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
me.ele.arch.emonitor.blink.DataStreamApplication.main(DataStreamApplication.java:48)
 ~[?:?]
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_66]
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_66]
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_66]
 at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_66]
 at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
 ~[flink-dist-1.16.0.jar:1.16.0]
 ... 7 more
Caused by: org.apache.flink.util.FlinkException: Could not upload job files.
 at 
org.apache.flink.runtime.client.ClientUtils.uploadJobGraphFiles(ClientUtils.java:86)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.runtime.client.ClientUtils.extractAndUploadJobGraphFiles(ClientUtils.java:62)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor.lambda$submitJob$6(EmbeddedExecutor.java:177)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981)
 ~[?:1.8.0_66]
 at 
java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) 
~[?:1.8.0_66]
 at 
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor.submitJob(EmbeddedExecutor.java:174)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor.submitAndGetJobClientFuture(EmbeddedExecutor.java:134)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor.execute(EmbeddedExecutor.java:104)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2188)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:206)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:144)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2049)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:2027)
 ~[flink-dist-1.16.0.jar:1.16.0]
 at 
me.ele.arch.emonitor.blink.DataStreamApplication.main(DataStreamApplication.java:48)
 ~[?:?]
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_66]
 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_66]
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_66]
 at