Re: flink json ddl解析

Hi， 如果声明为 ARRAY 是否可以满足你的需求呢？如果可以的话，你可以在 1.12之后使用这个feature[1]. [1] https://issues.apache.org/jira/browse/FLINK-18002 zilong xiao 于2020年9月1日周二 下午5:49写道： > 问题大概懂了，坐等Flink大佬回复 > > Dream-底限 于2020年9月1日周二 下午4:43写道： > > > hi > > 就是json数组如果是这种：[1,2,3]，我可以直接array解析 > > > > >

?????? ??????????????????????????????????????

taochang [17:00,18:00)watermark??5 (event time - 5) = 18:00 ---- ??:

?????? ????savepoint

??V1.10.1?? ---- ??: "user-zh"

Re: Flink 涉及到 state 恢复能从 RocketMQ 直接转到 Kafka 吗？

可以，保证 RokcetMQ source 算子的 uid 和原本的 Kafka source 算子的 uid 不同就行。 另外启动要设置参数 -n 或 —allowNonRestoredState 。 Best, Paul Lam > 2020年9月2日 17:21，wangl...@geekplus.com 写道： > > > 有一个 flink streaming 的程序，读 RocketMQ，中间有一些复杂度计算逻辑用 RocksDB state 存储. > 程序有小的更新直接 cancel -s 取消再 run -s 恢复 > > 现在我们需要用 Kafka 替换掉

Re: 无法savepoint

看官方文档 cancel 语法格式是：Syntax: cancel [OPTIONS] ，所以-yid xxx是不是要放到job id之前？ 另外文档中有提示到Cancel with a savepoint (deprecated)， 建议使用stop语法，见： https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/cli.html 希望对你有帮助，祝好~ x <35907...@qq.com> 于2020年9月3日周四 上午11:30写道： > /flink/flink-1.10.1/bin/flink

Re: Fail to deploy Flink on minikube

HI Yang, I update taskmanager-session-deployment.yaml like this: apiVersion: apps/v1 kind: Deployment metadata: name: flink-taskmanager spec: replicas: 1 selector: matchLabels: app: flink component: taskmanager template: metadata: labels: app: flink

????savepoint

/flink/flink-1.10.1/bin/flink cancel -s hdfs://nameservice1/user/flink_1.10.1/flink-savepoints f95e51fd12a83907b6ca8c52eb2614ae -yid application_1583831804206_1106301 Unrecognized option: -yid

Re: FileSystemHaServices and BlobStore

Hi Alexey, Thanks for the feedback. You are right. StatefulSet + PersistentVolume + FileSystemHaService could be another bundle of services for Flink HA support on K8s. The user jars could be built into the image or downloaded by init-container or mount via the PV. So they do not need to be

Re: flink-sql-gateway hive ddl 语句parse报错 godfrey he 能帮忙看下吗？

TABLE_SQL_DIALECT这个key在tableConfig里对应的就是table.sql-dialect ，设置hive dialect这个是没有问题的，需要在yml配置文件里加上configuration,我在tableenv里查看configuration的table.sql-dialect已经变成hive了，但是hive DDL语句仍然过不了sqlparse。 -- Sent from: http://apache-flink.147419.n8.nabble.com/

Re: flink-1.11 使用 application 模式时 jobid 问题

目前HA模式下，application模式还不能支持多job，所以就固定是0了 主要的原因是recover的问题还没有解决好 Best, Yang chenkaibit 于2020年9月2日周三 下午7:29写道： > hi: > 我在测试 flink-1.11 application 模式时发现 开启 HA 后 jobID 总是 > ；关闭 HA 后是个随机字符（和之前版本相同）。这个是个 bug 还是就是这么设计的？ > 求大神解答。 > > -- > > Best, yuchuan

Re: Re: 请指教一个关于时间窗的问题，非常感谢！

您好，谢谢回复！ 这个窗口在flink1.11.1里是关闭不了，要等下一个时间窗的一条数据来了才会触发。 发件人： taochanglian 发送时间： 2020-09-03 10:35 收件人： user-zh; samuel@ubtrobot.com 主题： Re: 请指教一个关于时间窗的问题，非常感谢！ 没有问题的，时间窗口是左闭右开，你的窗口按照org.apache.flink.streaming.api.windowing.windows.TimeWindow按照

Re: Fail to deploy Flink on minikube

I guess something is wrong with your kube proxy, which causes TaskManager could not connect to JobManager. You could verify this by directly using JobManager Pod ip instead of service name. Please do as follows. * Edit the TaskManager deployment(via kubectl edit flink-taskmanager) and update the

Re: 请指教一个关于时间窗的问题，非常感谢！

org.apache.flink.streaming.api.windowing.windows.TimeWindow getWindowStartWithOffset??17-182020-09-01 18:00:00.0 2020-09-01 18:00:00.0 ??2020-09-01

Re: Flink 1.10.1 cdh-hdfs启用HA时，flink job提交报错

访问HDFS出现UnknownHostException（其实是nameservice，不是真正的host）的原因大多都是因为 没有正确加载hdfs-site.xml的配置，你可以确认一下JM的classpath里面有没有hadoop配置，没有的话 就是因为Yarn没有透传给container，需要在Yarn层面解决 Best, Yang storm 于2020年9月2日周三 下午4:05写道： > > 各位老师好，在HDFS上开启HA的时候，向yarn提交任务的时候，遇到点问题。 > cdh版本：5.15.2 > hdfs版本：2.6.0 >

Re: flink-sql-gateway hive ddl 语句parse报错 godfrey he 能帮忙看下吗？

不知道gateway模式是不是支持hive dialect。不过你确定设置方言的写法正确么？在SQL client里设置hive方言应该是 set table.sql-dialect=hive On Thu, Sep 3, 2020 at 9:28 AM shougou <80562...@qq.com> wrote: > 使用flink-sql-gate-way 最新的master 版本，对应flink版本1.11.1 > > 当使用 SET TABLE_SQL_DIALECT=hive 设置方言后 > > 大概看了代码，感觉是SqlParser 的问题，是不是应该切到Flink

Re: 请教 hive streaming 报错

可以试试不把hive-exec和PB 3.5.1的jar放到lib下面，而是通过命令行参数的方式指定这两个依赖？ On Wed, Sep 2, 2020 at 5:52 PM liangck wrote: > 最后加了好多jar包到 flink/lib > > 下，任务跑起来了。但是hive-exec中依赖的protobuf版本是2.5.0而且打进了jar包里，和其他任务里依赖的protobuf版本3.5.1不兼容。。请问下大佬们有什么好办法吗？ > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/

Re: Use of slot sharing groups causing workflow to hang

Hi, The failure of requesting slots usually because of the lack of resources. If you put part of the workflow to a specific slot sharing group, it may require more slots to run the workflow than before. Could you share logs of the ResourceManager and SlotManager, I think there are more clues in

Re: 从Savepoint/Checkpoint恢复时 keyedstate中部分数据无法被读取

Hi 按理说 checkpoint/savepoint 有的数据，正常恢复后是可以读取到的。 1 正常从 checkpoint/savepoint 恢复了吗？ 2 获取 state 的时候，key 是同一个 key 吗？ Best, Congxian Liu Rising 于2020年9月3日周四 上午9:28写道： > 版本: 1.9 > > 问题： > 当从savepoint或者checkpoint恢复flink job时，发现部分keyedState中的数据丢失。 > 这里我们使用的是ListState，里面存储的是ObjectNode(Jackson

Re: Fail to deploy Flink on minikube

Hi Till, I find something may be helpful. The kubernetes Dashboard show job-manager ip 172.18.0.5, task-manager ip 172.18.0.6 When I run command 'kubectl exec -ti flink-taskmanager-74c68c6f48-jqpbn -- /bin/bash’ && ‘ping 172.18.0.5’ I can get response But when I ping flink-jobmanager ,there is

flink-sql-gateway hive ddl 语句parse报错 godfrey he 能帮忙看下吗？

使用flink-sql-gate-way 最新的master 版本，对应flink版本1.11.1 当使用 SET TABLE_SQL_DIALECT=hive 设置方言后 大概看了代码，感觉是SqlParser 的问题，是不是应该切到Flink Parser ？ 执行hive ddl sql校验过不去， { "statement": "CREATE TABLE hive_table ( user_id STRING, order_amount DOUBLE ) PARTITIONED BY (dt STRING, hr STRING) STORED AS

从Savepoint/Checkpoint恢复时 keyedstate中部分数据无法被读取

版本: 1.9 问题： 当从savepoint或者checkpoint恢复flink job时，发现部分keyedState中的数据丢失。 这里我们使用的是ListState，里面存储的是ObjectNode(Jackson DataBinding)类型的对象。 查log发现部分key的 listState.get() 返回空的iterator。 然而使用State Process API确认State的内容时， 发现上述这些key对应的数据是存在于State中的。 求问各位大佬这种情况是怎么回事？应该如何排查 谢谢 -- Sent from:

????: ????????????????????????????????????????????

?? ??Flink1.11.1?? package com.ubtechinc.dataplatform.flink.etl.exception.monitor;/* * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information *

Re: Fail to deploy Flink on minikube

Hi Till, This is the taskManager log As you see, the logs print ‘line 92 -- Could not connect to flink-jobmanager:6123’ then print ‘line 128 --Could not resolve ResourceManager address akka.tcp://flink@flink-jobmanager:6123/user/rpc/resourcemanager_*, retrying in 1 ms: Could not connect to

Use of slot sharing groups causing workflow to hang

Hi all, I’ve got a streaming workflow (using Flink 1.11.1) that runs fine locally (via Eclipse), with a parallelism of either 3 or 6. If I set up part of the workflow to use a specific (not “default”) slot sharing group with a parallelism of 3, and the remaining portions of the workflow have

Re: Default Flink Metrics Graphite

Hi Till, *Info below, also I have a question at the end. * pretty much what was told earlier, for 1.10.0 use: metrics.reporter.grph.class: org.apache.flink.metrics.graphite.GraphiteReporter And then Looks like my Graphite installation has a problem or I am missing something. Just to make sure

Re: Task Chaining slots performance

Thanks for the information Till Regards, Vijay > > On Sep 2, 2020, at 2:21 AM, Till Rohrmann wrote: > >  > Hi Vijayendra, > > in the general case, I believe that chaining will almost always give you > better performance since you consume fewer resources, avoid context switches > between

Re: PyFlink - detect if environment is a StreamExecutionEnvironment

Hi Manas, As Till said, you need to check whether the execution environment used is LocalStreamEnvironment. You need to get the class object corresponding to the corresponding java object through py4j. You can take a look at the example I wrote below, I hope it will help you ``` from

Re: TM SLOT资源共享

多谢回复，有看到云邪大佬的这篇文章 http://wuchong.me/blog/2016/05/09/flink-internals-understanding-execution-resources/ 我再研究研究，多谢 -- Sent from: http://apache-flink.147419.n8.nabble.com/

kafka metric lag统计

在flink 1.11.0中任务未开启checkpoint时，消费kafka，已开启autocommit，kafka端可以正常查看lag，但这时候从flink web ui的metric中可以看到current offset，但是commite offset一直是Long的最小值 有以下两个问题，还请大佬们赐教 1、“这时候从flink web ui的metric中可以看到current offset，但是commite offset一直是Long的最小值”，这种是正常的吗？是否是少了什么配置，还是flink的bug？只从flink任务的这些metric应该是计算不出来lag的

Re: A couple of question for Stateful Functions

Thanks for your quick reply. /Dan Den ons 2 sep. 2020 kl 12:24 skrev Igal Shilman : > Hi Dan, let me try to answer your questions: > > >> I guess my question is if one can >> freely mix Flink core with SF's code with regards to performance, >> fault-tolerance, and checkpointing? > > > The main

Re: Job Manager taking long time to upload job graph on remote storage

The logs don't look suspicious. Could you maybe check what the write bandwidth to your GCS bucket is from the machine you are running Flink on? It should be enough to generate a 200 MB file and write it to GCS. Thanks a lot for your help in debugging this matter. Cheers, Till On Wed, Sep 2, 2020

Re: Fail to deploy Flink on minikube

Hmm, this is indeed strange. Could you share the logs of the TaskManager with us? Ideally you set the log level to debug. Thanks a lot. Cheers, Till On Wed, Sep 2, 2020 at 12:45 PM art wrote: > Hi Till, > > The full information when I run command ' kubectl get all’ like this: > > NAME

flink-1.11 使用 application 模式时 jobid 问题

hi: 我在测试 flink-1.11 application 模式时发现 开启 HA 后 jobID 总是 ；关闭 HA 后是个随机字符（和之前版本相同）。这个是个 bug 还是就是这么设计的？ 求大神解答。 -- Best, yuchuan

回复： flink-1.11 sql写ES6问题

放到flink/lib下了 在2020年09月2日 16:52，Danny Chan 写道： Es connector 的包放到哪个目录下了 ？ Best, Danny Chan 在 2020年9月2日 +0800 PM3:38，酷酷的浑蛋 ，写道： Caused by: java.lang.ClassNotFoundException: org.elasticsearch.client.RestClientBuilder Flink sql 写入ES：总是报上面的错误，我检查了依赖并没有冲突这个类啊,而且我解压了jar，里面是有这个类的啊

Flink如何实现至多一次（At Most Once）

我们有一些场景，对实时性要求高，同时对数据重复会有比较大大影响。 我想关闭checkpoint，这样是不是能不能保证“至多一次” (At Most Once) ? 这里会不会有什么坑？ 另外：我们允许丢失数据。 -- ** tivanli **

Re: Job Manager taking long time to upload job graph on remote storage

Hi, Thanks for the response. Yes, we are running Flink in HA mode. We checked there are no such quota limits for GCS for us. Please find the logs below, here you can see the copying of blob started at 11:50:39,455 and it got JobGraph submission at 11:50:46,400. 2020-09-01 11:50:37,061 DEBUG

????????????????????????????????????????????

??flink ---- ??: "user-zh"

Re: Fail to deploy Flink on minikube

Hi Till, The full information when I run command ' kubectl get all’ like this: NAME READY STATUSRESTARTS AGE pod/flink-jobmanager-85bdbd98d8-ppjmf1/1 Running 0 2m34s pod/flink-taskmanager-74c68c6f48-6jb5v 1/1 Running 0

Re: flink1.10集成PrometheusPushGateway，日志报错Failed to push metrics to PushGateway with jobName

你 pushgateway 是什么版本？版本降到 0.8.0 试一下 wangl...@geekplus.com Sender: Jim Chen Send Time: 2020-09-02 17:43 Receiver: flink_user_zh Subject: flink1.10集成PrometheusPushGateway，日志报错Failed to push metrics to PushGateway with jobName Hi: 我的环境是flink1.10.1，是基于yarn的per

Re: A couple of question for Stateful Functions

Hi Dan, let me try to answer your questions: > I guess my question is if one can > freely mix Flink core with SF's code with regards to performance, > fault-tolerance, and checkpointing? The main limitations at the moment is that, currently SF requires a processing time watermark semantics

Re: 请教 hive streaming 报错

最后 加了好多包到 flink/lib 下。我的任务是好了。hive-exec包中依赖的protobuf-java是2.5.0而且是直接把 protobuf-java 打进了jar包。我们这边有其他的任务依赖的protobuf版本是3.5.1，不兼容，服务起不来。。。请问下大佬们有什么好的办法吗。 -- Sent from: http://apache-flink.147419.n8.nabble.com/

Re: 请教 hive streaming 报错

最后加了好多jar包到 flink/lib 下，任务跑起来了。但是hive-exec中依赖的protobuf版本是2.5.0而且打进了jar包里，和其他任务里依赖的protobuf版本3.5.1不兼容。。请问下大佬们有什么好办法吗？ -- Sent from: http://apache-flink.147419.n8.nabble.com/

flink1.10集成PrometheusPushGateway，日志报错Failed to push metrics to PushGateway with jobName

Hi: 我的环境是flink1.10.1，是基于yarn的per job模式运行的。现在集成了PrometheusPushGateway，但是日志中，一直提示Failed to push metrics to PushGateway with jobName。 具体报错日志为： 2020-09-02 15:11:21.901 application_1598509186865_0129 172.22.64.72 wx11-dsj-flink004 [Flink-MetricRegistry-thread-1] WARN o.a.f.m.p.

Re: Fail to deploy Flink on minikube

Hi art, could you check what `kubectl get services` returns? Usually if you run `kubectl get all` you should also see the services. But in your case there are no services listed. You have see something like service/flink-jobmanager otherwise the flink-jobmanager service (K8s service) is not

Flink 涉及到 state 恢复能从 RocketMQ 直接转到 Kafka 吗？

有一个 flink streaming 的程序，读 RocketMQ，中间有一些复杂度计算逻辑用 RocksDB state 存储. 程序有小的更新直接 cancel -s 取消再 run -s 恢复 现在我们需要用 Kafka 替换掉 RocketMQ，消息内容都是一样的， flink 程序需要改一下改为读 Kafka 我可以直接 cancel -s 后再 run -s 复用之前的 state 吗？ wangl...@geekplus.com

Re: Task Chaining slots performance

Hi Vijayendra, in the general case, I believe that chaining will almost always give you better performance since you consume fewer resources, avoid context switches between threads and if object reuse is enabled even avoid serialization when records are passed from one operator to another. The

Re: Using S3 as a streaming File source

OK thanks for the notice on the cost point. I will check the cost calculations. This already does have SNS enabled for another solution to this problem, but I'm trying to use the minimal amount of different software components at this stage of the pipeline. My prefered approach would have been

Re: Default Flink Metrics Graphite

Hi Vijayendra, what was the problem in the end? Sharing this knowledge might help other community member who might run into the same issue. Cheers, Till On Wed, Sep 2, 2020 at 3:34 AM Vijayendra Yadav wrote: > Thanks all, I could see the metrics. > > On Thu, Aug 27, 2020 at 7:51 AM Robert

Re: PyFlink - detect if environment is a StreamExecutionEnvironment

Hi Manas, I am not entirely sure but you might try to check whether env._j_stream_execution_environment is an instance of gateway.jvm.org.apache.flink.streaming.api.environment.LocalStreamEnvironment via Python's isinstance function. Cheers, Till On Wed, Sep 2, 2020 at 5:46 AM Manas Kale

Re: [DKIM Failure] Re: 【闫云鹏】Flink sql 写入es实现object嵌套形式

Hi: 那使用sql怎么表示了？ 查询的结果怎么放入 arrya>中 我刚才尝试使用GROUP_CONCAT来实现sql的拼接，但是flinksql不支持 闫云鹏 在 2020/9/2 16:51，“Danny Chan” 写入: [ { f:f },{ g:g } ] 可否用 Array> 来表达？ Best, Danny Chan 在 2020年9月2日 +0800 PM3:54，user-zh@flink.apache.org，写道：

sql-training????????

hi?? flink-sql-training flink sql??docker-compose.ymldocker-compose.ymldocker-compose up -d ??ERROR: build path /root/flink-sql-training/images/flink-image either does not exist, is not accessible, or is not a

Re: flink-1.11 sql写ES6问题

可能是jar包冲突。 -- Sent from: http://apache-flink.147419.n8.nabble.com/

Re: flink-1.11 sql写ES6问题

Es connector 的包放到哪个目录下了 ？ Best, Danny Chan 在 2020年9月2日 +0800 PM3:38，酷酷的浑蛋 ，写道： > Caused by: java.lang.ClassNotFoundException: > org.elasticsearch.client.RestClientBuilder > Flink sql 写入ES：总是报上面的错误，我检查了依赖并没有冲突这个类啊,而且我解压了jar，里面是有这个类的啊 > > >

Re:Re: 回复：flink1.9.1采用Prometheus Pushgateway监控，当任务被kill掉，但grafana还是可以监控到数据

hi： 你可以尝试下 prometheus-pushgateway-cleaner，支持 docker 运行 https://github.com/jorinvo/prometheus-pushgateway-cleaner 可以定时清理指定时间未更新的 pushgateway metric，希望可以帮到你 -- Best, yuchuan 在 2020-09-02 11:25:25，"Jim Chen" 写道： >能不能过restful api拿到pushgateway中metric上报的时间？通过这个时间动态判断删除group > >xiao cai

Re: A couple of question for Stateful Functions

Hi Dan, thanks for reaching out to the Flink community. I'm pulling in Gordon and Igal who will be able to answer your questions. Cheers, Till On Wed, Sep 2, 2020 at 8:22 AM danp wrote: > Hi, > > Nice to see the progress of Stateful functions. > > I have a few questions that I hope you can

Re: 【闫云鹏】Flink sql 写入es实现object嵌套形式

[ { f:f },{ g:g } ] 可否用 Array> 来表达？ Best, Danny Chan 在 2020年9月2日 +0800 PM3:54，user-zh@flink.apache.org，写道： > > [ > { > f:f > },{ > g:g > } > ]

Re: FlinkKafkaConsumer problem

The reason two Flink jobs using a Kafka consumer with the same consumer group are seeing the same events is that Flink's FlinkKafkaConsumer does not participate in Kafka's consumer group management. Instead Flink manually assigns all partitions to the source operators (on a per job basis). The

Re: Re:Flink 1.10.1 cdh-hdfs启用HA时，flink job提交报错

现在的配置是这样的，没有添加namenode+ip； jobmanager.archive.fs.dir: hdfs:///completed-jobs/ 需要改成： hdfs://nameservice2/completed-jobs/ 这样的吗？ 感觉是创建fs的时候错了。看到这部分异常： createNonHAProxy -- Sent from: http://apache-flink.147419.n8.nabble.com/

Re: Job Manager taking long time to upload job graph on remote storage

Hi Prakhar, have you enabled HA for your cluster? If yes, then Flink will try to store the job graph to the configured high-availability.storageDir in order to be able to recover it. If this operation takes long, then it is either the filesystem which is slow or storing the pointer in ZooKeeper.

Re: Fail to deploy Flink on minikube

Hi art, could you verify that the jobmanager-service has been started? It looks as if the name flink-jobmanager is not resolvable. It could also help to know the Minikube and K8s version you are using. Cheers, Till On Wed, Sep 2, 2020 at 9:50 AM art wrote: > Hi，I’m going to deploy flink on

Flink 1.10.1 cdh-hdfs启用HA时，flink job提交报错

各位老师好，在HDFS上开启HA的时候，向yarn提交任务的时候，遇到点问题。 cdh版本：5.15.2 hdfs版本：2.6.0 启动模式：flink-on-yarn 配置了HADOOP_CONF_DIR=/etc/hadoop/conf 命令： ./bin/flink run -m yarn-cluster -yt /yarn-conf -p 3 -ytm 2048 -ys 1 -ynm xxx /jars/flink10.jar xxx HDFS不启用HA的时候，能正常提交。 提交任务到yarn的时候，出现如下异常：nameservice2

Fail to deploy Flink on minikube

Hi，I’m going to deploy flink on minikube referring to https://ci.apache.org/projects/flink/flink-docs-release-1.11/zh/ops/deployment/kubernetes.html ; kubectl create -f

Re:Flink 1.10.1 cdh-hdfs启用HA时，flink job提交报错

你好， 应该是flink访问hdfs的时候，没有找到namespace 如果做flink任务提交的机器是CDH集群的网关节点，可以将flink-conf.yaml中hdfs有关的配置都配成类似： hdfs:///flink/ha/ 去掉namespace和端口的配置，再尝试一下。 Best Regards Huang Li --Original-- From: "Storm☀️"http://apache-flink.147419.n8.nabble.com/

Job Manager taking long time to upload job graph on remote storage

Hi, We are currently running Flink 1.9.0. We see a delay of around 20 seconds in order to start a job on a session Flink cluster. We start the job using Flink's monitoring REST API where our jar is already uploaded on Job Manager. Our jar file size is around 200 MB. We are using memory state

??????????????????????????????????????

flink SQL,tumble window //eventtime??watermark DataStream> withTimestampsAndWatermarksDS = singleDS.assignTimestampsAndWatermarks( WatermarkStrategy

FlinkKafkaConsumer problem

hi, i am confused about consumer group of FlinkKafkaConsumer, i have two applications,with the same code like this: //--- val bsEnv = StreamExecutionEnvironment.getExecutionEnvironment Env.setRestartStrategy(RestartStrategies.noRestart()) val consumerProps = new

flink-1.11 sql写ES6问题

Caused by: java.lang.ClassNotFoundException: org.elasticsearch.client.RestClientBuilder Flink sql 写入ES：总是报上面的错误，我检查了依赖并没有冲突这个类啊,而且我解压了jar，里面是有这个类的啊

Flink 1.10.1 cdh-hdfs启用HA时，flink job提交报错

各位老师，好。 Flink on yarn 模式提交 hadoop：2.6.0 cdh 5.15.2 HADOOP_CONF_DIR=/etc/hadoop/conf 在cdh上开启hdfs的HA之后提交任务报错,不开启HA能正常提交任务。 启动方式： /bin/flink run -m yarn-cluster -yt /yarn-conf -p 3 -ytm 2048 -ys 1 -ynm xxx /jars/flink10.jar xx 报错信息r如下： 2020-09-02 14:53:08,118 DEBUG

Re: Editing Rowtime for SQL Table

Yes, the new TableSource API allows to emit retractions. However, it does not give you direct access to DataStream API. FLIP-136 [1] might help you in the near future. We hope it can be part of 1.12. Regards, Timo [1]

ceshi

测试内容 -- Sent from: http://apache-flink.147419.n8.nabble.com/

【闫云鹏】Flink sql 写入es实现object嵌套形式

Hi 我使用flink sql mysql作为原表，多表join后将结果写入es的时候如何实现这种Object结构 { a:a, b:{ c:c d:d }, e:[ { f:f },{ g:g } ] } 看到了sql中支持的数据类型是有map 或者 row这种来支持的，但是测试了几次都没成功，能给个实例？ 闫云鹏

Re: Debezium Flink EMR

This worked, thanks! Looking forward to the future releases :) On Mon, Aug 31, 2020 at 5:06 PM Marta Paes Moreira wrote: > Hey, Rex! > > This is likely due to the tombstone records that Debezium produces for > DELETE operations (i.e. a record with the same key as the deleted row and a > value

A couple of question for Stateful Functions

Hi, Nice to see the progress of Stateful functions. I have a few questions that I hope you can reply to. My first question is regarding the newly implemented StatefulFunctionDataStreamBuilder. Is there anything to pay attention to if one first union a couple of streams and performs a sort via

Re: 回复：flink1.9.1采用Prometheus Pushgateway监控，当任务被kill掉，但grafana还是可以监控到数据

您好，可以请教一下如何清理Pushgateway中的数据么？我是想主动调用pushgateway的delete方法来删除pushgetway的metrics，但是Pushgateway中的metrics因为设置了randomJobNameSuffix：true参数，导致Pushgateway中的job名称都是随机生成的，那么该如何清理呢？ - Best Wishes -- Sent from:

?????? Flink ???? RocksDB CPU ????

?? subtask https://drive.google.com/file/d/1uiH2vNi0kMGHuiHOW5Wq-m053ys4rFHu/view?usp=sharing : contains CPU ?? 5.92%?? CPU MapState ?? contains CPU ??

Cannot load user class

Hi all， 我们对kafka connector flink 进行了扩展，flink-connector-kafka-base包中新增了类，在功能迁移到1.11.1中，但是sql-cli中测试运行时报了无法加载类的异常，1.10.1版本是ok的，是不是1.11版本对类加载做了什么改动？ 求大佬解惑，谢谢 异常如下： Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: