Hi all!
I'm trying to understand the logic of saving checkpoint files and from the
exchange dump with ceph I see the following requests
HEAD
/checkpoints/example-job//shared/9701fae2-0de3-4d6c-b08b-0a92fb7285c9
HTTP/1.1
HTTP/1.1 404 Not Found
HEAD
Thank you for the tips, I will try these out
From: Josh Mahonin
Sent: 18 January 2024 21:07
To: Qing Lim
Cc: Jun Qin ; User
Subject: Re: Use different S3 access key for different S3 bucket
Oops my syntax was a bit off there, as shown in the Hadoop docs, it looks like:
fs.s3a.bucket..
Josh
Glad to hear this!
Best,
Zakelly
On Fri, Jan 19, 2024 at 9:22 AM Konstantinos Karavitis
wrote:
> I would like again to thank you as we managed to fix this strange issue we
> had by moving all the state initializations into the open method of
> ProcessFunction!
>
> On Thu, Jan 18, 2024 at
Please send email to user-unsubscr...@flink.apache.org and
user-zh-unsubscr...@flink.apache.org if you want to unsubscribe the mail
from user@flink.apache.org and user...@flink.apache.org, you can refer
[1][2] for more details.
Best,
Junrui
[1]
Hi,
oracle cdc connector 已经接入增量快照读框架，动态加表也是可以统一去实现的。
可以去社区创建issue，也欢迎直接贡献。
Best,
Jiabao
On 2024/01/19 04:46:21 "casel.chen" wrote:
>
>
>
>
>
>
> 想知道oracle cdc connector不支持动态加表的原因是什么？可否自己扩展实现呢？
>
>
>
>
>
>
>
>
>
>
>
> 在 2024-01-19 11:53:49，"Jiabao Sun" 写道：
> >Hi,
> >
> >Oracle
想知道oracle cdc connector不支持动态加表的原因是什么？可否自己扩展实现呢？
在 2024-01-19 11:53:49，"Jiabao Sun" 写道：
>Hi,
>
>Oracle CDC connector[1] 目前是不支持动态加表的。
>
>Best,
>Jiabao
>
>[1]
>https://ververica.github.io/flink-cdc-connectors/release-2.4/content/connectors/oracle-cdc.html
>
>
>On 2024/01/19
Please send email to user-unsubscr...@flink.apache.org if you want to
unsubscribe the mail from u...@flink.apache.org, and you can refer [1][2]
for more details.
请发送任意内容的邮件到 user-unsubscr...@flink.apache.org 地址来取消订阅来自
u...@flink.apache.org 邮件组的邮件，你可以参考[1][2] 管理你的邮件订阅。
Best,
Jiabao
[1]
Hi,
Oracle CDC connector[1] 目前是不支持动态加表的。
Best,
Jiabao
[1]
https://ververica.github.io/flink-cdc-connectors/release-2.4/content/connectors/oracle-cdc.html
On 2024/01/19 03:37:41 Jiabao Sun wrote:
> Hi,
>
> 请提供一下 flink cdc 的版本，使用的什么连接器。
> 如果方便的话，也请提供一下日志。
> 另外，table 的正则表达式可以匹配到新增的表吗？
>
>
退订
退订
Hi,
请提供一下 flink cdc 的版本，使用的什么连接器。
如果方便的话，也请提供一下日志。
另外，table 的正则表达式可以匹配到新增的表吗？
Best,
Jiabao
[1]
https://ververica.github.io/flink-cdc-connectors/release-3.0/content/connectors/mysql-cdc%28ZH%29.html#id15
On 2024/01/19 03:27:22 王凯 wrote:
> 在使用flink
Hi Alexandre,
I couldn't find the image apache/flink-statefun-playground:3.3.0-1.0 in Docker
Hub.
You can temporarily use the release-3.2 version.
Hi Martijn, did we ignore pushing it to the docker registry?
Best,
Jiabao
[1] https://hub.docker.com/r/apache/flink-statefun-playground/tags
On
I would like again to thank you as we managed to fix this strange issue we
had by moving all the state initializations into the open method of
ProcessFunction!
On Thu, Jan 18, 2024 at 11:53 PM Konstantinos Karavitis <
kkaravi...@gmail.com> wrote:
> Thank you very much Zakelly for taking the time
Thank you very much Zakelly for taking the time to answer to my question. I
appreciate it a lot.
Unfortunately, I cannot share the source code as it is confidential and
owned by the company that I co-operate with.
But, yes you are right that inside the code, I can see that the state
initialization
Oops my syntax was a bit off there, as shown in the Hadoop docs, it looks
like:
fs.s3a.bucket..
Josh
>
Hi Qing,
You may have some luck with using per-bucket S3 configuration. Assuming
you're using the flink-s3-fs-hadoop plugin, you should be able to apply
different access keys to different buckets, eg:
Hi,
I am trying to run the example provided here:
https://github.com/apache/flink-statefun-playground/tree/release-3.3/python/greeter
1 - Following the read.me, with docker (that I installed):
"docker-compose build" works well. But "docker-compose up" returns an error:
[image: image.png]
2 -
Hi, I am suddenly starting to get
* java.lang.IllegalArgumentException: The minBucketMemorySize is not valid!
It comes from `org.apache.flink.table.runtime.util.collections.binary.
AbstractBytesHashMap`
I believe the actually value is coming from generated code, any advice on what
we
Hi Yang,
You can run `StandaloneAutoscalerEntrypoint`, and the scale report will print
in log (info level) by LoggingEventHandler[2].
[1]
flink-kubernetes-operator/flink-autoscaler-standalone/src/main/java/org/apache/flink/autoscaler/standalone/StandaloneAutoscalerEntrypoint.java
at main ·
Hi,
I have a question about how to correctly set up a test that will read
input from locally provided collection in bounded mode and provide
outputs at the end of the computation. My test case looks something like
the following:
String[] lines = ...;
try (StreamExecutionEnvironment env =
Hi Jun
I am indeed talking about processing two different tables, but I don’t see any
option that allow configuring credentials at Flink table level, do you know
where is it documented?
Today we are setting the credentials via Flink conf yaml, which is documented
here:
Hi Qing
The S3 credentials are associated with Flink SQL tables.
I assume you are talking about processing/joining from two different tables,
backed up by two different S3 buckets. If so, you can provide different
credentials for different tables, then use the two tables in your pipeline.
Jun
Hi Sun,
As Dulce said, running in a cluster is typically recommended. However if for
some reason you need to run in standalone mode, why do you recreate Cluster on
each job, can you try to reuse the MiniCluster?
I happen to have a similar setup when we are running in standalone mode, and
run
Hello dear flink community,
I noticed that there's a scaling report feature (specifically, the strings
defined in AutoscalerEventHandler) in the Flink operator autoscaler.
However, I'm unable to find this information in the Flink operator logs.
Could anyone guide me on how to access or visualize
Hi, I am using Flink SQL to create table backed by S3 buckets.
We are not using AWS S3, so we have to use access key and secret for Auth.
My pipeline depends on 2 different buckets, each requires different
credentials, can flink support this?
Qing Lim | Marshall Wace LLP, George House, 131
25 matches
Mail list logo
