Re: Issue with file system implementation

2022-09-01 Thread yidan zhao
Do you place the s3 jar to plugins/s3 dir? Darius Žalandauskas 于2022年8月31日周三 03:11写道: > > Hello apache-flink team, > For the last week I am really struggling with setting up EMR to store stream > output to AWS S3. > According to the documentation, if running flink with emr, no manual > adjustme

Re: Why this example does not save anything to file?

2022-09-01 Thread podunk
Thank you. So I understand 'this is normal Flink property - live with it' :-)   M.   Sent: Wednesday, August 31, 2022 at 9:37 PM From: "David Anderson" To: pod...@gmx.com Cc: "user" Subject: Re: Why this example does not save anything to file? With the parallelism set to 2, you will get 2 files

Re: Issue with file system implementation

2022-09-01 Thread Darius Žalandauskas
Hello, I place it according to documentation under /plugins/s3-fs-hadoop mkdir ./plugins/s3-fs-hadoop > cp ./opt/flink-s3-fs-hadoop-1.14.4.jar ./plugins/s3-fs-hadoop/ Pagarbiai/Respectfuly | *Darius Žalandauskas* | Phone: +37063082356 | eMail: darius.zalandaus...@gmail.com On Thu, 1 Sept 202

回复: Re:Re: get NoSuchMethodError when using flink flink-sql-connector-hive-2.2.0_2.11-1.14.4.jar

2022-09-01 Thread Liting Liu (litiliu)
Tks for your suggestion xuyang. I have solved this problem by packing a new flink-sql-connector-hive-2.2.0_2.11-1.14.4.jar with the conflict jar shaded. 发件人: Xuyang 发送时间: 2022年9月1日 0:27 收件人: user@flink.apache.org 抄送: luoyu...@alumni.sjtu.edu.cn ; Liting Liu (

Kafka source stops consuming messages from topic after some minutes

2022-09-01 Thread alfredo.vasquez.spglobal.com via user
Hello, Im using flink-connector-kafka version 1.15.2 to consume messages from a kafka topic which has 3 partitions that later its connected to to another kafka source and then processed in a BroadcastProcessFunction. The Kafka source is created as follows: Properties properties = new Propertie

Kubernetes operator expose UI rest service as NodePort instead of default clusterIP

2022-09-01 Thread Vignesh Kumar Kathiresan via user
Hello Flink community, Need some help with "flink kubernetes operator" based cluster setup. My flink cluster is set up using the flink-kubernetes-operator in AWS EKS. The required resources(deployments, pods, services, configmaps etc) are created as expected. But the service "*-rest" is created a

Re: Kubernetes operator expose UI rest service as NodePort instead of default clusterIP

2022-09-01 Thread Yang Wang
I am afraid the current flink-kubernetes-operator always overrides the "REST_SERVICE_EXPOSED_TYPE" to "ClusterIP". Could you please share why the ingress[1] could not meet your requirements? Compared with NodePort, I think it is a more graceful implementation. [1]. https://nightlies.apache.org/fli

Re: [E] Re: Kubernetes operator expose UI rest service as NodePort instead of default clusterIP

2022-09-01 Thread Vignesh Kumar Kathiresan via user
Hi Yang, Yeah, I gathered that from the operator code soon after posting. I am using the aws alb ingress class [1]. There under considerations it is mentioned if the alb target type is "instance" which is the default traffic mode, the kubernetes service type has to be nodeport or loadbalancer. Al

flink ci build run longer than the maximum time of 310 minutes.

2022-09-01 Thread hjw
I commit a pr to Flink Github . A error happened in building ci. [error]The job running on agent Azure Pipelines 6 ran longer than the maximum time of 310 minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134 How to solve this problem? How to triigle the ci building