Hello,

I hope you have the context of use case clear now as described in below mail as 
well as here again –

Flink application will be opening a server socket as part of custom source over 
a fixed port and client will be opening client sockets for Flink server socket. 
To read data from each client socket, Flink server socket will open a thread 
after accepting the connection which can be 1000 in nos.

Flink server socket application is running over K8s environment.

Do let me know please that it is ok to open multiple threads for accepting 
client socket connections inside flink application?

Rgds,
Kamal

From: Kamal Mittal via user <user@flink.apache.org>
Sent: 03 August 2023 09:27 AM
To: Paul Lam <paullin3...@gmail.com>
Cc: user@flink.apache.org
Subject: RE: Flink operator task opens threads internally

Hello,

We have a client sending TCP traffic towards Flink application and to support 
that there is server socket (Configurable port) opened which is accepting 
socket connections.

So to accept multiple client connections we have used a thread pool 
(configurable size) which will execute in a flink task slot.

Rgds,
KAmal

From: Paul Lam <paullin3...@gmail.com<mailto:paullin3...@gmail.com>>
Sent: 03 August 2023 08:59 AM
To: Kamal Mittal <kamal.mit...@ericsson.com<mailto:kamal.mit...@ericsson.com>>
Cc: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Re: Flink operator task opens threads internally

Hi Kamal,

It’s okay if you don’t mind the data order.

But it’s not very commonly seen to accept client sockets from Flink jobs, as
the socket server address is dynamic and requires service discovery.

Would you like to share more about the background?

Best,
Paul Lam

2023年8月3日 10:26,Kamal Mittal via user 
<user@flink.apache.org<mailto:user@flink.apache.org>> 写道:

Hello Community,

Please share views for the below mail.

Rgds,
Kamal

From: Kamal Mittal via user 
<user@flink.apache.org<mailto:user@flink.apache.org>>
Sent: 02 August 2023 08:19 AM
To: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Flink operator task opens threads internally

Hello Community,

I have a operator pipeline like as below, is it ok if “source” task opens 
threads by using java thread pool and parallelize the work?

This is needed for accepting multiple client socket connections in “single 
custom source server socket function”.

Single Custom source server socket function – ServerSocket serversocket = new 
ServerSocket(9999);
Now using thread pool accept multiple connections in separate threads = new 
Runnable () -> serversocket.accept();


<image001.png>

Rgds,
Kamal

Reply via email to