Hello, I hope you have the context of use case clear now as described in below mail as well as here again –
Flink application will be opening a server socket as part of custom source over a fixed port and client will be opening client sockets for Flink server socket. To read data from each client socket, Flink server socket will open a thread after accepting the connection which can be 1000 in nos. Flink server socket application is running over K8s environment. Do let me know please that it is ok to open multiple threads for accepting client socket connections inside flink application? Rgds, Kamal From: Kamal Mittal via user <user@flink.apache.org> Sent: 03 August 2023 09:27 AM To: Paul Lam <paullin3...@gmail.com> Cc: user@flink.apache.org Subject: RE: Flink operator task opens threads internally Hello, We have a client sending TCP traffic towards Flink application and to support that there is server socket (Configurable port) opened which is accepting socket connections. So to accept multiple client connections we have used a thread pool (configurable size) which will execute in a flink task slot. Rgds, KAmal From: Paul Lam <paullin3...@gmail.com<mailto:paullin3...@gmail.com>> Sent: 03 August 2023 08:59 AM To: Kamal Mittal <kamal.mit...@ericsson.com<mailto:kamal.mit...@ericsson.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Flink operator task opens threads internally Hi Kamal, It’s okay if you don’t mind the data order. But it’s not very commonly seen to accept client sockets from Flink jobs, as the socket server address is dynamic and requires service discovery. Would you like to share more about the background? Best, Paul Lam 2023年8月3日 10:26，Kamal Mittal via user <user@flink.apache.org<mailto:user@flink.apache.org>> 写道： Hello Community, Please share views for the below mail. Rgds, Kamal From: Kamal Mittal via user <user@flink.apache.org<mailto:user@flink.apache.org>> Sent: 02 August 2023 08:19 AM To: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Flink operator task opens threads internally Hello Community, I have a operator pipeline like as below, is it ok if “source” task opens threads by using java thread pool and parallelize the work? This is needed for accepting multiple client socket connections in “single custom source server socket function”. Single Custom source server socket function – ServerSocket serversocket = new ServerSocket(9999); Now using thread pool accept multiple connections in separate threads = new Runnable () -> serversocket.accept(); <image001.png> Rgds, Kamal
