Hello, Can you please share views about below mail?
Rgds, Kamal From: Kamal Mittal via user <user@flink.apache.org> Sent: 28 July 2023 07:59 AM To: Martijn Visser <mvis...@confluent.io> Cc: user@flink.apache.org Subject: RE: Custom TCP server socket source Hello Martijn, I followed the same link and created Enumerator but that enumerator looks like opens ServerSocket in task manager via custom source rather than at job manager. Attached the very rough custom source code. FlinkClientApp – Main program SSLServerCustomSocketStreamFunction_Copy – custom source SocketEnumerator – Implementation of SplitEnumerator In below diagram taken from flink custom source link only, it is said that enumerator works over job manager but how it is attached/scheduled over job manager? I am thinking to open ServerSocket at job manager and then accept Socket connections (Socket object) as split (unbounded) in respective source readers (task managers). Please share your views. [cid:image001.png@01D9C387.49D33500] Rgds, Kamal From: Martijn Visser <mvis...@confluent.io<mailto:mvis...@confluent.io>> Sent: 27 July 2023 06:19 PM To: Kamal Mittal <kamal.mit...@ericsson.com<mailto:kamal.mit...@ericsson.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Custom TCP server socket source Hi Kamal, The best starting point would be to look at how to write a custom source connector. Have a look at https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/sources/ which also includes links to the various classes that you'll need. Please let us know what else you've tried already and what issues you've encountered. Best regards, Martijn On Thu, Jul 27, 2023 at 6:54 AM Kamal Mittal via user <user@flink.apache.org<mailto:user@flink.apache.org>> wrote: Hello, I need to write “Custom server socket source” which accepts client connections over a port. 1. How to scale it across task managers with parallelism <= no. of task managers and with same single port 2. This is needed w.r.t Kubernetes POD deployment model where each POD is running a task manager Doing it with in same task manager causes port collision which is expected. Rgds, Kamal
