Hi Kamal, It confuses me a little that what's the purpose of opening a server socket in SplitEnumerator? Currently there will be only one SplitEnumerator instance in JobManager for each one source not each source subtask. If there's only one source in your job, no matter how much parallelism this source has, there will be only one SplitEnumerator instance in JM which can connect and send events to the source subtasks.
Additionally, if you want to read data from a netty source, you can implement your NettySplitEnumerator to get splits from netty source and assign them to a NettySourceReader which will read data from netty source according to the splits. Best, Shammon FY On Tue, Aug 1, 2023 at 12:02 PM Kamal Mittal via user <user@flink.apache.org> wrote: > Hello, > > > > I created a custom server socket source in which opened server socket in > split enumerator, source parallelism is =2 and it gives error while running > - “Address is already in use” i.e. it tried to open two server sockets as > per parallelism which is not correct as parallelism is applicable for > source operator and not for split enumerator? > > > > Please correct me if above understanding is not correct. > > > > Rgds, > > Kamal > > > > *From:* Hang Ruan <ruanhang1...@gmail.com> > *Sent:* 01 August 2023 08:55 AM > *To:* Kamal Mittal <kamal.mit...@ericsson.com> > *Cc:* liu ron <ron9....@gmail.com>; user@flink.apache.org > *Subject:* Re: Flink netty connector for TCP source > > > > Hi, Kamal. > > > > The SplitEnumerator is contained in the SourceCoordinator. They are only > used in JM. > > > > Best, > > Hang > > > > Kamal Mittal via user <user@flink.apache.org> 于2023年8月1日周二 10:43写道： > > Thanks. > > > > I looked at the link for custom data sources, one query here that how to > make sure for Split enumerator to execute on Job Manager rather than at > Task manager? > > > > *From:* liu ron <ron9....@gmail.com> > *Sent:* 31 July 2023 10:06 PM > *To:* user@flink.apache.org > *Subject:* Re: Flink netty connector for TCP source > > > > Hi, Kamal > > Currently, we don't provide the connector like TCP source in Flink main > repo. If you need this connector, you can try to implement it refer to the > FLIP-27 source docs > > > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/sources/ > > > > Best, > > Ron > > > > Shammon FY <zjur...@gmail.com> 于2023年7月27日周四 11:23写道： > > I cannot find any information about netty source in flink website and it > is not in the connector list[1], so I'm think that it is not supported by > flink community > > > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/overview/ > > > > Best, > > Shammon FY > > > > On Thu, Jul 27, 2023 at 10:53 AM Kamal Mittal <kamal.mit...@ericsson.com> > wrote: > > Hello Shammon, > > > > Yes socket text stream I am aware of but was thinking if something like as > ‘https://github.com/apache/bahir-flink/tree/master/flink-connector-netty > <https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-ffc853ff875d9ed3&q=1&e=30c3fa9e-7c08-4486-8017-0d78c3c714ac&u=https%3A%2F%2Fgithub.com%2Fapache%2Fbahir-flink%2Ftree%2Fmaster%2Fflink-connector-netty>’ > is also supported by Flink? > > > > Rgds, > > Kamal > > > > *From:* Shammon FY <zjur...@gmail.com> > *Sent:* 27 July 2023 08:15 AM > *To:* Kamal Mittal <kamal.mit...@ericsson.com> > *Cc:* user@flink.apache.org > *Subject:* Re: Flink netty connector for TCP source > > > > Hi Kamal, > > > > There's socket text stream in `DataStream` and you can refer to [1] for > more details. > > > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/overview/#example-program > > > > Best, > > Shammon FY > > > > On Wed, Jul 26, 2023 at 4:26 PM Kamal Mittal via user < > user@flink.apache.org> wrote: > > Hello, > > > > Does flink provides netty connector for custom TCP source? > > > > Any documentation details please share? > > > > Rgds, > > Kamal > >
