Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-24 Thread Lan Liang
Actually, we really have same case in production. We have not use pulsar function or pulsar io, just implemented it from ourself code. Best Regards, Lan Liang On 11/18/2021 04:22,Neng Lu wrote: @sijie the case here might be tricky. They may want to move data across pulsar clusters operate

Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-17 Thread Neng Lu
@sijie the case here might be tricky. They may want to move data across pulsar clusters operated by different org or teams. Remember we previously added the ability to send messages to external pulsar clusters for pulsar function but got reverted. I think this is the case they are trying to tackle

Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-17 Thread Sijie Guo
I don't think you need a separate connector. An identity function should be able to do the job for you. - Sijie On Mon, Nov 15, 2021 at 3:34 PM Neng Lu wrote: > Just did a quick search, it's interesting we don't have a pulsar connector > to move data among pulsar clusters. > I guess people usu

Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-15 Thread Neng Lu
Just did a quick search, it's interesting we don't have a pulsar connector to move data among pulsar clusters. I guess people usually write their own pulsar client to move data around. On Mon, Nov 15, 2021 at 3:11 PM ZhangJian He wrote: > Yes, move data across different pulsar clusters which be

Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-15 Thread ZhangJian He
Yes, move data across different pulsar clusters which belongs to different company or organization Thanks ZhangJian He Neng Lu 于2021年11月16日周二 上午2:50写道: > Hi, > > What's your new connector used for in the customer use cases? > A `pulsar-io-kafak-connector` is used for moving data between kafka a

Re: [DISCUSS] Add Pulsar io Pulsar connector

2021-11-15 Thread Neng Lu
Hi, What's your new connector used for in the customer use cases? A `pulsar-io-kafak-connector` is used for moving data between kafka and pulsar. But in your case, a `pulsar-io-pulsar-connector`, do you mean you want to move data across different pulsar clusters? On Mon, Nov 15, 2021 at 6:51 AM

[DISCUSS] Add Pulsar io Pulsar connector

2021-11-15 Thread ZhangJian He
Dear all My team are suggesting some of our customers use pulsar instead of kafka for their needs. Before, my team used a pulsar-io-kafka-connector, now my team wants to use a pulsar-io-to-pulsar-connector server for these customers. And I notice now we don't have a pulsar-io-pulsar-connector. S