Re: Pinning clients to specific brokers

2017-08-24 Thread Mohit Chawla 
Hi Stephen, Hans,

This can help in keeping the traffic isolated in an availability zone, and
avoid inter AZ communication.

Thanks,
Mohit

On Thursday, August 24, 2017, Hans Jespersen  wrote:

> We (Confluent) run Kafka as a SaaS-based cloud offering and we do not see
> any reason for this feature so I just don’t understand the motivation for
> it. Please explain.
>
> -hans
>
> --
> /**
>  * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
>  * h...@confluent.io  (650)924-2670
>  */
>
>
>
>
> > On Aug 23, 2017, at 12:42 AM, Mohit Chawla <
> mohit.chawla.bin...@gmail.com > wrote:
> >
> > Hey Hans,
> >
> > Thanks for your reply.
> >
> > In a cloud environment this can be useful. Perhaps if partitioning and
> > replicas are selected appropriately, this could be possible ?
> >
> > Thanks,
> > Mohit
> >
> > On Tuesday, August 22, 2017, Hans Jespersen  > wrote:
> >
> >> Doing that doesn't really make sense in a Kafka cluster because the
> topic
> >> partitions and their replicas are spread out across many brokers in the
> >> cluster. That's what enables the parallel processing and fault tolerance
> >> features of Kafka.
> >>
> >> -hans
> >>
> >>> On Aug 22, 2017, at 3:14 AM, Mohit Chawla <
> mohit.chawla.bin...@gmail.com 
> >> > wrote:
> >>>
> >>> Hi folks,
> >>>
> >>> Is it possible to pin kafka clients to use only specific brokers
> >> throughout
> >>> their lifetime and not just for the initial bootstrapping ?
> >>>
> >>> Thanks,
> >>> Mohit
> >>
>
>

Re: Pinning clients to specific brokers

2017-08-23 Thread Hans Jespersen 
We (Confluent) run Kafka as a SaaS-based cloud offering and we do not see any 
reason for this feature so I just don’t understand the motivation for it. 
Please explain.

-hans

-- 
/**
 * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
 * h...@confluent.io (650)924-2670
 */




> On Aug 23, 2017, at 12:42 AM, Mohit Chawla  
> wrote:
> 
> Hey Hans,
> 
> Thanks for your reply.
> 
> In a cloud environment this can be useful. Perhaps if partitioning and
> replicas are selected appropriately, this could be possible ?
> 
> Thanks,
> Mohit
> 
> On Tuesday, August 22, 2017, Hans Jespersen  wrote:
> 
>> Doing that doesn't really make sense in a Kafka cluster because the topic
>> partitions and their replicas are spread out across many brokers in the
>> cluster. That's what enables the parallel processing and fault tolerance
>> features of Kafka.
>> 
>> -hans
>> 
>>> On Aug 22, 2017, at 3:14 AM, Mohit Chawla > > wrote:
>>> 
>>> Hi folks,
>>> 
>>> Is it possible to pin kafka clients to use only specific brokers
>> throughout
>>> their lifetime and not just for the initial bootstrapping ?
>>> 
>>> Thanks,
>>> Mohit
>>

Re: Pinning clients to specific brokers

2017-08-23 Thread Stephen Durfey 
Mohit,

Can you describe your use case around why you want this to happen?

Thanks


From: Joao Reis 
Sent: Wednesday, August 23, 2017 11:08:02 AM
To: users@kafka.apache.org
Subject: Re: Pinning clients to specific brokers

Hey Mohit,

I agree with Hans, and additionally you may have trouble when the cluster needs 
to rebalance its partitions or when partition leadership changes node.

Also you may loose high-availability and cluster resilience in case of 
fail-over.

Cheers
João Reis
On 23 Aug 2017, at 08:42, Mohit Chawla 
mailto:mohit.chawla.bin...@gmail.com>> wrote:

Hey Hans,

Thanks for your reply.

In a cloud environment this can be useful. Perhaps if partitioning and
replicas are selected appropriately, this could be possible ?

Thanks,
Mohit

On Tuesday, August 22, 2017, Hans Jespersen 
mailto:h...@confluent.io>> wrote:

Doing that doesn't really make sense in a Kafka cluster because the topic
partitions and their replicas are spread out across many brokers in the
cluster. That's what enables the parallel processing and fault tolerance
features of Kafka.

-hans

On Aug 22, 2017, at 3:14 AM, Mohit Chawla 
mailto:mohit.chawla.bin...@gmail.com>
> wrote:

Hi folks,

Is it possible to pin kafka clients to use only specific brokers
throughout
their lifetime and not just for the initial bootstrapping ?

Thanks,
Mohit



__
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit 
http://www.symanteccloud.com<http://www.symanteccloud.com/>
__


__
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
__

Re: Pinning clients to specific brokers

2017-08-23 Thread Joao Reis 
Hey Mohit,

I agree with Hans, and additionally you may have trouble when the cluster needs 
to rebalance its partitions or when partition leadership changes node.

Also you may loose high-availability and cluster resilience in case of 
fail-over.

Cheers
João Reis
On 23 Aug 2017, at 08:42, Mohit Chawla 
mailto:mohit.chawla.bin...@gmail.com>> wrote:

Hey Hans,

Thanks for your reply.

In a cloud environment this can be useful. Perhaps if partitioning and
replicas are selected appropriately, this could be possible ?

Thanks,
Mohit

On Tuesday, August 22, 2017, Hans Jespersen 
mailto:h...@confluent.io>> wrote:

Doing that doesn't really make sense in a Kafka cluster because the topic
partitions and their replicas are spread out across many brokers in the
cluster. That's what enables the parallel processing and fault tolerance
features of Kafka.

-hans

On Aug 22, 2017, at 3:14 AM, Mohit Chawla 
mailto:mohit.chawla.bin...@gmail.com>
> wrote:

Hi folks,

Is it possible to pin kafka clients to use only specific brokers
throughout
their lifetime and not just for the initial bootstrapping ?

Thanks,
Mohit



__
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit 
http://www.symanteccloud.com
__


__
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
__

Re: Pinning clients to specific brokers

2017-08-23 Thread Mohit Chawla 
Hey Hans,

Thanks for your reply.

In a cloud environment this can be useful. Perhaps if partitioning and
replicas are selected appropriately, this could be possible ?

Thanks,
Mohit

On Tuesday, August 22, 2017, Hans Jespersen  wrote:

> Doing that doesn't really make sense in a Kafka cluster because the topic
> partitions and their replicas are spread out across many brokers in the
> cluster. That's what enables the parallel processing and fault tolerance
> features of Kafka.
>
> -hans
>
> > On Aug 22, 2017, at 3:14 AM, Mohit Chawla  > wrote:
> >
> > Hi folks,
> >
> > Is it possible to pin kafka clients to use only specific brokers
> throughout
> > their lifetime and not just for the initial bootstrapping ?
> >
> > Thanks,
> > Mohit
>

Re: Pinning clients to specific brokers

2017-08-22 Thread Hans Jespersen 
Doing that doesn't really make sense in a Kafka cluster because the topic 
partitions and their replicas are spread out across many brokers in the 
cluster. That's what enables the parallel processing and fault tolerance 
features of Kafka.

-hans

> On Aug 22, 2017, at 3:14 AM, Mohit Chawla  
> wrote:
> 
> Hi folks,
> 
> Is it possible to pin kafka clients to use only specific brokers throughout
> their lifetime and not just for the initial bootstrapping ?
> 
> Thanks,
> Mohit

Pinning clients to specific brokers

2017-08-22 Thread Mohit Chawla 
Hi folks,

Is it possible to pin kafka clients to use only specific brokers throughout
their lifetime and not just for the initial bootstrapping ?

Thanks,
Mohit