RE: Broker Interceptors

2019-12-04 Thread Thomas Aley
ind as well. > > Best regards, > Sönke > > Thomas Aley schrieb am Di., 3. Dez. 2019, 10:45: > > > Hi M. Manna, > > > > Thank you for your feedback, any and all thoughts on this are appreciated > > from the community. > > > > I think it i

RE: Broker Interceptors

2019-12-03 Thread Thomas Aley
intercept for their use case. Tom Aley thomas.a...@ibm.com From: "M. Manna" To: Kafka Users Cc: d...@kafka.apache.org Date: 02/12/2019 11:31 Subject:[EXTERNAL] Re: Broker Interceptors Hi Tom, On Mon, 2 Dec 2019 at 09:41, Thomas Aley wrote: > Hi Kafka commun

Broker Interceptors

2019-12-02 Thread Thomas Aley
Hi Kafka community, I am hoping to get some feedback and thoughts about broker interceptors. KIP-42 Added Producer and Consumer interceptors which have provided Kafka users the ability to collect client side metrics and trace the path of individual messages end-to-end. This KIP also mentioned

Re: Kafka logs are getting deleted too soon

2019-07-17 Thread Thomas Aley
Hi Sachin, Try adding --from-beginning to your console consumer to view the historically produced data. By default the console consumer starts from the last offset. Tom Aley thomas.a...@ibm.com From: Sachin Nikumbh To: Kafka Users Date: 17/07/2019 16:01 Subject:[EXTERNAL] K

Re: Kafka behind NAT

2018-05-22 Thread Thomas Aley
Try setting: advertised.listeners=EXTERNAL://:9093,INTERNAL://:9092 inter.broker.listener.name=INTERNAL listener.security.protocol.map=EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT Then you should be able to use :9093 as your bootstrap.servers from outside the network or :9092 from inside. Obviously th

Re: Hardware Guidance

2018-03-01 Thread Thomas Aley
Hi Adrien, Without asking the author directly I can't give the exact answer but I would interpret that as per broker. Kafka will make use of as much hardware as you give it so it's not uncommon to see many CPU cores and lots or RAM per broker. That being said it's completely down to your use c

Re: When a broker down, Producer LOST messages!

2018-03-01 Thread Thomas Aley
Hi, The log is not attached. I'm assuming your topic has a replication factor greater than 1 so that it is available from another Broker if the partition leader fails. Try adding props.put("acks", "all"); to your producer and run your experiment again. If you configured your topic to have --

Re: broker properties explanations

2018-02-21 Thread Thomas Aley
Hi Adrien, log.dirs exists to facilitate multiple data directories which allows more than one disk to be used without the need for RAID. This increases throughput but beware of naive load balancing that may fill up one disk way before another. When log.flush.interval.ms is null the log.flush.i