Keys and partitions

2020-07-06 Thread Victoria Zuberman
Hi, I have userId as a key. Many users have moderate amounts of data but some users have more and some users have huge amount of data. I have been thinking about the following aspects of partitioning: 1. If two or more large users will fall into same partition I might end up with large

Re: Latest Logstash 7.8 and compatibility with latest Kafka 2.5.0

2020-07-06 Thread allen chan
Best is to read the changelog of the plugin https://github.com/logstash-plugins/logstash-integration-kafka/blob/master/CHANGELOG.md they are up to 2.4.1 per 10.1.0 notes and you have to see what version is packaged with the release. If it is not the right version, you need to use automation or

Re: Latest Logstash 7.8 and compatibility with latest Kafka 2.5.0

2020-07-06 Thread Sebastjan Vodušek
Hi, thank you for replay. I have read this on elastic site but I had some problems in production with latest Logstash 7.8 and Kafka 2.2 in production so I had to revert Logstash to one of previous versions. That is why I wanna be extra careful before upgrade. Br, Sebastjan On Jul 6 2020, at

Problem with replication?!

2020-07-06 Thread s.schabbach
Hi there, I just have a problem with my kafka brokers, maybe a firewall issue, but I don’t know. I have got 3 Brokers at three different Servers (each with another IP) and on the first server running zookeeper: Server1:9092 (zookeeper:2182) Server2:9092 Server3:9092 And I have got

Re: Latest Logstash 7.8 and compatibility with latest Kafka 2.5.0

2020-07-06 Thread Pere Urbón Bayes
HI Sebastian, I would check for that https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html, where it says: This plugin uses Kafka Client 2.1.0. For broker compatibility, see the official Kafka compatibility reference

Latest Logstash 7.8 and compatibility with latest Kafka 2.5.0

2020-07-06 Thread Sebastjan Vodušek
Hello, does anyone know if latest Logstash 7.8 input plugin is compatible with latest Kafka 2.5.0? Thank you and best regards, Sebastjan