We are using Kafka 0.8.1.1 (2.9.2) rebuilt with Snappy 1.1.0.1, and don't
have problems with producers or consumers.
Chi
On Thu, Jun 5, 2014 at 9:29 PM, Vinay Gupta
wrote:
> Hi,
> I am using
> kafka_2.9.2-0.8.1and
> snappy-java-1.1.0.1.jar
>
>
> I have been able to successfully use gzip
Hi,
I am running Kafka 0.8.1.1 with one Zookeeper and one broker. I created a
partition 'test2' as below:
/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 3 --topic test2
I noticed this exception in the state-change.log. Why is this occuring ?
Should I be
Hi,
I’m glad there’s so much thought into getting security right!
But as a user of Kafka who doesn’t need Enterprise Security sort of features,
I would ask whether doing such a large project built into Kafka is the
appropriate
use of developer time at this point in its lifecycle.
For example, eve
I've tried to run mirrormaker tools in async mode and I get
WARN Produce request with correlation id 263 failed due to
[benchmark1,30]: kafka.common.MessageSizeTooLargeException
(kafka.producer.async.DefaultEventHandler)
I don't get error in sync mode. My message.max.bytes is default
(100). As
Yes, I realized last night that I needed to be clearer in what I was
saying. Encryption should ABSOLUTELY not be handled server-side. I think
it¹s a good idea to enable use of it in the consumer/producer, but doing
it server side will not solve many use cases for needing encryption
because the serv
It would be nice to have Alcatraz on-disk security for the discriminating
client.
Thanks,
Rob
> On Jun 6, 2014, at 11:51 AM, Pradeep Gollakota wrote:
>
> I'm actually not convinced that encryption needs to be handled server side
> in Kafka. I think the best solution for encryption is to handle
I'm actually not convinced that encryption needs to be handled server side
in Kafka. I think the best solution for encryption is to handle it
producer/consumer side just like compression. This will offload key
management to the users and we'll still be able to leverage the sendfile
optimization for
On consideration, if we have 3 different access groups (1 for
production WRITE and 2 consumers) they all need to decode the same
encryption and so all need the same public/private keycerts won't
work, unless you write a CertAuthority to build multiple certs with
the same keys. Better s
This is quite interesting to me and it is an excelent opportunity to
promote a slightly different security scheme. Object-capabilities are
perfect for online security and would use ACL style authentication to
gain capabilities filtered to those allowed resources for allow
actions (READ/WRI
It is not easy to get the fetcher thread id from kafka stream. We do have
the mapping as a topicThreadIdAndQueues, but it is private to the consumer
and not exposed to callers.
Guozhang
On Fri, Jun 6, 2014 at 9:00 AM, Bogdan Dimitriu (bdimitri) <
bdimi...@cisco.com> wrote:
> I see. And can I so
I see. And can I somehow reliably get the ID of the fetcher thread that is
providing data for a KafkaStream? Specifically I¹d like to know that ID
from the thread where I¹m consuming (where I iterate through the stream).
Thank you,
Bogdan
On 06/06/2014 16:16, "Guozhang Wang" wrote:
>Bogdan,
>
>
Libo,
I have double checked the code. With sync producers all failures should be
either thrown as exceptions or logged in warning/error log entries.
Guozhang
On Thu, Jun 5, 2014 at 6:38 PM, Libo Yu wrote:
> Not really. The issue was reported by a client. I added a lot of logging
> to make sur
Bogdan,
The kafka stream does not have an ID itself, the one you mentioned is the
ID of the fetcher thread that put data into the stream. Although there is a
one-to-one mapping between the fetcher thread and the stream, the Ids of
the fetcher cannot be accessed from the kafka stream itself.
Guozh
Hello folks,
I’m using Kafka 0.8.0 with the high level consumer and I have a situation where
I need to obtain the ID for each of the KafkaStreams that I create.
The KafkaStream class has a method called “clientId()” that I expected would
give me just that, but unfortunately it returns the name o
14 matches
Mail list logo
