I'm firing up a KafkaServer (using some EmbeddedKafkaBroker code that I
found on Github) so that I can run an end-to-end test ingesting data
through a kafka topic with consumers in Spark Streaming pushing to
Accumulo.
Thus far, my code is doing this:
1) Creating a MiniAccumuloCluster and
as the leader but it's strange that the log messages
above seem like they are missing the data. New topic creation callback for
seems like it should be listing a topic and not blank.
Any ideas?
On Thu, May 14, 2015 at 1:00 PM, Corey Nolet cjno...@gmail.com wrote:
I'm firing up a KafkaServer (using some
of it that are making it
unparseable once pulled from zookeeper.
Any ideas to what this could be? I'm using 0.8.2.0- this is really what's
holding me back right now from getting my tests functional.
On Thu, May 14, 2015 at 4:29 PM, Corey Nolet cjno...@gmail.com wrote:
I raised the log levels to try to figure
Json encoded blob definitely appears to be going in as a json string. The
partition assignment json seems to be the only thing that is being prefixed
by these bytes. Any ideas?
On Thu, May 14, 2015 at 5:17 PM, Corey Nolet cjno...@gmail.com wrote:
I think I figured out what the problem
I have a cluster of 3 nodes and I've created a topic with some number of
partitions and some number of replicas, let's say 10 and 2, respectively.
Later, after I've got my 3 nodes fairly consumed with data in the 10
partitions, I want to add 2 more nodes to the mix to help balance out the
. The batching in new producer is
per topic partition, the batch size it is controlled by both max batch
size and linger time config.
Jiangjie (Becket) Qin
On 3/9/15, 10:10 AM, Corey Nolet cjno...@gmail.com wrote:
I'm curious what type of batching Kafka producers do at the socket layer
I'm new to Kafka and I'm trying to understand the version semantics. We
want to use Kafka w/ Spark but our version of Spark is tied to 0.8.0. We
were wondering what guarantees are made about backwards compatbility across
0.8.x.x. At first glance, given the 3 digits used for versions, I figured
I'm curious what type of batching Kafka producers do at the socket layer.
For instance, if I have a partitioner that round robin's n messages to a
different partition, am I guaranteed to get n different messages sent over
the socket or is there some micro-batching going on underneath?
I am trying
