You can replay the messages with the high level consumer you can even
start at whatever position you want.
Prior to your consumers starting call
ZkUtils.maybeDeletePath(zkClientConnection, "/consumers/" + groupId)
make sure you have in your consumer properties
auto.offset.reset="smallest"
AFAIK, we can not replay the messages with high level consumer. We need to
use simple consumer.
On Sun, Jan 18, 2015 at 12:15 AM, Christopher Piggott
wrote:
> Thanks. That helped clear a lot up in my mind.
>
> I'm trying to high-level consumer now. Occasionally I need to do a replay
> of the s
Thanks. That helped clear a lot up in my mind.
I'm trying to high-level consumer now. Occasionally I need to do a replay
of the stream. The example is:
KafkaStream.iterator();
which starts at wherever zookeeper recorded as where you left off.
With the high level interface, can you request
Hi,
1. In SimpleConsumer, you must keep track of the offsets in your
application.
In the example code, "readOffset" variable can be saved in
redis/zookeeper.
You should plugin this logic in your code. High Level Consumer stores
the last
read offset information in ZooKeeper.
2. You wil
Thanks Jun,
I completely missed your earlier reply in this thread. Changing to use the
javaapi makes the process a lot cleaner.
Thanks,
Chris
On Mon, Dec 3, 2012 at 11:26 AM, Jun Rao wrote:
> kafka.javaapi.consumer.SimpleConsumer has an api to get the topic metadata.
>
> Thanks,
>
> Jun
>
>
There is no reason the ClientUtils.fetchTopicMetadata should take broker
instances, that is totally unusable. I have a patch for this included in
KAFKA-642.
-Jay
On Mon, Dec 3, 2012 at 7:53 AM, Chris Curtin wrote:
> Hi,
>
> I was able to implement my own lookup code but have a few concerns abo
kafka.javaapi.consumer.SimpleConsumer has an api to get the topic metadata.
Thanks,
Jun
On Mon, Dec 3, 2012 at 7:53 AM, Chris Curtin wrote:
> Hi,
>
> I was able to implement my own lookup code but have a few concerns about
> this long term:
> - the Broker class is marked as 'private' in the Sc
Hi,
I was able to implement my own lookup code but have a few concerns about
this long term:
- the Broker class is marked as 'private' in the Scala code. IntelliJ gives
me an error about using it, but the runtime lets me use it and get the
host/port out.
- I have to know a lot about the structure
The clientid is used to identify a particular client application. This
is used by the server side request logging to identify the client
sending a particular request. The clientid is also used to give
meaningful names to the mbeans for producer/consumer clients.
Also, there are 2 ways to send the
The offset now begins at 0 and increases sequentially for each partition.
The offset is identical across all replicas of that partition on different
brokers, but amongst different partitions the offsets are independent (as
before). The offset of a committed message is unique within that
topic/parti
To use SimpleConsumer, you need to send TopicMetadataRequest (available in
SimpleConsumer) to figure out the leader of each partition before making
the fetch requests.
In both 0.7 and 0.8, a fetch request fetches data starting at the provided
offset. In 0.8, offset is a sequential and evergrowing
11 matches
Mail list logo
