Yes you are correct. I mis-understood the feature.
auto.create.topics.enable has an interesting side effect. Say your kafka
configuration file allows for 10 partitions of a topic. If the topic
auto-creates you get 10 partitions. Doing it the way I did it gives you
control of the number of partition
auto.create.topics.enable
is true by default. For this test I relied on that property.I don't
think a real production class should rely on that though.. Too easy to mess
things up with a typo- cb
On Wed, Nov 6, 2013 at 9:28 PM, Edward Capriolo wrote:
> One thing I noticed about your c
One thing I noticed about your code. I thought in kafka 0.8.0 topics are
not created automatically on first message. I do not see anywhere in your
code which creates the topics.
I am creating the topic as part of my tests.
Before I was not setting that property, and getting some of the messages.
Do you need to use that configuration to get the tests (as currently
checked in) to pass ?I did not find i needed that particular knob
(although it is a good one to know about).
and Sorry about your suffering... I can sympathize !
- cb
On Wed, Nov 6, 2013 at 7:38 PM, Edward Capriolo wrote
After about 5 days of relentless head pounding, and twittling about
everything under the sun, I figured it out.
If you read to the bottom of this page:
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
You find:
consumerProps.put("auto.offset.reset", "smallest");
Now I ca
Hi, Edward..
yup .. you are correct.. when we get to a little over 1000 messages the
program was failing with the exception stack trace i included below.
I fixed the test so it passes as long as the consumer gets all messages
sent by the producer.. even if an exception is thrown during shut dow
Also I have my own version of a test driver
https://github.com/edwardcapriolo/IronCount/blob/iron-ng/src/test/java/com/jointhegrid/ironcount/IntegrationTest.java
For Kafka 0.7.0/2 I had a number of integration tests where I could put
messages on a topic and pull them off later. I remember having
I have success when the number of messages is less then ~1200. With more
then 1200 it never completes.
Try changing the program to this:
tkp = new TestKafkaProducer(
theTopic,
"localhost:" + zookeeperTestServer.getPort(),
4000);
E
Hi Ed:
regarding the test the testtopology exampe program on my github ...
do you see the words " successful completion" printed out towards the
end of the test Run?
I assuming that you ran mvn:exec java ...etc.. as specified in the Read Me
file.. is that correct?
On Nov 4, 2013 7:52 PM, "Edw
I am using:
org.apache.kafka
kafka_2.9.2
0.8.0-beta1
I am trying to rebuild iron-count
https://github.com/edwardcapriolo/IronCount/tree/iron-ng
against kafka 0.8.0
I am having considerable issues getting tests to run correctly.
If you run
10 matches
Mail list logo
