Hi,
Many thanks for the explanation.
I just wanted to do some tests and look around, I think the perf tool will be
OK for me.
Indeed, it’s a bit tricky with the console producer. To do a commit
after every new line (enter) sounds not right. Something like
begin; message, message… commit; would be better but how would that look like
in a console producer ☺
or maybe simply a commit every n seconds.
BTW. Is there maybe any plan to implement the exactly once semantics in to the
mirrormaker?
BR,
Marcin
On 03.08.17, 21:01, "Apurva Mehta" <apu...@confluent.io> wrote:
Ismael raises good questions about what transactions would mean for the
console producer.
However, the kafka-producer-perf-test script has transactions enabled. It
enables you to generate transactions of a certain duration (like 50ms,
100ms). It produces messages of specified size and commits them
transactionally in a periodic manner, enabling you to at least have a look
at the transaction log, etc.
Thanks,
Apurva
On Thu, Aug 3, 2017 at 6:22 AM, Ismael Juma <ism...@juma.me.uk> wrote:
> Hi Marcin,
>
> The console producer hasn't been updated to invoke the appropriate methods
> if transactions are enabled. It also requires a bit of thinking on how it
> should work. Would there be a way to start and commit the transaction via
> the console or would the console producer do it periodically? What was
your
> intent?
>
> Ismael
>
> On Thu, Aug 3, 2017 at 9:50 AM, Bienek, Marcin <mbie...@team.mobile.de>
> wrote:
>
> > Hi,
> >
> > I’m trying to test the new exactly once transaction feature. Doing
> simple
> > test like:
> >
> > /opt/kafka/bin/kafka-console-producer.sh --request-required-acks "all"
> > --producer-property "transactional.id=777" --producer-property="enable.
> idempotence=true"
> > --broker-list broker1:9092 --topic bla
> >
> > Fails with:
> >
> > java.lang.IllegalStateException: Cannot perform a 'send' before
> > completing a call to initTransactions when transactions are enabled.
> > at org.apache.kafka.clients.producer.internals.
> TransactionManager.
> > failIfNotReadyForSend(TransactionManager.java:253)
> > at org.apache.kafka.clients.producer.internals.
> TransactionManager.
> > maybeAddPartitionToTransaction(TransactionManager.java:233)
> > at org.apache.kafka.clients.producer.KafkaProducer.doSend(
> > KafkaProducer.java:745)
> > at org.apache.kafka.clients.producer.KafkaProducer.send(
> > KafkaProducer.java:701)
> > at kafka.producer.NewShinyProducer.send(BaseProducer.scala:47)
> > at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:61)
> > at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
> >
> > I suspect that somehow the producer is not able to trigger the creation
> of
> > the internal transaction topic ?
> >
> >
> > BR,
> > Marcin
> >
>
