Hi Anindya, Currently, setting transaction.id in producer props won't enable transactions. Users need to set --transaction-duration-ms to enable transactions in kafka-producer-perf-test.
We recognize this can be confusing, so there's a JIRA ticket for it: https://issues.apache.org/jira/browse/KAFKA-16900 I've submitted a pull request to address this issue. With the update, users can enable transactions by either: - Setting transaction.id=<id> via --producer-props - Setting transaction.id=<id> in the config file via --producer.config - Setting --transaction-id <id> - Setting --transaction-duration-ms=<ms> I believe this will resolve your issue. If you have any thoughts or suggestions, please feel free to share them on the PR or JIRA. On 2019/10/18 19:18:15 Anindya Haldar wrote: > Anyone who might have experienced this, or have a known solution? Would > appreciate some insights here. > > Sincerely, > Anindya Haldar > Oracle Responsys > > > > On Oct 17, 2019, at 5:38 PM, Anindya Haldar <anindya.hal...@oracle.com> > > wrote: > > > > I am trying to test a 3 node Kafka cluster using the producer and consumer > > test perf scripts that ship with the binaries. When I am trying to enable > > transaction in the producer script it runs into errors. For example: > > > > # bin/kafka-producer-perf-test.sh --topic testtopic1 --num-records 10000 > > --throughput -1 --record-size 256 --producer-props > > bootstrap.servers=172.18.0.4:9092,172.18.0.5:9092,172.18.0.6:9092 acks=all > > transactional.id=1111 enable.idempotence=true > > Exception in thread "main" java.lang.IllegalStateException: Cannot perform > > a 'send' before completing a call to initTransactions when transactions are > > enabled. > > at > > org.apache.kafka.clients.producer.internals.TransactionManager.failIfNotReadyForSend(TransactionManager.java:362) > > at > > org.apache.kafka.clients.producer.internals.TransactionManager.maybeAddPartitionToTransaction(TransactionManager.java:341) > > at > > org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:915) > > at > > org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856) > > at > > org.apache.kafka.tools.ProducerPerformance.main(ProducerPerformance.java:143) > > > > A non-transactional test succeeds though: > > > > bin/kafka-producer-perf-test.sh --topic testtopic1 --num-records 10000 > > --throughput -1 --record-size 256 --producer-props > > bootstrap.servers=172.18.0.4:9092,172.18.0.5:9092,172.18.0.6:9092 acks=all > > 10000 records sent, 6997.900630 records/sec (1.71 MB/sec), 112.32 ms avg > > latency, 499.00 ms max latency, 101 ms 50th, 206 ms 95th, 213 ms 99th, 214 > > ms 99.9th. > > > > The version I am using is: kafka_2.12-2.3.0 > > > > Does the producer perf test script support transactions? If it does, how do > > we run the producer with transaction support? > > > > Sincerely, > > Anindya Haldar > > Oracle Responsys > > > >
