Rajiv, 0.9.0 Java producers have a few minor bug fixes from 0.8.2, and add another API function "flush()": https://issues.apache.org/jira/browse/KAFKA-1865
You can look through its changes by just searching "producer" in the release notes: http://mirror.stjschools.org/public/apache/kafka/0.9.0.0/RELEASE_NOTES.html Guozhang On Tue, Jan 12, 2016 at 6:00 PM, Rajiv Kurian <ra...@signalfx.com> wrote: > Thanks Guozhan. I have upgraded to 0.9.0 now. Are there are any other > producer changes to be aware of? My understanding is there were no big > producer changes made from 0.8.2 to 0.9.0. > > Thanks, > Rajiv > > On Mon, Jan 11, 2016 at 5:52 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Hi Rajiv, > > > > This warning could be ignored and is indeed done in 0.9.0, where we > > downgrade the logging level for it from WARN to DEBUG. So if you upgrade > to > > 0.9.0 Java producer you should not see this warning. > > > > A bit more context on the EOFException, a socket closure could result > this; > > and a server could actively close a socket under some cases, for example > 1) > > if it is idle for some time and server would decide to close it based on > > the idle management config, or 2) if producer use ack=0 and there is an > > error processing the request, so server just close the socket to "notify" > > the client, etc. > > > > Guozhang > > > > > > > > > > On Mon, Jan 11, 2016 at 1:08 PM, Rajiv Kurian <ra...@signalfx.com> > wrote: > > > > > We have recently upgraded some of our applications to use the Kafka > 0.8.2 > > > Java producers from the old Java wrappers over Scala producers. > > > > > > We've noticed these log messages on our application since the upgrade: > > > > > > 2016-01-11T20:56:43.023Z WARN [roducer-network-thread | producer-2] > > > [s.o.a.kafka.common.network.Selector ] {}: Error in I/O with > > > my_kafka_host/some_ip > > > > > > java.io.EOFException: null > > > > > > at > > > > > > > > > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) > > > ~[kafka_2.10-0.8.2.2.jar:na] > > > > > > at > > org.apache.kafka.common.network.Selector.poll(Selector.java:248) > > > ~[kafka_2.10-0.8.2.2.jar:na] > > > > > > at > > > org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) > > > [kafka_2.10-0.8.2.2jar:na] > > > > > > at > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) > > > [kafka_2.10-0.8.2.2.jar:na] > > > > > > at > > > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) > > > [kafka_2.10-0.8.2.2.jar:na] > > > > > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] > > > > > > They don't occur too often and may be harmless but it is pretty > alarming > > to > > > see these. It happens with all the brokers we connect to so it doesn't > > seem > > > like a problem with a single broker. Our producer config looks a bit > like > > > this: > > > > > > final Properties config = new Properties(); > > > > > > config.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, > > > OUR_KAFKA_CONNECT_STRING); > > > > > > config.put(ProducerConfig.BLOCK_ON_BUFFER_FULL_CONFIG, false); // non > > > blocking > > > > > > config.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 10 * 1024 * 1024); > // > > 10 > > > MB > > > > > > config.put(ProducerConfig.BATCH_SIZE_CONFIG, 16384); // 16 KB > > > > > > config.put(ProducerConfig.LINGER_MS_CONFIG, 50); // 50 ms > > > > > > > > > Thanks, > > > > > > Rajiv > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
