Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps jay.kr...@gmail.commailto:jay.kr...@gmail.com wrote: I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? On Oct 30, 2014, at 9:51 AM, Jun Rao jun...@gmail.commailto:jun...@gmail.com wrote: Yes, we can change this to two methods in 0.8.2 final. Thanks. After some experimentation, I think there isn’t actually a form of this code that can compile under both kafka 0.8.1.1 and 0.8.2-beta. This form fails to compile against kafka-0.8.1.1: connector.commitOffsets() [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: Unit does not take parameters [error] connector.commitOffsets() [error]^ This form fails to compile against kafka-0.8.2-beta: connector.commitOffsets [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ -- Jack Foy j...@whitepages.commailto:j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Yes, this is actually indeed an incompatible change. In 0.8.1.x, ConsumerConnector has the following api: def commitOffsets This is changed to the following in 0.8.2 and breaks compatibility def commitOffsets(retryOnFailure: Boolean = true) Filed KAFKA-1743 as an 0.8.2 blocker. Thanks, Jun On Fri, Oct 31, 2014 at 3:07 PM, Jack Foy j...@whitepages.com wrote: On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps jay.kr...@gmail.commailto: jay.kr...@gmail.com wrote: I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? On Oct 30, 2014, at 9:51 AM, Jun Rao jun...@gmail.commailto: jun...@gmail.com wrote: Yes, we can change this to two methods in 0.8.2 final. Thanks. After some experimentation, I think there isn’t actually a form of this code that can compile under both kafka 0.8.1.1 and 0.8.2-beta. This form fails to compile against kafka-0.8.1.1: connector.commitOffsets() [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: Unit does not take parameters [error] connector.commitOffsets() [error]^ This form fails to compile against kafka-0.8.2-beta: connector.commitOffsets [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ -- Jack Foy j...@whitepages.commailto:j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the scala rule (http://docs.scala-lang.org/style/method-invocation.html) for omitting parentheses when calling arity-0 methods is that the methods in question have no side effects. In this case, commitOffsets() clearly has side effect and should have been called with parentheses. Thanks, Jun On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
But Jun, That change breaks peoples code who weren't calling in that way. I don't think we should be making breaking changes in a point release like this. I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? -Jay On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao jun...@gmail.com wrote: Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the scala rule (http://docs.scala-lang.org/style/method-invocation.html) for omitting parentheses when calling arity-0 methods is that the methods in question have no side effects. In this case, commitOffsets() clearly has side effect and should have been called with parentheses. Thanks, Jun On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
+1 on the two methods suggestion -JW On Oct 30, 2014, at 9:20 AM, Jay Kreps jay.kr...@gmail.com wrote: But Jun, That change breaks peoples code who weren't calling in that way. I don't think we should be making breaking changes in a point release like this. I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? -Jay On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao jun...@gmail.com wrote: Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the scala rule (http://docs.scala-lang.org/style/method-invocation.html) for omitting parentheses when calling arity-0 methods is that the methods in question have no side effects. In this case, commitOffsets() clearly has side effect and should have been called with parentheses. Thanks, Jun On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
+1 on two methods -- Colin Clark +1-320-221-9531 On Oct 30, 2014, at 11:24 AM, Jonathan Weeks jonathanbwe...@gmail.com wrote: +1 on the two methods suggestion -JW On Oct 30, 2014, at 9:20 AM, Jay Kreps jay.kr...@gmail.com wrote: But Jun, That change breaks peoples code who weren't calling in that way. I don't think we should be making breaking changes in a point release like this. I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? -Jay On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao jun...@gmail.com wrote: Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the scala rule (http://docs.scala-lang.org/style/method-invocation.html) for omitting parentheses when calling arity-0 methods is that the methods in question have no side effects. In this case, commitOffsets() clearly has side effect and should have been called with parentheses. Thanks, Jun On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Yes, we can change this to two methods in 0.8.2 final. Thanks, Jun On Thu, Oct 30, 2014 at 9:20 AM, Jay Kreps jay.kr...@gmail.com wrote: But Jun, That change breaks peoples code who weren't calling in that way. I don't think we should be making breaking changes in a point release like this. I think we should treat this like a bug for 0.8.2 final, we should be able to add two commitOffsets methods with and without the param which should fix the problem, right? -Jay On Thu, Oct 30, 2014 at 8:59 AM, Jun Rao jun...@gmail.com wrote: Jack, The commit offset api is changed slightly from def commitOffsets() in 0.8.1.x to def commitOffsets(retryOnFailure: Boolean = true) in 0.8.2.x. If you have been calling the method with parentheses like commitOffsets(), then the code will compile in both 0.8.1.x and 0.8.2.x. In general, the scala rule (http://docs.scala-lang.org/style/method-invocation.html) for omitting parentheses when calling arity-0 methods is that the methods in question have no side effects. In this case, commitOffsets() clearly has side effect and should have been called with parentheses. Thanks, Jun On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Which version of Scala did you use for Kafka 0.8.2-beta? On Wed, Oct 29, 2014 at 12:40 PM, Jack Foy j...@whitepages.com wrote: My Scala project built against kafka 0.8.1.1 commits consumer offsets as follows: connector.commitOffsets This compiles without warnings. When I bumped the library dependency to 0.8.2-beta, the compiler started emitting this error: [error] src/main/scala/com/whitepages/kafka/consumer/Connector.scala:21: missing arguments for method commitOffsets in trait ConsumerConnector; [error] follow this method with `_' if you want to treat it as a partially applied function [error] connector.commitOffsets [error] ^ [error] one error found [error] (compile:compile) Compilation failed The following change resolved the error: -connector.commitOffsets +connector.commitOffsets() Should we expect compilation-breaking changes moving from 0.8.1.1 to 0.8.2-beta? -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
On Oct 29, 2014, at 1:46 PM, Neha Narkhede neha.narkh...@gmail.com wrote: Which version of Scala did you use for Kafka 0.8.2-beta? The problem reproduces under both Scala 2.10.4 and Scala 2.11.2. -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
On Oct 29, 2014, at 3:03 PM, Jack Foy j...@whitepages.com wrote: On Oct 29, 2014, at 1:46 PM, Neha Narkhede neha.narkh...@gmail.com wrote: Which version of Scala did you use for Kafka 0.8.2-beta? The problem reproduces under both Scala 2.10.4 and Scala 2.11.2. Sorry, to be clearer: I am using SBT with the following dependency line, to pull in the specific package for the version of Scala I’m running. libraryDependencies ++= Seq( org.apache.kafka %% kafka % 0.8.2-beta exclude(javax.jms, jms) exclude(com.sun.jdmk, jmxtools) exclude(com.sun.jmx, jmxri) exclude(log4j, log4j) ) I tried the following combinations: Scala 2.10.4, kafka 0.8.1.1, my old code - compiled Scala 2.10.4, kafka 0.8.2, my old code - failed Scala 2.11.2, kafka 0.8.2, my old code - failed Scala 2.10.4, kafka 0.8.2, my new code - compiled Scala 2.11.2, kafka 0.8.2, my new code - compiled -- Jack Foy j...@whitepages.com
Re: Compile failure going from kafka 0.8.1.1 to 0.8.2
Scala 2.10.4, kafka 0.8.1.1, my old code - compiled Scala 2.10.4, kafka 0.8.2, my old code - failed If that's the case, then we probably need to fix the issue. Can you please file a JIRA? On Wed, Oct 29, 2014 at 3:15 PM, Jack Foy j...@whitepages.com wrote: On Oct 29, 2014, at 3:03 PM, Jack Foy j...@whitepages.com wrote: On Oct 29, 2014, at 1:46 PM, Neha Narkhede neha.narkh...@gmail.com wrote: Which version of Scala did you use for Kafka 0.8.2-beta? The problem reproduces under both Scala 2.10.4 and Scala 2.11.2. Sorry, to be clearer: I am using SBT with the following dependency line, to pull in the specific package for the version of Scala I’m running. libraryDependencies ++= Seq( org.apache.kafka %% kafka % 0.8.2-beta exclude(javax.jms, jms) exclude(com.sun.jdmk, jmxtools) exclude(com.sun.jmx, jmxri) exclude(log4j, log4j) ) I tried the following combinations: Scala 2.10.4, kafka 0.8.1.1, my old code - compiled Scala 2.10.4, kafka 0.8.2, my old code - failed Scala 2.11.2, kafka 0.8.2, my old code - failed Scala 2.10.4, kafka 0.8.2, my new code - compiled Scala 2.11.2, kafka 0.8.2, my new code - compiled -- Jack Foy j...@whitepages.com
