from:"Jonathan Natkins"

Re: Official Kafka Gitter Room?

2015-07-19 Thread Jonathan Natkins

Slack results are only searchable to a point if you're using the free
version. I think they allow for 10K messages to be stored, but unlimited is
paid.

On Sunday, July 19, 2015, Gwen Shapira  wrote:

> Good point.
>
> Slack has searchable archives, but I don't think it shows up in Google
> like the mailing list does.
>
> On Sun, Jul 19, 2015 at 12:37 AM, Harsha >
> wrote:
> > one of the benefits of keeping conversations in mailing lists is it can
> > be searchable later. It will benefit users who might seen the same error
> > which was answered on the mailing lists before. slack/glitter can do
> > that?
> >
> >
> > On Sat, Jul 18, 2015, at 09:59 PM, Gwen Shapira wrote:
> >> FYI:
> >> Joining Slack requires an invitation. We can take invite-requests in
> >> the mailing lists (like we do for KIP hangout).
> >> I wanted to point this out as this may be considered non-Apache
> >> (although I didn't find any specific wording against invite-only
> >> channels in addition to the mailing list).
> >>
> >> So, if you want to try it out, yell for an invite :)
> >>
> >> On Sat, Jul 18, 2015 at 6:26 PM, Gwen Shapira  >
> >> wrote:
> >> > So, as an experiment, I created:
> >> > https://apachekafka.slack.com
> >> >
> >> > I figured we'll give it a whirl for a week or two for dev discussions,
> >> > see how it goes and if we have activity we can add this to the website
> >> > and announce on the lists.
> >> >
> >> > Gwen
> >> >
> >> > On Sat, Jul 18, 2015 at 6:18 PM, Gwen Shapira  > wrote:
> >> >> Slack sounds good and I'm not a fan of gitter (we use it for Apache
> >> >> Sqoop, I dislike the GUI and the guilt-inducing emails about all the
> >> >> messages I missed...)
> >> >> Do they have free rooms for open source projects?
> >> >>
> >> >> On Sat, Jul 18, 2015 at 6:10 PM, Andrew Psaltis
> >> >> > wrote:
> >> >>> +1 for slack
> >> >>>
> >> >>> On Saturday, July 18, 2015, Neha Narkhede  > wrote:
> >> >>>
> >>  FWIW, if the apache-kafka channel was on Slack, I'd hang out there
> :-)
> >> 
> >>  On Sat, Jul 18, 2015 at 7:20 AM, Ismael Juma  
> >>  > wrote:
> >> 
> >>  > Hi Grant,
> >>  >
> >>  > I have been considering proposing the same thing myself for
> similar
> >>  > reasons, but I also have some doubts. More comments inline.
> >>  >
> >>  > On Fri, Jul 17, 2015 at 7:28 PM, Grant Henke <
> ghe...@cloudera.com 
> >>  > wrote:
> >>  >
> >>  > > I think it could be useful to have a place to chat that is
> associated
> >>  > with
> >>  > > the Kafka repo.
> >>  >
> >>  >
> >>  > Agreed.
> >>  >
> >>  >
> >>  > > Note that we do currently have an IRC channel, but from my
> >>  > > experience its a ghost town.
> >>  > >
> >>  >
> >>  > I've recently joined #apache-kafka and activity is low, yes, but
> there is
> >>  > some[1]. The bigger issue, perhaps, is that contributors are not
> there.
> >>  Do
> >>  > we think that this would be different with Gitter? And, if so,
> what is
> >>  the
> >>  > reason? Is it because no application needs to be installed? In
> theory,
> >>  > IRCCloud provides something similar for IRC (although the
> limitations of
> >>  > the trial and the upgrade nagging can be annoying) and we could
> link to
> >>  the
> >>  > channel in GitHub's readme.
> >>  >
> >>  > Also, are there any Apache projects who use Gitter? There was a
> thread in
> >>  > the Spark mailing list, but the response wasn't positive[2]. We
> would
> >>  need
> >>  > Apache Infra to set-up Gitter for us as they are the only ones
> that have
> >>  > write access to the repository (the set-up is really easy
> though, which
> >>  is
> >>  > definitely a positive).
> >>  >
> >>  > Gitter may be the right answer, I just want to make sure it
> actually
> >>  helps
> >>  > us have a more active channel instead of splitting the existing
> activity
> >>  > between two channels.
> >>  >
> >>  > Thoughts?
> >>  >
> >>  > Best,
> >>  > Ismael
> >>  >
> >>  > [1]
> https://botbot.me/freenode/apache-kafka/2015-07-06/?tz=Europe/London
> >>  > [2]
> >>  >
> >>  >
> >> 
> http://apache-spark-developers-list.1001551.n3.nabble.com/Gitter-chat-room-for-Spark-td11636.html
> >>  >
> >> 
> >> 
> >> 
> >>  --
> >>  Thanks,
> >>  Neha
> >> 
>


-- 
Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin

Re: Kafka Out of Memory error

2015-01-19 Thread Jonathan Natkins

Hi Pranay,

I think the JIRA you're referencing is a bit orthogonal to the OOME that
you're experiencing. Based on the stacktrace, it looks like your OOME is
coming from a consumer request, which is attempting to allocate 200MB.
There was a thread (relatively recently) that discussed what I think is
your issue:

http://mail-archives.apache.org/mod_mbox/kafka-users/201412.mbox/%3CCAG1fNJDHHGSL-x3wp=pPZS1asOdOBrQ-Ge3kiA3Bk_iz7o=5...@mail.gmail.com%3E

I suspect that the takeaway is that the way Kafka determines the required
memory for a consumer request is (#partitions in the topic) x
(replica.fetch.max.bytes), and seemingly you don't have enough memory
allocated to handle that request. The solution is likely to increase the
heap size on your brokers or to decrease your max fetch size.

Thanks,
Natty

Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin 


On Mon, Jan 19, 2015 at 2:10 PM, Pranay Agarwal 
wrote:

> Hi All,
>
> I have a kafka cluster setup which has 2 topics
>
> topic1 with 10 partitions
> topic2 with 1000 partitions.
>
> While, I am able to consume messages from topic1 just fine, I get following
> error from the topic2. There is a resolved issue here on the same thing
> https://issues.apache.org/jira/browse/KAFKA-664
>
> I am using latest kafka server version, and I have used kafka command line
> tools to consumer from the topics.
>
> 
> [2015-01-19 22:08:10,758] ERROR OOME with size 201332748
> (kafka.network.BoundedByteBufferReceive)
>
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
> at
>
> kafka.network.BoundedByteBufferReceive.byteBufferAllocate(BoundedByteBufferReceive.scala:80)
> at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:63)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:81)
> at
>
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:109)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:109)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:109)
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:108)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:108)
> at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:108)
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:107)
> at
>
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96)
> at
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:88)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> <<<
>
>
> Thanks
> -Pranay
>

Re: Kafka Out of Memory error

2015-01-19 Thread Jonathan Natkins

The fetch.message.max.size is actually a client-side configuration. With
regard to increasing the number of threads, I think the calculation may be
a little more subtle than what you're proposing, and frankly, it's unlikely
that your servers can handle allocating 200MB x 1000 threads = 200GB of
memory at a single time.

I believe that if you have every partition on a single broker, and all of
your consumer threads are requesting data simultaneously, then yes, the
broker would attempt to allocate 200GB of heap, and probably you'll hit an
OOME. However, since each consumer is only reading from one partition,
those 1000 threads should be making requests that are spread out over the
entire Kafka cluster. Depending on the memory on your servers, you may need
to increase the number of brokers in your cluster to support the 1000
threads. For example, I would expect that you can support this with 10
brokers if each broker has something north of 20GB of heap allocated.

Some of this is a little bit of guess work on my part, and I'm not super
confident of my numbers...Can anybody else on the list validate my math?

Thanks,
Natty

Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin <http://www.linkedin.com/in/nattyice>


On Mon, Jan 19, 2015 at 2:34 PM, Pranay Agarwal 
wrote:

> Thanks Natty.
>
> Is there any config which I need to change on the client side as well?
> Also, currently I am trying with only 1 consumer thread. Does the equation
> changes to
> (#partitions)*(fetchsize)*(#consumer_threads) in case I try to read with
> 1000 threads from from topic2(1000 partitions)?
>
> -Pranay
>
> On Mon, Jan 19, 2015 at 2:26 PM, Jonathan Natkins 
> wrote:
>
> > Hi Pranay,
> >
> > I think the JIRA you're referencing is a bit orthogonal to the OOME that
> > you're experiencing. Based on the stacktrace, it looks like your OOME is
> > coming from a consumer request, which is attempting to allocate 200MB.
> > There was a thread (relatively recently) that discussed what I think is
> > your issue:
> >
> >
> >
> http://mail-archives.apache.org/mod_mbox/kafka-users/201412.mbox/%3CCAG1fNJDHHGSL-x3wp=pPZS1asOdOBrQ-Ge3kiA3Bk_iz7o=5...@mail.gmail.com%3E
> >
> > I suspect that the takeaway is that the way Kafka determines the required
> > memory for a consumer request is (#partitions in the topic) x
> > (replica.fetch.max.bytes), and seemingly you don't have enough memory
> > allocated to handle that request. The solution is likely to increase the
> > heap size on your brokers or to decrease your max fetch size.
> >
> > Thanks,
> > Natty
> >
> > Jonathan "Natty" Natkins
> > StreamSets | Customer Engagement Engineer
> > mobile: 609.577.1600 | linkedin <http://www.linkedin.com/in/nattyice>
> >
> >
> > On Mon, Jan 19, 2015 at 2:10 PM, Pranay Agarwal <
> agarwalpran...@gmail.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I have a kafka cluster setup which has 2 topics
> > >
> > > topic1 with 10 partitions
> > > topic2 with 1000 partitions.
> > >
> > > While, I am able to consume messages from topic1 just fine, I get
> > following
> > > error from the topic2. There is a resolved issue here on the same thing
> > > https://issues.apache.org/jira/browse/KAFKA-664
> > >
> > > I am using latest kafka server version, and I have used kafka command
> > line
> > > tools to consumer from the topics.
> > >
> > > >>>>
> > > [2015-01-19 22:08:10,758] ERROR OOME with size 201332748
> > > (kafka.network.BoundedByteBufferReceive)
> > >
> > > java.lang.OutOfMemoryError: Java heap space
> > > at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> > > at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
> > > at
> > >
> > >
> >
> kafka.network.BoundedByteBufferReceive.byteBufferAllocate(BoundedByteBufferReceive.scala:80)
> > > at
> > >
> > >
> >
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:63)
> > > at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> > > at
> > >
> > >
> >
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> > > at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> > > at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:81)
> > > at
> > >
> > >
> >
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$se

Re: Kafka ecosystem licensing question

2015-01-21 Thread Jonathan Natkins

Hi Eliot,

I can't speak for the Kafka PMC, but as a general rule, if the code is
going to be contributed to the Kafka project itself, it must be
Apache-licensed. What you can do, and what many organizations do is release
code separately via a public Github account, which would allow you to
choose whatever license you prefer (for example:
https://github.com/linkedin/camus or any of the myriad clients
https://cwiki.apache.org/confluence/display/KAFKA/Clients)

However, I think that Apache/BSD/MIT are the safest licenses to use if you
really want people to use your code. AGPL is a particular contentious one,
especially if you want to use the code in a corporate setting, because it
requires that you open-source any code changes you make, and I think it has
some other fairly serious implications in terms of what must be
open-sourced if you include the code in a larger project.

That said, these are primarily my own opinions, and I am a) not a lawyer,
and b) not an Apache committer.

Thanks,
Natty

Jonathan "Natty" Natkins
StreamSets | Customer Engagement Engineer
mobile: 609.577.1600 | linkedin 

On Wed, Jan 21, 2015 at 8:48 AM, Weitz, Eliot 
wrote:

> Hello,
>
> I lead a group of developers at our company, ViaSat, who are building a
> set of stream processing services on top of Kafka.  We would very much like
> to open source our work and become part of the Kafka “ecosystem”
> contributing back to the community.
>
> Our company is fairly new to participating in open source projects and are
> wondering about licensing.  If we used something other than an Apache 2
> license (such as a copyleft license like AGPL), do you think it would it be
> viewed negatively by your developers or others in the Kafka ecosystem and
> become a barrier to contribute to our project?
>
> I’d appreciate any insights.
>
> Good work on Kafka!
>
> Regards,
>
> Eliot Weitz

Review Request 24006: Patch for KAFKA-1420

2014-07-28 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in all unit tests


Diffs
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---


Thanks,

Jonathan Natkins

Review Request 24008: Patch for KAFKA-1420

2014-07-28 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24008/
---

Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in all unit tests


Diffs
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24008/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-07-28 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated July 28, 2014, 8:52 p.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in all unit tests


Diffs
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing (updated)
---

Automated


Thanks,

Jonathan Natkins

Review Request 24007: Patch for KAFKA-1420

2014-07-28 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24007/
---

Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in all unit tests


Diffs
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24007/diff/


Testing
---


Thanks,

Jonathan Natkins

Review Request 24012: Patch for KAFKA-1560

2014-07-28 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24012/
---

Review request for kafka.


Bugs: KAFKA-1560
https://issues.apache.org/jira/browse/KAFKA-1560


Repository: kafka


Description
---

KAFKA-1560 Make arguments to jira-python API more explicit in 
kafka-patch-review's get_jira()


Diffs
-

  kafka-patch-review.py dc45549f886440f1721c60aab9aa0a4af9b4cbef 

Diff: https://reviews.apache.org/r/24012/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated July 30, 2014, 6:18 p.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description (updated)
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in unit tests


Diffs (updated)
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated July 30, 2014, 6:24 p.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in unit tests


Diffs (updated)
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-07-30 Thread Jonathan Natkins



> On July 30, 2014, 12:22 a.m., Guozhang Wang wrote:
> > core/src/test/scala/unit/kafka/admin/AdminTest.scala, line 314
> > <https://reviews.apache.org/r/24006/diff/1/?file=643839#file643839line314>
> >
> > Is there a specific reason we want to use 10 seconds instead of default 
> > 5 seconds?

Sorry, I'd added this in the midst of debugging, and forgotten to remove it. 
I've actually changed this call, because I realized that it didn't necessarily 
assure me that broker 0 had caught up to the ISR yet. The test has been changed 
to be more reliable.


> On July 30, 2014, 12:22 a.m., Guozhang Wang wrote:
> > core/src/test/scala/unit/kafka/admin/AdminTest.scala, line 317
> > <https://reviews.apache.org/r/24006/diff/1/?file=643839#file643839line317>
> >
> > Is this println intended?

Removed


> On July 30, 2014, 12:22 a.m., Guozhang Wang wrote:
> > core/src/test/scala/unit/kafka/utils/TestUtils.scala, line 186
> > <https://reviews.apache.org/r/24006/diff/1/?file=643842#file643842line186>
> >
> > Could we just set the default value of configs parameter to null, 
> > instead of creating a separate function?

The reason I'd done this is that the Scala compiler complained because there's 
another implementation of createTopic that defines default parameter values. 
However, I was able to change the calls to this API to the one that uses 
numPartitions and replicationFactor, and stuck the Properties parameter over 
there.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/#review49049
---


On July 30, 2014, 6:18 p.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24006/
> ---
> 
> (Updated July 30, 2014, 6:18 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1420
> https://issues.apache.org/jira/browse/KAFKA-1420
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
> with TestUtils.createTopic in unit tests
> 
> 
> Diffs
> -
> 
>   core/src/test/scala/unit/kafka/admin/AdminTest.scala 
> e28979827110dfbbb92fe5b152e7f1cc973de400 
>   core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
> 29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
>   core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
> f44568cb25edf25db857415119018fd4c9922f61 
>   core/src/test/scala/unit/kafka/utils/TestUtils.scala 
> c4e13c5240c8303853d08cc3b40088f8c7dae460 
> 
> Diff: https://reviews.apache.org/r/24006/diff/
> 
> 
> Testing
> ---
> 
> Automated
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Re: Review Request 24006: Patch for KAFKA-1420

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/#review49139
---



core/src/test/scala/unit/kafka/admin/AdminTest.scala
<https://reviews.apache.org/r/24006/#comment85979>

Will fix whitespace



core/src/test/scala/unit/kafka/admin/AdminTest.scala
<https://reviews.apache.org/r/24006/#comment85980>

Will fix whitespace here


- Jonathan Natkins


On July 30, 2014, 6:24 p.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24006/
> ---
> 
> (Updated July 30, 2014, 6:24 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1420
> https://issues.apache.org/jira/browse/KAFKA-1420
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
> with TestUtils.createTopic in unit tests
> 
> 
> Diffs
> -
> 
>   core/src/test/scala/unit/kafka/admin/AdminTest.scala 
> e28979827110dfbbb92fe5b152e7f1cc973de400 
>   core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
> 29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
>   core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
> f44568cb25edf25db857415119018fd4c9922f61 
>   core/src/test/scala/unit/kafka/utils/TestUtils.scala 
> c4e13c5240c8303853d08cc3b40088f8c7dae460 
> 
> Diff: https://reviews.apache.org/r/24006/diff/
> 
> 
> Testing
> ---
> 
> Automated
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Review Request 24113: Patch for KAFKA-1562

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24113/
---

Review request for kafka.


Bugs: KAFKA-1562
https://issues.apache.org/jira/browse/KAFKA-1562


Repository: kafka


Description
---

KAFKA-1562 kafka-topics.sh alter add partitions resets cleanup.policy


Diffs
-

  core/src/main/scala/kafka/admin/AdminUtils.scala 
b5d8714e964fee8b29b05db04d79fd6ac84f3e48 
  core/src/main/scala/kafka/admin/TopicCommand.scala 
8d5c2e7088fc6e8bf69e775ea7f5893b94580fdf 
  core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/24113/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24113: Patch for KAFKA-1562

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24113/
---

(Updated July 30, 2014, 8:18 p.m.)


Review request for kafka.


Bugs: KAFKA-1562
https://issues.apache.org/jira/browse/KAFKA-1562


Repository: kafka


Description
---

KAFKA-1562 kafka-topics.sh alter add partitions resets cleanup.policy


Diffs (updated)
-

  core/src/main/scala/kafka/admin/AdminUtils.scala 
b5d8714e964fee8b29b05db04d79fd6ac84f3e48 
  core/src/main/scala/kafka/admin/TopicCommand.scala 
8d5c2e7088fc6e8bf69e775ea7f5893b94580fdf 
  core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/24113/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24113: Patch for KAFKA-1562

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24113/
---

(Updated July 30, 2014, 8:18 p.m.)


Review request for kafka.


Bugs: KAFKA-1562
https://issues.apache.org/jira/browse/KAFKA-1562


Repository: kafka


Description
---

KAFKA-1562 kafka-topics.sh alter add partitions resets cleanup.policy


Diffs
-

  core/src/main/scala/kafka/admin/AdminUtils.scala 
b5d8714e964fee8b29b05db04d79fd6ac84f3e48 
  core/src/main/scala/kafka/admin/TopicCommand.scala 
8d5c2e7088fc6e8bf69e775ea7f5893b94580fdf 
  core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/24113/diff/


Testing (updated)
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24113: Patch for KAFKA-1562

2014-07-30 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24113/
---

(Updated July 30, 2014, 8:51 p.m.)


Review request for kafka.


Bugs: KAFKA-1562
https://issues.apache.org/jira/browse/KAFKA-1562


Repository: kafka


Description
---

KAFKA-1562 kafka-topics.sh alter add partitions resets cleanup.policy


Diffs (updated)
-

  core/src/main/scala/kafka/admin/AdminUtils.scala 
b5d8714e964fee8b29b05db04d79fd6ac84f3e48 
  core/src/main/scala/kafka/admin/TopicCommand.scala 
8d5c2e7088fc6e8bf69e775ea7f5893b94580fdf 
  core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/24113/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-02 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated Aug. 2, 2014, 6:04 p.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in unit tests


Diffs (updated)
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-02 Thread Jonathan Natkins



> On Aug. 1, 2014, 9:27 p.m., Guozhang Wang wrote:
> > core/src/test/scala/unit/kafka/admin/AdminTest.scala, line 134
> > <https://reviews.apache.org/r/24006/diff/3/?file=646111#file646111line134>
> >
> > What I was actually thinking is that probably we can define these two 
> > variables at the top of testTopicCreationInZK and crate the 
> > expectedReplicaAssignment and leaderForPartitionMap programmatically 
> > instead of hand-written them.

This would admittedly be easier in later versions of Scala, which have some 
combinatorial functionality, but I've written a little code to do this. Take a 
look at it...I'm not totally sure it's what you had in mind, so let me know if 
you just meant for me to use AdminTools.assignReplicasToBrokers or something 
like that.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/#review49382
-------


On Aug. 2, 2014, 6:04 p.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24006/
> ---
> 
> (Updated Aug. 2, 2014, 6:04 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1420
> https://issues.apache.org/jira/browse/KAFKA-1420
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
> with TestUtils.createTopic in unit tests
> 
> 
> Diffs
> -
> 
>   core/src/test/scala/unit/kafka/admin/AdminTest.scala 
> e28979827110dfbbb92fe5b152e7f1cc973de400 
>   core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
> 29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
>   core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
> f44568cb25edf25db857415119018fd4c9922f61 
>   core/src/test/scala/unit/kafka/utils/TestUtils.scala 
> c4e13c5240c8303853d08cc3b40088f8c7dae460 
> 
> Diff: https://reviews.apache.org/r/24006/diff/
> 
> 
> Testing
> ---
> 
> Automated
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Re: Review Request 24113: Patch for KAFKA-1562

2014-08-02 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24113/
---

(Updated Aug. 2, 2014, 6:10 p.m.)


Review request for kafka.


Bugs: KAFKA-1562
https://issues.apache.org/jira/browse/KAFKA-1562


Repository: kafka


Description
---

KAFKA-1562 kafka-topics.sh alter add partitions resets cleanup.policy


Diffs (updated)
-

  core/src/main/scala/kafka/admin/AdminUtils.scala 
b5d8714e964fee8b29b05db04d79fd6ac84f3e48 
  core/src/main/scala/kafka/admin/TopicCommand.scala 
8d5c2e7088fc6e8bf69e775ea7f5893b94580fdf 
  core/src/test/scala/unit/kafka/admin/TopicCommandTest.scala PRE-CREATION 

Diff: https://reviews.apache.org/r/24113/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Issues getting IntelliJ set up

2014-08-04 Thread Jonathan Natkins

Hi,

I've been having some issues getting IntelliJ set up...I followed all the
instructions on the wiki, and I've successfully imported the project, and
run the jar Gradle target successfully. However, when I try to run a test
in the IDE, I get a number of errors:

/Users/Natty/apache/kafka-new/core/src/main/scala/kafka/tools/KafkaMigrationTool.java
Error:(21, 30) java: package kafka.javaapi.producer does not exist
Error:(22, 22) java: package kafka.producer does not exist
Error:(23, 22) java: package kafka.producer does not exist
Error:(24, 19) java: cannot find symbol
  symbol:   class Utils
  location: package kafka.utils
Error:(303, 39) java: cannot find symbol
  symbol:   class KeyedMessage
  location: class kafka.tools.KafkaMigrationTool.MigrationThread

And so on.

The two classes that seem to be causing trouble are KafkaMigrationTool and
ConsumerConnector. Has anybody run into this? Anyone know how to get around
this issue?

Thanks a lot,
Natty

Re: Issues getting IntelliJ set up

2014-08-04 Thread Jonathan Natkins

I did. I actually tried this from a completely clean repo (cloned a new
repo from github, changed gradle.properties, ran `gradlew idea`, then
imported into IntelliJ)


On Mon, Aug 4, 2014 at 12:18 PM, Timothy Chen  wrote:

> Hi Johnathan,
>
> Did you update your scala version before you run gradle idea?
>
> Also try cleaning up all the artifacts and try it again, as perhaps
> your intellij is not picking up the right version and from the right
> build folder.
>
> Tim
>
> On Mon, Aug 4, 2014 at 12:09 PM, Jonathan Natkins 
> wrote:
> > Hi,
> >
> > I've been having some issues getting IntelliJ set up...I followed all the
> > instructions on the wiki, and I've successfully imported the project, and
> > run the jar Gradle target successfully. However, when I try to run a test
> > in the IDE, I get a number of errors:
> >
> >
> /Users/Natty/apache/kafka-new/core/src/main/scala/kafka/tools/KafkaMigrationTool.java
> > Error:(21, 30) java: package kafka.javaapi.producer does not exist
> > Error:(22, 22) java: package kafka.producer does not exist
> > Error:(23, 22) java: package kafka.producer does not exist
> > Error:(24, 19) java: cannot find symbol
> >   symbol:   class Utils
> >   location: package kafka.utils
> > Error:(303, 39) java: cannot find symbol
> >   symbol:   class KeyedMessage
> >   location: class kafka.tools.KafkaMigrationTool.MigrationThread
> >
> > And so on.
> >
> > The two classes that seem to be causing trouble are KafkaMigrationTool
> and
> > ConsumerConnector. Has anybody run into this? Anyone know how to get
> around
> > this issue?
> >
> > Thanks a lot,
> > Natty
>

Re: Issues getting IntelliJ set up

2014-08-04 Thread Jonathan Natkins

Yep, it looks like you win the game. Managed to get a test running by doing
this. Thanks a bunch, Gwen!


On Mon, Aug 4, 2014 at 1:57 PM, Gwen Shapira  wrote:

> I think I found it :)
>
> Go to "project settings"
> Pick your module (I picked "core")
> "Paths" tab
> And change the paths to:
> /Users/Natty/apache/kafka-new/core/build/classes/main
> and
> /Users/Natty/apache/kafka-new/core/build/classes/test
>
> Don't use the project inherited paths, because we have a build dir per
> module.
>
> Gwen
>
> On Mon, Aug 4, 2014 at 12:54 PM, Jonathan Natkins 
> wrote:
> > I did. I actually tried this from a completely clean repo (cloned a new
> > repo from github, changed gradle.properties, ran `gradlew idea`, then
> > imported into IntelliJ)
> >
> >
> > On Mon, Aug 4, 2014 at 12:18 PM, Timothy Chen  wrote:
> >
> >> Hi Johnathan,
> >>
> >> Did you update your scala version before you run gradle idea?
> >>
> >> Also try cleaning up all the artifacts and try it again, as perhaps
> >> your intellij is not picking up the right version and from the right
> >> build folder.
> >>
> >> Tim
> >>
> >> On Mon, Aug 4, 2014 at 12:09 PM, Jonathan Natkins  >
> >> wrote:
> >> > Hi,
> >> >
> >> > I've been having some issues getting IntelliJ set up...I followed all
> the
> >> > instructions on the wiki, and I've successfully imported the project,
> and
> >> > run the jar Gradle target successfully. However, when I try to run a
> test
> >> > in the IDE, I get a number of errors:
> >> >
> >> >
> >>
> /Users/Natty/apache/kafka-new/core/src/main/scala/kafka/tools/KafkaMigrationTool.java
> >> > Error:(21, 30) java: package kafka.javaapi.producer does not exist
> >> > Error:(22, 22) java: package kafka.producer does not exist
> >> > Error:(23, 22) java: package kafka.producer does not exist
> >> > Error:(24, 19) java: cannot find symbol
> >> >   symbol:   class Utils
> >> >   location: package kafka.utils
> >> > Error:(303, 39) java: cannot find symbol
> >> >   symbol:   class KeyedMessage
> >> >   location: class kafka.tools.KafkaMigrationTool.MigrationThread
> >> >
> >> > And so on.
> >> >
> >> > The two classes that seem to be causing trouble are KafkaMigrationTool
> >> and
> >> > ConsumerConnector. Has anybody run into this? Anyone know how to get
> >> around
> >> > this issue?
> >> >
> >> > Thanks a lot,
> >> > Natty
> >>
>

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-10 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated Aug. 10, 2014, 9:11 p.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in unit tests


Diffs (updated)
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-10 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/#review50126
---



core/src/test/scala/unit/kafka/admin/AdminTest.scala
<https://reviews.apache.org/r/24006/#comment87702>

I wasn't totally sure I understood this comment, so I made a change that I 
think reflects what you were looking for. Let me know if I missed the mark.


- Jonathan Natkins


On Aug. 10, 2014, 9:11 p.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24006/
> ---
> 
> (Updated Aug. 10, 2014, 9:11 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1420
> https://issues.apache.org/jira/browse/KAFKA-1420
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
> with TestUtils.createTopic in unit tests
> 
> 
> Diffs
> -
> 
>   core/src/test/scala/unit/kafka/admin/AdminTest.scala 
> e28979827110dfbbb92fe5b152e7f1cc973de400 
>   core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
> 29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
>   core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
> f44568cb25edf25db857415119018fd4c9922f61 
>   core/src/test/scala/unit/kafka/utils/TestUtils.scala 
> c4e13c5240c8303853d08cc3b40088f8c7dae460 
> 
> Diff: https://reviews.apache.org/r/24006/diff/
> 
> 
> Testing
> ---
> 
> Automated
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-10 Thread Jonathan Natkins

Ah, gotcha. Given that, I think I made the right adjustment. Thanks for the
clarification!


On Sun, Aug 10, 2014 at 10:38 PM, Guozhang Wang  wrote:

>
>
> > On Aug. 10, 2014, 9:12 p.m., Jonathan Natkins wrote:
> > > core/src/test/scala/unit/kafka/admin/AdminTest.scala, line 114
> > > <
> https://reviews.apache.org/r/24006/diff/3-4/?file=646111#file646111line114
> >
> > >
> > > I wasn't totally sure I understood this comment, so I made a
> change that I think reflects what you were looking for. Let me know if I
> missed the mark.
>
> What I meant is that we need to guarantee the preferred replica would be
> the first replica in the list. For our case, it just that
>
> range(0, 10).map(i => (i -> i % brokers.size)).toMap
>
> gets the same result that it gets the first replica of the lists returned
> by
>
> combinations(brokers, replicationFactor)
>
> but it may not always be the case.
>
>
> - Guozhang
>
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24006/#review50126
> ---
>
>
> On Aug. 10, 2014, 9:11 p.m., Jonathan Natkins wrote:
> >
> > ---
> > This is an automatically generated e-mail. To reply, visit:
> > https://reviews.apache.org/r/24006/
> > ---
> >
> > (Updated Aug. 10, 2014, 9:11 p.m.)
> >
> >
> > Review request for kafka.
> >
> >
> > Bugs: KAFKA-1420
> > https://issues.apache.org/jira/browse/KAFKA-1420
> >
> >
> > Repository: kafka
> >
> >
> > Description
> > ---
> >
> > KAFKA-1420 Replace
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with
> TestUtils.createTopic in unit tests
> >
> >
> > Diffs
> > -
> >
> >   core/src/test/scala/unit/kafka/admin/AdminTest.scala
> e28979827110dfbbb92fe5b152e7f1cc973de400
> >   core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala
> 29cc01bcef9cacd8dec1f5d662644fc6fe4994bc
> >
> core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala
> f44568cb25edf25db857415119018fd4c9922f61
> >   core/src/test/scala/unit/kafka/utils/TestUtils.scala
> c4e13c5240c8303853d08cc3b40088f8c7dae460
> >
> > Diff: https://reviews.apache.org/r/24006/diff/
> >
> >
> > Testing
> > ---
> >
> > Automated
> >
> >
> > Thanks,
> >
> > Jonathan Natkins
> >
> >
>
>

Re: Review Request 24006: Patch for KAFKA-1420

2014-08-10 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24006/
---

(Updated Aug. 11, 2014, 6:03 a.m.)


Review request for kafka.


Bugs: KAFKA-1420
https://issues.apache.org/jira/browse/KAFKA-1420


Repository: kafka


Description
---

KAFKA-1420 Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK 
with TestUtils.createTopic in unit tests


Diffs (updated)
-

  core/src/test/scala/unit/kafka/admin/AdminTest.scala 
e28979827110dfbbb92fe5b152e7f1cc973de400 
  core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala 
29cc01bcef9cacd8dec1f5d662644fc6fe4994bc 
  core/src/test/scala/unit/kafka/integration/UncleanLeaderElectionTest.scala 
f44568cb25edf25db857415119018fd4c9922f61 
  core/src/test/scala/unit/kafka/utils/TestUtils.scala 
c4e13c5240c8303853d08cc3b40088f8c7dae460 

Diff: https://reviews.apache.org/r/24006/diff/


Testing
---

Automated


Thanks,

Jonathan Natkins

Review Request 24620: Patch for KAFKA-1580

2014-08-12 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/
---

Review request for kafka.


Bugs: KAFKA-1580
https://issues.apache.org/jira/browse/KAFKA-1580


Repository: kafka


Description
---

KAFKA-1580 Reject producer requests to internal topics


Diffs
-

  core/src/main/scala/kafka/server/KafkaApis.scala 
bb94673c8f3079a4053e0a7d58689e69775cdf8b 
  core/src/test/scala/unit/kafka/producer/ProducerTest.scala 
dd71d81041e1baa3e3ebdbc38f60f247ebd65948 

Diff: https://reviews.apache.org/r/24620/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-14 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/
---

(Updated Aug. 14, 2014, 11:50 p.m.)


Review request for kafka.


Bugs: KAFKA-1580
https://issues.apache.org/jira/browse/KAFKA-1580


Repository: kafka


Description
---

KAFKA-1580 Reject producer requests to internal topics


Diffs (updated)
-

  
clients/src/main/java/org/apache/kafka/common/errors/InternalTopicOperationException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3374bd98be8e565608c4e764ed10afdae383fb6f 
  core/src/main/scala/kafka/common/ErrorMapping.scala 
5559d26ba2b96059f719754a351fa4598ca8a70b 
  core/src/main/scala/kafka/common/InternalTopicOperationException.scala 
PRE-CREATION 
  core/src/main/scala/kafka/server/KafkaApis.scala 
bb94673c8f3079a4053e0a7d58689e69775cdf8b 
  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
789e74c776d99f204e0ef87eba1f55a3551abdf2 
  core/src/test/scala/unit/kafka/producer/ProducerTest.scala 
dd71d81041e1baa3e3ebdbc38f60f247ebd65948 

Diff: https://reviews.apache.org/r/24620/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-14 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/
---

(Updated Aug. 14, 2014, 11:56 p.m.)


Review request for kafka.


Bugs: KAFKA-1580
https://issues.apache.org/jira/browse/KAFKA-1580


Repository: kafka


Description
---

KAFKA-1580 Reject producer requests to internal topics


Diffs (updated)
-

  
clients/src/main/java/org/apache/kafka/common/errors/InternalTopicOperationException.java
 PRE-CREATION 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3374bd98be8e565608c4e764ed10afdae383fb6f 
  core/src/main/scala/kafka/common/ErrorMapping.scala 
5559d26ba2b96059f719754a351fa4598ca8a70b 
  core/src/main/scala/kafka/common/InternalTopicOperationException.scala 
PRE-CREATION 
  core/src/main/scala/kafka/server/KafkaApis.scala 
bb94673c8f3079a4053e0a7d58689e69775cdf8b 
  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
789e74c776d99f204e0ef87eba1f55a3551abdf2 

Diff: https://reviews.apache.org/r/24620/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-14 Thread Jonathan Natkins



> On Aug. 15, 2014, 12:12 a.m., Neha Narkhede wrote:
> > clients/src/main/java/org/apache/kafka/common/protocol/Errors.java, line 56
> > <https://reviews.apache.org/r/24620/diff/3/?file=661064#file661064line56>
> >
> > I meant adding the existing InvalidTopicException here. I'm not sure if 
> > there is value in differentiting amongst all the reasons the topic is 
> > invalid. We could just return the generic InvalidTopicException. 
> > 
> > Sorry if my suggestion wasn't clear enough.

Yeah, I understood the suggestion, I just have some mixed feelings about giving 
a relatively generic error message. I guess it's sufficient to have a more 
specific error message on the server, but it's a little unfortunate that 
there's not a way to pass a customized error message back to a Producer when 
something goes wrong. Makes for a more frustrating debugging session down the 
line, which is why I changed this to be more specific. Thoughts?


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/#review50669
-------


On Aug. 14, 2014, 11:56 p.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24620/
> ---
> 
> (Updated Aug. 14, 2014, 11:56 p.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1580
> https://issues.apache.org/jira/browse/KAFKA-1580
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1580 Reject producer requests to internal topics
> 
> 
> Diffs
> -
> 
>   
> clients/src/main/java/org/apache/kafka/common/errors/InternalTopicOperationException.java
>  PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
> 3374bd98be8e565608c4e764ed10afdae383fb6f 
>   core/src/main/scala/kafka/common/ErrorMapping.scala 
> 5559d26ba2b96059f719754a351fa4598ca8a70b 
>   core/src/main/scala/kafka/common/InternalTopicOperationException.scala 
> PRE-CREATION 
>   core/src/main/scala/kafka/server/KafkaApis.scala 
> bb94673c8f3079a4053e0a7d58689e69775cdf8b 
>   core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
> 789e74c776d99f204e0ef87eba1f55a3551abdf2 
> 
> Diff: https://reviews.apache.org/r/24620/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-14 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/
---

(Updated Aug. 15, 2014, 1:21 a.m.)


Review request for kafka.


Bugs: KAFKA-1580
https://issues.apache.org/jira/browse/KAFKA-1580


Repository: kafka


Description
---

KAFKA-1580 Reject producer requests to internal topics


Diffs (updated)
-

  
clients/src/main/java/org/apache/kafka/common/errors/InvalidTopicException.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3374bd98be8e565608c4e764ed10afdae383fb6f 
  core/src/main/scala/kafka/common/ErrorMapping.scala 
5559d26ba2b96059f719754a351fa4598ca8a70b 
  core/src/main/scala/kafka/server/KafkaApis.scala 
bb94673c8f3079a4053e0a7d58689e69775cdf8b 
  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
789e74c776d99f204e0ef87eba1f55a3551abdf2 

Diff: https://reviews.apache.org/r/24620/diff/


Testing
---


Thanks,

Jonathan Natkins

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-14 Thread Jonathan Natkins



> On Aug. 15, 2014, 12:12 a.m., Neha Narkhede wrote:
> > clients/src/main/java/org/apache/kafka/common/protocol/Errors.java, line 56
> > <https://reviews.apache.org/r/24620/diff/3/?file=661064#file661064line56>
> >
> > I meant adding the existing InvalidTopicException here. I'm not sure if 
> > there is value in differentiting amongst all the reasons the topic is 
> > invalid. We could just return the generic InvalidTopicException. 
> > 
> > Sorry if my suggestion wasn't clear enough.
> 
> Jonathan Natkins wrote:
> Yeah, I understood the suggestion, I just have some mixed feelings about 
> giving a relatively generic error message. I guess it's sufficient to have a 
> more specific error message on the server, but it's a little unfortunate that 
> there's not a way to pass a customized error message back to a Producer when 
> something goes wrong. Makes for a more frustrating debugging session down the 
> line, which is why I changed this to be more specific. Thoughts?

That said, I believe revision 4 is the patch you were hoping for


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/#review50669
---


On Aug. 15, 2014, 1:21 a.m., Jonathan Natkins wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24620/
> ---
> 
> (Updated Aug. 15, 2014, 1:21 a.m.)
> 
> 
> Review request for kafka.
> 
> 
> Bugs: KAFKA-1580
> https://issues.apache.org/jira/browse/KAFKA-1580
> 
> 
> Repository: kafka
> 
> 
> Description
> ---
> 
> KAFKA-1580 Reject producer requests to internal topics
> 
> 
> Diffs
> -
> 
>   
> clients/src/main/java/org/apache/kafka/common/errors/InvalidTopicException.java
>  PRE-CREATION 
>   clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
> 3374bd98be8e565608c4e764ed10afdae383fb6f 
>   core/src/main/scala/kafka/common/ErrorMapping.scala 
> 5559d26ba2b96059f719754a351fa4598ca8a70b 
>   core/src/main/scala/kafka/server/KafkaApis.scala 
> bb94673c8f3079a4053e0a7d58689e69775cdf8b 
>   core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
> 789e74c776d99f204e0ef87eba1f55a3551abdf2 
> 
> Diff: https://reviews.apache.org/r/24620/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jonathan Natkins
> 
>

Re: Review Request 24620: Patch for KAFKA-1580

2014-08-15 Thread Jonathan Natkins


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24620/
---

(Updated Aug. 15, 2014, 10:05 p.m.)


Review request for kafka.


Bugs: KAFKA-1580
https://issues.apache.org/jira/browse/KAFKA-1580


Repository: kafka


Description
---

KAFKA-1580 Reject producer requests to internal topics


Diffs (updated)
-

  
clients/src/main/java/org/apache/kafka/common/errors/InvalidTopicException.java 
PRE-CREATION 
  clients/src/main/java/org/apache/kafka/common/protocol/Errors.java 
3374bd98be8e565608c4e764ed10afdae383fb6f 
  core/src/main/scala/kafka/common/ErrorMapping.scala 
5559d26ba2b96059f719754a351fa4598ca8a70b 
  core/src/main/scala/kafka/server/KafkaApis.scala 
bb94673c8f3079a4053e0a7d58689e69775cdf8b 
  core/src/test/scala/integration/kafka/api/ProducerFailureHandlingTest.scala 
789e74c776d99f204e0ef87eba1f55a3551abdf2 

Diff: https://reviews.apache.org/r/24620/diff/


Testing
---


Thanks,

Jonathan Natkins

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-28 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114079#comment-14114079
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Wanted to bump this ticket so the patch doesn't get irreparably stale.

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>Assignee: Jun Rao
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch, KAFKA-1420_2014-08-10_23:03:46.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-21 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069394#comment-14069394
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

I was starting to work on this JIRA, but I'm hitting a small stumbling block. 
I've noticed some tests that create brokers using 
TestUtils.createBrokerConfigs() and mapping over the configs with 
TestUtils.createServer(), and other tests that create brokers using 
TestUtils.createBrokersInZk().

Where this becomes a little confusing is that both implementations of 
createTopic require a Seq[KafkaServer], but createServer() returns a 
KafkaServer via a Properties object and createBrokersInZk returns a 
Seq[Broker], and I don't see a particularly obvious way to go from a Broker to 
a KakfaServer.

Am I missing something obvious?

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in all unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
>
> This is a follow-up JIRA from KAFKA-1389



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-21 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069626#comment-14069626
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Thanks for the context, Guozhang. So in the cases where createBrokersInZk is 
used, should AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK still be 
used? In particular, I encountered this in 
AdminTest.testManualReplicaAssignment.

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in all unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
>
> This is a follow-up JIRA from KAFKA-1389



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-22 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070756#comment-14070756
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Hi Guozhang,

The particular case I'm confused by is a test like this:

{code:scala}
  @Test
  def testManualReplicaAssignment() {
val brokers = List(0, 1, 2, 3, 4)
TestUtils.createBrokersInZk(zkClient, brokers)

// duplicate brokers
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,0)))
}

// inconsistent replication factor
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,1), 1->Seq(0)))
}

// good assignment
val assignment = Map(0 -> List(0, 1, 2),
 1 -> List(1, 2, 3))
AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, "test", 
assignment)
val found = ZkUtils.getPartitionAssignmentForTopics(zkClient, Seq("test"))
assertEquals(assignment, found("test"))
  }
{code}

This test uses createOrUpdate, but creates brokers via createBrokersInZk, which 
doesn't return a reference to a KafkaServer.

The createTopic call, on the other hand, has two implementations:

{code:scala}
def createTopic(zkClient: ZkClient, topic: String, numPartitions: Int = 1, 
replicationFactor: Int = 1,
  servers: Seq[KafkaServer])

def createTopic(zkClient: ZkClient, topic: String, partitionReplicaAssignment: 
collection.Map[Int, Seq[Int]],
  servers: Seq[KafkaServer])
{scala}

In both cases, I need a Seq[KafkaServer]. It's unclear to me how to convert the 
aforementioned test to use createTopic, without materially changing the 
semantics of the test. Currently it doesn't use a real Kafka server, but to use 
createTopic would require me to get a hold of some real servers.

How would a conversion like this be done? Hopefully this clarifies why I'm 
confused.

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in all unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
>
> This is a follow-up JIRA from KAFKA-1389



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Comment Edited] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in all unit tests

2014-07-22 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070756#comment-14070756
 ] 

Jonathan Natkins edited comment on KAFKA-1420 at 7/22/14 7:30 PM:
--

Hi Guozhang,

The particular case I'm confused by is a test like this:

{code}
  @Test
  def testManualReplicaAssignment() {
val brokers = List(0, 1, 2, 3, 4)
TestUtils.createBrokersInZk(zkClient, brokers)

// duplicate brokers
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,0)))
}

// inconsistent replication factor
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,1), 1->Seq(0)))
}

// good assignment
val assignment = Map(0 -> List(0, 1, 2),
 1 -> List(1, 2, 3))
AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, "test", 
assignment)
val found = ZkUtils.getPartitionAssignmentForTopics(zkClient, Seq("test"))
assertEquals(assignment, found("test"))
  }
{code}

This test uses createOrUpdate, but creates brokers via createBrokersInZk, which 
doesn't return a reference to a KafkaServer.

The createTopic call, on the other hand, has two implementations:

{code}
def createTopic(zkClient: ZkClient, topic: String, numPartitions: Int = 1, 
replicationFactor: Int = 1,
  servers: Seq[KafkaServer])

def createTopic(zkClient: ZkClient, topic: String, partitionReplicaAssignment: 
collection.Map[Int, Seq[Int]],
  servers: Seq[KafkaServer])
{code}

In both cases, I need a Seq[KafkaServer]. It's unclear to me how to convert the 
aforementioned test to use createTopic, without materially changing the 
semantics of the test. Currently it doesn't use a real Kafka server, but to use 
createTopic would require me to get a hold of some real servers.

How would a conversion like this be done? Hopefully this clarifies why I'm 
confused.


was (Author: natty):
Hi Guozhang,

The particular case I'm confused by is a test like this:

{code:scala}
  @Test
  def testManualReplicaAssignment() {
val brokers = List(0, 1, 2, 3, 4)
TestUtils.createBrokersInZk(zkClient, brokers)

// duplicate brokers
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,0)))
}

// inconsistent replication factor
intercept[IllegalArgumentException] {
  AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, 
"test", Map(0->Seq(0,1), 1->Seq(0)))
}

// good assignment
val assignment = Map(0 -> List(0, 1, 2),
 1 -> List(1, 2, 3))
AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK(zkClient, "test", 
assignment)
val found = ZkUtils.getPartitionAssignmentForTopics(zkClient, Seq("test"))
assertEquals(assignment, found("test"))
  }
{code}

This test uses createOrUpdate, but creates brokers via createBrokersInZk, which 
doesn't return a reference to a KafkaServer.

The createTopic call, on the other hand, has two implementations:

{code:scala}
def createTopic(zkClient: ZkClient, topic: String, numPartitions: Int = 1, 
replicationFactor: Int = 1,
  servers: Seq[KafkaServer])

def createTopic(zkClient: ZkClient, topic: String, partitionReplicaAssignment: 
collection.Map[Int, Seq[Int]],
  servers: Seq[KafkaServer])
{scala}

In both cases, I need a Seq[KafkaServer]. It's unclear to me how to convert the 
aforementioned test to use createTopic, without materially changing the 
semantics of the test. Currently it doesn't use a real Kafka server, but to use 
createTopic would require me to get a hold of some real servers.

How would a conversion like this be done? Hopefully this clarifies why I'm 
confused.

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in all unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
>
> This is a follow-up JIRA from KAFKA-1389



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Status: Patch Available  (was: Open)

Patch available at https://reviews.apache.org/r/24006/

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: (was: KAFKA-1420.patch)

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (KAFKA-1560) Make arguments to jira-python API more explicit in kafka-patch-review's get_jira()

2014-07-28 Thread Jonathan Natkins (JIRA)

Jonathan Natkins created KAFKA-1560:
---

 Summary: Make arguments to jira-python API more explicit in 
kafka-patch-review's get_jira() 
 Key: KAFKA-1560
 URL: https://issues.apache.org/jira/browse/KAFKA-1560
 Project: Kafka
  Issue Type: Improvement
Reporter: Jonathan Natkins
Priority: Minor


I ran into an issue with kafka-patch-review.py as a result of a change in 
parameters for the JIRA() object between jira-python 0.28 and 0.29. A quick fix 
is to make parameters more explicit.

As a side note, there is a separate bug affecting the review tool, but I'm not 
sure it's fixable without modifying the jira-python source 
(https://bitbucket.org/bspeakmon/jira-python/issue/122/clientissue-function-needs-param-removal)

It seems the answer is that people should not use jira-python 0.29 yet, since 
it's a little broken.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1560) Make arguments to jira-python API more explicit in kafka-patch-review's get_jira()

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1560:


Status: Patch Available  (was: Open)

> Make arguments to jira-python API more explicit in kafka-patch-review's 
> get_jira() 
> ---
>
> Key: KAFKA-1560
> URL: https://issues.apache.org/jira/browse/KAFKA-1560
> Project: Kafka
>  Issue Type: Improvement
>    Reporter: Jonathan Natkins
>Priority: Minor
> Attachments: KAFKA-1560.patch
>
>
> I ran into an issue with kafka-patch-review.py as a result of a change in 
> parameters for the JIRA() object between jira-python 0.28 and 0.29. A quick 
> fix is to make parameters more explicit.
> As a side note, there is a separate bug affecting the review tool, but I'm 
> not sure it's fixable without modifying the jira-python source 
> (https://bitbucket.org/bspeakmon/jira-python/issue/122/clientissue-function-needs-param-removal)
> It seems the answer is that people should not use jira-python 0.29 yet, since 
> it's a little broken.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1560) Make arguments to jira-python API more explicit in kafka-patch-review's get_jira()

2014-07-28 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1560:


Attachment: KAFKA-1560.patch

> Make arguments to jira-python API more explicit in kafka-patch-review's 
> get_jira() 
> ---
>
> Key: KAFKA-1560
> URL: https://issues.apache.org/jira/browse/KAFKA-1560
> Project: Kafka
>  Issue Type: Improvement
>    Reporter: Jonathan Natkins
>Priority: Minor
> Attachments: KAFKA-1560.patch
>
>
> I ran into an issue with kafka-patch-review.py as a result of a change in 
> parameters for the JIRA() object between jira-python 0.28 and 0.29. A quick 
> fix is to make parameters more explicit.
> As a side note, there is a separate bug affecting the review tool, but I'm 
> not sure it's fixable without modifying the jira-python source 
> (https://bitbucket.org/bspeakmon/jira-python/issue/122/clientissue-function-needs-param-removal)
> It seems the answer is that people should not use jira-python 0.29 yet, since 
> it's a little broken.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1560) Make arguments to jira-python API more explicit in kafka-patch-review's get_jira()

2014-07-28 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14076957#comment-14076957
 ] 

Jonathan Natkins commented on KAFKA-1560:
-

Created reviewboard https://reviews.apache.org/r/24012/diff/
 against branch origin/trunk

> Make arguments to jira-python API more explicit in kafka-patch-review's 
> get_jira() 
> ---
>
> Key: KAFKA-1560
> URL: https://issues.apache.org/jira/browse/KAFKA-1560
> Project: Kafka
>  Issue Type: Improvement
>    Reporter: Jonathan Natkins
>Priority: Minor
> Attachments: KAFKA-1560.patch
>
>
> I ran into an issue with kafka-patch-review.py as a result of a change in 
> parameters for the JIRA() object between jira-python 0.28 and 0.29. A quick 
> fix is to make parameters more explicit.
> As a side note, there is a separate bug affecting the review tool, but I'm 
> not sure it's fixable without modifying the jira-python source 
> (https://bitbucket.org/bspeakmon/jira-python/issue/122/clientissue-function-needs-param-removal)
> It seems the answer is that people should not use jira-python 0.29 yet, since 
> it's a little broken.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420_2014-07-30_11:18:26.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079690#comment-14079690
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Updated reviewboard https://reviews.apache.org/r/24006/diff/
 against branch origin/trunk

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420_2014-07-30_11:24:55.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079699#comment-14079699
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Updated reviewboard https://reviews.apache.org/r/24006/diff/
 against branch origin/trunk

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079846#comment-14079846
 ] 

Jonathan Natkins commented on KAFKA-1562:
-

If it's alright, I was planning on working on this a bit. I think I know where 
the issue is, and I'm in the process of fixing it at the moment.

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1562:


Attachment: KAFKA-1562.patch

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1562:


Attachment: KAFKA-1562_2014-07-30_13:18:21.patch

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079897#comment-14079897
 ] 

Jonathan Natkins commented on KAFKA-1562:
-

Updated reviewboard https://reviews.apache.org/r/24113/diff/
 against branch origin/trunk

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079893#comment-14079893
 ] 

Jonathan Natkins commented on KAFKA-1562:
-

Created reviewboard https://reviews.apache.org/r/24113/diff/
 against branch origin/trunk

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1562:


Status: Patch Available  (was: Open)

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
>Assignee: Sriharsha Chintalapani
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079953#comment-14079953
 ] 

Jonathan Natkins commented on KAFKA-1562:
-

Updated reviewboard https://reviews.apache.org/r/24113/diff/
 against branch origin/trunk

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch, 
> KAFKA-1562_2014-07-30_13:51:25.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-07-30 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1562:


Attachment: KAFKA-1562_2014-07-30_13:51:25.patch

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch, 
> KAFKA-1562_2014-07-30_13:51:25.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-02 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420_2014-08-02_11:04:15.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-02 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083649#comment-14083649
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Updated reviewboard https://reviews.apache.org/r/24006/diff/
 against branch origin/trunk

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-08-02 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083651#comment-14083651
 ] 

Jonathan Natkins commented on KAFKA-1562:
-

Updated reviewboard https://reviews.apache.org/r/24113/diff/
 against branch origin/trunk

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch, 
> KAFKA-1562_2014-07-30_13:51:25.patch, KAFKA-1562_2014-08-02_11:10:34.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1562) kafka-topics.sh alter add partitions resets cleanup.policy

2014-08-02 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1562:


Attachment: KAFKA-1562_2014-08-02_11:10:34.patch

> kafka-topics.sh alter add partitions resets cleanup.policy
> --
>
> Key: KAFKA-1562
> URL: https://issues.apache.org/jira/browse/KAFKA-1562
> Project: Kafka
>  Issue Type: Bug
>Affects Versions: 0.8.1.1
>Reporter: Kenny
> Attachments: KAFKA-1562.patch, KAFKA-1562_2014-07-30_13:18:21.patch, 
> KAFKA-1562_2014-07-30_13:51:25.patch, KAFKA-1562_2014-08-02_11:10:34.patch
>
>
> When partitions are added to an already existing topic the 
> cleanup.policy=compact is not retained.
> {code}
> ./kafka-topics.sh --zookeeper localhost --create --partitions 1 
> --replication-factor 1 --topic KTEST --config cleanup.policy=compact
> ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:1ReplicationFactor:1 
> Configs:cleanup.policy=compact
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
> ./kafka-topics.sh --zookeeper localhost --alter --partitions 3 --topic KTEST 
> --config cleanup.policy=compact
>  ./kafka-topics.sh --zookeeper localhost --describe --topic KTEST
> Topic:KTEST   PartitionCount:3ReplicationFactor:1 Configs:
>   Topic: KTESTPartition: 0Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 1Leader: 0   Replicas: 0 Isr: 0
>   Topic: KTESTPartition: 2Leader: 0   Replicas: 0 Isr: 0
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-07 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089850#comment-14089850
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Wanted to try to make sure this doesn't fall too far off the radar, since the 
review is mostly done.

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-10 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092203#comment-14092203
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Updated reviewboard https://reviews.apache.org/r/24006/diff/
 against branch origin/trunk

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-10 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420_2014-08-10_14:12:05.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-10 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1420:


Attachment: KAFKA-1420_2014-08-10_23:03:46.patch

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch, KAFKA-1420_2014-08-10_23:03:46.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-10 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14092485#comment-14092485
 ] 

Jonathan Natkins commented on KAFKA-1420:
-

Updated reviewboard https://reviews.apache.org/r/24006/diff/
 against branch origin/trunk

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch, KAFKA-1420_2014-08-10_23:03:46.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-12 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Attachment: KAFKA-1580.patch

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
> Attachments: KAFKA-1580.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1580) Reject producer requests to internal topics

2014-08-12 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094895#comment-14094895
 ] 

Jonathan Natkins commented on KAFKA-1580:
-

Created reviewboard https://reviews.apache.org/r/24620/diff/
 against branch origin/trunk

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
> Attachments: KAFKA-1580.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-12 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Assignee: Jonathan Natkins
  Status: Patch Available  (was: Open)

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>    Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Assigned] (KAFKA-1420) Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with TestUtils.createTopic in unit tests

2014-08-12 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins reassigned KAFKA-1420:
---

Assignee: Jonathan Natkins

> Replace AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK with 
> TestUtils.createTopic in unit tests
> --
>
> Key: KAFKA-1420
> URL: https://issues.apache.org/jira/browse/KAFKA-1420
> Project: Kafka
>  Issue Type: Bug
>Reporter: Guozhang Wang
>    Assignee: Jonathan Natkins
>  Labels: newbie
> Fix For: 0.8.2
>
> Attachments: KAFKA-1420.patch, KAFKA-1420_2014-07-30_11:18:26.patch, 
> KAFKA-1420_2014-07-30_11:24:55.patch, KAFKA-1420_2014-08-02_11:04:15.patch, 
> KAFKA-1420_2014-08-10_14:12:05.patch, KAFKA-1420_2014-08-10_23:03:46.patch
>
>
> This is a follow-up JIRA from KAFKA-1389.
> There are a bunch of places in the unit tests where we misuse 
> AdminUtils.createOrUpdateTopicPartitionAssignmentPathInZK to create topics, 
> where TestUtils.createTopic needs to be used instead.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097925#comment-14097925
 ] 

Jonathan Natkins commented on KAFKA-1580:
-

Updated reviewboard https://reviews.apache.org/r/24620/diff/
 against branch origin/trunk

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Attachment: KAFKA-1580_2014-08-14_16:50:40.patch

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>    Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097934#comment-14097934
 ] 

Jonathan Natkins commented on KAFKA-1580:
-

Updated reviewboard https://reviews.apache.org/r/24620/diff/
 against branch origin/trunk

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Attachment: KAFKA-1580_2014-08-14_16:56:50.patch

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>    Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1560) Make arguments to jira-python API more explicit in kafka-patch-review's get_jira()

2014-08-14 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097996#comment-14097996
 ] 

Jonathan Natkins commented on KAFKA-1560:
-

No problem, thanks!

> Make arguments to jira-python API more explicit in kafka-patch-review's 
> get_jira() 
> ---
>
> Key: KAFKA-1560
> URL: https://issues.apache.org/jira/browse/KAFKA-1560
> Project: Kafka
>  Issue Type: Improvement
>    Reporter: Jonathan Natkins
>Assignee: Jonathan Natkins
>Priority: Minor
> Attachments: KAFKA-1560.patch, KAFKA-1560.patch
>
>
> I ran into an issue with kafka-patch-review.py as a result of a change in 
> parameters for the JIRA() object between jira-python 0.28 and 0.29. A quick 
> fix is to make parameters more explicit.
> As a side note, there is a separate bug affecting the review tool, but I'm 
> not sure it's fixable without modifying the jira-python source 
> (https://bitbucket.org/bspeakmon/jira-python/issue/122/clientissue-function-needs-param-removal)
> It seems the answer is that people should not use jira-python 0.29 yet, since 
> it's a little broken.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098026#comment-14098026
 ] 

Jonathan Natkins commented on KAFKA-1580:
-

Updated reviewboard https://reviews.apache.org/r/24620/diff/
 against branch origin/trunk

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch, KAFKA-1580_2014-08-14_18:21:38.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-14 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Attachment: KAFKA-1580_2014-08-14_18:21:38.patch

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>    Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch, KAFKA-1580_2014-08-14_18:21:38.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (KAFKA-1580) Reject producer requests to internal topics

2014-08-15 Thread Jonathan Natkins (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14099244#comment-14099244
 ] 

Jonathan Natkins commented on KAFKA-1580:
-

Updated reviewboard https://reviews.apache.org/r/24620/diff/
 against branch origin/trunk

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch, KAFKA-1580_2014-08-14_18:21:38.patch, 
> KAFKA-1580_2014-08-15_15:05:29.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (KAFKA-1580) Reject producer requests to internal topics

2014-08-15 Thread Jonathan Natkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/KAFKA-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Natkins updated KAFKA-1580:


Attachment: KAFKA-1580_2014-08-15_15:05:29.patch

> Reject producer requests to internal topics
> ---
>
> Key: KAFKA-1580
> URL: https://issues.apache.org/jira/browse/KAFKA-1580
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Reporter: Joel Koshy
>    Assignee: Jonathan Natkins
> Attachments: KAFKA-1580.patch, KAFKA-1580_2014-08-14_16:50:40.patch, 
> KAFKA-1580_2014-08-14_16:56:50.patch, KAFKA-1580_2014-08-14_18:21:38.patch, 
> KAFKA-1580_2014-08-15_15:05:29.patch
>
>
> Producer requests to internal topics (currently only __consumer_offset) can 
> be disastrous.
> E.g., if we allow a message to be appended to the offsets topic this could 
> lead to fatal exceptions when loading the offsets topic and when compacting 
> the log.
> Producer requests to these topics should be rejected.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

81 matches

Mail list logo