[jira] [Commented] (KAFKA-260) Add audit trail to kafka

2012-11-27 Thread Lorenz Knies (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504487#comment-13504487 ] Lorenz Knies commented on KAFKA-260: Hi, i like option 2. I wrote a generic consumer

[jira] [Created] (KAFKA-634) ConsoleProducer compresses messages and ignores the --compress flag

2012-11-27 Thread Paul Garner (JIRA)
Paul Garner created KAFKA-634: - Summary: ConsoleProducer compresses messages and ignores the --compress flag Key: KAFKA-634 URL: https://issues.apache.org/jira/browse/KAFKA-634 Project: Kafka

[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504815#comment-13504815 ] Jay Kreps commented on KAFKA-521: - 1. The point of the logic is to recompress all the

[jira] [Created] (KAFKA-635) Producer error when trying to send not displayed unless in DEBUG logging level

2012-11-27 Thread Chris Curtin (JIRA)
Chris Curtin created KAFKA-635: -- Summary: Producer error when trying to send not displayed unless in DEBUG logging level Key: KAFKA-635 URL: https://issues.apache.org/jira/browse/KAFKA-635 Project:

[jira] [Commented] (KAFKA-133) Publish kafka jar to a public maven repository

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504826#comment-13504826 ] Jay Kreps commented on KAFKA-133: - Hey Otis, that's awesome. Thanks for taking point on

Re: Kafka 0.8 trial version

2012-11-27 Thread S Ahmed
Nice! What has changed, is it only replication or other enhancements? On Mon, Nov 26, 2012 at 12:23 PM, Jun Rao jun...@gmail.com wrote: Hi, Everyone, Thanks to people who contributed to the project, we have made significant progress in Kafka replication. Now, we'd like people to give it a

Re: Kafka 0.8 trial version

2012-11-27 Thread Neha Narkhede
Following are the major changes - 1. Logical offsets The design document is here: https://cwiki.apache.org/confluence/display/KAFKA/Keyed+Messages+Proposal The JIRA has a number of technical details regarding the logical offsets: https://issues.apache.org/jira/browse/KAFKA-506 2.

[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504871#comment-13504871 ] Swapnil Ghike commented on KAFKA-521: - 1. I see, thanks for the clarification. If there

Re: understanding partitions based on wiki example of profile visits

2012-11-27 Thread S Ahmed
How does that work out though, I mean with 10 million users that is 10 million files at least. On Mon, Nov 26, 2012 at 2:02 PM, Jay Kreps jay.kr...@gmail.com wrote: Yeah a partition is physically implemented as a log (i.e. a sequence of files containing a bunch of messages indexed by

Re: understanding partitions based on wiki example of profile visits

2012-11-27 Thread Jay Kreps
We don't have a partition per user, there is no need for that. In the same way a distributed database doesn't have a partition per user. A partition is just a physical grouping of keys. -Jay On Tue, Nov 27, 2012 at 12:00 PM, S Ahmed sahmed1...@gmail.com wrote: How does that work out though, I

[jira] [Commented] (KAFKA-633) AdminTest.testShutdownBroker fails

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505095#comment-13505095 ] Jay Kreps commented on KAFKA-633: - Another annoying thing is that after AdminTest runs you

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v4.patch New patch, v4, contains two changes: 1. Remove unused imports Swapnil

[jira] [Updated] (KAFKA-622) Create mbeans per client

2012-11-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Ghike updated KAFKA-622: Attachment: kafka-622-v3.patch 1. I see your point, that's good to know. Now using a case class

[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Swapnil Ghike (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505148#comment-13505148 ] Swapnil Ghike commented on KAFKA-521: - Log.deleteOldSegments: Just a nitpick, I

[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Neha Narkhede (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505183#comment-13505183 ] Neha Narkhede commented on KAFKA-521: - Pretty good cleanup and great usage of Java data

[jira] [Created] (KAFKA-636) Make log segment delete asynchronous

2012-11-27 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-636: --- Summary: Make log segment delete asynchronous Key: KAFKA-636 URL: https://issues.apache.org/jira/browse/KAFKA-636 Project: Kafka Issue Type: Bug Reporter:

[jira] [Commented] (KAFKA-635) Producer error when trying to send not displayed unless in DEBUG logging level

2012-11-27 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505222#comment-13505222 ] Jun Rao commented on KAFKA-635: --- Chris, Thanks for reporting this. Do you really need to

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v5.patch New patch. Addresses your comments (see below) and also one more

Re: git wip incubator-kafka.git

2012-11-27 Thread Jun Rao
Chris, I plan to submit an infra ticket for the post graduation items. One of the subtasks will be to move our svn repo to a new location (w/o incubator). We could either reuse your ticket and ask for a new repo name or we could close your ticket and create a new one. Which one would you prefer?