[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504815#comment-13504815 ] Jay Kreps commented on KAFKA-521: - 1. The point of the logic is to recompress all the

[jira] [Commented] (KAFKA-133) Publish kafka jar to a public maven repository

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504826#comment-13504826 ] Jay Kreps commented on KAFKA-133: - Hey Otis, that's awesome. Thanks for taking point on

[jira] [Commented] (KAFKA-633) AdminTest.testShutdownBroker fails

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505095#comment-13505095 ] Jay Kreps commented on KAFKA-633: - Another annoying thing is that after AdminTest runs you

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v4.patch New patch, v4, contains two changes: 1. Remove unused imports Swapnil

[jira] [Created] (KAFKA-636) Make log segment delete asynchronous

2012-11-27 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-636: --- Summary: Make log segment delete asynchronous Key: KAFKA-636 URL: https://issues.apache.org/jira/browse/KAFKA-636 Project: Kafka Issue Type: Bug Reporter:

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-27 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v5.patch New patch. Addresses your comments (see below) and also one more

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v2.patch Updated patch. In addition to the items in v1 This has the following

[jira] [Commented] (KAFKA-521) Refactor Log subsystem

2012-11-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504066#comment-13504066 ] Jay Kreps commented on KAFKA-521: - This patch is ready for review. It is intended for

[jira] [Resolved] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-544. - Resolution: Fixed Retain key in producer and expose it in the consumer

[jira] [Created] (KAFKA-631) Implement log compaction

2012-11-26 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-631: --- Summary: Implement log compaction Key: KAFKA-631 URL: https://issues.apache.org/jira/browse/KAFKA-631 Project: Kafka Issue Type: New Feature Components:

[jira] [Commented] (KAFKA-631) Implement log compaction

2012-11-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504192#comment-13504192 ] Jay Kreps commented on KAFKA-631: - Here is a specific proposal: We will retain the

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v3.patch Updated patch. Only change is to make the maxOffset parameter in Log.read

[jira] [Resolved] (KAFKA-562) Non-failure System Test Log Segment File Checksums mismatched

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-562. - Resolution: Fixed Fix Version/s: 0.8 This was addressed a while back.

[jira] [Commented] (KAFKA-367) StringEncoder/StringDecoder use platform default character set

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502946#comment-13502946 ] Jay Kreps commented on KAFKA-367: - Eli, I was also working on the same area for KAFKA-544

[jira] [Updated] (KAFKA-367) StringEncoder/StringDecoder use platform default character set

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-367: Resolution: Fixed Fix Version/s: 0.8 Status: Resolved (was: Patch Available)

[jira] [Commented] (KAFKA-468) String#getBytes is platform dependent

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502947#comment-13502947 ] Jay Kreps commented on KAFKA-468: - I integrated the changes in KAFKA-367 into the patch for

[jira] [Updated] (KAFKA-468) String#getBytes is platform dependent

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-468: Resolution: Fixed Status: Resolved (was: Patch Available) String#getBytes is platform

[jira] [Commented] (KAFKA-344) migration tool from 0.7 to 0.8

2012-11-22 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502950#comment-13502950 ] Jay Kreps commented on KAFKA-344: - This is done now, right? migration

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-21 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Attachment: KAFKA-544-joel-comments.patch Hey Joel, those are good points. Here is a follow-up patch that

[jira] [Updated] (KAFKA-521) Refactor Log subsystem

2012-11-20 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-521: Attachment: KAFKA-521-v1.patch Changes FileMessageSet: - Improve docs - rename constructor argument

[jira] [Commented] (KAFKA-626) Produce requests dropped due to socket timeouts on get metadata requests

2012-11-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500441#comment-13500441 ] Jay Kreps commented on KAFKA-626: - Suspect the root cause is KAFKA-608. I am changing that

[jira] [Updated] (KAFKA-608) getTopicMetadata does not respect producer config settings

2012-11-19 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-608: Priority: Blocker (was: Major) Affects Version/s: 0.8 Fix Version/s: 0.8

[jira] [Commented] (KAFKA-619) Regression : System Test 900x (Migration Tool) - java.lang.ClassCastException: kafka.message.Message cannot be cast to [B

2012-11-16 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499175#comment-13499175 ] Jay Kreps commented on KAFKA-619: - I believe I have fixed this.

[jira] [Resolved] (KAFKA-619) Regression : System Test 900x (Migration Tool) - java.lang.ClassCastException: kafka.message.Message cannot be cast to [B

2012-11-16 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-619. - Resolution: Fixed Regression : System Test 900x (Migration Tool) - java.lang.ClassCastException:

[jira] [Created] (KAFKA-615) Avoid fsync on log segment roll

2012-11-15 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-615: --- Summary: Avoid fsync on log segment roll Key: KAFKA-615 URL: https://issues.apache.org/jira/browse/KAFKA-615 Project: Kafka Issue Type: Bug Reporter: Jay

[jira] [Commented] (KAFKA-615) Avoid fsync on log segment roll

2012-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498364#comment-13498364 ] Jay Kreps commented on KAFKA-615: - Here is a quick thought on how this might work to kick

[jira] [Created] (KAFKA-616) Implement acks=0

2012-11-15 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-616: --- Summary: Implement acks=0 Key: KAFKA-616 URL: https://issues.apache.org/jira/browse/KAFKA-616 Project: Kafka Issue Type: Bug Reporter: Jay Kreps For

[jira] [Updated] (KAFKA-616) Implement acks=0

2012-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-616: Description: For completeness it would be nice to handle the case where acks=0 in the produce request. The

[jira] [Resolved] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-545. - Resolution: Fixed Fix Version/s: 0.8 Assignee: Jay Kreps Add a Performance Suite for

[jira] [Created] (KAFKA-617) Refactor KafkaApis.handleOffsetRequest

2012-11-15 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-617: --- Summary: Refactor KafkaApis.handleOffsetRequest Key: KAFKA-617 URL: https://issues.apache.org/jira/browse/KAFKA-617 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-617) Refactor KafkaApis.handleOffsetRequest

2012-11-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498482#comment-13498482 ] Jay Kreps commented on KAFKA-617: - Evidence of code funk: def

[jira] [Commented] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497205#comment-13497205 ] Jay Kreps commented on KAFKA-544: - One other minor thing in this patch: 5. Both the key and

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Attachment: KAFKA-544-v2.patch Updated patch. This patch fixes on of the uglier things in the producer API.

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Attachment: KAFKA-544-v4.patch Final patch fixes a bug that effected the system tests. This patch is ready

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Attachment: KAFKA-544-v5.patch 30. Good point, added. 31. No, for some reason scala won't allow that since

[jira] [Resolved] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-588. - Resolution: Fixed Fixed. Index truncation doesn't seem to remove the last entry

[jira] [Resolved] (KAFKA-71) embed offset inside Message class

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-71. Resolution: Invalid (was: Unresolved) In 0.8 the iterator returns the offset. embed

[jira] [Updated] (KAFKA-179) Log files always touched when broker is bounced

2012-11-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-179: Labels: newbie (was: ) Log files always touched when broker is bounced

[jira] [Created] (KAFKA-608) getTopicMetadata does not respect producer config settings

2012-11-13 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-608: --- Summary: getTopicMetadata does not respect producer config settings Key: KAFKA-608 URL: https://issues.apache.org/jira/browse/KAFKA-608 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-13 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Summary: Retain key in producer and expose it in the consumer (was: Retain key in producer) Retain

[jira] [Updated] (KAFKA-544) Retain key in producer and expose it in the consumer

2012-11-13 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-544: Attachment: KAFKA-544-v1.patch This patch does the following: 1. Change Encoder and Decoder to map between

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490869#comment-13490869 ] Jay Kreps commented on KAFKA-593: - Looks good, two minor things: 1. Can we name resetSize

[jira] [Updated] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-545: Attachment: KAFKA-545.patch - Improve the existing TestLinearWriteSpeed.scala that does linear writes to a

[jira] [Updated] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-545: Attachment: KAFKA-545-v2.patch Oops, wrong base revision for that patch, fixed it in v2.

[jira] [Updated] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-545: Attachment: KAFKA-545-v3.patch One more tweak. Add a Performance Suite for the Log

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13490971#comment-13490971 ] Jay Kreps commented on KAFKA-593: - Discussed with Victor. (1) and (3) should be doable, but

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-11-05 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491082#comment-13491082 ] Jay Kreps commented on KAFKA-593: - +1 Empty log index file created when

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-11-02 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489500#comment-13489500 ] Jay Kreps commented on KAFKA-593: - trimOrReallocate(isReallocate: Boolean) is a bit of a

[jira] [Commented] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-01 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1344#comment-1344 ] Jay Kreps commented on KAFKA-545: - Yes, for the above data everything is done on a

[jira] [Commented] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-11-01 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488985#comment-13488985 ] Jay Kreps commented on KAFKA-545: - Okay, wrote some tests for lockin MappedByteBuffer and

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-10-31 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487878#comment-13487878 ] Jay Kreps commented on KAFKA-593: - Makes sense, clever. I thought a bit about this during

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-10-31 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487879#comment-13487879 ] Jay Kreps commented on KAFKA-593: - Err, that should read intra-messageset positions.

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-31 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188-v8.patch This failing test was due to a real bug in LogManager.shutdown that lead to

[jira] [Created] (KAFKA-597) Refactor KafkaScheduler

2012-10-31 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-597: --- Summary: Refactor KafkaScheduler Key: KAFKA-597 URL: https://issues.apache.org/jira/browse/KAFKA-597 Project: Kafka Issue Type: Bug Affects Versions: 0.8.1

[jira] [Commented] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-10-31 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488366#comment-13488366 ] Jay Kreps commented on KAFKA-545: - The initial set of results show the following: Big

[jira] [Commented] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-10-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487086#comment-13487086 ] Jay Kreps commented on KAFKA-588: - Okay, I fooled around a little more and figured it out.

[jira] [Commented] (KAFKA-595) Producer side compression is unnecessary

2012-10-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487102#comment-13487102 ] Jay Kreps commented on KAFKA-595: - I think saying it is unnecessary is perhaps overstating

[jira] [Updated] (KAFKA-545) Add a Performance Suite for the Log subsystem

2012-10-30 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-545: Attachment: KAFKA-545-draft.patch Attaching wip that has this command. Usage is: jkreps-mn:kafka-git

[jira] [Commented] (KAFKA-188) Support multiple data directories

2012-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486220#comment-13486220 ] Jay Kreps commented on KAFKA-188: - Okay, you and Neha are giving conflicting advice on

[jira] [Commented] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486293#comment-13486293 ] Jay Kreps commented on KAFKA-588: - Jun the OffsetIndex.truncateTo logic looks right to me

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188-v6.patch Okay this patch addresses Jun's comments: 50. The zeros are actually

[jira] [Updated] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-588: Attachment: KAFKA-588.patch I can't reproduce this issue, but this patch expands the tests and adds a

[jira] [Commented] (KAFKA-593) Empty log index file created when it shouldn't be empty

2012-10-29 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486631#comment-13486631 ] Jay Kreps commented on KAFKA-593: - I am not sure that this is a bug. The index entries are

[jira] [Assigned] (KAFKA-588) Index truncation doesn't seem to remove the last entry properly

2012-10-26 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-588: --- Assignee: Jay Kreps Index truncation doesn't seem to remove the last entry properly

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188-v2.patch Updated patch: 1. Split HWM file per data directory 2. Move to a least

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188-v3.patch Wups, missed two minor changes in that last patch. Support

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188-v4.patch Attached an updated patch with a few very minor changes: 1. If there is only

[jira] [Updated] (KAFKA-587) Refactor end-to-end latency test to work in system test framework

2012-10-25 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-587: Issue Type: Improvement (was: Bug) Refactor end-to-end latency test to work in system test framework

[jira] [Resolved] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-15 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-557. - Resolution: Fixed Fix Version/s: 0.8 Assignee: Jay Kreps Replica fetch thread

[jira] [Commented] (KAFKA-188) Support multiple data directories

2012-10-14 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475908#comment-13475908 ] Jay Kreps commented on KAFKA-188: - Yeah, I think I agree. It's a little more complex, be we

[jira] [Commented] (KAFKA-568) LogTest started failing after KAFKA-506

2012-10-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475137#comment-13475137 ] Jay Kreps commented on KAFKA-568: - I noticed this too. The problem seems to stem from a

[jira] [Assigned] (KAFKA-568) LogTest started failing after KAFKA-506

2012-10-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps reassigned KAFKA-568: --- Assignee: Jay Kreps LogTest started failing after KAFKA-506

[jira] [Updated] (KAFKA-569) Cleanup kafka.utils.Utils

2012-10-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-569: Attachment: KAFKA-569-v2.patch Neha, thanks for the quick review! New patch: - Added the comment you

[jira] [Commented] (KAFKA-569) Cleanup kafka.utils.Utils

2012-10-12 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475443#comment-13475443 ] Jay Kreps commented on KAFKA-569: - Oh my, good catch. BTW, in fixing this I found a rare

[jira] [Commented] (KAFKA-560) Garbage Collect obsolete topics

2012-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474254#comment-13474254 ] Jay Kreps commented on KAFKA-560: - Filed KAFKA-566 for this. Garbage

[jira] [Created] (KAFKA-566) Add last modified time to the TopicMetadataRequest

2012-10-11 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-566: --- Summary: Add last modified time to the TopicMetadataRequest Key: KAFKA-566 URL: https://issues.apache.org/jira/browse/KAFKA-566 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474345#comment-13474345 ] Jay Kreps commented on KAFKA-557: - Oh man, you are a tough reviewer: 1.2 The coding guide

[jira] [Created] (KAFKA-569) Cleanup kafka.utils.Utils

2012-10-11 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-569: --- Summary: Cleanup kafka.utils.Utils Key: KAFKA-569 URL: https://issues.apache.org/jira/browse/KAFKA-569 Project: Kafka Issue Type: Bug Affects Versions: 0.8

[jira] [Updated] (KAFKA-569) Cleanup kafka.utils.Utils

2012-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-569: Attachment: KAFKA-569.patch This patch has a large number of trivial changes. Here is a summary: - Make

[jira] [Updated] (KAFKA-188) Support multiple data directories

2012-10-11 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-188: Attachment: KAFKA-188.patch Okay, took a shot at this. Attached is a draft patch. It needs some tests

[jira] [Commented] (KAFKA-562) Non-failure System Test Log Segment File Checksums mismatched

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473426#comment-13473426 ] Jay Kreps commented on KAFKA-562: - Yeah, this is definitely a real problem that is leading

[jira] [Commented] (KAFKA-562) Non-failure System Test Log Segment File Checksums mismatched

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473463#comment-13473463 ] Jay Kreps commented on KAFKA-562: - Okay, this is not a bug exactly, I was mistaken. Here is

[jira] [Updated] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-557: Attachment: KAFKA-557.patch Attached is a patch that makes offset assignment by the log optional. This is a

[jira] [Updated] (KAFKA-506) Store logical offset in log

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-506: Attachment: KAFKA-506-neha-post-review-v2.patch This patch is identical to the previous Neha related patch

[jira] [Commented] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473716#comment-13473716 ] Jay Kreps commented on KAFKA-557: - I verified that this fixes KAFKA-562

[jira] [Commented] (KAFKA-562) Non-failure System Test Log Segment File Checksums mismatched

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473719#comment-13473719 ] Jay Kreps commented on KAFKA-562: - So the proposed fix for this is to special case appends

[jira] [Commented] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473790#comment-13473790 ] Jay Kreps commented on KAFKA-557: - 1.1 Fixed 1.2 The long lines are due to use of

[jira] [Updated] (KAFKA-557) Replica fetch thread doesn't need to recompute message id

2012-10-10 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-557: Attachment: KAFKA-557.patch Replica fetch thread doesn't need to recompute message id

[jira] [Commented] (KAFKA-551) Log.truncateTo() may need to trucate immutable log segment

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472460#comment-13472460 ] Jay Kreps commented on KAFKA-551: - I think (hope) that is what we are doing. Here is the

[jira] [Created] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand

2012-10-09 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-554: --- Summary: Move all per-topic configuration into ZK and add to the CreateTopicCommand Key: KAFKA-554 URL: https://issues.apache.org/jira/browse/KAFKA-554 Project: Kafka

[jira] [Updated] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-554: Issue Type: New Feature (was: Bug) Move all per-topic configuration into ZK and add to the

[jira] [Created] (KAFKA-555) Add a key-based log retention strategy

2012-10-09 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-555: --- Summary: Add a key-based log retention strategy Key: KAFKA-555 URL: https://issues.apache.org/jira/browse/KAFKA-555 Project: Kafka Issue Type: New Feature

[jira] [Resolved] (KAFKA-551) Log.truncateTo() may need to trucate immutable log segment

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps resolved KAFKA-551. - Resolution: Fixed Assignee: Jay Kreps Committed. Log.truncateTo() may need to

[jira] [Updated] (KAFKA-543) Metadata request from DefaultEventHandler.handle repeats same topic over and over

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-543: Attachment: KAFKA-543-v2.patch Hey Neha, I actually kind of agree with you. We need to dedupe on the server

[jira] [Created] (KAFKA-559) Garbage collect old consumer metadata entries

2012-10-09 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-559: --- Summary: Garbage collect old consumer metadata entries Key: KAFKA-559 URL: https://issues.apache.org/jira/browse/KAFKA-559 Project: Kafka Issue Type: New Feature

[jira] [Updated] (KAFKA-560) Garbage Collect obsolete topics

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-560: Description: Old junk topics tend to accumulate over time. Code may migrate to use new topics leaving the

[jira] [Created] (KAFKA-560) Garbage Collect obsolete topics

2012-10-09 Thread Jay Kreps (JIRA)
Jay Kreps created KAFKA-560: --- Summary: Garbage Collect obsolete topics Key: KAFKA-560 URL: https://issues.apache.org/jira/browse/KAFKA-560 Project: Kafka Issue Type: New Feature

[jira] [Commented] (KAFKA-502) Simplify setup / initialization in replication_basic_test.py

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472667#comment-13472667 ] Jay Kreps commented on KAFKA-502: - Sorry about that, should have given a heads up on that

[jira] [Commented] (KAFKA-537) expose clientId and correlationId in ConsumerConfig

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13472678#comment-13472678 ] Jay Kreps commented on KAFKA-537: - The correlation id is meant to be a per-connection,

[jira] [Updated] (KAFKA-556) Change MessageSet.sizeInBytes to Int

2012-10-09 Thread Jay Kreps (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Kreps updated KAFKA-556: Priority: Major (was: Blocker) Assignee: Jay Kreps Summary: Change MessageSet.sizeInBytes to Int

  1   2   3   >