[
https://issues.apache.org/jira/browse/KAFKA-521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Kreps resolved KAFKA-521.
-----------------------------
Resolution: Fixed
Fixed scaladoc and committed. This also fixes the transient failures in
LogManagerTest, I believe, though the test still contains timing assumptions.
> Refactor Log subsystem
> ----------------------
>
> Key: KAFKA-521
> URL: https://issues.apache.org/jira/browse/KAFKA-521
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jay Kreps
> Attachments: KAFKA-521-v1.patch, KAFKA-521-v2.patch,
> KAFKA-521-v3.patch, KAFKA-521-v4.patch, KAFKA-521-v5.patch
>
>
> There are a number of items it would be nice to cleanup in the log subsystem:
> 1. Misc. funky apis in Log and LogManager
> 2. Much of the functionality in Log should move into LogSegment along with
> corresponding tests
> 3. We should remove SegmentList and instead use a ConcurrentSkipListMap
> The general idea of the refactoring fall into two categories. First, improve
> and thoroughly document the public APIs. Second, have a clear delineation of
> responsibility between the various layers:
> 1. LogManager is responsible for the creation and deletion of logs as well as
> the retention of data in log segments. LogManager is the only layer aware of
> partitions and topics. LogManager consists of a bunch of individual Log
> instances and interacts with them only through their public API (mostly true
> today).
> 2. Log represents a totally ordered log. Log is responsible for reading,
> appending, and truncating the log. A log consists of a bunch of LogSegments.
> Currently much of the functionality in Log should move into LogSegment with
> Log interacting only through the Log interface. Currently we reach around
> this a lot to call into FileMessageSet and OffsetIndex.
> 3. A LogSegment consists of an OffsetIndex and a FileMessageSet. It supports
> largely the same APIs as Log, but now localized to a single segment.
> This cleanup will simplify testing and debugging because it will make the
> responsibilities and guarantees at each layer more clear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
