[
https://issues.apache.org/jira/browse/KAFKA-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Koshy updated KAFKA-391:
-----------------------------
Attachment: KAFKA-391-v3.patch
Overview of changes in v3:
(3.1) - (20) - This actually caused bulk of the changes in v3. I did this
for just the topic-partition pairs in producer/consumer request handling;
same for producer response status. There are a lot more places where we
could move from tuples to case classes. I think (as mentioned on the mailing
list) it would be good to do this as part of cleanup but we should defer
that for later since such changes cut across a lot of files. Going forward I
think this brings out a convention that we might want to follow. The "scala
book" has a reasonable guideline. I'll send out an email to kafka-dev for
discussion and add it to our coding convention depending on how that pans
out.
(3.2) (21)-(23) - Thanks for catching the javaapi issue. Couple of changes
here:
a - Added javaapi for FetchRequest. (I needed to provide both java/non-java
fetchrequest to the simpleconsumer since FetchBuilder returns a scala
FetchRequest.)
b - Java map for all the Fetch/Produce request/response in the javaapi.
c - Removed SyncProducer: agreed that is unnecessary since Producer supports
both sync and async; made the hadoop producer code use the high level
producer. I think that's safe - i.e., I don't see a good reason why anyone
would "depend" on the data going to a single broker.
d - Got rid of the unreferenced ProducerConsumerTestHarness in javaapi.
e - Fixed the equals method in javaapi ProducerRequest; added one to
FetchResponse - actually we can abstract this out into a trait, but that is
a minor improvement that I punted on.
f - Made the ProducerRequest use Java map in javaapi.
g - (I did not add Java versions of ProducerResponse since the SyncProducer
has been removed.)
(3.3) (24) - added the helper method, although I don't think I'm using it
anywhere.
(3.4) - Got rid of OffsetDetail.
For (25)(26) - I tried to use the pattern wherever I could, but may have
missed a few.
I did not rebase this (i.e., it still applies on r1381858). I'll rebase
after we are done with reviews.
> Producer request and response classes should use maps
> -----------------------------------------------------
>
> Key: KAFKA-391
> URL: https://issues.apache.org/jira/browse/KAFKA-391
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Joel Koshy
> Priority: Blocker
> Labels: optimization
> Fix For: 0.8
>
> Attachments: KAFKA-391-draft-r1374069.patch, KAFKA-391-v2.patch,
> KAFKA-391-v3.patch
>
>
> Producer response contains two arrays of error codes and offsets - the
> ordering in these arrays correspond to the flattened ordering of the request
> arrays.
> It would be better to switch to maps in the request and response as this
> would make the code clearer and more efficient (right now, linear scans are
> used in handling producer acks).
> We can probably do the same in the fetch request/response.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
