[ https://issues.apache.org/jira/browse/KAFKA-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927449#comment-15927449 ]
Grant Henke commented on KAFKA-4906: ------------------------------------ [~ijuma] Following up with a summary of some of our out of band chats here. In smoke testing of a WIP patch it appeared I was able to send messages from a trunk client to a 0.9 broker and receive the from a trunk consumer. We were a bit confused by this since the message format had changed and should not be parsable. I think since I was using uncompressed messages and a regular topic, the message could pass through without the format really being parsed or validated. However, that is likely not the case for a compacted topic or a compressed message set. More testing would be needed to be sure. Regardless the safest approach would likely be to ensure the message format matches the producer message version. (Produce v1 = Message Format 0, and Produce V2 = Message Format 1). I will investigate further and see how large of a change is required before posting anything further to do that. > Support 0.9 brokers with a newer Producer or Consumer version > ------------------------------------------------------------- > > Key: KAFKA-4906 > URL: https://issues.apache.org/jira/browse/KAFKA-4906 > Project: Kafka > Issue Type: Improvement > Components: clients > Affects Versions: 0.10.2.0 > Reporter: Grant Henke > Assignee: Grant Henke > > KAFKA-4507 added the ability for newer Kafka clients to talk to older Kafka > brokers if a new feature supported by a newer wire protocol was not > used/required. > We currently support brokers as old as 0.10.0.0 because thats when the > ApiVersionsRequest/Response was added to the broker (KAFKA-3307). > However, there are relatively few changes between 0.9.0.0 and 0.10.0.0 on the > wire, making it possible to support another major broker version set by > assuming that any disconnect resulting from an ApiVersionsRequest is from a > 0.9 broker and defaulting to legacy protocol versions. > Supporting 0.9 with newer clients can drastically simplify upgrades, allow > for libraries and frameworks to easily support a wider set of environments, > and let developers take advantage of client side improvements without > requiring cluster upgrades first. > Below is a list of the wire protocol versions by release for reference: > {noformat} > 0.10.x > Produce(0): 0 to 2 > Fetch(1): 0 to 2 > Offsets(2): 0 > Metadata(3): 0 to 1 > OffsetCommit(8): 0 to 2 > OffsetFetch(9): 0 to 1 > GroupCoordinator(10): 0 > JoinGroup(11): 0 > Heartbeat(12): 0 > LeaveGroup(13): 0 > SyncGroup(14): 0 > DescribeGroups(15): 0 > ListGroups(16): 0 > SaslHandshake(17): 0 > ApiVersions(18): 0 > 0.9.x: > Produce(0): 0 to 1 (no response timestamp from v2) > Fetch(1): 0 to 1 (no response timestamp from v2) > Offsets(2): 0 > Metadata(3): 0 (no cluster id or rack info from v1) > OffsetCommit(8): 0 to 2 > OffsetFetch(9): 0 to 1 > GroupCoordinator(10): 0 > JoinGroup(11): 0 > Heartbeat(12): 0 > LeaveGroup(13): 0 > SyncGroup(14): 0 > DescribeGroups(15): 0 > ListGroups(16): 0 > SaslHandshake(17): UNSUPPORTED > ApiVersions(18): UNSUPPORTED > 0.8.2.x: > Produce(0): 0 (no quotas from v1) > Fetch(1): 0 (no quotas from v1) > Offsets(2): 0 > Metadata(3): 0 > OffsetCommit(8): 0 to 1 (no global retention time from v2) > OffsetFetch(9): 0 to 1 > GroupCoordinator(10): 0 > JoinGroup(11): UNSUPPORTED > Heartbeat(12): UNSUPPORTED > LeaveGroup(13): UNSUPPORTED > SyncGroup(14): UNSUPPORTED > DescribeGroups(15): UNSUPPORTED > ListGroups(16): UNSUPPORTED > SaslHandshake(17): UNSUPPORTED > ApiVersions(18): UNSUPPORTED > {noformat} > Note: Due to KAFKA-3088 it may take up to request.timeout.time to fail an > ApiVersionsRequest and failover to legacy protocol versions unless we handle > that scenario specifically in this patch. The workaround would be to reduce > request.timeout.time if needed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)