Hi devs,

I'm trying to replace deprecated poll(long) with poll(Duration), and
realized there's no alternative which behaves exactly same as poll(0), as
poll(0) has been used as a hack to only update metadata instead of pulling
records. poll(Duration.ZERO) wouldn't behave same since even updating
metadata will be timed-out. So now end users would need to give more
timeout and even pull some records even they're only interested in metadata.

I looked back some KIPs which brought the change, and "discarded" KIP
(KIP-288 [1]) actually proposed a new API which only pulls metadata.
KIP-266 [2] is picked up instead but it didn't cover all the things what
KIP-288 proposed. I'm seeing some doc explaining poll(0) hasn't been
supported officially, but the hack has been widely used and they can't be
ignored.

Kafka test code itself relies on either deprecated poll(0),
or updateAssignmentMetadataIfNeeded, which seems to be private API only for
testing.
(Btw, I'd try out replacing poll(0) to updateAssignmentMetadataIfNeeded as
avoiding deprecated method - if it works I'll submit a PR.)

I'm feeling that it would be ideal to expose
`updateAssignmentMetadataIfNeeded` to the public API, maybe with renaming
as `waitForAssignment` which was proposed in KIP-288 if it feels too long.

What do you think? If it sounds feasible I'd like to try out contribution
on this. I'm new to contribute Kafka community, so not sure it would
require a new KIP or not.

Thanks,
Jungtaek Lim (HeartSaVioR)

1.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-288%3A+%5BDISCARDED%5D+Consumer.poll%28%29+timeout+semantic+change+and+new+waitForAssignment+method
2.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior

Reply via email to