vl-kp commented on PR #12609:
URL: https://github.com/apache/iceberg/pull/12609#issuecomment-2760257234
> I was indicating to that only, if that log is not showing up means you
consumer group in not in stable state and it is in dead state and you are
definitely in a no co-ordinator scenario, using which you can debug.
I believe not every iceberg connector user could know this as when i first
use it, it makes me confuse as there is no warning or error why the connector
didn't save the metadata file even the data was saved by worker
> Currently we are retrieving the members only when the group is stable and
with this change it will always check the members.
> Also if we have configured the connect-group-id properly it will
ultimately lead to co-ordinator election and we will end up with the logs.
if there is concern of always check the members, could we send a warning log
before return false, at least it could lead the user to catch the problem,
like
```
LOG.warning(
"Consumer group {} is in {} state with {} members - waiting for
group to stabilize",
config.connectGroupId(),
groupDesc.state(),
members.size());
return false;
```
> I can see from the issue you referred
https://github.com/databricks/iceberg-kafka-connect/issues/245, you are using
Kafka-connect from data bricks.
This is just a reference as this issue happens in both repo
https://github.com/databricks/iceberg-kafka-connect/blob/main/kafka-connect/src/main/java/io/tabular/iceberg/connect/channel/CoordinatorThreadFactoryImpl.java#L72-L90
This is the issue https://github.com/apache/iceberg/issues/11703 reported by
someone else here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
