lkindere opened a new issue, #12549:
URL: https://github.com/apache/iceberg/issues/12549
### Query engine
Kafka Connect IcebergSinkConnector
### Question
Hello,
Every so often I noticed that the IcebergSinkConnector is missing some
records which can be seen when when querying the iceberg tables.
For example we store the Kafka partition and Kafka offset in the tables for
debugging.
And with this query:
SELECT * FROM <table>
WHERE
__kafka_metadata.__kafka_partition = 1
AND
__kafka_metadata.__kafka_offset >= 175807
AND
__kafka_metadata.__kafka_offset <= 175809
We would expect to see 3 rows, with offsets 175807, 175808 and 175809,
however the record with the offset 175808 is missing and we only see 175807 and
175809.
Checking the logs by ingestion time we do not see any abnormalities or
errors.
Has anyone run into something similar before and knows what may be the cause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
