tranhan02 opened a new issue, #12547:
URL: https://github.com/apache/iceberg/issues/12547
I am using kafka-connect-iceberg to connect to an Iceberg catalog backed by
Hive Metastore (HMS) with SSL enabled. Below is my connector configuration:
```
class: io.tabular.iceberg.connect.IcebergSinkConnector
config:
topics: test123
iceberg.catalog.type: hive
iceberg.catalog.uri: 'thrift://...:9083'
iceberg.catalog.warehouse: 's3a://...'
iceberg.catalog.io-impl: org.apache.iceberg.aws.s3.S3FileIO
iceberg.catalog.s3.endpoint: '...'
iceberg.catalog.s3.access-key-id: admin
iceberg.catalog.s3.secret-access-key: admin
iceberg.control.commit.timeout-ms: '10000'
iceberg.tables.auto-create-enabled: 'true'
iceberg.catalog: icebergtest
iceberg.tables: "test.test123"
iceberg.tables.evolve-schema-enabled: 'true'
iceberg.control.commit.interval-ms: '50000'
iceberg.catalog.client.region: us-east-1
iceberg.tables.upsert-mode-enabled: 'false'
iceberg.catalog.s3.path-style-access: 'true'
key.converter: org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable: 'false'
value.converter: org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable: 'false'
iceberg.catalog.hive.metastore.use.SSL: true
iceberg.catalog.hive.metastore.truststore.type: 'PKCS12'
iceberg.catalog.hive.metastore.truststore.path: '/opt/truststore.p12'
iceberg.catalog.hive.metastore.truststore.password: 'test'
iceberg.catalog.hive.metastore.client.auth.mode: 'PLAIN'
iceberg.catalog.hive.metastore.client.plain.username: 'abc'
iceberg.catalog.hive.metastore.client.plain.password: 'abc'
```
When running the connector, I encounter the following error:
```
2025-03-17 04:36:13,673 INFO [test|task-0] Found configuration file null
(org.apache.hadoop.hive.conf.HiveConf) [task-thread-test-0]
2025-03-17 04:36:14,292 WARN [test|task-0] Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
(org.apache.hadoop.util.NativeCodeLoader) [task-thread-test-0]
2025-03-17 04:36:14,306 INFO [test|task-0] Trying to connect to metastore
with URI thrift://192.168.11.11:1111/ (hive.metastore) [task-thread-test-0]
2025-03-17 04:36:14,328 INFO [test|task-0] Opened a connection to metastore,
current connections: 1 (hive.metastore) [task-thread-test-0]
2025-03-17 04:36:14,462 WARN [test|task-0] set_ugi() not successful, Likely
cause: new client talking to old server. Continuing without it.
(hive.metastore) [task-thread-test-0]
org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
2025-03-17 04:36:15,746 ERROR [test|task-0] WorkerSinkTask{id=test-0} Task
threw an uncaught and unrecoverable exception. Task is being killed and will
not recover until manually restarted
(org.apache.kafka.connect.runtime.WorkerTask) [task-thread-test-0]
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due
to unrecoverable exception.
at
org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:614)
at
org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:336)
Caused by: java.lang.RuntimeException: Failed to get table info from
metastore test.test123
at
org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:159)
at
org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:97)
at
org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:80)
at
org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:49)
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
```
Has anyone encountered this issue before? If so, how did you resolve it? I
would really appreciate any guidance or suggestions to help me fix this
problem. Thank you in advance for your support!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
