I have noticed some flakiness with Flink and Spark tests both locally and
in CI checks. @zhangjun0x01 also reported the same problem with
iceberg-spark3-extensions. Below is a full stack trace from a local run
for Flink tests.
The flakiness might be recent regression, as the tests were stable for me
until recently. Any recent hive dep change? Anyone have any ideas?
org.apache.iceberg.flink.source.TestIcebergSourceReaderDeletes >
testMixedPositionAndEqualityDeletes[fileFormat=ORC] FAILED
java.lang.RuntimeException: Failed to get table info from metastore
default.test
at
org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:142)
at
org.apache.iceberg.BaseMetastoreTableOperations.refresh(BaseMetastoreTableOperations.java:86)
at
org.apache.iceberg.BaseMetastoreTableOperations.current(BaseMetastoreTableOperations.java:69)
at
org.apache.iceberg.BaseMetastoreCatalog.loadTable(BaseMetastoreCatalog.java:92)
at
org.apache.iceberg.flink.TableLoader$CatalogTableLoader.loadTable(TableLoader.java:113)
at
org.apache.iceberg.flink.source.TestIcebergSourceReaderDeletes.rowSet(TestIcebergSourceReaderDeletes.java:90)
Caused by:
org.apache.thrift.transport.TTransportException:
java.net.SocketException: Broken pipe (Write failed)
at
org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:161)
at
org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at
org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_table_req(ThriftHiveMetastore.java:1561)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:1553)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1350)
at
org.apache.iceberg.hive.HiveTableOperations.lambda$doRefresh$0(HiveTableOperations.java:130)
at org.apache.iceberg.hive.ClientPool.run(ClientPool.java:65)
at
org.apache.iceberg.hive.HiveTableOperations.doRefresh(HiveTableOperations.java:130)
... 5 more
Caused by:
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at
java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at
java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at
org.apache.thrift.transport.TIOStreamTransport.flush(TIOStreamTransport.java:159)
... 13 more
