test flakiness with SocketException of broken pipe in HiveMetaStoreClient

2021-01-06 Thread Steven Wu
I have noticed some flakiness with Flink and Spark tests both locally and in CI checks. @zhangjun0x01 also reported the same problem with iceberg-spark3-extensions. Below is a full stack trace from a local run for Flink tests. The flakiness might be recent regression, as the tests were stable for

Re: test flakiness with SocketException of broken pipe in HiveMetaStoreClient

2021-01-06 Thread Ryan Blue
I've noticed this too. I haven't had a chance to track down what's causing it yet. I've seen it in Spark tests, so it looks like there may be a problem that affects both. Probably a connection leak in the common code. On Wed, Jan 6, 2021 at 3:44 PM Steven Wu wrote: > I have noticed some flakines

Re: test flakiness with SocketException of broken pipe in HiveMetaStoreClient

2021-01-06 Thread OpenInx
I encountered a similar issue when supporting hive-site.xml for flink hive catalog. Here is the discussion and solution before: https://github.com/apache/iceberg/pull/1586#discussion_r509453461 It's a connection leak issue. On Thu, Jan 7, 2021 at 10:06 AM Ryan Blue wrote: > I've noticed this