Probably not a good thread to ask, but encounter the issue again during
verification of RC so asking here:
I'm consistently encountering multiple test failures due to HMS. It
shouldn't matter as others verified the UTs, but if someone is aware of the
issue and the resolution (or at least where to look at), shall I be OK to
ask it? I suspect messing up HMS between multiple usages (possibly) but
don't know why the UT got affected even after `./gradlew clean`.
Caused by:
> java.lang.RuntimeException: Metastore operation failed for
> db.test_table
> at
> org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:194)
> at
> org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:118)
> at
> org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:225)
> at
> org.apache.iceberg.BaseMetastoreCatalog.createTable(BaseMetastoreCatalog.java:58)
> at
> org.apache.iceberg.CachingCatalog.lambda$createTable$0(CachingCatalog.java:103)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2337)
> at
> java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2335)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2318)
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:111)
> at
> com.github.benmanes.caffeine.cache.LocalManualCache.get(LocalManualCache.java:54)
> at
> org.apache.iceberg.CachingCatalog.createTable(CachingCatalog.java:101)
> at
> org.apache.iceberg.flink.FlinkCatalog.createTable(FlinkCatalog.java:360)
> at
> org.apache.flink.table.catalog.CatalogManager.lambda$createTable$10(CatalogManager.java:632)
> at
> org.apache.flink.table.catalog.CatalogManager.execute(CatalogManager.java:791)
> ... 5 more
>
> Caused by:
> org.apache.thrift.TApplicationException: Internal error
> processing lock
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:4678)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:4665)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:2162)
> at
> org.apache.iceberg.hive.HiveTableOperations.lambda$acquireLock$5(HiveTableOperations.java:311)
> at
> org.apache.iceberg.hive.ClientPool.run(ClientPool.java:54)
> at
> org.apache.iceberg.hive.HiveTableOperations.acquireLock(HiveTableOperations.java:311)
> at
> org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:152)
> ... 19 more
>
Other than that I verified GPG and SHA, and extracted the archive.
On Mon, Nov 2, 2020 at 1:04 PM Simon Su <barley0...@gmail.com> wrote:
> + 1 (non-binding)
> 1. Build code pass all UTs.
> 2. Test Flink iceberg sink failover, test exactly-once.
>
> Junjie Chen <chenjunjied...@gmail.com> 于2020年11月2日周一 上午11:35写道：
>
>> + 1 (non-binding)
>>
>> I ran step 1-7 on my cloud virtual machine (centos 7, java 1.8.0_171),
>> all passed.
>>
>>
>>
>>
>>
>> On Mon, Nov 2, 2020 at 10:36 AM OpenInx <open...@gmail.com> wrote:
>>
>>> +1 for 0.10.0 RC2
>>>
>>> 1. Download the source tarball, signature (.asc), and checksum
>>> (.sha512): OK
>>> 2. Import gpg keys: download KEYS and run gpg --import
>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK
>>> 3. Verify the signature by running: gpg --verify
>>> apache-iceberg-xx-incubating.tar.gz.asc: OK
>>> 4. Verify the checksum by running: sha512sum -c
>>> apache-iceberg-xx-incubating.tar.gz.sha512 : OK
>>> 5. Untar the archive and go into the source directory: tar xzf
>>> apache-iceberg-xx-incubating.tar.gz && cd apache-iceberg-xx-incubating: OK
>>> 6. Run RAT checks to validate license headers: dev/check-license: OK
>>> 7. Build and test the project: ./gradlew build (use Java 8) : OK
>>>
>>> BTW, I think we may need a bash script to do the above verification
>>> automatically , so I created an issue for it:
>>> https://github.com/apache/iceberg/issues/1700
>>>
>>> Thanks all for the work.
>>>
>>> On Sun, Nov 1, 2020 at 5:40 AM Dongjoon Hyun <dongjoon.h...@gmail.com>
>>> wrote:
>>>
>>>> +1 for 0.10.0 RC2 (non-binding)
>>>>
>>>> I checked the followings.
>>>> 1. Checksum and GPG signatures
>>>> 2. Gradle build and tests on Java 1.8.0_272
>>>> 3. Manual integration tests with Hive Metastore 2.3.7 and Apache Spark
>>>> 2.3.7/3.0.1.
>>>>
>>>> Thank you!
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Fri, Oct 30, 2020 at 3:25 PM Russell Spitzer <
>>>> russell.spit...@gmail.com> wrote:
>>>>
>>>>> +1 (non-binding) Downloaded and ran build with Java HotSpot(TM) 64-Bit
>>>>> Server VM 18.9 (build 11.0.7+8-LTS, mixed mode). All tests passed :)
>>>>>
>>>>> On Fri, Oct 30, 2020 at 4:05 PM Anton Okolnychyi
>>>>> <aokolnyc...@apple.com.invalid> wrote:
>>>>>
>>>>>> Here is the link to steps we normally use to validate a release
>>>>>> candidate:
>>>>>>
>>>>>> https://lists.apache.org/thread.html/rd5e6b1656ac80252a9a7d473b36b6227da91d07d86d4ba4bee10df66%40%3Cdev.iceberg.apache.org%3E
>>>>>> <https://lists.apache.org/thread.html/rd5e6b1656ac80252a9a7d473b36b6227da91d07d86d4ba4bee10df66@%3Cdev.iceberg.apache.org%3E>
>>>>>>
>>>>>> - Anton
>>>>>>
>>>>>> On 30 Oct 2020, at 14:03, Anton Okolnychyi <
>>>>>> aokolnyc...@apple.com.INVALID> wrote:
>>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I propose the following RC to be released as official Apache Iceberg
>>>>>> 0.10.0 release.
>>>>>>
>>>>>> The commit id is 37f21b72fb55503e6e40b1555b7ea1af61dfdfc7
>>>>>> * This corresponds to the tag: apache-iceberg-0.10.0-rc2
>>>>>> * https://github.com/apache/iceberg/commits/apache-iceberg-0.10.0-rc2
>>>>>> *
>>>>>> https://github.com/apache/iceberg/tree/37f21b72fb55503e6e40b1555b7ea1af61dfdfc7
>>>>>>
>>>>>> The release tarball, signature, and checksums are here:
>>>>>> *
>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-0.10.0-rc2/
>>>>>>
>>>>>> You can find the KEYS file here (make sure to import the new key that
>>>>>> was used to sign the release):
>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS
>>>>>>
>>>>>> Convenience binary artifacts are staged in Nexus. The Maven
>>>>>> repository URL is:
>>>>>> *
>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1011/
>>>>>>
>>>>>> This release includes important changes:
>>>>>>
>>>>>> * Flink support
>>>>>> * Hive read support
>>>>>> * ORC support fixes and improvements
>>>>>> * Application of row-level delete files on read
>>>>>> * Snapshot partition summary
>>>>>> * Ability to load LocationProvider dynamically
>>>>>> * Sort spec
>>>>>>
>>>>>> Please download, verify, and test.
>>>>>>
>>>>>> Please vote in the next 72 hours.
>>>>>>
>>>>>> [ ] +1 Release this as Apache Iceberg 0.10.0
>>>>>> [ ] +0
>>>>>> [ ] -1 Do not release this because…
>>>>>>
>>>>>> Thanks,
>>>>>> Anton
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>
>> --
>> Best Regards
>>
>
