Eugene Koifman created HIVE-21146:
-------------------------------------
Summary: Enforce TransactionBatch size=1 for blob stores
Key: HIVE-21146
URL: https://issues.apache.org/jira/browse/HIVE-21146
Project: Hive
Issue Type: Bug
Components: Streaming, Transactions
Affects Versions: 3.0.0
Reporter: Eugene Koifman
Streaming Ingest API supports a concept of {{TransactionBatch}} where N
transactions can be opened at once and the data in all of them will be written
to the same delta_x_y directory where each transaction in the batch can be
committed/aborted independently. The implementation relies on
{{FSDataOutputStream.hflush()}} (called from OrcRecordUpdater}} which is
available on HDFS but is often implemented as no-op in Blob store backed
{{FileSystem}} objects.
Need to add a check to {{HiveStreamingConnection()}} constructor to raise an
error if {{builder.transactionBatchSize > 1}} and the target table/partitions
are backed by something that doesn't support {{hflush()}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
