Sushanth Sowmyan created HIVE-9381:
--------------------------------------
Summary: HCatalog hardcodes maximum append limit to 1000.
Key: HIVE-9381
URL: https://issues.apache.org/jira/browse/HIVE-9381
Project: Hive
Issue Type: Bug
Components: HCatalog
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
When HCatalog appends to a table, it tries to avoid fileclashes while moving by
appending a "_a_NNN" suffix to a file if there is a clash, where the NNN is a
number from 0 to 999.
This limit is hardcoded and this causes issues with some users who have tables
with a large number of files. Ideally, we should push back on those users to
change their usecase, or run some manner of compactions on their table (a
CREATE TABLE AS SELECT from this table to another table, followed by a drop and
re creating as select from the other table would suffice, for example). But,
for users where those are not viable solutions, we need to help them get
unstuck. One way is by making that hardcoded parameter a configuration
parameter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
