-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22766/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-7218
https://issues.apache.org/jira/browse/HIVE-7218
Repository: hive-git
Description
-------
HIVE-2372 was originally created in response to this error message, however
that patch was merely a work-around to handle the condition where
mapred.input.dir is too long.
Any other environment variable that is "too long" for the host OS will still
cause a job failure.
In my case:
While creating a table with a large number of columns, a large hive variable is
temporarily created using SET, the variable contains the columns and column
descriptions.
A CREATE TABLE statement then successfully uses that large variable.
After successfully creating the table the hive script attempts to load data
into the table using a TRANSFORM script, triggering the error:
java.io.IOException: error=7, Argument list too long
Since the variable is no longer used after the table is created, the hive
script was updated to SET the large variable to empty.
After setting the variable empty the second statement in the hive script ran
fine.
Hive should more gracefully notify the user as to the cause of the problem and
offer a configurable approach for automatically handling the condition.
In this case, originally identifying the cause of the issue was somewhat
confusing since the portion of the hive script that referenced the long
variable ran successfully, and the portion of the script that failed didn't
even use/reference the variable that was causing that portion to fail.
Since HIVE-2372 has already been "Fixed" this JIRA re-opens the issue since the
original issue was worked around, not resolved...
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/ScriptOperator.java 5b857e2
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 75394f3
ql/src/java/org/apache/hadoop/hive/ql/parse/SelectClauseParser.g 1855d7f
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 6cdaedb
ql/src/java/org/apache/hadoop/hive/ql/plan/ScriptDesc.java 4f7c0da
ql/src/test/queries/clientpositive/script_env_var3.q PRE-CREATION
ql/src/test/results/clientpositive/script_env_var3.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/22766/diff/
Testing
-------
Thanks,
Navis Ryu
