Re: fails to alter table concatenate

[patcharee]

Actually it works on mr. So the problem is from tez. thanks!

BR,
Patcharee

On 30. juni 2015 10:23, Nitin Pawar wrote:

can you try doing same by changing the query engine from tez to mr1? not sure if its hive bug or tez bug

On Tue, Jun 30, 2015 at 1:46 PM, patcharee <patcharee.thong...@uni.no <mailto:patcharee.thong...@uni.no>> wrote:

    Hi,

    I am using hive 0.14. It fails to alter table concatenate
    occasionally (see the exception below). It is strange that it
    fails from time to time not predictable. Is there any suggestion/clue?

    hive> alter table 4dim partition(zone=2,z=15,year=2005,month=4)
    CONCATENATE;
    
--------------------------------------------------------------------------------
            VERTICES      STATUS  TOTAL  COMPLETED  RUNNING PENDING
    FAILED  KILLED
    
--------------------------------------------------------------------------------

File Merge FAILED -1 0 0 -1 0 0

    
--------------------------------------------------------------------------------
    VERTICES: 00/01  [>>--------------------------] 0% ELAPSED TIME:
    1435651968.00 s
    
--------------------------------------------------------------------------------
    Status: Failed
    Vertex failed, vertexName=File Merge,
    vertexId=vertex_1435307579867_0041_1_00, diagnostics=[Vertex
    vertex_1435307579867_0041_1_00 [File Merge] killed/failed due
    to:ROOT_INPUT_INIT_FAILURE, Vertex Input:
    
[hdfs://service-10-0.local:8020/apps/hive/warehouse/wrf_tables/4dim/zone=2/z=15/year=2005/month=4]
    initializer failed, vertex=vertex_1435307579867_0041_1_00 [File
    Merge], java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.io
    
<http://org.apache.hadoop.hive.ql.io>.HiveInputFormat.init(HiveInputFormat.java:265)
        at org.apache.hadoop.hive.ql.io
    
<http://org.apache.hadoop.hive.ql.io>.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:452)
        at
    
org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:441)
        at
    
org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:295)
        at
    
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:124)
        at
    
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245)
        at
    
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at
    
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at
    
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239)
        at
    
org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at
    
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
    
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
    ]
    DAG failed due to vertex failure. failedVertices:1 killedVertices:0
    FAILED: Execution Error, return code 2 from
    org.apache.hadoop.hive.ql.exec.DDLTask

    BR,
    Patcharee




--
Nitin Pawar