Date: Mon, Feb 1, 2010 at 11:02 AM
Subject: Re: HIVE-74 and CombineFileInputFormat on pre-0.20 hadoop
To: Namit Jain
Cc: "hive-u...@hadoop.apache.org"
Reviving this old thread...just found the time to work on this...
I have a patch for using MultiFIleInputFormat in hado
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HIVE-74:
---
Description:
There are cases when the input to a Hive job are thousands of small files. In
this case, there is
Job = job_200909301537_0068
> with
> > errors
> > 2009-10-01 10:40:58,622 ERROR ql.Driver
> (SessionState.java:printError(248))
> > - FAILED: Execution Error, return code 2 from
> > org.apache.hadoop.hive.ql.exec.ExecDriver
> >
> >
> >
> > On Wed, S
seems OK ?
>> Can you get the stack trace from /tmp//hive.log ?
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Matt Pestritto [mailto:m...@pestritto.com]
>> Sent: Wednesday, September 30, 2009 6:51 AM
>> To: hive-dev@hadoop.
log ?
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Matt Pestritto [mailto:m...@pestritto.com]
>> Sent: Wednesday, September 30, 2009 6:51 AM
>> To: hive-dev@hadoop.apache.org; hive-u...@hadoop.apache.org
>> Subject: Fwd: Hive-74
k trace from /tmp//hive.log ?
>
>
>
>
>
> -Original Message-
> From: Matt Pestritto [mailto:m...@pestritto.com]
> Sent: Wednesday, September 30, 2009 6:51 AM
> To: hive-dev@hadoop.apache.org; hive-u...@hadoop.apache.org
> Subject: Fwd: Hive-74
>
> Including
What you are doing seems OK ?
Can you get the stack trace from /tmp//hive.log ?
-Original Message-
From: Matt Pestritto [mailto:m...@pestritto.com]
Sent: Wednesday, September 30, 2009 6:51 AM
To: hive-dev@hadoop.apache.org; hive-u...@hadoop.apache.org
Subject: Fwd: Hive-74
Including
Including hive-user in case someone has any experience with this..
Thanks
-Matt
-- Forwarded message --
From: Matt Pestritto
Date: Tue, Sep 29, 2009 at 5:26 PM
Subject: Hive-74
To: hive-dev@hadoop.apache.org
Hi-
I'm having a problem using CombineHiveInputSplit. I believe
Hi-
I'm having a problem using CombineHiveInputSplit. I believe this was
patched in http://issues.apache.org/jira/browse/HIVE-74
I'm currently running hadoop 20.1 using hive trunk.
hive-default.xml has the following property:
hive.input.format
The default input format, if
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raghotham Murthy updated HIVE-74:
-
Resolution: Fixed
Fix Version/s: (was: 0.4.0)
0.5.0
Release Note
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755087#action_12755087
]
Raghotham Murthy commented on HIVE-74:
--
looks good. will commit once tests pass.
&g
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754825#action_12754825
]
Namit Jain commented on HIVE-74:
@Raghu, verified that the file CombineFileRecordReader.
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-74:
---
Attachment: hive.74.2.patch
> Hive can use CombineFileInputFormat for when the input are many small fi
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753928#action_12753928
]
Namit Jain commented on HIVE-74:
talked with Raghu offline - will load a new patch a
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12753791#action_12753791
]
Raghotham Murthy commented on HIVE-74:
--
Why does Hadoop18Shims.java have
{
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-74:
---
Attachment: hive.74.1.patch
> Hive can use CombineFileInputFormat for when the input are many small fi
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-74:
---
Status: Patch Available (was: Open)
> Hive can use CombineFileInputFormat for when the input are many sm
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-74:
---
Status: Open (was: Patch Available)
> Hive can use CombineFileInputFormat for when the input are many sm
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-74:
---
Status: Patch Available (was: Open)
I was busy with some other things - will get back to it soon
> Hive
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749152#action_12749152
]
Jeff Hammerbacher commented on HIVE-74:
---
What is the status of this patch?
> H
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain reassigned HIVE-74:
--
Assignee: Namit Jain (was: dhruba borthakur)
> Hive can use CombineFileInputFormat for when the in
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701036#action_12701036
]
dhruba borthakur edited comment on HIVE-74 at 4/20/09 8:4
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dhruba borthakur updated HIVE-74:
-
Attachment: hiveCombineSplit2.patch
This combines multiple blocks from files into a single split
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johan Oskarsson updated HIVE-74:
Fix Version/s: (was: 0.2.0)
0.4.0
> Hive can use CombineFileInputFormat
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674769#action_12674769
]
Joydeep Sen Sarma commented on HIVE-74:
---
where are the pools for
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674742#action_12674742
]
Joydeep Sen Sarma commented on HIVE-74:
---
Is it possible to do this in a way that
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dhruba borthakur updated HIVE-74:
-
Attachment: hiveCombineSplit.patch
Allow Hive to use CombineInputFormat. This will not compile
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Thusoo updated HIVE-74:
--
Component/s: Query Processor
> Hive can use CombineFileInputFormat for when the input are many sm
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649490#action_12649490
]
Joydeep Sen Sarma commented on HIVE-74:
---
looks like in the right direction. only t
[
https://issues.apache.org/jira/browse/HIVE-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dhruba borthakur updated HIVE-74:
-
Attachment: hiveCombineSplit.patch
This code is for review purposes only. I would rather create a
Hive can use CombineFileInputFormat for when the input are many small files
---
Key: HIVE-74
URL: https://issues.apache.org/jira/browse/HIVE-74
Project: Hadoop Hive
31 matches
Mail list logo
