Hive Metatool experts,
Trying to use the hive "metatool" to update the locations for all of our tables
after a configuration change did not do this automatically. The tool gets a
heap error even though we upped the JVM to 4 GB. Does anyone know of a
workaround?
Thanks,
Mark
Hive has built-in support for gzipped text files so you don't need to do
anything special with them (meaning you just load them like regular files).
However, there is one important limitation with this approach these files
are not splittable. Depending on the size of your log files and how many of
Thanks Navis, that got me past this exception!
Ross
On Mon, Nov 11, 2013 at 6:03 PM, Navis류승우 wrote:
> in handling PARTIAL1,
>
> inputOI = (StandardListObjectInspector) parameters[0];
> return ObjectInspectorFactory.getStandardListObjectInspector(inputOI);
>
> 1.
> inputOI is not guaranteed to
Hi all,
I am using Hadoop cdh3u6 and apache hive 0.10. I run the summary sql as
follow :
SELECT pt,
count(1),
sum(flow_byte),
count(distinct t.user_id),
count(distinct t.phone_no),
max(t.user_ip),
min(t.user_ip),
max(t.phone_no),
Awesome! Congrats, Prasad!
On Mon, Nov 11, 2013 at 4:00 PM, Zhang Xiaoyu wrote:
> Congrats, Prasad!
>
> Johnny
>
>
> On Mon, Nov 11, 2013 at 10:01 AM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
>
>> Congrats Prasad!
>>
>> Thanks
>> Prasanth Jayachandran
>>
>> On Nov 10, 2013
Hi,
I have a web log files (text format). I want to load these files to a Hive
table in compressed format. How do I do it ?
Should I compress the text file (using any Linux utilities) and then create the
Hive table?
Can any one provide me the Hive syntax for loading the compressed file?
Thank
in handling PARTIAL1,
inputOI = (StandardListObjectInspector) parameters[0];
return ObjectInspectorFactory.getStandardListObjectInspector(inputOI);
1.
inputOI is not guaranteed to be a StandardListObjectInspector.
Use ListObjectInspector instead.
2.
ObjectInspectorFactory.getStandardListObjectIn
On 12 Nov 2013, at 0:01, Sunita Arvind wrote:
Just in case this acts as a workaround for someone:
The issue is resolved if I eliminate the "where" clause in the query
(just
keep "where $CONDITIONS"). So 2 workarounds I can think of now are:
1. Create views in Oracle and query without the where
Hi Sunita,
Sqoop specific questions are better asked on Sqoop user mailing list
u...@sqoop.apache.org. You can find instructions how to subscribe to that at
[1].
I would suggest to take a look into the failed map task log as that log usually
contain entire exception including all the chained ex
Congrats, Prasad!
Johnny
On Mon, Nov 11, 2013 at 10:01 AM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:
> Congrats Prasad!
>
> Thanks
> Prasanth Jayachandran
>
> On Nov 10, 2013, at 10:16 PM, Vaibhav Gumashta
> wrote:
>
> > Congrats Prasad!
> >
> >
> > On Sun, Nov 10, 2013 at
Just in case this acts as a workaround for someone:
The issue is resolved if I eliminate the "where" clause in the query (just
keep "where $CONDITIONS"). So 2 workarounds I can think of now are:
1. Create views in Oracle and query without the where clause in the sqoop
import command
2. Import every
Hi Jie,
Seems that there is an error related to:
No FileSystem for scheme: pfile
Have you tried to replace the line?
pfile://
With:
/test
So that it would work in this format:
CREATE TABLE rename_partition_table (key STRING, value STRING) PARTITIONED BY
(part STRING)
STORED AS RCFILE
L
Hello,
I am using sqoop to import data from oracle into hive. Below is my SQL:
nohup sqoop import --connect "jdbc:oracle:thin:@(DESCRIPTION = (ADDRESS =
(PROTOCOL = TCP)(HOST = xxx)(PORT = )) (CONNECT_DATA = (SERVER =
DEDICATED) (SERVICE_NAME = CDWQ.tms.toyota.com) (FAILOVER_MODE=
(TYPE=s
Hello,
I'm writing a generic UDAF function that closely resembles SUM() with the
main difference being that it accepts an array datatype parameter and
returns an array datatype.
I've already done this for a GenericUDF successfully. I believe I am having
difficulty coding the proper ObjectInspecto
Congrats Prasad!
Thanks
Prasanth Jayachandran
On Nov 10, 2013, at 10:16 PM, Vaibhav Gumashta
wrote:
> Congrats Prasad!
>
>
> On Sun, Nov 10, 2013 at 8:17 PM, Lefty Leverenz
> wrote:
>
>> Congratulations Prasad!
>>
>> -- Lefty
>>
>>
>> On Sun, Nov 10, 2013 at 11:04 PM, Brock Noland wrot
As Owen noted, max run for version 0.11 is 130. 3 is minimum run for RLE to be
used. So max value that can be interpreted from 7 bits is 130.
Thanks
Prasanth Jayachandran
On Nov 11, 2013, at 9:51 AM, Owen O'Malley wrote:
> Hi,
> The RLE in ORC is a tradeoff (as is all compression) between t
Hi,
The RLE in ORC is a tradeoff (as is all compression) between tight
representations for commonly occurring patterns and longer representations
for rarely occurring patterns. The question at hand is how to use the bits
available to reduce the average size of the column. In Hive 0.12, ORC
gained
Runs of 1M is not common case. I am not sure how vertica stores the run
lengths. It seems like variable length integers are used.
ORC does not use variable length integers for storing run length. Using
variable length integer has advantage of storing much longer runs but for
repeating shorter r
In vertica, if I have a column sorted, and the same value repeat 1M times,
it only used very small storage as it only stores (value, 1M). But in ORC,
looks like the max length is less than 200 ( not very sure, but at about
the same level of hundreds), why restrict the max run length?
19 matches
Mail list logo
