Re: show table throwing strange error

2013-06-19 Thread Mapred Learn
Can you also check your hive site XML ? Is it properly formatted and connection strings correct ? Sent from my iPhone On Jun 19, 2013, at 6:30 PM, Mohammad Tariq donta...@gmail.com wrote: Hello Anurag, Thank you for the quick response. Log files is full of such lines along with a trace

Re: Create Index Map/Reduce failure

2012-10-27 Thread Mapred Learn
Try: Export HADOOP_HEAPSIZE=1000 ( 1gb) Before running your hive query And keep keep increasing the size to needed value Another option is setting Xmx in hive-env.sh Sent from my iPhone On Oct 27, 2012, at 12:21 PM, Peter Marron peter.mar...@trilliumsoftware.com wrote: Hi, I have a

How to run big queries in optimized way ?

2012-09-20 Thread Mapred Learn
Hi, We have datasets which are about 10-15 TB in size. We want to run hive queries on top of this input data. What are ways to reduce stress on our cluster for running many such big queries( include joins too) in parallel ? How to enable compression etc for intermediate hive output ? How to

Re: Creating Hive table by pulling data from mainFrames

2012-07-26 Thread Mapred Learn
Sqoop is a nice tool to get dAta to/from DB2 to hive and then you can run hive queries on top of it. Lot of people are using this for traditional DBs and hadoop connectivity. Sent from my iPhone On Jul 26, 2012, at 11:32 AM, Siddharth Tiwari siddharth.tiw...@live.com wrote: Hey Team, We

Re: Text file with ctrl chat as delimiter

2012-06-20 Thread Mapred Learn
a table with one field and create a view split function on top . I dont want this additional overheard, if I can make the input format work with the ctrl char as the delimiter. Mapred Learn, Yes I did have the word 'external' in the create table statement. Thanks, Sam On Jun

Re: Text file with ctrl chat as delimiter

2012-06-19 Thread Mapred Learn
Did you add the word external in create table I.e. Create external table(...blah...blah...) Sent from my iPhone On Jun 19, 2012, at 4:15 PM, Sam William sa...@stumbleupon.com wrote: Hi, I have a data file that is exactly equivalent to a CSV , except that the field delimiter is a control

Re: Want to improve the performance for execution of Hive Jobs.

2012-05-07 Thread Mapred Learn
Try setting this value to your block Size, for 128 mb block size, set mapred.min.split.size=128000 Sent from my iPhone On May 7, 2012, at 10:11 PM, Bhavesh Shah bhavesh25s...@gmail.com wrote: Thanks Nitin for your reply. In short my Task is 1) Initially I want to import the data from MS

Re: Sequence generated Id in Hive

2012-01-27 Thread Mapred Learn
Hive 0.8.0 has row_sequence UDF but it generates unique seq ids only per mapper, not across the job. Sent from my iPhone On Jan 27, 2012, at 8:37 AM, Anson Abraham anson.abra...@gmail.com wrote: Does Hive support automated sequence Id generation, or does a udf have to be created per each

Re: Changing Hive Job Log Path

2012-01-26 Thread Mapred Learn
Try changing one of following property in file listed before them : hive-exec-log4j.properties:hive.log.dir=/tmp/${user.name} hive-log4j.properties:hive.log.dir=/tmp/${user.name} On Thu, Jan 26, 2012 at 3:40 PM, Tucker, Matt matt.tuc...@disney.comwrote: We’ve started noticing that some of

hive case and group-by statement

2011-12-04 Thread Mapred Learn
Hi, I have a following smaple query: select A, CASE WHEN B IN(1,2) THEN 'Type A' ELSE 'Type B' END AS B, C from table_a groupby A, B, C; But when i run this query, it gives error: FAILED: Error in semantic analysis: Line 95:0 Invalid table alias or column reference entity This error is from B

Re: LOAD data into hive table

2011-10-05 Thread Mapred Learn
Hive external table is your solution Sent from my iPhone On Oct 5, 2011, at 11:29 AM, Navin Gupta navin.gu...@kindsight.net wrote: Hi, The loading of data into hive table currently moves the data if the location is hdfs (NON LOCAL). Is there an option that would allow to copy the

How to write a UDAF ?

2011-07-09 Thread Mapred Learn
Hi, Could somebody point me to some wiki abt how to create a UDF In hive ? My problem is how to include hive code in my build path to import exec.UDAF. I downloaded hive code from cloudera's distribution. Any help is appreciated ! -JJ Sent from my iPhone

Re: does hive support Sequence File format ?

2011-06-28 Thread Mapred Learn
...@yahoo.com wrote: I have a requirement to support data from the SequenceFile KEY (not the VALUE) to be used by Hive table. How can I do this. From the code, it looks like the VALUE part is available for Hive. Please help. Regards. From: Mapred Learn

Re: does hive support Sequence File format ?

2011-06-28 Thread Mapred Learn
: https://cwiki.apache.org/confluence/pages/listpages-dirview.action?key=Hive Tim On Tue, Jun 28, 2011 at 9:54 AM, Mapred Learn mapred.le...@gmail.comwrote: Hi, Looks the documentation link you guys provided earlier is moved to some other location: http://wiki.apache.org/hadoop/Hive

Re: Resend - how to load sequence file with decimal data

2011-06-27 Thread Mapred Learn
? On Fri, Jun 24, 2011 at 5:12 PM, Steven Wong sw...@netflix.com wrote: Not sure if this is what you’re asking for: Hive has a LOAD DATA command. There is no decimal data type. ** ** ** ** *From:* Mapred Learn [mailto:mapred.le...@gmail.com] *Sent:* Thursday, June 23, 2011 7:25 AM

Loading seq file into hive

2011-06-27 Thread Mapred Learn
Hi, I have seq files with key as line number and value is ctrl B delimited text. a sample value is: 45454^B567^Brtrt^B-7.8 56577^B345^Bdrtd^B-0.9 when I create a table like: create table temp_seq (no. int, code string, rank string, amt string) row format delimited fields terminated by '\002'

Resend - how to load sequence file with decimal data

2011-06-23 Thread Mapred Learn
Hi, I have a sequence file where The value is text with delimited data and some fields are decimal fields. For eg: decimal(16,6). Sample value : 123.456735. How do I upload such a sequence file in hive and what shud I give in table definition for decimal values as above ? Thanks in

How to load a sequence file with decimal data to hive ?

2011-06-22 Thread Mapred Learn
Hi, I have a sequence file where I have delimited data and some data is decimal fields. For eg: decimal(16,6). Sample value : 123.456735. How do I upload such a sequence file and what shud I give in table definition for decimal values as above ? Thanks in advance ! Sent from my iPhone

Re: does hive support Sequence File format ?

2011-02-17 Thread Mapred Learn
Thanks Ted ! Just found it few minutes ago. On Feb 17, 2011, at 1:46 PM, Ted Yu yuzhih...@gmail.com wrote: Look under http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Create_Table On Thu, Feb 17, 2011 at 12:00 PM, Mapred Learn mapred.le...@gmail.com wrote: Hi, I was wondering if hive