Re: Generate Hive DDL

2013-02-15 Thread Dean Wampler
If you're using Hive v0.10, this implements the new SHOW CREATE TABLE mytable command. On Fri, Feb 15, 2013 at 12:07 AM, Anandha L Ranganathan analog.s...@gmail.com wrote: 1) download the jar from from Apache JIRA https://issues.apache.org/jira/browse/HIVE-967 2) run the following

Re: Map join optimization issue

2013-02-15 Thread Mayuresh Kunjir
Thanks Aniket. I actually had not specified the map-join hint though. Sorry for providing the wrong information earlier. I had only set hive.auto.convert.join=true before firing my join query. ~Mayuresh On Thu, Feb 14, 2013 at 10:44 PM, Aniket Mokashi aniket...@gmail.comwrote: I think

CREATE EXTERNAL TABLE Fails on Some Directories

2013-02-15 Thread Joseph D Antoni
I'm trying to create a series of external tables for a time series of data (using the prebuilt Cloudera VM). The directory structure in HDFS is as such: /711 /712 /713 /714 /715 /716 /717 Each directory contains the same set of files, from a different day. They were all put into HDFS using

Re: CREATE EXTERNAL TABLE Fails on Some Directories

2013-02-15 Thread Dean Wampler
You confirmed that 715 is an actual directory? It didn't become a file by accident? By the way, you don't need to include the file name in the LOCATION. It will read all the files in the directory. dean On Fri, Feb 15, 2013 at 10:29 AM, Joseph D Antoni jdant...@yahoo.comwrote: I'm trying to

Re: CREATE EXTERNAL TABLE Fails on Some Directories

2013-02-15 Thread Joseph D Antoni
[cloudera@localhost data]$ hdfs dfs -ls 715 Found 13 items -rw-r--r--   1 cloudera cloudera    7853975 2013-02-15 00:41 715/file.csv (didn't leave rest of ls results) Thanks on the directory--wasn't clear on that.. Joey From: Dean Wampler

Re: CREATE EXTERNAL TABLE Fails on Some Directories

2013-02-15 Thread Dean Wampler
Something's odd about this output; why is there no / in front of 715? I always get the full path when I run a -ls command. I would expect either: /715/file.csv or /user/me/715/file.csv Or is that what you meant by (didn't leave rest of ls results)? dean On Fri, Feb 15, 2013 at 10:45 AM, Joseph

Re: Map join optimization issue

2013-02-15 Thread bejoy_ks
Hi In later versions of hive you actually don't need a map joint hint in your query. Just the following would suffice the purpose Set hive.auto.convert.join=true Regards Bejoy KS Sent from remote device, Please excuse typos -Original Message- From: Mayuresh Kunjir

Re: CREATE EXTERNAL TABLE Fails on Some Directories

2013-02-15 Thread bejoy_ks
Hi Joseph There are differences in the following ls commands cloudera@localhost data]$ hdfs dfs -ls /715 This would list out all the contents in /715 in hdfs, if it is a dir Found 1 items -rw-r--r--   1 cloudera supergroup    7853975 2013-02-14 17:03 /715 The output clearly defines it is

RE: Nullpointer Exception when using UDF

2013-02-15 Thread Christopher Wirt
Hi Neelesh, I've just upgraded our cluster from hadoop-0.20/hive-0.10 to hadoop-1.03(mapr2.1)/hive0.10 and started running into this issue immediately. Strangely is doesn't appear to happen all of the time or be local to any nodes or set of nodes. The only work around I've

Hive Queries

2013-02-15 Thread Cyrille Djoko
I am looking for a relatively efficient way of transferring data between a remote server and Hive without going through the hassle of storing the data first on memory before loading it to Hive. From what I have read so far there is no such command but it would not hurt to ask. Is it possible to

Re: Install / Download of Hive 0.7.0 or 0.7.1

2013-02-15 Thread Dean Wampler
Any particular reason you want to use such an old version? I see it's not even listed in the Apache releases anymore. http://hive.apache.org/releases.html (and following the links to the ftp sites doesn't show it either...) dean On Fri, Feb 15, 2013 at 1:08 PM, Vince George

Re: Hive Queries

2013-02-15 Thread Jarek Jarcec Cecho
Hi Cyrille, I'm not exactly sure what exactly you mean, so I'm more or less blindly shooting, but maybe Apache Sqoop [1] might help you? Jarcec Links: 1: http://sqoop.apache.org/ On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote: I am looking for a relatively efficient way of

Re: Install / Download of Hive 0.7.0 or 0.7.1

2013-02-15 Thread Edward Capriolo
All releases are found on http://archive.apache.org On Fri, Feb 15, 2013 at 2:12 PM, Dean Wampler dean.wamp...@thinkbiganalytics.com wrote: Any particular reason you want to use such an old version? I see it's not even listed in the Apache releases anymore.

Re: Hive Queries

2013-02-15 Thread Cyrille Djoko
Hi Jarcec, I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop 1.0.4 But I keep running on the following exception. Exception in thread main java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.JobContext, but interface was expected So I wrote a

Re: Hive Queries

2013-02-15 Thread Jarek Jarcec Cecho
[-user@hive, +user@sqoop] Hi Cyrille, this seems to me more a Sqoop issue than Hive issue, so I've moved this email to user@sqoop mailing list. I'm keeping user@hive in Bcc so that the mailing list will get the memo. Please join the user@sqoop mailing list [1] to receive additional feedback.

Re: Map join optimization issue

2013-02-15 Thread Aniket Mokashi
I have tested that the parameter hive.mapjoin.smalltable.filesize works well with 0.8. What version of hive are you on? On Fri, Feb 15, 2013 at 8:57 AM, bejoy...@yahoo.com wrote: ** Hi In later versions of hive you actually don't need a map joint hint in your query. Just the following

Re: Map join optimization issue

2013-02-15 Thread Mayuresh Kunjir
I am on 0.9. If I have a selectivity condition on small table, does Hive try to estimate filtered data size before deciding the join algorithm? If it is the case, it makes sense to use map join even when the small table(before filter) is larger than the hive.mapjoin.smalltable.filesize parameter.

NULLable STRUCTs

2013-02-15 Thread Michael Malak
It seems that all Hive columns (at least those of primitive types) are always NULLable? What about columns of type STRUCT? The following: echo 1,2 twovalues.csv hive CREATE TABLE tc (x INT, y INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','; LOAD DATA LOCAL INPATH 'twovalues.csv' INTO TABLE

Re: Generate Hive DDL

2013-02-15 Thread Murtaza Doctor
Thanks Dean Anandha, we are on Hive 0.10 and the command of course works well. Not sure how much this feature is documented. From: Dean Wampler dean.wamp...@thinkbiganalytics.commailto:dean.wamp...@thinkbiganalytics.com Reply-To: user@hive.apache.orgmailto:user@hive.apache.org