Re: Hive built-in functions not working

2013-06-21 Thread Ramki Palle
Try to add the jar explicitly from hive prompt and see if that works. Regards, Ramki. On Fri, Jun 21, 2013 at 7:32 AM, fab wol darkwoll...@gmail.com wrote: I'm using Hadoop 0.20.2 with Hive 0.11. I have succesfully inserted into hive/hdfs some csv-files in seperate tables. selects and joins

Re: Is there a mechanism similar to hadoop -archive in hive (add archive is not apparently)

2013-06-20 Thread Ramki Palle
In the *Attempt two, *are you not supposed to use hivetry as the directory? May be you should try giving the full path /opt/am/ver/1.0/hive/hivetry/classifier_wf.py and see if it works. Regards, Ramki. On Thu, Jun 20, 2013 at 9:28 AM, Stephen Boesch java...@gmail.com wrote: Stephen: would

Re: Hive External Table issue

2013-06-20 Thread Ramki Palle
1. I was under the impression that you cannot refer the table location to a file. But, it looks like it works. Please see the discussion in the thread http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/% 3c556325346ca26341b6f0530e07f90d96017084360...@gbgh-exch-cms.sig.ads%3e 2. It

Re: Hive External Table issue

2013-06-20 Thread Ramki Palle
Nitin, Can you go through the thread with subject S3/EMR Hive: Load contents of a single file on Tue, 26 Mar, 17:11 at http://mail-archives.apache.org/mod_mbox/hive-user/201303.mbox/thread?1 This gives the whole discussion about the topic of table location pointing to a filename vs.

Re: LZO compression implementation in Hive

2013-06-17 Thread Ramki Palle
Hi Sanjay, Can you quickly give your insights on thip topic, if possible? Regards, Ramki. On Mon, May 20, 2013 at 2:51 PM, Sanjay Subramanian sanjay.subraman...@wizecommerce.com wrote: Hi Programming Hive Book authors Maybe a lot of u have already successfully implemented this but only

Re: Table present in HDFS but 'show table' Returns Empty

2013-04-23 Thread Ramki Palle
May be you are using derby as your metastore. It creates the metastore in the current directory from where you started your hive session. You may have started your hive session from a different directory next time. Please use either mysql as your metastore or set a definite directory in your

Re: Exception comes out when counting the rows

2013-04-23 Thread Ramki Palle
Normally, you do not use load data when you use external table. You either specify location in the create external table query or use alter table tablename set location syntax to set the location of the files that the external table stores the data. In this case, try describe extended

Re: Partition performance

2013-04-05 Thread Ramki Palle
...@apache.org wrote: See slide #9 from my Optimizing Hive Queries talk http://www.slideshare.net/oom65/optimize-hivequeriespptx . Certainly, we will improve it, but for now you are much better off with 1,000 partitions than 10,000. -- Owen On Thu, Apr 4, 2013 at 4:21 PM, Ramki Palle ramki.pa

Re: Partition performance

2013-04-04 Thread Ramki Palle
Is it possible for you to send the explain plan of these two queries? Regards, Ramki. On Thu, Apr 4, 2013 at 4:06 PM, Sanjay Subramanian sanjay.subraman...@wizecommerce.com wrote: The slow down is most possibly due to large number of partitions. I believe the Hive book authors tell us to

Re: S3/EMR Hive: Load contents of a single file

2013-03-26 Thread Ramki Palle
First of all, you cannot point a table to a file. Each table will have a corresponding table. If you want to have all the in the table contains in only one file, simply copy that one file into the directory. The table does not need to know the name of the file. It only matters whether the

Re: Using TABLESAMPLE on inner queries

2013-03-20 Thread Ramki Palle
You may use percent based (block sampling) sampling for non-bucketed tables, though there are some restrictions. https://cwiki.apache.org/Hive/languagemanual-sampling.html Regards, Ramki. On Wed, Mar 20, 2013 at 12:27 PM, Mark Grover grover.markgro...@gmail.comwrote: Hey Dean, I am not a

Re: how to handle variable format data of text file?

2013-03-18 Thread Ramki Palle
One way you can try is to make your ldata as a map field as it contains variable formatted data and write a UDF to get whatever information you need get. Regards, Ramki. On Mon, Mar 18, 2013 at 1:23 AM, Zhiwen Sun pens...@gmail.com wrote: As u defined in create table hql: fields delimited by

Re: hive issue with sub-directories

2013-03-11 Thread Ramki Palle
One way it was solved by an user earlier was by subclassing the InputFormat class and overriding the listStatus method so that you can ignore subdirectories. This was done in 0.7.1 version. Not sure if there is any better way in later versions. At least you can use this approach until someone

Re: Error while table creation

2013-03-10 Thread Ramki Palle
and we are not using if not exists clause On Sun, Mar 10, 2013 at 4:37 AM, Ramki Palle ramki.pa...@gmail.comwrote: Just wondering if your create table syntax include if not exists such as CREATE EXTERNAL TABLE IF NOT EXISTS my_table ( ... ... ... ) On Sun, Mar 10, 2013 at 2:34 AM

Re: java.lang.NoClassDefFoundError: com/jayway/jsonpath/PathUtil

2013-03-10 Thread Ramki Palle
When you execute the following query, hive select * from twitter limit 5; Hive runs it in local mode and not use MapReduce. For the query, hive select tweet_id from twitter limit 5; I think you need to add JSON jars to overcome this error. You might have added these in a previous session. If

Re: java.lang.NoClassDefFoundError: com/jayway/jsonpath/PathUtil

2013-03-10 Thread Ramki Palle
about by passing MR, why would we do it and what is the use of it. Will appreciate any input. Thanks Sai -- *From:* Ramki Palle ramki.pa...@gmail.com *To:* user@hive.apache.org; Sai Sai saigr...@yahoo.in *Sent:* Sunday, 10 March 2013 4:22 AM *Subject:* Re

Re: HIVE issues when using large number of partitions

2013-03-09 Thread Ramki Palle
Check this for your first question: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Recoverpartitions Please post if you find any solution for your 2nd and 3rd questions. Regards, Ramki. On Thu, Mar 7, 2013 at 8:01 PM, Suresh Krishnappa

Re: difference between add jar in hive session and hive --auxpath

2013-03-09 Thread Ramki Palle
I am not sure whether it works but try this: Put your ADD JAR commands into a file and invoke hive with -i file option. or insert your ADD JAR commands in your $HOME/.hiverc file and start hive. -Ramki. On Fri, Mar 8, 2013 at 11:55 PM, Edward Capriolo edlinuxg...@gmail.comwrote:

Re: Rename external table, including HDFS directory

2013-03-09 Thread Ramki Palle
As you can run the hadoop dfs commands from inside Hive, you can put both the commands in a script file and run the script file using hive. These are still two different commands but are together at one place and can be executed in the same environment in one go. -Ramki. On Fri, Mar 8, 2013 at

Re: Error while table creation

2013-03-09 Thread Ramki Palle
Just wondering if your create table syntax include if not exists such as CREATE EXTERNAL TABLE IF NOT EXISTS my_table ( ... ... ... ) On Sun, Mar 10, 2013 at 2:34 AM, Viral Bajaria viral.baja...@gmail.comwrote: Is this is a new installation of Hive or did you upgrade ? How many tables do

Re: Hive sample test

2013-03-08 Thread Ramki Palle
If any of the 100 rows that the sub-query returns do not satisfy the where clause, there would be no rows in the overall result. Do we still consider that the Hive query is verified in this case? Regards, Ramki. On Wed, Mar 6, 2013 at 1:14 AM, Dean Wampler dean.wamp...@thinkbiganalytics.com

Re: Find current db we r using in Hive

2013-03-07 Thread Ramki Palle
Sai, I do not think there is any command to show the current db in Hive. One alternative for you is to set a property so that the current database is shown as part of the prompt: set hive.cli.print.current.db=true; This one shows your current db as part of your hive prompt. Regards, Ramki.