Re: "Subject" etiquette

2012-11-22 Thread Alexander Alten-Lorenz
+1 On Nov 22, 2012, at 10:53 PM, Ruslan Al-Fakikh wrote: > +1 > > > On Thu, Nov 22, 2012 at 6:27 PM, Mohammad Tariq wrote: > >> +1 >> >> Regards, >>Mohammad Tariq >> >> >> >> On Thu, Nov 22, 2012 at 7:47 PM, Dean Wampler < >> dean.wamp...@thinkbiganalytics.com> wrote: >> >>> As a se

Re: Multiuser setup on Hive

2012-11-22 Thread Austin Chungath
Thanks dean. On Thu, Nov 22, 2012 at 7:44 PM, Dean Wampler < dean.wamp...@thinkbiganalytics.com> wrote: > If you go the route of locking down permissions at the HDFS level, then it > will help if everyone works in his or her own database, since all the > tables will be rooted at a directory for e

Re: hive query not running in cron job

2012-11-22 Thread wd
Add the following line before your crontab config source ~/.bashrc On Thu, Nov 22, 2012 at 5:59 PM, Chunky Gupta wrote: > Hi, > I have a python script :- > > ---cron_script.py--- > > import os > import sys > from subprocess import call > print 'starting'

Re: "Subject" etiquette

2012-11-22 Thread Ruslan Al-Fakikh
+1 On Thu, Nov 22, 2012 at 6:27 PM, Mohammad Tariq wrote: > +1 > > Regards, > Mohammad Tariq > > > > On Thu, Nov 22, 2012 at 7:47 PM, Dean Wampler < > dean.wamp...@thinkbiganalytics.com> wrote: > >> As a service to everyone on this list, please fill in the "Subject" field >> when you post t

RE: Effecient partitions usage in join

2012-11-22 Thread Dima Datsenko
Hi Benny, The udf solution sounds like a plan. Much better than generating hive query with hardcoded partition out of table B. Can you please provide a sample of what you’re doing there? Thanks, Dima From: Bennie Schut [mailto:bsc...@ebuddy.com] Sent: יום ה 22 נובמבר 2012 16:28 To: user@hive.a

RE: Effecient partitions usage in join

2012-11-22 Thread Bennie Schut
Unfortunately at the moment partition pruning is a bit limited in hive. When hive creates the query plan it decides what partitions to use. So if you put hardcoded list of partition_id items in the where clause it will know what to do. In the case of a join (or a subquery) it would have to run t

Re: Multiuser setup on Hive

2012-11-22 Thread Dean Wampler
If you go the route of locking down permissions at the HDFS level, then it will help if everyone works in his or her own database, since all the tables will be rooted at a directory for each db. dean On Thu, Nov 22, 2012 at 2:26 AM, Austin Chungath wrote: > Shreepadam, > So what do you recommen

Effecient partitions usage in join

2012-11-22 Thread Dima Datsenko
Hi Guys, I wonder if you could help me. I have a huge Hive table partitioned by some field. It has thousands of partitions. Now I have another small table containing tens of partitions id. I'd like to get the data only from those partitions. However when I run Select * from A join B on (A.part

Re: Multiuser setup on Hive

2012-11-22 Thread Austin Chungath
Typo, I meant "user2 logs into hive and he is able to see and delete database data1" On Thu, Nov 22, 2012 at 12:16 PM, Austin Chungath wrote: > Hi, > > I had been trying to set up a multi user environment for hive. > I have set up the hive metastore db in MySQL and hive works. > > Consider this s

Re: how to transform the date format in hive?

2012-11-22 Thread Ruslan Al-Fakikh
Hi, also take a look at Hive date functions: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions On Thu, Nov 22, 2012 at 3:59 AM, Tom Hubina wrote: > Could convert them to unix time which will give you two bigints that you > can subtract to get se

hive query not running in cron job

2012-11-22 Thread Chunky Gupta
Hi, I have a python script :- ---cron_script.py--- import os import sys from subprocess import call print 'starting' call(['hive', '-f', '/mnt/user/test_query'],stderr=open('/mnt/user/tmp/error','w'), stdout=open('/mnt/user/tmp/output','w')) --

Re: Multiuser setup on Hive

2012-11-22 Thread Alexander Alten-Lorenz
That means a separate metastore per User / different port. Please have in mind, anyone should maintain this. On top, the user has to choose the right JDBC connection. I have my doubt on such a installation ;) cheers, Alex On Nov 22, 2012, at 10:48 AM, Austin Chungath wrote: > Thanks Alex, >

Re: Multiuser setup on Hive

2012-11-22 Thread Austin Chungath
Thanks Alex, But unfortunately I don't have kerberos implementation right now to try it out. I was wondering if we can create multiple metastore dbs in mysql and then for each user group make separate hive-site.xml which has the username and jdbc connection details. Do I make any sense? is somethin

Re:

2012-11-22 Thread Nitin Pawar
getting hive to work on windows is really pain. This has been discussed many times. Its better to have a linux vm and then try your hands on On Thu, Nov 22, 2012 at 2:43 PM, imen Megdiche wrote: > Hello, > > I tried to execute the example of worcount with hive but I have had this > error . > >

Re: Multiuser setup on Hive

2012-11-22 Thread Alexander Alten-Lorenz
You could use SASL / kerberos implementation within HiveServer2. Depends on a kerberosized cluster, too. Hive's metastore server provides the same mechanism, but isn't fully multi connect ready. Here's a link: http://ben-tech.blogspot.de/2012/10/hive-server-2-in-cdh41.html - Alex On Nov 22, 201

Re: Multiuser setup on Hive

2012-11-22 Thread Austin Chungath
Shreepadam, So what do you recommend for this? What are the current best practices for deploying hive in a multi-user environment? Thanks, Austin On Thu, Nov 22, 2012 at 1:10 PM, Shreepadma Venugopalan < shreepa...@cloudera.com> wrote: > Hi Austin, > > Hive authorization in its current form has