Re: Are there any examples of simple UDAFs?

2010-12-07 Thread Tim Robertson
What about the count or max? http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCount.java http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFMax.java I've not used UDAFs, but I only got

Are there any examples of simple UDAFs?

2010-12-07 Thread Leo Alekseyev
I am trying to write a very simple aggregation function which seems like an overkill for using GenericUDAF as described on the wiki. However, I can't get the code to run. It always throws an exception of the form java.lang.ClassNotFoundException: com.hadoopbook.hive.Maximum$MaximumIntUDAFEvaluato

RE: Ctrl C and Hive ?

2010-12-07 Thread Tali K
Thanks a lot for your quick reply!! Can you also explain also why command hive -e 'select" produces output, prints OK, but give me a prompt only after 7-10 min? if I run hive shell, and do queries inside of shell I don't have such a behaviour. I see Ok , and sec after that I'll have pr

Re: Ctrl C and Hive ?

2010-12-07 Thread Edward Capriolo
On Tue, Dec 7, 2010 at 3:18 PM, Tali K wrote: > 1) When I cancel hive job with Ctrl C, I noticed that java/hive processes > still run on some of my nodes. > I shutdown hadfoop, and restarted it, but noticed that  2 or 3 java/hadoop > processes were still running on each node. > So we went to each

Ctrl C and Hive ?

2010-12-07 Thread Tali K
1) When I cancel hive job with Ctrl C, I noticed that java/hive processes still run on some of my nodes. I shutdown hadfoop, and restarted it, but noticed that 2 or 3 java/hadoop processes were still running on each node. So we went to each node and did a 'killall java' - in some cases I had to

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ning Zhang
Ryan, I wonder why setting 'hive.merge.mapfiles=false' could solve the issue. The issue seems to be a metastore related (drop table could not find default.test_table). This is probably due to the database support newly introduced in 0.6 (see JIRA HIVE-675). On Dec 7, 2010, at 10:21 AM, Ryan LeC

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
Sorry for all of the messages! Stupid gmail client. It turns out that I was able to resolve the failures by setting "hive.merge.mapfiles=false" (by default it is true). I think this is because I'm running Hadoop version 0.20.1, r810220. This thread is what lead me to try this: http://www.mail-ar

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
Digging even further, here's what I see: NOTE: We have a table in Hive called "test_table" but this seems to look for "default.test_table" ? ) 2010-12-07 00:52:24,600 ERROR metadata.Hive (Hive.java:getTable(357)) - NoSuchObjectException(message:default.test_table table not found) at org.a

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
Digging even further, here's what I see: NOTE: We have a table in Hive called "test_table" but this seems to look for "default.test_table" ? ) 2010-12-07 00:52:24,600 ERROR metadata.Hive (Hive.java:getTable(357)) - NoSuchObjectException(message:default.test_table table not found) at org.a

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
Digging even further, here's what I see: NOTE: We have a table in Hive called "test_table" but this seems to look for "default.test_table" ? ) 2010-12-07 00:52:24,600 ERROR metadata.Hive (Hive.java:getTable(357)) - NoSuchObjectException(message:default.test_table table not found) at org.a

Re: Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
I just put the Hive log4j config file on DEBUG, and here is the error that I'm seeing: 2010-12-07 12:16:50,281 WARN mapred.JobClient (JobClient.java:configureCommandLineOptions(539)) - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2010-12-07

Failure when using "insert overwrite" after upgrading to Hive 0.6.0

2010-12-07 Thread Ryan LeCompte
Hello all, This morning we upgraded our cluster to use Hive 0.6.0 from Hive 0.5.0. I ran the Derby DB migration script before upgrading. All of our queries are running fine, however our INSERT OVERWRITE queries are failing immediately. I am not sure what the problem is. When we submit an INSERT

Re: Protecting hive-site.xml and hive-defaults.xml

2010-12-07 Thread Edward Capriolo
On Tue, Dec 7, 2010 at 3:20 AM, Rakesh Davanum wrote: > Hi, > I have a question regarding securing the hive configuration files. The > hive-site.xml has credentials to connect to the mysql server. I will have > many users who would like to use Hive from the CLI. But I do not want them > to be able

Protecting hive-site.xml and hive-defaults.xml

2010-12-07 Thread Rakesh Davanum
Hi, I have a question regarding securing the hive configuration files. The hive-site.xml has credentials to connect to the mysql server. I will have many users who would like to use Hive from the CLI. But I do not want them to be able to see the configurations. 1) Is there any way I can secure th