Re: Problem with using Postgres as hive meta store DB.

2013-01-23 Thread wd
On Wed, Jan 23, 2013 at 4:46 PM, Ehsan Haq ehsan@klarna.com wrote: ERROR: invalid escape string Hint: Escape string must be empty or one character..) You can set standard_conforming_strings = off in postgresql.conf to avoid this.

Re: hive query not running in cron job

2012-11-23 Thread wd
This problem was caused by some env values not include when run by crontab, it's a common error for shell script writing . :)

Re: hive query not running in cron job

2012-11-22 Thread wd
Add the following line before your crontab config source ~/.bashrc On Thu, Nov 22, 2012 at 5:59 PM, Chunky Gupta chunky.gu...@vizury.comwrote: Hi, I have a python script :- ---cron_script.py--- import os import sys from subprocess import call

Re: Please help : Hive + postgres integration. Drop table query is hanging.

2012-08-28 Thread wd
http://www.mail-archive.com/user@hive.apache.org/msg01293.html Maybe the same error? On Wed, Aug 29, 2012 at 12:27 PM, rohithsharma rohithsharm...@huawei.com wrote: Hi I am using PostgreSQl-9.0.7 as metastore and + Hive-0.9.0. I integrated postgres with hive. Few queries are working fine.

Hive partition problem

2012-05-31 Thread wd
hi, We setup a new hive 0.9 client, Found some sql did not work, for example hive create table t1(a int) partitioned by ( dt int ); OK Time taken: 0.097 seconds hive load data local inpath '/tmp/t' into table t1 partition (dt=111); Copying data from file:/tmp/t Copying file: file:/tmp/t Loading

Re: Hive partition problem

2012-05-31 Thread wd
didn't saw that. Really Sorry. On Thu, May 31, 2012 at 12:53 PM, Bhavesh Shah bhavesh25s...@gmail.com wrote: Hello wd, Try this one... I am not sure about this ALTER TABLE t1 DROP PARTITION(dt = '111') -- Regards, Bhavesh Shah On Thu, May 31, 2012 at 12:49 PM, wd w...@wdicc.com wrote

Re: Hive partition problem

2012-05-31 Thread wd
Nothing output in hive history file, is there an other log file or an option to output detail log ? On Thu, May 31, 2012 at 3:34 PM, Aniket Mokashi aniket...@gmail.com wrote: You should look at hive log and find exact exception. That will give you a hint. On Thu, May 31, 2012 at 12:33 AM, wd

Re: Hive partition problem

2012-05-31 Thread wd
Is this a bug? Should I report a bug to hive? On Thu, May 31, 2012 at 3:56 PM, wd w...@wdicc.com wrote: Still no useful output. But we found the problem. When the partition col type is int, it can't be droped. After change it to string, it can be droped. On Thu, May 31, 2012 at 3:48 PM

Re: Hive partition problem

2012-05-31 Thread wd
partition for an hour for a particular day, it seemed to drop all of the hour partitions for the day. IMO it's a bug. We just moved to using text Malc -Original Message- From: wd [mailto:w...@wdicc.com] Sent: 31 May 2012 09:32 To: user@hive.apache.org Subject: Re: Hive partition

Re: subquery in select

2012-05-28 Thread wd
group by category On Mon, May 28, 2012 at 2:20 PM, shan s mysub...@gmail.com wrote: (select category, count(*) as count from gt group by cat) a,

Re: subquery in select

2012-05-28 Thread wd
(*) as totalCount from gt) b ; On Mon, May 28, 2012 at 1:55 PM, wd w...@wdicc.com wrote: group by category On Mon, May 28, 2012 at 2:20 PM, shan s mysub...@gmail.com wrote: (select category, count(*) as count from gt group by cat) a,

Re: subquery in select

2012-05-27 Thread wd
select a.id, a.count, a.count/b.val from (select id, count(*) as count from data group by id) a, (select val from tableY ) b You don't have a join condition, this maybe output very large data. On Mon, May 28, 2012 at 11:45 AM, shan s mysub...@gmail.com wrote: Thanks Edward. But I didn't get

Re: Is there a metastore schema script for postgresql for Hive version 0.9.0

2012-05-12 Thread wd
Hive can auto create metadata tables, maybe you can try it out. On Sat, May 12, 2012 at 2:28 PM, Xiaobo Gu guxiaobo1...@gmail.com wrote: I can't find it in the release package. Xiaobo Gu

Re: Rank implementation in hive

2012-05-05 Thread wd
Maybe someone can setup a site to accept user upload UDF or UDAF jars. Like cpan for perl, aur for archlinux. :D On Sun, May 6, 2012 at 1:28 AM, Edward Capriolo edlinuxg...@gmail.com wrote: Hey all, We (m6d.com) have released a implementation of the rank feature to github:

Re: Hive command line- select

2012-01-02 Thread wd
Because 'select *' will not run map reduce job, may be you should check if your hadoop cluster is work On Mon, Jan 2, 2012 at 10:37 AM, Aditya Kumar adityakumar...@yahoo.com wrote: Hi, I am able to install hive, and create a table (external) and map it to my Hbasetable. I am able to do

Re: How to make hive handle/show Chinese words

2011-11-04 Thread wd
use utf8 or have you try to use insert overwrite to local directory and chek the output file? On Wed, Nov 2, 2011 at 9:39 PM, Bing Li lib...@yahoo.com.cn wrote: Hi, guys I want to load some data files which including Chinese words. Currently, I found Hive can't display it well. Is there some

Re: Hive and HBase integration in Cloudera CDH3U1

2011-11-04 Thread wd
the problem you've meet? On Thu, Nov 3, 2011 at 1:56 AM, Weishung Chung weish...@gmail.com wrote: Hi, I am trying to integrate Hive and HBase using Cloudera cdh3u1, but still can't get it to work? Anyone has any success? Thank you, Wei Shung

Re: Latest hive source code

2011-10-27 Thread wd
http://www.apache.org/dyn/closer.cgi/hive/ On Fri, Oct 28, 2011 at 9:02 AM, trang van anh anh...@vtc.vn wrote: Dear all, Anybody show me how to get latest hive source code? Thanks in advance. Trang.

Re: Problem build hive index table

2011-10-11 Thread wd
You can look at your hadoop jobtracker log for the detail of the error. On Tue, Oct 11, 2011 at 5:04 PM, trang van anh anh...@vtc.vn wrote: Dear Experts, I have problem with building index table in hive, step by step : 1. i created table  named pv_users (pageid int, age int ) that has

Re: Best practices for storing data on Hive

2011-09-04 Thread wd
Hive support more than one partitions, have your tried? Maybe you can create to partitions named as date and user. Hive 0.7 also support index, maybe you can have a try. On Sat, Sep 3, 2011 at 1:18 AM, Mark Grover mgro...@oanda.com wrote: Hello folks, I am fairly new to Hive and am wondering

Re: RE: Why a sql only use one map task?

2011-08-24 Thread wd
What about your total Map Task Capacity? you may check it from http://your_jobtracker:50030/jobtracker.jsp 2011/8/24 Daniel,Wu hadoop...@163.com: I checked my setting, all are with the default value.So per the book of Hadoop the definitive guide, the split size should be 64M. And the file size

Re: Setting up stats database

2011-08-18 Thread wd
find it. I looked at the stats source and everything looks fine. My conn string is : jdbc:mysql://ip:3306/TempStatsStoreamp;user=nameamp;password=pwd. Am I missing something? Thanks On Thu, Aug 18, 2011 at 8:19 AM, wd w...@wdicc.com wrote: The error in log

Re: slow performance when using udf

2011-08-16 Thread wd
) { return null; } } //public static void main(String args[]) { //String t = %E5%A4%AA%E5%8E%9F-%E4%B8%89%E4%BA%9A; //System.out.println( getString(t) ); //} } On Tue, Aug 16, 2011 at 10:47 AM, wd w...@wdicc.com wrote: Thanks for all your advise, I'll try it out

Setting up stats database

2011-08-15 Thread wd
hi, I'm try to use postgres as stats database. And made following settings in hive-site.xml property namehive.stats.dbclass/name valuejdbc:postgresql/value descriptionThe default database that stores temporary hive statistics./description /property property

Re: Setting up stats database

2011-08-15 Thread wd
oh, found hive only support mysql and hbase. I'll try hbase. On Mon, Aug 15, 2011 at 3:09 PM, wd w...@wdicc.com wrote: hi, I'm try to use postgres as stats database. And made following settings in hive-site.xml property  namehive.stats.dbclass/name  valuejdbc:postgresql/value

Re: Setting up stats database

2011-08-15 Thread wd
HBase Publisher/Aggregator classes cannot be loaded. need to configure publisher/aggregator for hbase...there is only one way, that is use mysql .. does stats database will optimize hive query? Consider whether or not setup a mysql for this. On Mon, Aug 15, 2011 at 3:17 PM, wd w...@wdicc.com

Re: slow performance when using udf

2011-08-15 Thread wd
with performance. On Mon, Aug 15, 2011 at 1:49 AM, wd w...@wdicc.com wrote: hi, I create a udf to decode urlencoded things, but found the speed for mapred is 3 times(73sec - 213 sec) as before. How to optimize it? package com.test.hive.udf; import org.apache.hadoop.hive.ql.exec.UDF; import

Re: how to load data to partitioned table

2011-08-12 Thread wd
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML 2011/8/12 Daniel,Wu hadoop...@163.com suppose the table is partitioned by period_key, and the csv file also has a column named as period_key. The csv file contains multiple days of data, how can we load it in the the table?

Use spacial chars in hive split

2011-05-17 Thread wd
Hi, Can I use a spacial char like '^B' as the split pattern ? I've tried '\002', '^B', '0x02', all failed.

What does 'TempStatsStore' do ?

2011-05-10 Thread wd
hi, After upgrade to hive 0.7, I found there will a file named 'derby.log' and directory named 'TempStatsStore' be created every time I run the hive job . What does these files do? Is there a way to prevent them created ?

Re: Can't drop table

2011-04-28 Thread wd
| 37438 But there is no table named IDXS exists. So I think this problem is cased by these missing tables. Then I migrate a upgrade sql file from mysql, and upgrade my postgress, the problem solved. The attachment is the upgrade SQLs to run. 2011/1/6 wd w...@wdicc.com hi, I've setup

Can't drop table

2011-01-06 Thread wd
hi, I've setup a single node hadoop and hive. And can create table in hive, but can't drop table, hive cli will hang there, nothing more infos. hive-0.6.0-bin hadoop-0.20.2 jre1.6.0_23 postgresql-9.0-801.jdbc4.jar (have tried postgresql-8.4-701.jdbc4.jar) pgsql 9.0.2 How to find what's wrong

Re: Can't drop table

2011-01-06 Thread wd
more logging info try hive -hiveconf hive.root.logger=DEBUG,console Thanks. Carl On Thu, Jan 6, 2011 at 1:29 AM, wd w...@wdicc.com wrote: hi, I've setup a single node hadoop and hive. And can create table in hive, but can't drop table, hive cli will hang there, nothing more infos. hive