Re: hive comment in Chinese

2013-01-17 Thread 曹坤
Well I create a issue here : https://issues.apache.org/jira/browse/HIVE-3914 the patch simpley use “outStream.writeUTF” instead of “ outStream.writeBytes ” and solved this problem 2012/8/23 Edward Capriolo > Currently what is a legal column or table name is a very limited set > of ascii chara

Re: question about machine learning on Hive

2013-01-17 Thread Robin Morris
In a similar way, ML algorithms can be put into a Hive UDAF. I'm working on this at the moment, and it's proved quite straightforward to integrate liblinear into a UDAF. As Igor notes, by setting the number of reducers, you can set the number of parallel learners. Robin www.baynote.com From:

Re: question about machine learning on Hive

2013-01-17 Thread Igor Tatarinov
Here is how Twitter does it with Pig: http://www.umiacs.umd.edu/~jimmylin/publications/Lin_Kolcz_SIGMOD2012.pdf We use a similar approach and I think that Pig, being somewhat lower-level with better support of nested objects, is a better tool than Hive. It should be possible to do something simila

question about machine learning on Hive

2013-01-17 Thread qiaoresearcher
How to run machine learning algorithms (whatever ML algorithms) directly in Hive? assume the input and output already stored as Hive tables. ps: I know mahout is available there, but would prefer run machine learning algorithms directly in Hive many thanks,

ClassCastException during reduce-side join, but not map-side join

2013-01-17 Thread Anthony Urso
I am getting an exception when joining two tables with Amazon's Hive 0.8.1 on Amazon EMR, and I've run out of ideas on how to fix it. The query is something along the lines of Q1: SELECT count(*) FROM t1 x JOIN t2 y ON (x.id = y.x_id); Which ends up throwing an exception like this in some of the

Re: Incremental Data Processing With Hive UDAF

2013-01-17 Thread buddhika chamith
Hi All, Greatly appreciate any feedback on this. May be this may sound infeasible. Just wanted check with the experts on this. Anyway the problem of incremental data processing is a very interesting one if it can be accommodated for. Best Regards Buddhika On Wed, Jan 16, 2013 at 12:36 PM, buddhi

HIVE-2915: Partitioned Tables in Hive Metastore

2013-01-17 Thread Matt Tucker
Hi, I'm currently using CDH3u3 and Hive 0.7.1, and I'm looking into how the metadata is stored for partitioned tables within the RDBMS. The issue that I see is that for tables with multiple partitioning columns, there's no good way to determine which PARTITION_KEY_VALS record maps to it's logical

Re: Configure Hive in Cluster

2013-01-17 Thread venkatramanan
Can you suggest me the mandatory hive parameters and clustering configuration steps On Thursday 17 January 2013 12:56 PM, Nitin Pawar wrote: looks like a very small cluster with very limited memory to run mapreduce jobs also number of map/reduce slots on nodes are less so at a time only one ma