Re: SerDe and Rows

On Jun 3, 2010, at 12:36 PM, Sanjit Jhala wrote: > I'm wondering why the Split class needs to extend FileSplit and also why the > InputFormat needs to call FileInputFormat.getInputPaths(job) in getSplits. Is > this because of legacy code that needs to be cleaned up or does it get used > somewhe

Re: SerDe and Rows

I'm wondering why the Split class needs to extend FileSplit and also why the InputFormat needs to call FileInputFormat.getInputPaths(job) in getSplits. Is this because of legacy code that needs to be cleaned up or does it get used somewhere? -Sanjit On Wed, Jun 2, 2010 at 12:59 PM, Edward Capriol

Re: java.lang.OutOfMemoryError: PermGen space when running as a service.

Thanks for the detailed report Bennie. There might be memory leak in jdbc or hiveserver itself. Are your queries roughly the same (in terms of query size and number of partitions involved)? Better off can you come up a simple test case (serious of queries) that expose the mem leak problem? Than

Re: Error while using Hive JDBC to execute a create temporary UDF

Just closing the loop here. Turns out it was a bug on our side... We had single quotes in the initial ADD JAR command, which was causing the subsequent create temporary function call to fail. Removed the quotes and now we're all set. Thanks, Ryan On Wed, Jun 2, 2010 at 4:44 PM, Ryan LeCompte wr

Carriage return handling in Hive

Hi, Hive uses TextInputFormat by default and which treats '\n' AND '\r' as a line separator. However I don't want '\r' to be treated as a separator. Does Hive provide a way to set custom InputFormat? I tried 'STORED AS file_format' to do this but it seems it didn't work for me. I use 0.4.0. Thank

java.lang.OutOfMemoryError: PermGen space when running as a service.

Hi guys, When I run hive as a service like this: "hive --service hiveserver" I get these errors after about a day of running a lot of queries: java.lang.OutOfMemoryError: PermGen space I use: trunk hive from about a week ago with "-XX:MaxPermSize=128m". hadoop 0.20.2 mysql 5.1.45 meta store Some

Re: Data Type Validation in Hive?

Thanks yongqiang he. Is there any alternative method to do data (type) validation in Hive? ~ Thanks, Pavan * On Thu, Jun 3, 2010 at 12:25 PM, yongqiang he wrote: > Right now Hive does not do data validation when loading data into a table. > If you load some wrong data which does not conform t