Apache Hive Contributor Day

2011-06-08 Thread John Sichi
Hey, signups are now open for this event: http://hivecontribday2011.eventbrite.com/ It's free, but please don't sign up unless you're sure you can make it, because seating is limited. Contributions come in many shapes and forms (not just code), so anyone interested in helping to advance the pr

Hive error with hive.exec.parallel=true

2011-06-08 Thread Steven Wong
I get the following ClosedByInterruptException often - but not always - when running a query with hive.exec.parallel=true. It seems to happen only when 2 MR jobs are being launched in parallel. I doubt I'm the first person to have seen this error in this scenario, but googling didn't help me. An

Re: Index

2011-06-08 Thread Martin Konicek
Oh I had very little data (a few MB) - I am just testing whether features work at all before I try it on bigger data (for example compression does not work for me http://wiki.apache.org/hadoop/Hive/CompressedStorage ; also not sure whether indexes work on compressed files). The index got used

Re: Index

2011-06-08 Thread Guillaume WEILL
Thanks for your reply. Could you say me how much time have you saved with the index and the time used on a query without query? The amount of data of your table could be helpful too. This is to verify your point of view about the amount of data because I am really not impressed by performance of

unit of work with load data

2011-06-08 Thread W S Chung
I cannot find much written about how hive deal with unit of work control when data is loaded. If when a client crashes in the middle of a 'load data inpath' call, says if the file is not in hdfs, will the records in the file be partially loaded?

Re: Index

2011-06-08 Thread Martin Konicek
Hi, I was testing indexes today as well and the index definitely got used. You should be able to see this when you run two separate queries: INSERT OVERWRITE DIRECTORY "/tmp/index-result2" ... SELECT ... The SELECT was faster for me than without the index. In your case the time might be spent

pivot table in hive

2011-06-08 Thread Reju R
Hi Experts, I need one solution for hive in my project. Actually I have to create a pivot table type of concept in hive. I don't know how to do the same. But I have studied and implemented in hive for the same. But by looking into it, I am not satisfied with the solution and I am not kn

Index

2011-06-08 Thread Guillaume WEILL
Hi, I want to test the use of indexes in hive. For this I created anindex, I launched a first query above, I changed the settings on Hive and ran my query on my database table: CREATE INDEX index ON TABLE table(key) as 'COMPACT' WITH DEFERRED REBUILD; ALTER INDEX index ON table REBUILD; INSERT OV