Re: Reg:Column Statistics with Parquet

2014-07-25 Thread Suma Shivaprasad
Hi , I tried the same with compute statistics for columns a, b,c as above and still seeing the same results in explain plan. How do I confirm if its generating all the column stats for a given column. If this is confirmed, we can debug why Hive is still not using it? Thanks Suma On Thu, Jul

RE: Reg:Column Statistics with Parquet

2014-07-25 Thread Navdeep Agrawal
Well not the correct way ,you can check the statistics in mysql part_col_stats like tables in mysql data base if you are using mysql stat database . Or the other way is calling max,min,distinct on int columns ,largest length on string columns etc,if they run whole map reduce on these operation

Re: Hive shell code exception, urgent help needed

2014-07-25 Thread Sarfraz Ramay
Hi, I have 4 instances on ec2 1 master with namenode and YARN running on it 1 for secondary namenode and 2 slaves I used t2.medium instance for the master node only and left the rest as they were and still i got the same exception. t2.medium is a decent instance with 4GB RAM and 2 CPUs so i

Re: Using Parquet and Thrift in Hive

2014-07-25 Thread Abhishek Agarwal
+ Re-sending as delivery of earlier mail failed. On Fri, Jul 25, 2014 at 5:14 PM, Abhishek Agarwal abhishc...@gmail.com wrote: Hi All, Is it possible to create a table with Parquet as storage mechanism, with schema being supplied from thrift IDL rather than metastore? Something like below,

Re: Hive User Group Meeting

2014-07-25 Thread Xuefu Zhang
Dear Hive users and developers, As an update, the hive user group meeting during Hadoop World will be held on Oct. 15th, from 6:30pm to 9:00pm at about.com's office at 1500 Broadway, 6th floor, New York, NY 10036.. Here is the schedule: 6:30 pm: Doors open 6:30 pm-7:00pm: Networking and

HiveServer2 Availability

2014-07-25 Thread Raymond Lau
Has anyone had any experience with a multiple-machine HiveServer2 setup? Hive needs to be available at all times for our use-case, so if for some reason, one of our HiveServer2 machines goes down or the connection drops, the user should be able to just re-connect to another machine. In the end,

Re: [parquet-dev] Re: Using Parquet and Thrift in Hive

2014-07-25 Thread Brock Noland
This is not possible today. Can you file an enhancement and describe the motivation? Also Parquet has moved to apache: http://parquet.incubator.apache.org/ All questions and discussions should now be sent to d...@parquet.incubator.apache.org please subscribe by emailing

Re: HiveServer2 Availability

2014-07-25 Thread Edward Capriolo
I have put stAndard load balancers infront before and wrote a thrift show tables as a test. On Friday, July 25, 2014, Raymond Lau r...@ooyala.com wrote: Has anyone had any experience with a multiple-machine HiveServer2 setup? Hive needs to be available at all times for our use-case, so if for