Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"

On Fri, Oct 1, 2010 at 12:24 PM, Ning Zhang wrote: > If your success rate is increased by increasing > hive.metastore.connect.retries, mostly likely your Thrift server is too busy > to get connected. Can you check your /tmp//hive.log in your local > machine and see if exception related to metas

Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"

We are seeing this exception: Internal error processing create_table java.lang.RuntimeException: Commit is called, but transaction is not active. Either there are mismatching open and close calls or rollback was called in the same transaction Carl, this suggests that the transaction is not active

Re: java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

Thank you for answering, Is there any way to check and fix types? Zoltan 2010/10/1 Steven Wong > I am not familiar with JsonSerde, but your data might contain doubles > that you didn’t know about. LOAD does not check whether the data’s actual > types match the types specified in the schema. >

Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"

Hi Venkatesh, I filed HIVE-1679 which covers the issue I described earlier as well as HIVE-1681 which I think is the real root cause of the problem you are seeing. Please see HIVE-1681 for more information. Thanks. Carl On Fri, Oct 1, 2010 at 12:51 AM, S. Venkatesh wrote: > We are seeing this

Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"

Hi Venkatesh, I attached an interim patch to HIVE-1681. Please try applying this and let me know if it fixes your problem. Thanks. Carl On Fri, Oct 1, 2010 at 1:31 AM, Carl Steinbach wrote: > Hi Venkatesh, > > I filed HIVE-1679 which covers the issue I described earlier as well > as HIVE-1681

Re: Create table as select fails with error "Error in metadata: org.apache.thrift.TApplicationException"

Hi Carl, Awesome, we'll try this and let you know. Sincerely appreciated, Venkatesh On Fri, Oct 1, 2010 at 2:21 PM, Carl Steinbach wrote: > Hi Venkatesh, > I attached an interim patch to HIVE-1681. Please try applying this and let > me know if it fixes your problem. > Thanks. > Carl > > On Fri,

Select * from table fails : Killed : java.lang.NullPointerException

Hi, I am running Select * from table fails . But reduce task fails giving error :- Ended Job = job_201010011018_0001 with errors FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask 2010-10-01 10:01:38,915 FATAL org.apache.hadoop.mapred.TaskTracker: Task: attempt

Re: Tables using custom SerDe doesn't return any data when queried

Dear all, I managed to fix this by starting from scratch and re-creating the table and loading data into it. There must have been something odd about the way I created my original table. thanks Jamie On 30 September 2010 10:20, Jamie Cockrill wrote: > Dear hive-user's, > > I've written my own

dynamic partition query dies with LeaseExpiredException

We're trying to insert into a table, using a dynamic partition, but the query runs for a while and then dies with a LeaseExpiredException. The hadoop details & some discussion is at https://issues.apache.org/jira/browse/HDFS-198 Is there a way to configure hive, or our query, to work around this?

Re: dynamic partition query dies with LeaseExpiredException

Hi Dave, There may be 2 issues here: 1) according to the HDFS JIRA it seems to be a bug introduced in Hadoop 0.18 and was fixed by another hadoop JIRA in 0.20.2 suggested by Hairong, you may want to try 0.20.2 if that's possible. 2) the fact that the lease expired seems to be caused by the fac

wrong number of records loaded to a table is returned by Hive

Hi, I ran a simple Hive query inserting data into a target table from a source table. The number of records loaded to the target table (say number A), which is returned by running this query, is different with the number (say number B) returned by running a query "select count(1) from target". I

Re: wrong number of records loaded to a table is returned by Hive

I had such issues on different versions of hadoop/hive: The version of hadoop/hive I am using now is hadoop 0.20.2/hive 0.7. The version of hadoop/hive I once used is hadoop 0.20.0/hive 0.5 Ping On Fri, Oct 1, 2010 at 10:23 AM, Ping Zhu wrote: > Hi, > > I ran a simple Hive query inserting dat

Re: wrong number of records loaded to a table is returned by Hive

Ping, this is a known issue. The number reported at the end of INSERT OVERWRITE is obtained by means of Hadoop counters, which is not very reliable and subject to inaccuracy due to failed tasks and speculations. If you are using the latest trunk, you may want to try the feature of automatically

RE: java.lang.RuntimeException: java.lang.Double cannot be cast to java.lang.String

One way is to inspect the data files in the table to see if there's any wrongly typed data in them. First, find the directory of your table where the files are, using Hive's "describe extended ". Then, use hadoop fs -ls to find the files and hadoop fs -cat (or -get or whatever your favorite way

Re: sampling conditionality from a large table

Thanks, I ended up writing a scala program which uses the hive JDBC connector. Performance was still reasonable. @tommychheng Programmer and UC Irvine Graduate Student Find a great grad school based on research interests: http://gradschoolnow.com On 9/27/10 11:13 PM, Guru Prasad wrote: Hi, P

RE: Select * from table fails : Killed : java.lang.NullPointerException

I can't tell what the exact problem is, but it looks like your Hadoop cluster (not Hive) has a setup/configuration problem. Googling ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents reveals that other people reported this exception before; see if they found solutions. From: va

Re: wrong number of records loaded to a table is returned by Hive

Hi Ning, I also see the same behavior. Below is some data for your reference. This behavior is observed for large values. I believe HIVE is recording non-final values at the end of insert query: Since hive reads the HIVE History file counters, it may be printing non-final values. Relevant fun

Re: dynamic partition query dies with LeaseExpiredException

Thanks Ning, Our TRANSFORM script is pretty quick, the whole query only took 30 seconds when we added "and project='abc' " to limit the amount of input data. So it seems like the slowdown is somewhere in handling the data before (or after?) it gets to the transform script. I can try adding some

Re: wrong number of records loaded to a table is returned by Hive

One more data point: in Hive History: org.apache.hadoop.hive.ql.exec.FileSinkOperator$TableIdEnum.TABLE_ID_1_ROWCOUNT: 26002996 in JT: org.apache.hadoop.hive.ql.exec.FileSinkOperator$TableIdEnumTABLE_ID_1_ROWCOUNT031,208,09931,208,099 From: gaurav jain To

hive query doesn't seem to limit itself to partitions based on the WHERE clause

Hi, >From looking at the hive log output, it seems that my job is accessing many more partitions than it needs to? For example, my query is something like: INSERT OVERWRITE TABLE daily_fact PARTITION (dt='2010-09-29') SELECT 20100929 as stamp, tagtype, country, sum(num_requests) AS num_r

Re: Hive table export?

Hi, Carl, I got sometime today to install sqoop-1.0.0+3 pointed out by you, installation is successful, thanks, however, when I tried to use it and type: sqoop import --connect jdbc::mysql://mysql_host/my_db --table mytable --username xxx --password yyy *10/10/01 16:23:32 ERROR sqoop.Sqoop: Got e

RE: wrong number of records loaded to a table is returned by Hive

Based on my cursory code inspection, the non-final row count is set when ExecDriver.progress calls ss.getHiveHistory().setCounters(...) inside the while loop, and we need to add the same call after the while loop (after the last updateCounters call at the end) to set the final row count. From: