Help IN CHD
Hey All, Please tell me where to enter datanode IP's in CHD3U2 , actally i installed all the components in namenode and datanode but confuse where to put datanode IPS in namenode so thet they get connected. -- With Regards Vikas Srivastava DWH Analytics Team Mob:+91 9560885900 One97 | Let's get talking !
Re: Help IN CHD
Vikas, This question belongs to Hadoop's lists. I'm moving it to hdfs-u...@hadoop.apache.org. To answer your question: DN hostnames must exist in the dfs.hosts pointed file if you want selective inclusion. Else you just have to start the DN with the right config and network access to the NN, and it will automatically join the cluster. On Thu, Aug 11, 2011 at 6:26 PM, Vikas Srivastava vikas.srivast...@one97.net wrote: Hey All, Please tell me where to enter datanode IP's in CHD3U2 , actally i installed all the components in namenode and datanode but confuse where to put datanode IPS in namenode so thet they get connected. -- With Regards Vikas Srivastava DWH Analytics Team Mob:+91 9560885900 One97 | Let's get talking ! -- Harsh J
Reducer Issue in New Setup
Hello, We have just setup Hive on a new Hadoop cluster. When I run a select * on a table, it works fine but when I run any query which needs a reducer, like count(1) or a where condition, the query just sits there doing nothing (map 0%). I see some message like no reducers to run. How do I fix this? The configuration et al is exactly like another environment that is working fine. We are running CDH2 with Hadoop 0.20.1 and Hive 0.4.1 Cheers!
RE: Reducer Issue in New Setup
Have you checked your logs? These are often the best places to start. Look at the running job and click on the running count, the current task, then the task logs. Sometimes they're helpful, sometimes they're not. http://hadoop-master:50030/jobtracker.jsp Travis Powell / tpow...@tealeaf.com From: hadoop n00b [mailto:new2h...@gmail.com] Sent: Thursday, August 11, 2011 9:32 AM To: user@hive.apache.org Subject: Reducer Issue in New Setup Hello, We have just setup Hive on a new Hadoop cluster. When I run a select * on a table, it works fine but when I run any query which needs a reducer, like count(1) or a where condition, the query just sits there doing nothing (map 0%). I see some message like no reducers to run. How do I fix this? The configuration et al is exactly like another environment that is working fine. We are running CDH2 with Hadoop 0.20.1 and Hive 0.4.1 Cheers!
Re: Reducer Issue in New Setup
Can you run normal MR jobs, like the example Pi calculation? Sometimes a no-reducer problem stems from DNS issues— reducers use node names, not IP addresses, so you need to have each machine knows how to resolve the names of all the other machines in the cluster. If it's a new cluster, you may want to look at CDH3, as it is a pretty big leap forward. On Aug 11, 2011, at 9:35 AM, Travis Powell wrote: Have you checked your logs? These are often the best places to start. Look at the running job and click on the running count, the current task, then the task logs. Sometimes they’re helpful, sometimes they’re not. http://hadoop-master:50030/jobtracker.jsp Travis Powell / tpow...@tealeaf.com From: hadoop n00b [mailto:new2h...@gmail.com] Sent: Thursday, August 11, 2011 9:32 AM To: user@hive.apache.org Subject: Reducer Issue in New Setup Hello, We have just setup Hive on a new Hadoop cluster. When I run a select * on a table, it works fine but when I run any query which needs a reducer, like count(1) or a where condition, the query just sits there doing nothing (map 0%). I see some message like no reducers to run. How do I fix this? The configuration et al is exactly like another environment that is working fine. We are running CDH2 with Hadoop 0.20.1 and Hive 0.4.1 Cheers!
RE: Reducer Issue in New Setup
Are you using a custom scheduler? I have seen issues with jobs having 0 mappers and 1 reducer with Fair scheduler. From: hadoop n00b [mailto:new2h...@gmail.com] Sent: Thursday, August 11, 2011 9:32 AM To: user@hive.apache.org Subject: Reducer Issue in New Setup Hello, We have just setup Hive on a new Hadoop cluster. When I run a select * on a table, it works fine but when I run any query which needs a reducer, like count(1) or a where condition, the query just sits there doing nothing (map 0%). I see some message like no reducers to run. How do I fix this? The configuration et al is exactly like another environment that is working fine. We are running CDH2 with Hadoop 0.20.1 and Hive 0.4.1 Cheers!
Running Hive from Eclipse
Hi folks, I am trying to run Hive from eclipse. I've set it up correctly and it is building the jars and stuff. However I face execeptions when I try to run hive queries like show tables etc. There has been a discussion on this in the mailing list previously but there was no solution provided. It runs perfectly from command line . I am making a few changes to the hive source and every time I need to jar it from the command line and run it .Is there some way to run it directly from eclipse? Please help, Thanks, JS
Re: Running Hive from Eclipse
Hi John, Can you please include the error messages/exceptions that you're encountering? Thanks. Carl On Thu, Aug 11, 2011 at 1:40 PM, john smith js1987.sm...@gmail.com wrote: Hi folks, I am trying to run Hive from eclipse. I've set it up correctly and it is building the jars and stuff. However I face execeptions when I try to run hive queries like show tables etc. There has been a discussion on this in the mailing list previously but there was no solution provided. It runs perfectly from command line . I am making a few changes to the hive source and every time I need to jar it from the command line and run it .Is there some way to run it directly from eclipse? Please help, Thanks, JS
multiple tables join with only one hug table.
if the retailer fact table is sale_fact with 10B rows, and join with 3 small tables: stores (10K), products(10K), period (1K). What's the best join solution? In oracle, it can first build hash for stores, and hash for products, and hash for stores. Then probe using the fact table, if the row matched in stores, that row can go up further to map with products by hashing check, if pass, then go up further to try to match period. In this way, the sale_fact only needs to be scanned once which save lots of disk IO. Is this doable in hive, if doable, what hint need to use?
Re: Running Hive from Eclipse
Hi Carl, This is the stack trace I get .. http://pastebin.com/3pASqvDq I configured mysql as my metastore and its perfectly getting updated when ever I am adding tables via commandline. Also one more thing is ..I am not getting any log statements while using command line . I haven't messed up with log4j props but I wonder why this is happening. THanks On Fri, Aug 12, 2011 at 2:12 AM, Carl Steinbach c...@cloudera.com wrote: Hi John, Can you please include the error messages/exceptions that you're encountering? Thanks. Carl On Thu, Aug 11, 2011 at 1:40 PM, john smith js1987.sm...@gmail.comwrote: Hi folks, I am trying to run Hive from eclipse. I've set it up correctly and it is building the jars and stuff. However I face execeptions when I try to run hive queries like show tables etc. There has been a discussion on this in the mailing list previously but there was no solution provided. It runs perfectly from command line . I am making a few changes to the hive source and every time I need to jar it from the command line and run it .Is there some way to run it directly from eclipse? Please help, Thanks, JS
Re: Running Hive from Eclipse
Hi, See in the line that log4j props is not in found .. I added Hive_conf dir to the classpath while running and now I get this trace .. http://pastebin.com/vXs98aZ5 I am completely clueless ! Thanks JS On Fri, Aug 12, 2011 at 9:54 AM, john smith js1987.sm...@gmail.com wrote: Hi Carl, This is the stack trace I get .. http://pastebin.com/3pASqvDq I configured mysql as my metastore and its perfectly getting updated when ever I am adding tables via commandline. Also one more thing is ..I am not getting any log statements while using command line . I haven't messed up with log4j props but I wonder why this is happening. THanks On Fri, Aug 12, 2011 at 2:12 AM, Carl Steinbach c...@cloudera.com wrote: Hi John, Can you please include the error messages/exceptions that you're encountering? Thanks. Carl On Thu, Aug 11, 2011 at 1:40 PM, john smith js1987.sm...@gmail.comwrote: Hi folks, I am trying to run Hive from eclipse. I've set it up correctly and it is building the jars and stuff. However I face execeptions when I try to run hive queries like show tables etc. There has been a discussion on this in the mailing list previously but there was no solution provided. It runs perfectly from command line . I am making a few changes to the hive source and every time I need to jar it from the command line and run it .Is there some way to run it directly from eclipse? Please help, Thanks, JS
