whirr properties for non ec2

Hi experts, From the configuration guide (http://archive.cloudera.com/cdh/3/whirr/configuration-guide.html), it seems that some properties' defaults are none. What does the none mean? For whirr.location-id, for example, it says an arbitrary location. What does this arbitrary location mean? Is

Re: whirr properties for non ec2

Hi all, Please forget about this. From whirr quick-start-guide (http://whirr.apache.org/docs/0.7.1/quick-start-guide.html), I found an account with a cloud provider is pre-requisites. Thanks, Manhee - Original Message - From: Manhee Jo j...@nttdocomo.com To: common-user

Re: tar or hadoop archive

do you know how to set the number of map/reduce tasks rather than 1 during hadoop archiving? i've tried -Dmapred.map.tasks=2 (we are using 0.19.2 actually :( ) but in vain. thanks, manhee - Original Message - From: Joey Echeverria j...@cloudera.com To: common-user@hadoop.apache.org

is HDFS-788 resolved?

Hi there, Is https://issues.apache.org/jira/browse/HDFS-788 resolved? What actually happens if the smaller partition of some datanodes get full while writing a block? Is it possible that the datanodes are recognized as dead making replication storm among some hundreds of machines? Thanks,

two questions regarding hadoop fair scheduler

Hi all, 1. How can I assing a job to a specific pool from command line? I succeeded it from MR programming by including configuration code below conf.set(mapred.queue.name, testpool) I've tried % hadoop jar -D mapred.queue.name=testpool my.jar myclass input output from command line, which is

Re: Task process exit with nonzero status of 1 - deleting userlogshelps

Hi, I've also encountered the same nonzero status of 1 error before. What did you set to mapred.child.ulimit and mapred.child.java.opts? mapred.child.ulimit must be greater than the -Xmx passed to JavaVM, else the VM might not start. That's wat MR tutorial says. Setting bigger ulimit, I could

mapred.jobtracker.retirejob.interval

Hi, When I ran a job (containing some hundreds of thousands tasks) over our hadoop-0.19.2 cluster, I got OutOfMemoryError at JobTracker. Monitoring memory usage at the JobTracker with Ganglia, it looks like that memory space of the JobTracker is released every 24 hours, which is the default

mrmetrics.log collision on write

arbitration mechanisms to synchronize the write? Thanks, Manhee Jo

A question on dfs.safemode.threshold.pct

Hi all, Why isn't the dfs.safemode.threshold.pct 1 by default? When dfs.replication.min=1 with dfs.safemode.threshold.pct=0.999, there might be chances for a NameNode to check in with incomplete data in its file system. Am I right? Is it permissible? Or is it assuming that replication would be

Re: A question on dfs.safemode.threshold.pct

are slower to start up. When the replication is 1, you don't have that issue. A block either exists or does not. Raghu 2009/10/5 Manhee Jo j...@nttdocomo.com Hi all, Why isn't the dfs.safemode.threshold.pct 1 by default? When dfs.replication.min=1 with dfs.safemode.threshold.pct=0.999, there might

Re: A question on dfs.safemode.threshold.pct

on dfs.safemode.threshold.pct I am not sure what the real concern is... You can set it to 1.0 (or even 1.1 :)) if you prefer. Many admins do. Raghu. On Tue, Oct 6, 2009 at 5:20 PM, Manhee Jo j...@nttdocomo.com wrote: Thank you, Raghu. Then, when the percentage is below 0.999, how can you tell if some datanodes

fuse-dfs then samba mount

Hi all, I've succeeded in sharing hdfs files from windows xp through fuse-dfs then samba mount. When I tried to copy (read and write) 1GB text file from fuse-dfs over samba, it took around 50 secs. Then, I tried dfs get the same file to a data node's local file system and tried to copy the