Thank you for the responses guys! First, to Patrick, I didn't set it in the code, though I will try it because that's a really good idea to set it there, so I shall play around with that.
Long: I should have clarified, I am using 0.20.1, and so this is a bit different. I set the parameter in mapred-site.xml and for some reason it's just not getting implemented. Thank you anyways, though! -Mark -----Original Message----- From: Long Van Nguyen Dinh [mailto:munt...@gmail.com] Sent: Tuesday, December 22, 2009 12:17 PM To: common-user@hadoop.apache.org Subject: Re: io.sort.mb configuration? Hadoop has a default file (hadoop-default.xml - version 19) for all configuration, don't change the values in that file (they won't be affected), copy the parameter to the file hadoop-site.xml where you set up the cluster and set the value you want there. Long Van On Tue, Dec 22, 2009 at 11:40 AM, Patrick Angeles <patrickange...@gmail.com> wrote: > You can also set that param per-job. Maybe you called some code that did > that behind the scenes? > > On Tue, Dec 22, 2009 at 11:10 AM, Mark Vigeant <mark.vige...@riskmetrics.com >> wrote: > >> Hey Everyone- >> >> I've been playing around with Hadoop and Hbase for a while and I noticed >> that when running a program to upload data into an HTable I saw the output: >> >> INFO mapred.MapTask: io.sort.mb = 100 >> >> Which is the default value, but in the mapred configuration on all machines >> in my cluster I set this value to 250. Could it be that my program is not >> accessing the configuration properly? Is that too large a value? Or is it >> most likely just a foolish syntax error on my part? >> >> Thank you very much, all input is appreciated. >> >> Mark Vigeant >> RiskMetrics Group, Inc. >> >> >> This email message and any attachments are for the sole use of the intended >> recipients and may contain proprietary and/or confidential information which >> may be privileged or otherwise protected from disclosure. Any unauthorized >> review, use, disclosure or distribution is prohibited. If you are not an >> intended recipient, please contact the sender by reply email and destroy the >> original message and any copies of the message as well as any attachments to >> the original message. >> > This email message and any attachments are for the sole use of the intended recipients and may contain proprietary and/or confidential information which may be privileged or otherwise protected from disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy the original message and any copies of the message as well as any attachments to the original message.
