Re: assign tasks to specific nodes

2013-09-11 Thread Mark Olimpiati
-- *From:* Mark Olimpiati markq2...@gmail.com *To:* user@hadoop.apache.org *Sent:* Friday, September 6, 2013 1:47 PM *Subject:* assign tasks to specific nodes Hi guys, I'm wondering if there is a way for me to assign tasks to specific machines or at least assign priorities

Re: assign tasks to specific nodes

2013-09-11 Thread Mark Olimpiati
. http://hortonworks.com/ On Sep 11, 2013, at 10:09 AM, Mark Olimpiati wrote: Thanks for replying Rev, but the link is talking about reducers which seems to be like a similar case but what if I assigned priorities to the data partitions (eg. partition B=1, partition C=2, partition A=3

assign tasks to specific nodes

2013-09-06 Thread Mark Olimpiati
Hi guys, I'm wondering if there is a way for me to assign tasks to specific machines or at least assign priorities to the tasks to be executed in that order. Any suggestions? Thanks, Mark

Re: Multi-threaded map task

2013-01-14 Thread Mark Olimpiati
Never mind, depends on plantform, in my case would work fine. Thanks guys! Mark On Mon, Jan 14, 2013 at 12:23 PM, Mark Olimpiati markq2...@gmail.comwrote: Thanks Bertrand, I shall try it and hope to gain some speed. One last question though, do you think the threads used are user-level

Re: Multi-threaded map task

2013-01-13 Thread Mark Olimpiati
to process the same split, wouldn't that be faster with enough cores? Mark On Sun, Jan 13, 2013 at 10:34 PM, Nitin Pawar nitinpawar...@gmail.comwrote: Thats because its distributed processing framework over network On Jan 14, 2013 11:27 AM, Mark Olimpiati markq2...@gmail.com wrote: Hi

Re: Maps split size

2012-10-28 Thread Mark Olimpiati
Dechoux decho...@gmail.comwrote: Hi Mark, I think http://wiki.apache.org/hadoop/HowManyMapsAndReduces might interest you. If you require more information, feel free to ask after reading it. Regards Bertrand On Fri, Oct 26, 2012 at 10:47 PM, Mark Olimpiati markq2...@gmail.com wrote: Hi

Re: Reading Sequence File from Hadoop Distributed Cache ..

2012-10-15 Thread Mark Olimpiati
I'll try that thanks for the suggestion Steve! Mark On Fri, Oct 12, 2012 at 11:27 AM, Steve Loughran ste...@hortonworks.comwrote: On 11 October 2012 20:53, Mark Olimpiati markq2...@gmail.com wrote: Thanks for the reply Harsh, but as I said I tried locally too by using the following

Re: Reading Sequence File from Hadoop Distributed Cache ..

2012-10-11 Thread Mark Olimpiati
, 2012 at 5:15 AM, Mark Olimpiati markq2...@gmail.com wrote: Hi, I'm storing sequence files in the distributed cache which seems to be stored somewher under each node's /tmp .../local/archive/ ... path. In mapper code, I tried using SequenceFile.Reader with all possible

Reading Sequence File from Hadoop Distributed Cache ..

2012-10-10 Thread Mark Olimpiati
Hi, I'm storing sequence files in the distributed cache which seems to be stored somewher under each node's /tmp .../local/archive/ ... path. In mapper code, I tried using SequenceFile.Reader with all possible configurations (locally, distribtued) however, it can't find it. Are sequence files

Re: Metrics ..

2012-08-29 Thread Mark Olimpiati
, publish_avg_time=0.0, publish_stdev_time=0.0, publish_imin_time=0.0, publish_imax_time=1.401298464324817E-45, publish_min_time=0.0, publish_max_time=1.401298464324817E-45, dropped_pub_all=0 David Wong -Original Message- From: Mark Olimpiati [mailto:markq2...@gmail.com] Sent

Metrics ..

2012-08-29 Thread Mark Olimpiati
Hi, I enabled the metrics.properties to use FileContext, in which jvm metrics values are written to a file as follows: jvm.metrics: hostName= localhost, processName=MAP, sessionId=, gcCount=10, gcTimeMillis=130, logError=0, logFatal=0, logInfo=21, logWarn=0, memHeapCommittedM=180.1211,