MapReduce Streaming on Solaris

Hi all, I have a 20 node cluster that is running on Solaris x86 (OpenIndian). I'm not really familiar with OpenIndiana having moved from Solaris to Linux many years ago, but it's the OS of choice for the systems administrator at my company. Each worker has 24 700xGB drives, 24 cores and 96 GB of

Re: ListWritable In Hadoop

No, but hadoop 2.2 has ArrayWritable. https://hadoop.apache.org/docs/r2.2.0/api/org/apache/hadoop/io/ArrayWritable.html Would you provide more information about your use case you can describe? There maybe an alternative that will meet your needs. On Thu, Jul 10, 2014 at 12:31 AM, unmesha sreev

Upgrading from 1.1.2 to 2.2.0

Has anyone upgraded directly from 1.1.2 to 2.2.0? If so, is there anything I should be concerned about? Thanks, Rich -- *Kernighan's Law* "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not sm

Re: MR JOB

File copy operations do not run as map reduce jobs. All hadoop fs commands are run as operations against HDFS and do not use the MapReduce. On Fri, Jul 18, 2014 at 10:49 AM, Ashish Dobhal wrote: > Does the normal operations of hadoop such as uploading and downloading a > file into the HDFS run

Re: MR JOB

HDFS handles the splitting of files into multiple blocks. It's a file system operation that is transparent to the user. On Fri, Jul 18, 2014 at 11:07 AM, Ashish Dobhal wrote: > Rich Haase Thanks, > But if the copy ops do not occur as a MR job then how does the splitting >

Re: unsubscribe

To unsubscribe from this list send an email to user-unsubscribe@hadoop. apache.org On Fri, Jul 18, 2014 at 10:54 AM, Zilong Tan wrote: > > > -- *Kernighan's Law* "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you

Re: No such file or directory

Try the same commands, but set the config path. e.g. $ hadoop --config /path/to/hdfs/config/dir dfs ... On Tue, Jul 29, 2014 at 12:16 PM, Bhupathi, Ramakrishna wrote: > Folks, > > > > Can you help me with this ? I am not sure why I am getting this “No such > file or directory” for every hd

Re: Datanode can not start with error "Error creating plugin: org.apache.hadoop.metrics2.sink.FileSink"

The reason you can't launch your datanode is: *2014-09-04 10:20:01,677 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain* *java.net.BindException: Port in use: 0.0.0.0:50075 * It appears that you already have a datanode instance listening on por

Re: Map job not finishing

How many tasktrackers do you have setup for your single node cluster? Oozie runs each action as a java program on an arbitrary cluster node, so running a workflow requires a minimum of two tasktrackers. On Fri, Sep 5, 2014 at 7:33 AM, Charles Robertson < charles.robert...@gmail.com> wrote: > Hi

Re: Map job not finishing

rles > > > On 5 September 2014 16:44, Rich Haase wrote: > >> How many tasktrackers do you have setup for your single node cluster? >> Oozie runs each action as a java program on an arbitrary cluster node, so >> running a workflow requires a minimum of two tasktr

Re: Regular expressions in fs paths?

HDFS doesn't support he full range of glob matching you will find in Linux. If you want to exclude all files from a directory listing that meet a certain criteria try doing your listing and using grep -v to exclude the matching records.

Re: Hadoop Smoke Test: TERASORT

You can set the number of reducers used in any hadoop job from the command line by using -Dmapred.reduce.tasks=XX. e.g. hadoop jar hadoop-mapreduce-examples.jar terasort -Dmapred.reduce.tasks=10 /terasort-input /terasort-output

Re: Writing output from streaming task without dealing with key/value

In python, or any streaming program just set the output value to the empty string and you will get something like "key"\t"". On Wed, Sep 10, 2014 at 12:03 PM, Susheel Kumar Gadalay wrote: > If you don't want key in the final output, you can set like this in Java. > > job.setOutputKeyClass(NullWr

Re: Writing output from streaming task without dealing with key/value

t tab character if you don't want it during the post processing steps you want to perform with *nix commands. On Wed, Sep 10, 2014 at 12:12 PM, Dmitry Sivachenko wrote: > > On 10 сент. 2014 г., at 22:05, Rich Haase wrote: > > > In python, or any streaming program just set the ou

Re: UNSUBSCRIBE

Email user-unsubscr...@hadoop.apache.org to unsubscribe. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com From: Naveen teja J N V mailto:naveen.teja.j@gmail.com>> Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" mailto:

Re: to all this unsubscribe sender

+1 Sent from my iPhone On Dec 5, 2014, at 08:29, Ted Yu mailto:yuzhih...@gmail.com>> wrote: +1 On Fri, Dec 5, 2014 at 7:22 AM, Aleks Laz mailto:al-userhad...@none.at>> wrote: +1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Sye

Re: What happens to data nodes when name node has failed for long time?

The remaining cluster services will continue to run. That way when the namenode (or other failed processes) is restored the cluster will resume healthy operation. This is part of hadoop’s ability to handle network partition events. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146

Re: Copying files to hadoop.

file system between the VM and host. This is probably the easiest solution since it will allow you to see the files you have on OS X in your Linux VM and then you can use the hdfs/hadoop/yarn commands on linux (which you already have configured). Cheers, Rich Rich Haase | Sr. Software Engineer

Re: Copying files to hadoop.

Anil, Happy to help! Cheers, Rich Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com From: Anil Jagtap mailto:anil.jag...@gmail.com>> Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" mailto:user@hadoop.apache.org>>

Re: Enable symlinks

Hadoop does not currently support symlinks. Hence the “Symlinks not supported” exception message. You can follow progress on making symlinks production ready via this JIRA: https://issues.apache.org/jira/browse/HADOOP-10019 Cheers, Rich From: Tang mailto:shawndow...@gmail.com>> Reply-To: "us

Re: Cloudera Manager Installation is failing

Try posting this question on the Cloudera forum. http://community.cloudera.com/ On Mar 2, 2015, at 3:21 PM, Krish Donald mailto:gotomyp...@gmail.com>> wrote: Hi, I am trying to install Cloudera manager but it is failing and below is the log file: I have uninstalled postgres and tried again but

Re: Cloudera monitoring Services not starting

Please ask cloudera related questions on Cloudera’s forums. community.cloudera.com On Mar 5, 2015, at 11:56 AM, Krish Donald wrote: > Hi, > > I have setup a 4 node cliuster , 1 namenode and 3 datanode using cloudera > manager 5.2 . > But it is not starting Cloudra Monitorinf service and for

Re:

I’m not a Sqoop user, but it looks like you have an error in your SQL. -> Caused by: java.sql.SQLSyntaxErrorException: ORA-00907: missing right parenthesis On May 7, 2015, at 11:34 PM, Kumar Jayapal mailto:kjayapa...@gmail.com>> wrote: ORA-00907 Rich Haase| Sr. Software Engineer |

Re:

If Sqoop is generating the SQL for your import then you may have hit a bug in the way the SQL for Oracle is being generated. I’d recommend emailing the Sqoop user mailing list: u...@sqoop.apache.org<mailto:u...@sqoop.apache.org>. On May 7, 2015, at 11:45 PM, Rich Haase mail

Re: Jr. to Mid Level Big Data jobs in Bay Area

an opportunity where I can work all day or most of the day with big data technologies, and contribute and learn from the project at hand. Thanks if anyone can share any information, Adam Rich Haase| Sr. Software Engineer | Pandora m (303) 887-1146 | rha...@pandora.com<mailto:rha...@pandora.com>

Re: Move blocks between Nodes

ts the movements happen to the 100 nodes. But I need to disable write_op for those nodes to be decommissioned. Is there a way to make DataNode in safe mode (read only) mode? Of course, regarding to blocks movement between nodes, Any thoughts will be appreciated. -- Bing Jiang Rich Haa

Re: println in MapReduce job

To unsubscribe from this list send an email to user-unsubscr...@hadoop.apache.org. https://hadoop.apache.org/mailing_lists.html On Sep 24, 2015, at 9:40 AM, sukesh kumar mailto:s724...@gmail.com>> wrote: unsubscribe On Thu, Sep 24, 2015 at 8:49 PM, x

Re: unsubscribe

Please see https://hadoop.apache.org/mailing_lists.html for unsubscribe instructions. On Oct 13, 2015, at 3:55 AM, shanthi k mailto:kshanthi...@gmail.com>> wrote: Who is thiz? On 13 Oct 2015 15:18, "MANISH SINGLA" mailto:coolmanishh...@gmail.com>> wrote: -- Regards Manish Singla

Re: Unsubscribe

Sath, Please read the attached link for instructions to properly unsubscribe from the list. https://hadoop.apache.org/mailing_lists.html Sent from my iPhone On Dec 1, 2015, at 7:48 PM, Sath mailto:neiodavi...@gmail.com>> wrote: Namikaze Its not professional to use fowl language in a forum l

Re: 2 items - RE: Unsubscribe & A better ListSrv for beginners.

hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. Rich Haase| Sr. Software Engineer | Pandora m (303) 887-1146 | rha...@pandora.com<mailto:rha...@pandora.com>

Re: 2 items - RE: Unsubscribe & A better ListSrv for beginners.

Here’s a link to the most recent edition: http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/1491901632/ref=dp_ob_title_bk I didn’t realize the 4th edition was out. :) On Dec 2, 2015, at 1:13 PM, Rich Haase mailto:rha...@pandora.com>> wrote: Hi Carl, I would recommend pickin