Re: Question on adding Hadoop XML to Configuration Object

2012-08-09 Thread Dave Beech
Instead of setting the xml files programmatically, why not set the HADOOP_CONF_DIR env variable to '/usr/local/hadoop-1.0.2/conf/'? That way, you can just create a new Configuration() object and the files will be loaded for you without any extra work. On 9 August 2012 09:28, Chandra Mohan, Ananda

Re: Question on adding Hadoop XML to Configuration Object

2012-08-09 Thread Dave Beech
Also - make sure the fs.default.name parameter is actually in core-site.xml, not hdfs-site.xml On 9 August 2012 09:32, Mohammad Tariq wrote: > Try this and let me know if it works, > > Configuration conf = new Configuration(); > conf.addResource(new > Path("YOUR_H

Re: sym Links in hadoop

2012-10-08 Thread Dave Beech
Hi, The FileUtil.symlink command does nothing more than call the unix "ln" command, so it has no knowledge of how to work with Hadoop archive files, only plain files and directories. Is your archive on local disk, or in HDFS? Cheers, Dave On 8 October 2012 13:43, Visioner Sadak wrote: > I tried u

Re: sym Links in hadoop

2012-10-09 Thread Dave Beech
Colin >>> >>> >>> On Mon, Oct 8, 2012 at 7:20 AM, Visioner Sadak >>> wrote: >>> > thanks dave its in hdfs onlyany other methods of creating a symlink >>> > >>> > >>> > On Mon, Oct 8, 2012 at 7:00 PM, Dave Beech

Re: sym Links in hadoop

2012-10-09 Thread Dave Beech
different. Symlink support for HDFS, as far as I'm aware, isn't contained in any "stable" release of Hadoop (i.e. 1.x.x) On 9 October 2012 13:40, Dave Beech wrote: > A lot of code in Hadoop is marked "Deprecated". This doesn't mean you > shouldn't

Re: GroupingComparator

2012-10-15 Thread Dave Beech
Hi Alberto The iterator you are looping over in your reduce method isn't a self-contained list of values. What's actually happening is that you're iterating through *part* of the sorted key/value set that was sent to that reduce node, and it is the grouping comparator that decides when to break th

Re: GroupingComparator

2012-10-15 Thread Dave Beech
chieve the same behavior as the old api? I need the > second field of the first key object to stay the same among the > iterations, in order to compare it with other objects. Do I have to > clone the object? > > > Thanks. > > On 15 October 2012 21:27, Dave Beech wrote: >>

Re: GroupingComparator

2012-10-16 Thread Dave Beech
e there is no possibility to avoid the "new" operator, right? > > > > On 15 October 2012 22:49, Dave Beech wrote: >> Well, if all you need is the tag (the 1 or 2), why not just use a Text >> or IntWritable instance variable. You wouldn't need to clone the wh

Re: Hadoop installation on mac

2012-10-16 Thread Dave Beech
+1 Installing from tarball by usual method is fine for mac os. One issue to be aware of is https://issues.apache.org/jira/browse/HADOOP-7489 (but even that doesn't stop it working) On 16 October 2012 11:56, Harsh J wrote: > Suneel, > > What version are you trying to run? Following regular tarball

Re: Hadoop installation on mac

2012-10-16 Thread Dave Beech
Instructions for single node operation: http://hadoop.apache.org/docs/r0.21.0/single_node_setup.html Instructions for cluster: http://hadoop.apache.org/docs/r0.21.0/cluster_setup.html Cheers, Dave On 16 October 2012 12:02, suneel hadoop wrote: > Hi Harsh, > > Thanks for ur quick turn around, >

Re: Subscription to the mailing list

2012-10-23 Thread Dave Beech
I guess you could set up gmail filters to do this. Set it up so all mail from the list gets immediately archived / deleted / whatever, but flag your queries as "important" so they skip the filter and hit your inbox. On 22 October 2012 19:15, Jay Vyas wrote: > I would simply use "Stackoverflow" fo

Re: MapReduce APIs

2012-11-27 Thread Dave Beech
AK - look again at that javadoc. Job does a have getConfiguration() method. You may have missed it the first time because it's inherited from a parent class, JobContext. On 27 November 2012 14:23, Kartashov, Andy wrote: > Thank man for the response. Much appreciated. > > > > Why? Because Job o