Hadoop + Cygwin , IOException, /TMP dir

2011-10-27 Thread Masoud
Hi, I installed cygwin on win7, when i run hadoop examples its makes /tmp dir in C:/ (win install dir) not in c:/cygwin (cygwin install dir), so java IOException happened. any solution? Thanks, BS

Re: How to create Output files of about fixed size

2011-10-27 Thread Mapred Learn
Hi Shevek, Thanks for the explanation ! Can you point me to some documentatino for specifying size in output format ? If i say size as 200 MB, then after 200 mb, it would do this per split or overall ? I mena would I end up with 200 mb and a 50 mb from 1st mapper and then, say 200 mb and 10 mb fr

Re: writing to hdfs via java api

2011-10-27 Thread Arpit Gupta
hdfs scheme should work but you will have to change the port. To find the correct port # look for fs.default.name prop in the core-site.xml or the namenode ui should also state the port. -- Arpit On Oct 27, 2011, at 10:52 PM, Jay Vyas wrote: > I found a way to connect to hadoop via hftp, and it

Re: writing to hdfs via java api

2011-10-27 Thread Harsh J
Jay, Using the hdfs:// scheme is the right way, as you have determined. However… A few things you need to ensure while using the Java FileSystem API to do your HDFS tasks: - Connect to NameNode's RPC port, not the web port. Default RPC port is usually 8020, but your fs.default.name config will t

writing to hdfs via java api

2011-10-27 Thread Jay Vyas
I found a way to connect to hadoop via hftp, and it works fine, (read only) : uri = "hftp://172.16.xxx.xxx:50070/";; System.out.println( "uri: " + uri ); Configuration conf = new Configuration(); FileSystem fs = FileSystem.get( URI.create( uri ), conf ); fs.printStatistics();

Re: Hadoop 0.20.204 eclipse plugin

2011-10-27 Thread Jignesh Patel
I am using with 3.7 and 3.6 both. On Oct 27, 2011, at 3:30 AM, Harsh J wrote: > Yes it should be fine with 3.7, as well as 3.6 from my experience. > > On Thursday, October 27, 2011, Masoud wrote: >> Hi, >> >> I'm trying to setup a hadoop development node on windows. >> as you know an eclipse pl

Re: access to build machines

2011-10-27 Thread giridharan kesavan
ooo y'day. here is the info gkesavan@asf000:~$ lsb_release -a No LSB modules are available. Distributor ID:Ubuntu Description:Ubuntu 10.04.3 LTS Release:10.04 Codename:lucid gkesavan@asf000:~$ uname -a Linux asf000.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20 17:4

Re: correct way to reserve space

2011-10-27 Thread Harsh J
Rita/Alexander, You may be interested in continuing the discussion for this feature at https://issues.apache.org/jira/browse/HDFS-1564 Sorry I could not post that link earlier, as I was on my phone. On Thu, Oct 27, 2011 at 1:15 PM, Alexander C.H. Lorenz wrote: > Hi Harsh, > > ah, nice to know,

Re: correct way to reserve space

2011-10-27 Thread Alexander C.H. Lorenz
Hi Harsh, ah, nice to know, thanks ;) best, Alex On Thu, Oct 27, 2011 at 9:27 AM, Harsh J wrote: > The percentage opt is not valid on 0.20 iirc. > > On Thursday, October 27, 2011, Alexander C.H. Lorenz < > wget.n...@googlemail.com> wrote: > > Hi, > > > > in hdfs-site.xml: > > > > > > dfs.da

Re: Hadoop 0.20.204 eclipse plugin

2011-10-27 Thread Harsh J
Yes it should be fine with 3.7, as well as 3.6 from my experience. On Thursday, October 27, 2011, Masoud wrote: > Hi, > > I'm trying to setup a hadoop development node on windows. > as you know an eclipse plugin released by Hadoop 0.20.204, > Do you know this plugin is match with which version of

Re: correct way to reserve space

2011-10-27 Thread Harsh J
The percentage opt is not valid on 0.20 iirc. On Thursday, October 27, 2011, Alexander C.H. Lorenz < wget.n...@googlemail.com> wrote: > Hi, > > in hdfs-site.xml: > > > dfs.datanode.du.reserved > VALUE_HERE (maybe 300) > > > > dfs.datanode.du.pct > 0.9f > > > But not available for