On Thu, Jan 31, 2013 at 8:53 AM, Artem Ervits wrote:
> Does anyone run Hadoop on Dell R720 model of servers? Dell site lists C2100
> model of servers as best fit for Hadoop workloads. What does community
> recommend?
The R720 supports up to 2 xeon CPUs and 8 drives in 2U. If configured
appropriat
Moving from general@ to user@. The general list is not for technical
questions, it's to discuss project-wide issues.
On Tue, Jan 22, 2013 at 1:03 PM, Mohit Vadhera
wrote:
> Namenode switches into safemode when it has low disk space on the root fs /
> i have to manually run a command to leave it.
Some of the unit tests fail with 1.7. HDFS and MR mostly work OK, but
if you run into problems with 1.7 the first question will be "does it
work in 1.6?".
-andy
On Thu, Jan 17, 2013 at 11:19 AM, Chris Mawata wrote:
> Do you know what causes 1.7 to fail? I am running 1.7 and so far have not
> don
ill, the mappers on node2 are the
> ones runs fastest.
>
> Is it possible that it is because of cache distribution(disk cache?) Does
> data node hold any cache of recently accessed data?
>
> Regards,
> Haifeng
>
>
> -Original Message-
> From: Andy Isaacson [m
On Tue, Jan 8, 2013 at 7:37 PM, Lewis John Mcgibbney
wrote:
> Over @Nutch, we are experiencing failed builds on our nightly CI builds
> after the above upgrade.
> This happens on the Jenkins (Solaris and Ubuntu) builds but NOT on local
> operating systems. This has been verified by numerous member
However, note that DFSShell currently has some issues with
percent-encoded characters.
https://issues.apache.org/jira/browse/HDFS-4329
-andy
On Wed, Dec 26, 2012 at 1:35 PM, Ted Dunning wrote:
> The colon is a reserved character in a URI according to RFC 3986[1].
>
> You should be able to perce
Your output shows that node2 has 13 mappers and the reducer, while
node3 and node4 had only 8 mappers each. So I'd expect some
disparity. Since it's hard to correlate the mapper throughput against
the reducer throughput, it's possible that node3 got just as much work
done.
That doesn't explain wh
On Fri, Dec 14, 2012 at 7:47 AM, anand sharma wrote:
> Hi i am following this tutorial for installing hadoop alpha 2.0.2 on ubuntu
> 12.10 and am sure home variables for java and hadoop are properly
> configured.
>
> http://jugnu-life.blogspot.in/2012/05/hadoop-20-install-tutorial-023x.html?showCo
Rack awareness will help, but it's a "best effort" rather than
guaranteed replication. Over time the cluster will converge to having
at least one replica on each rack, but even just normal block churn
can result in significant time periods where rack replication policy
is violated. The issue become
What kernel did you see this on? Was there significant swap traffic
(si/so in vmstat output) during the high-system-time period?
BTW, you don't need to nor do you want to run sync(1) when
manipulating drop_caches, it just causes additional noise and
slowdown. drop_caches doesn't have any impact on
On Tue, Nov 27, 2012 at 1:35 PM, Uri Laserson wrote:
> This URL gives me a 404
> http://hadoop.apache.org/docs/current/file_system_shell.html
The 2.0 docs are not correctly being generated, currently.
https://issues.apache.org/jira/browse/HADOOP-8427
As a workaround you can refer to a previous v
On Fri, Nov 16, 2012 at 10:55 AM, Pankaj Gupta wrote:
> The Hadoop Definitive Guide provides comparison with regular file systems
> and indicates the advantage being lower number of seeks(as far as I
> understood it, may be I read it incorreclty, if so I apologize). But, as I
> understand, the dat
Andy's points:
>
> To be clarify: I think 0.23 does not claim HA feature.
> Also Hadoop-2 HA is Active-Standby model.
>
> Regards,
> Uma
> ____
> From: Andy Isaacson [a...@cloudera.com]
> Sent: Thursday, November 15, 2012 8:19 AM
>
On Wed, Nov 14, 2012 at 4:35 AM, mailinglist
wrote:
> does anyone know, if it possible to setup an active-active-NameNode in hadoop
> 1.0 ? Or how can i provide a HA-NameNode?
HA is not present in hadoop 1.0. You'll have to upgrade to a release
on branch 2.0 or 0.23.
-andy
On Tue, Nov 13, 2012 at 1:40 PM, Jay Vyas wrote:
> 1) but I thought that this sort of thing (yes even on linux) becomes
> important when you have large amounts of data - because the way files are
> written can cause issues on highly packed drives.
If you're running any filesystem at 99% full with
On Thu, Nov 8, 2012 at 1:48 PM, Kartashov, Andy wrote:
> When running examples, you bring them into HDFS. Say, you need to make some
> correction to a file, you need to make them on local FS and run $hadoop fs
> -put ... again. You cannot just make changes to files inside HDFS except for
> "tou
The 404 link is due to
https://issues.apache.org/jira/browse/HADOOP-8427 and
https://issues.apache.org/jira/browse/HADOOP-8860 which we
unfortunately haven't finished fixing yet. Apologies for the
inconvenience.
The 1.0.3 version of the docs at
http://hadoop.apache.org/docs/r1.0.3/hdfs_user_guide.
Moving the thread to user@. The general@ list is not used for
technical questions.
On Fri, Nov 2, 2012 at 1:59 AM, zjl208399617 wrote:
> When i running Hive query option：
> there often throw Error from Reduce Tasks：
>
> Error: java.io.IOException: File too large
> at java.io.FileOutputStr
On Fri, Oct 26, 2012 at 11:47 AM, Kartashov, Andy
wrote:
> I successfully ran a job on a cluster on foo1 in pseudo-distributed mode and
> are now trying to try fully-dist'ed one.
>
> a. I created another instance foo2 on EC2.
It seems like you're trying to use the start-dfs.sh style startup
scri
On Fri, Oct 26, 2012 at 9:40 AM, Kartashov, Andy wrote:
> Gents,
We're not all male here. :) I prefer "Hadoopers" or "hi all,".
> 1.
> - do you put Master's node under fs.default.name in core-site.xml
> on the slave machines or slaves' hostnames?
Master. I have a 4-node cluster, named foo1
On Thu, Oct 25, 2012 at 7:01 AM, wrote:
> Hi Brahma,
>
> I am on Mac OS X it dosent have copy cmd i.e
>
> sh-copy-id -i
>
> I copyed it as
>
> mediaadmin$ cat ~/.ssh/id_rsa.pub | ssh pluto@10.203.33.80 'cat >>
> ~/.ssh/authorized_keys'
> Password:
>
> and did
> ssh 10.203.33.80 and it asked for
BTW, while verifying this behavior I noticed that there is a bug in
Hadoop 2.0 "dfs -test -d", it generates error messages when it should
be silent. I filed the bug at
https://issues.apache.org/jira/browse/HDFS-4104
-andy
On Mon, Oct 22, 2012 at 2:57 PM, Andy Isaacson wrote:
>
The result is reported in the shell exit code, not as a textual output
from the command, just like the UNIX /usr/bin/test command.
The intended use case is something like the following Bourne/bash shell example:
if test -d /opt; then
... install in /opt
else
... install in /usr
fi
or using
On Wed, Oct 17, 2012 at 2:04 PM, Sundeep Kambhmapati
wrote:
> I am trying to install Hadoop 0.20.2 on a cluster on two virtual machines.
> One acting as master other as slave.
> I am able to ssh from master to slave and vice verse. But when I run
> start-dfs.sh namenode is not starting.
> I checke
sk array as JBOD", we plan to use disk array
> with RAID5 and make LUN of 1T.
> so we have many LUN of the size of 1T. and we mkfs on every LUN,so we
> have 12 fs /data1./data12, which will be put into HDFS.
>
>
> Best R.
>
> beatls
>
> On Tue, Oct 9, 2012 at
Also, note that JVM startup overhead, etc, means your -ls time is not
completely unreasonable. Using OpenJDK on a cluster of VMs, my "hdfs
dfs -ls" takes 1.88 seconds according to time (and 1.59 seconds of
user CPU time).
I'd be much more concerned about your slow transfer times. On the
same clus
ts firewalled off
using AWS security policies.
There are unsavoury people who scan EC2 IP ranges looking for
unsecured services like Hadoop, so don't think that "I didn't publish
my IP, I am safe" will save you!
-andy
>
> Andy Kartashov
>
>
> -Original Messa
kering Parkway, Pickering, L1V 0C4
> 1 Phone : (905) 837 6269
> 6 Mobile: (416) 722 1787
> andy.kartas...@mpac.ca
>
> -Original Message-
> From: Andy Isaacson [mailto:a...@cloudera.com]
> Sent: Friday, October 12, 2012 6:24 PM
> To: user@hadoop.apache.org
> Subject: Re: Iss
&D, Co-op
> 1340 Pickering Parkway, Pickering, L1V 0C4
> 1 Phone : (905) 837 6269
> 6 Mobile: (416) 722 1787
> andy.kartas...@mpac.ca
>
>
> -Original Message-
> From: Andy Isaacson [mailto:a...@cloudera.com]
> Sent: Friday, October 12, 2012 4:31 PM
> To: user@hado
hings work much better when I delete that line from /etc/hosts.
-andy
> Cheers,
> Andy
>
>
> -Original Message-
> From: Andy Isaacson [mailto:a...@cloudera.com]
> Sent: Friday, October 12, 2012 2:10 PM
> To: user@hadoop.apache.org
> Subject: Re: Issue when clicki
From: Serge Blazhiyevskyy [mailto:serge.blazhiyevs...@nice.com]
>> Sent: Thursday, October 11, 2012 5:35 PM
>> To: user@hadoop.apache.org; murthy nvvs
>> Subject: Re: Issue when clicking on BrowseFileSystem
>>
>> Take a look at your conf files. Normally it is in hdfs-
What Hadoop version did you install? Can you provide the .address
settings for your NN and one of your DNs? If you visit /dfshealth.jsp
on your NN, how many live nodes and dead nodes does it report?
-andy
On Wed, Oct 10, 2012 at 9:54 PM, murthy nvvs wrote:
> Hi,
>
> I had made Hadoop set
Moving this to user@ since it's not appropriate for general@.
On Fri, Sep 28, 2012 at 11:16 PM, Xiang Hua wrote:
> Hi,
> i want to select 4(600G) local disks combined with 3*800G disks form
> diskarray in one datanode.
> is there any problem? performance ?
The recommended configuration wo
On Tue, Sep 25, 2012 at 9:28 AM, Peter Sheridan
wrote:
> We're using Hadoop 1.0.3. We need to pick up a set of large (4+GB) files
> when they've finished being written to HDFS by a different process.
The common way to solve this problem is to modify the writing
application to write to a temporar
On Thu, Sep 20, 2012 at 5:24 AM, Michael Segel
wrote:
> Why is it that when anyone asks a question about IBM Tom wants to take it off
> line?
To be fair, most vendors tend to redirect distro-specific discussion
to non-apache.org forums. Cloudera has a cdh-user list, MapR has web
forums and a sup
On Mon, Sep 3, 2012 at 5:09 AM, Hemanth Yamijala wrote:
> Is there a reason why Yarn's directory paths are not defaulting to be
> relative to hadoop.tmp.dir.
>
> For e.g. yarn.nodemanager.local-dirs defaults to /tmp/nm-local-dir.
> Could it be ${hadoop.tmp.dir}/nm-local-dir instead ? Similarly for
Hi folks,
Replying to this thread is not going to get you unsubscribed and will
just annoy everyone else who's subscribed. To unsubscribe please send
an email to user-unsubscr...@hadoop.apache.org from your subscribed
address.
For more info please visit
http://hadoop.apache.org/common/mailing_lis
37 matches
Mail list logo
