It's better if you post this question on the zookeeper mailing list.
But based on my past usage of Zookeeper, what you described seems more like
what you would want to do. You can put the configuration on zookeeper and
your distributed nodes can read it from ever so often and update their
configur
I don't think there's a data loss here. However, I think you may have been
affected by https://issues.apache.org/jira/browse/HDFS-4301 due to a large
fsimage size perhaps, which you can workaround by increasing the default
timeout via property dfs.image.transfer.timeout (default 6ms, i.e. 1
min
The speed shown there is to be taken with a grain of salt. It is an
average value measured from the initiation of its phase. So if the
reduce is waiting for more map outputs to be available, having started
earlier (default is at 5% of maps completed) then the wait period is
also counted into this r
Good points all,
The mapreduce jobs are, well. intensive. We've got a whole variety, but
typically I see them use a lot of CPU, a lot of Disk, and upon occasion a
whole bunch of Network bandwidth. Duh right? J
The master node is mostly CPU intensive right? We're using LXC to segregate
(ps
Hi,
I am trying to do an outer join on to input files.
But while joining the TupleWritable value in the mapper is not getting cleaned
up and so is using the previous values of a different key.
The code I used is : ( 'plist' is containing the set of paths to be taken as
input )
jobConf.setInp
Hi,
When I try to run the package test.org.apache.hadoop.fs.s3native as Junit Test
is shows the following error:
Class not found
org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
java.lang.ClassNotFoundException:
org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemCo
2013/3/19 Tapas Sarangi
>
> On Mar 19, 2013, at 5:00 AM, Алексей Бабутин
> wrote:
>
> node A=12TB
> node B=72TB
> How many A nodes and B from 200 do you have?
>
>
> We have more number of A nodes than B. The ratio of the number is about
> 80, 20. Note that not all the B nodes are 72TB, that's a
Hi,
I was using Hadoop 1.0.4 in a single-node mode and everything was working
fine.But then I had to restart my computer because of some reason.And now
when I am trying to start Namenode and Datanode I can find only Datanode
running.Could anyone tell me the possible reason of this problem??Or am I
Have you added these properties in your hdfs-site.xml file :
dfs.name.dir
/hadoop/hdfs/name
dfs.data.dir
/hadoop/hdfs/data
If not please do that. The values of these properties default to the /tmp
directory which gets emptied at each reboot and you end up loosing your
data and metad
hank you Mohammed
I'll restart my computer and try to start NameNode whithout format it
2013/3/20 Mohammad Tariq
> Have you added these properties in your hdfs-site.xml file :
>
> dfs.name.dir
> /hadoop/hdfs/name
>
>
> dfs.data.dir
> /hadoop/hdfs/data
>
>
>
> If not please do tha
Hi Everyone,
I am trying to implement Secondary Sort Algorithm on mydata. But I am
having a trouble with my Combiner.
When I donot use Combiner, grouping is done well, I mean one reduce task is
running for every pair, sharing the same first element.
However, when I set Combiner as Reducer class
Hi Jens
These are not complete version of Hadoop.
*1) hadoop-0.20-mapreduce-0.20.2+1341 (has only MRv1)
2) hadoop-2.0.0+922 (has HDFS+ Yarn)*
I request you to read the comments in this link
https://issues.cloudera.org/browse/DISTRO-447
On Tue, Mar 19, 2013 at 1:17 PM, Jens Scheidtmann <
jens
Can you guys please take this thread to CDH mailing list?
Sent from phone
On Mar 20, 2013, at 2:48 PM, rohit sarewar wrote:
> Hi Jens
>
> These are not complete version of Hadoop.
> 1) hadoop-0.20-mapreduce-0.20.2+1341 (has only MRv1)
> 2) hadoop-2.0.0+922 (has HDFS+ Yarn)
>
> I request you t
Hi All,
Can you help me to copy one hbase table to another cluster hbase
(Table copy) .
Regards,
samir
Hi Samir,
Is this what you are looking for?
http://hbase.apache.org/book/ops_mgt.html#copytable
What kind of help do you need?
JM
2013/3/20 samir das mohapatra :
> Hi All,
> Can you help me to copy one hbase table to another cluster hbase (Table
> copy) .
>
> Regards,
> samir
Thanks, for reply
I need to copy the hbase table into another cluster through the java code.
Any example will help to me
On Wed, Mar 20, 2013 at 8:48 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Hi Samir,
>
> Is this what you are looking for?
>
> http://hbase.apache.org/book/op
Hi Samir,
Have you looked at the link I sent you?
You have a command line for that, you have an example, and if you need
to do it in Java, you san simply open the
org.apache.hadoop.hbase.mapreduce.CopyTable, look into it, and do
almost the same thing for your needs?
JM
2013/3/20 samir das mohap
Thanks for your reply. Some follow up questions below :
On Mar 20, 2013, at 5:35 AM, Алексей Бабутин wrote:
>
>
> dfs.balance.bandwidthPerSec in hdfs-site.xml.I think balancer cant help
> you,because it makes all the nodes equal.They can differ only on balancer
> threshold.Threshold =10 by d
yes, yes just i thought same thing.
many many thanks.
On Wed, Mar 20, 2013 at 8:55 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Hi Samir,
>
> Have you looked at the link I sent you?
>
> You have a command line for that, you have an example, and if you need
> to do it in Java, you
The answer to second question would be subjective.
Do you have specific use case in mind ?
Thanks
On Wed, Mar 20, 2013 at 9:07 AM, oualid ait wafli wrote:
> Hi,
>
> Which is the best HBase or Cassandra ?
> Which are the criteria to compare those tools( HBase and Cassandra)
>
> Thanks
>
Yes I have a data source which contains log files, I want to analyze those
files and store them
any idea ?
thanks
2013/3/20 Ted Yu
> The answer to second question would be subjective.
>
> Do you have specific use case in mind ?
>
> Thanks
>
>
> On Wed, Mar 20, 2013 at 9:07 AM, oualid ait wafli
What the hell, people.
Unsubscribe the way our ancestor did: USE THE UNSUBSCRIBE ADDRESS!
Fabio
Although I do agree with Fabio's sentiments on this, why don't we just
ignore these messages. Let people do some work to find how to unsubscribe J
From: Fabio Pitzolu [mailto:fabio.pitz...@gr-ci.com]
Sent: Wednesday, March 20, 2013 12:23 PM
To: user@hadoop.apache.org
Subject: RE: unsubscribe
Totally off topic, but kind'a not. Why the hell are we still using
something our ancestors used? I didn't even know listservs were still
in existence until I started using Apache open source software. I was
like, "listservs...really?"
On Wed, Mar 20, 2013 at 9:23 AM, Fabio Pitzolu wrote:
> Wh
unsubscribe
From: turboc...@gmail.com [turboc...@gmail.com] on behalf of John Conwell
[j...@iamjohn.me]
Sent: Wednesday, March 20, 2013 12:31 PM
To: user@hadoop.apache.org
Subject: Re: unsubscribe
Totally off topic, but kind'a not. Why the hell are we st
LOL. WTF.
Original Message
Subject:
From: "Jensen, Daniel"
Date: Wed, March 20, 2013 1:17 pm
To: "user@hadoop.apache.org"
unsubscribe
From: turboc...@gmail.com [turboc...@gmail.com] on behalf of John
Conwell [j...@iamjohn.me]
Sent
touche :)
On Wed, Mar 20, 2013 at 10:22 AM, Chris Folsom wrote:
>
>
> LOL. WTF.
>
>
> Original Message
> Subject:
> From: "Jensen, Daniel"
> Date: Wed, March 20, 2013 1:17 pm
> To: "user@hadoop.apache.org"
>
> unsubscribe
>
> From: tur
I'm sorry... I'm sure it can be interesting...
Here is the link to unsubscribe:
http://hadoop.apache.org/mailing_lists.html
From: Chris Folsom [jcfol...@pureperfect.com]
Sent: Wednesday, March 20, 2013 1:22 PM
To: user@hadoop.apache.org
Subject: RE:
LOL.
Hi Kishore,
50010 is the datanode port. Does your lsof indicate that the sockets are in
CLOSE_WAIT? I had come across an issue like this where that was a symptom.
-Sandy
On Wed, Mar 20, 2013 at 4:24 AM, Krishna Kishore Bonagiri <
write2kish...@gmail.com> wrote:
> Hi,
>
> I am running a date c
Hi,
I'm starting a project to build a 10 node cluster grid.
I've already successfully built a 10 node grid with hadoop 1.0.4.
This next grid would preferrably be the 0.23.X branch, which I think would
be the best version to smoothly transition to 2.0.3 release (right?)
When I was working with t
Can you give us more information about your use case ?
e.g. approximate ratio between write vs. read load, amount of log, etc.
Cheers
On Wed, Mar 20, 2013 at 9:22 AM, oualid ait wafli wrote:
> Yes I have a data source which contains log files, I want to analyze those
> files and store them
> an
The WebHDFS wiki has bad information regarding curl. According to RFC 2616
Section 10.3.8, a web client shall not automatically follow a 307 redirect
unless the response is from a GET or HEAD request. As jetty returns a 307
response for the PUT request against the WebHDFS api, curl does the
You're probably running into
https://issues.apache.org/jira/browse/MAPREDUCE-3310. There was also a
discussion on this IIRC but I cannot find the archive now. One aside
mention on this from Chris is at
http://search-hadoop.com/m/RH5AP11ob2o1.
On Wed, Mar 20, 2013 at 6:10 PM, Fatih Haltas wrote:
>
You are correct about your idea of clients. To talk to HDFS, they need
to be allowed to talk to the NN's ports as well as the DN's ports. To
talk to YARN/MR, they need access to both RM and NM ports (as well as
the JobHistoryServer's web port).
Aside of just a local install, they'll also need the
Really Thanks. It works!!
-- --
??: "ken barclay";
: 2013??3??20??(??) 2:19
??: "user";
: Re: Put a file to HDFS with WebHDFS ?
It didn't fail. If you hit the URL it returned, it will do the create.
--- On Wed, 3/20/13,
Thanks for your reply.
-- --
??: "Adam Faris";
: 2013??3??21??(??) 4:42
??: "";
: Re: Put a file to HDFS with WebHDFS ?
The WebHDFS wiki has bad information regarding curl. According to RFC 2616
Section 10.3.8, a web c
There was an issue related to hung connections (HDFS-3357). But the JIRA
indicates the fix is available in Hadoop-2.0.0-alpha. Still, would be worth
checking on Sandy's suggestion
On Wed, Mar 20, 2013 at 11:09 PM, Sandy Ryza wrote:
> Hi Kishore,
>
> 50010 is the datanode port. Does your lsof ind
Hi Hemanth & Sandy,
Thanks for your reply. Yes, that indicates it is in close wait state,
exactly like below:
java 30718 dsadm 200u IPv4 1178376459 0t0
TCP *:50010 (LISTEN)
java 31512 dsadm 240u IPv6 1178391921 0t0
TCP node1:51342->node1:5
38 matches
Mail list logo
