check zkfc logs first ; try checking the HDFS ha and zookeeper timeouts,
its better to have a dedicated disk for journal node service (similar to
zookeeper)
On Sat, Dec 19, 2015 at 9:29 AM, Sandeep Nemuri
wrote:
> What does the logs say ?
> ᐧ
>
> On Sat, Dec 19, 2015 at 10:08 PM, Marcin Tustin
+1 Zhijie, or if that doesn't work may be you can run
*ps -aef | grep hadoop*
on terminal and check the value of *-Dyarn.log.dir*,
that should give you where are logs getting printed.
On Thu, Jul 2, 2015 at 10:13 AM, Zhijie Shen wrote:
> That's just some warnings from web component. It shou
. I am using hadoop version 2.5.1, if that helps.
Has anyone seen this before? Any ideas on how do I go about this one?
thanks,
Nikhil
-
2015-02-24 16:47:37,555 INFO org.apache.hadoop.ha.ActiveStandbyElector:
Yielding from election
2015-02-24 16:47:37,555 INFO
>>cache 1 entries in the state store.
May I know what are 1 entries you are talking about here? Any details
please.
On Tue, Feb 10, 2015 at 9:59 PM, Suma Shivaprasad <
sumasai.shivapra...@gmail.com> wrote:
> We are planning to deploy Hadoop 2.6.0 with a default configuration to
> cache 10
Hi All,
I have a system where files are coming in hdfs at regular intervals and I
perform an operation everytime the directory size goes above a particular point.
My Question is that when I submit a map reduce job, would it only work on the
files present at that point ??
Regards,
Nikhil Kandoi
solution that I think would help solve the problem.
Regards,
Nikhil
From: Shekhar Sharma [mailto:shekhar2...@gmail.com]
Sent: Tuesday, December 17, 2013 9:12 PM
To: user@hadoop.apache.org
Subject: Re: Estimating the time of my hadoop jobs
Apart from what Devin has suggested there are other
I know this foolish of me to ask this, because there are a lot of factors that
affect this,
but why is it taking so much time, can anyone suggest possible reasons for it,
or if anyone has faced such issue before
Thanks,
Nikhil Kandoi
P.S - I am Hadoop-1.0.3 for this application, so I wonder
l nodes on Virtual
Machines) , all nodes have 4GB ram.
But when I try to run the dataset (which is a sample dataset at this point ) of
about 30 GB, it takes about 3 hrs to process all of it.
I would like to know is it normal for this kind of infrastructure to take this
amount of time.
Thank you
Nikhil Kandoi/
Hi,
Thank you for your reply. One simple answer can be to reduce the time taken for
ingesting the data in HDFS.
Regards,
Nikhil
From: Sanjay Subramanian [mailto:sanjay.subraman...@wizecommerce.com]
Sent: Friday, May 31, 2013 12:50 PM
To:
Cc: user@hadoop.apache.org
Subject: Re: MapReduce on
Hi Rahul,
Can you please explain what do you mean by “filer directory mounted to all the
DN” ? Do you mean a NFS-mount? If yes then I want to avoid NFS-mount. With
NFS-mount it is possible to do it.
Thanks & Regards,
Nikhil
From: Rahul Bhattacharjee [mailto:rahul.rec@gmail.com]
ter" node.
Please let me know if it is possible to do this.
Thanks & Regards,
Nikhil
temporary directories. which needs to
be read by all TaskTrackers. So, how does HDFS share the contents? Does it use
nfs mount or ?
Thanks & Regards,
Nikhil
that entire file A was
transferred to the other node?
Regards,
Nikhil
-Original Message-
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Thursday, May 16, 2013 11:47 AM
To:
Subject: Re: Map Tasks do not obey data locality principle
The scheduling is done based on block locations
No, it does not. I have kept the granularity at file level rather than a
block. I do not think that should affect the mapping of tasks ?
Regards,
Nikhil
-Original Message-
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Thursday, May 16, 2013 2:31 AM
To:
Subject: Re: Map Tasks do
is submitted to TaskTracker running on
slave2 then it implies that file A is being fetched over the network by slave2.
How do I avoid this from happening?
Thanks,
Nikhil
Hi,
I got it. The log info is printed in userlogs folder in slave nodes, in the
file syslog.
Thanks,
Nikhil
-Original Message-
From: Agarwal, Nikhil
Sent: Monday, May 13, 2013 4:10 PM
To: 'user@hadoop.apache.org'
Subject: RE: How to combine input files for a MapReduce job
.
Regards,
Nikhil
-Original Message-
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Monday, May 13, 2013 1:28 PM
To:
Subject: Re: How to combine input files for a MapReduce job
Yes I believe the branch-1 patch attached there should apply cleanly to 1.0.4.
On Mon, May 13, 2013 at 1:25 PM
overheads which will bog you down.
For "which file is being submitted to which" question: Having
https://issues.apache.org/jira/browse/MAPREDUCE-3678 in the
version/distribution of Apache Hadoop you use would help.
On Mon, May 13, 2013 at 12:50 PM, Agarwal, Nikhil
wrote:
> Hi,
>
for your time.
Regards,
Nikhil
sets were
exactly processed by each of the two slaves.
Can you please tell me some way to find out what exactly did each of my
tasktracker do during the entire job execution? I am using Hadoop-1.0.4 source
code.
Thanks & Regards,
Nikhil
>From http://archive.cloudera.com/cdh/3/hadoop/hdfs_user_guide.html
(Assuming you are using Cloudera Hadoop Distribution 3)
$ hadoop dfsadmin -refreshNodes # would help do the same.
-refreshNodes : Updates the set of hosts allowed to connect to namenode.
Re-reads the config file to update values
master node (where JobTracker is running) is linux then can I have
slave node as windows machine and start Tasktracker on it without putting
Hadoop source code?
Thank you for the help.
Regards,
Nikhil
4312. Please suggest where am I going wrong.
Also, if you feel cascading would be better for this then please do let me know.
Thanks & Regards,
Nikhil
From: Agarwal, Nikhil
Sent: Tuesday, March 26, 2013 2:49 PM
To: 'user@hadoop.apache.org'
Subject: How to tell my Hadoop cluster to read d
?
Thanks & Regards,
Nikhil
org.apache.http does not
exist" although I have included the required jars.
Can anyone suggest what could have possibly gone wrong?
Thanks & Regards,
Nikhil
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Can you please suggest what might have gone wrong.
Thanks,
Nikhil
From: Agarwal, Nikhil
Sent: Monday, March 18, 2013 5:16 PM
To: 'user@hadoop.apache.org'
Subject:
Hi,
Thanks for the quick reply. In order to test the class
TestInMemoryNativeS3FileSystemContract and its functions what should be the
value of parameter sin my configuration files (core-site, mapred, etc.)?
Regards,
Nikhil
From: Agarwal, Nikhil
Sent: Monday, March 18, 2013 1:55 PM
To
in advance.
Regards,
Nikhil
by the function getFileBlockLocations
(FileStatus file, long start, long len).
Thank you very much for your time and helping me out.
Regards,
Nikhil
From: Agarwal, Nikhil
Sent: Thursday, February 21, 2013 4:19 PM
To: 'user@hadoop.apache.org'
Subject: How to add another file system in H
-site.xml or other
configuration files to use CDMI file system. Where all do I need to make
changes to enable CDMI file system become a part of Hadoop ?
Thanks a lot in advance.
Regards,
Nikhil
Thanks Manoj for your answer. :)
That helped.
From: Agarwal, Nikhil
Sent: Tuesday, February 19, 2013 4:53 PM
To: 'user@hadoop.apache.org'
Subject: Which class or method is called first when i run a command in hadoop
Hi All,
Thanks for your answers till now. I was trying to de
executed.
Thanks & Regards,
Nikhil
ing
compute to data).
Thanks,
Nikhil
From: Agarwal, Nikhil
Sent: Sunday, February 17, 2013 11:53 AM
To: 'user@hadoop.apache.org'
Subject: Can I perfrom a MR on my local filesystem
Hi,
Recently I followed a blog to run Hadoop on a single node cluster.
I wanted to ask that in a sing
?
In the Hadoop source code I saw there are implementations of other file systems
too like S3, KFS, FTP, etc. so how does exactly a MR happen on S3 data store ?
How does JobTracker or Tasktracker run in S3 ?
I would be very thankful to get a reply to this.
Thanks & Regards,
Nikhil
store.
Thanks & Regards,
Nikhil
.
Can you please guide me as how to do so. I am using Ubuntu 12.04. Please give
me some resource by which I can get this done.
Thanks & Regards,
Nikhil
36 matches
Mail list logo
