Dear Hadoop community,
A strange problem happened in our fresh installed hdfs cluster. The
datanode cannot start because two threads tried to load storage, the latter
would fail and lead a FATAL error.
As the following log shown, the blue text
means DataStorage::addStorageLocations is called, the
D
matches. --> "attempt to load an used block storage:
/vdir/mnt/disk2/hadoop/data/current/BP-1558632922-172.16.2.238-1587535931027"
On 22/4/20 9:50 pm, Wenqi Ma wrote:
Dear Hadoop community,
A strange problem happened in our fresh installed hdfs cluster. The
datanode cannot start becau
Hi,
I am setting up HDFS security with Kerberos:
When I manually started the first datanode, I got the following messages (the
namenode is started):
1) INFO org.apache.hadoop.security.UserGroupInformation: Login successful for
user
2) ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
Secure DN needs to be started as root (it runs as proper user, but
needs to be started as root to grab reserved ports), and needs a
proper jsvc binary (for your arch/OS) available. Are you using
tarballs or packages (and if packages, are they from Bigtop)?
On Mon, Nov 26, 2012 at 5:21 PM, a...@hsk
Hi Harsh,
Thank you very much for your reply, got it!
Thanks
ac
On 26 Nov 2012, at 8:32 PM, Harsh J wrote:
> Secure DN needs to be started as root (it runs as proper user, but
> needs to be started as root to grab reserved ports), and needs a
> proper jsvc binary (for your arch/OS) available. A
Hi,
A question:
I started Secure DN then ran JPS as root, I could not find any running DN:
16152
16195 Jps
However, when I tried to start the secure DN again, I got:
Warning: $HADOOP_HOME is deprecated.
datanode running as process 16117. Stop it first.
Does it mean JPS is no longer a tool to c
The 16152 should be the DN JVM I think. This is a jps limitation, as
seen at http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jps.html
and jsvc (which secure mode DN uses) is such a custom launcher.
"The jps command uses the java launcher to find the class name and
arguments passed to the m
Hi,
Thanks for your reply.
However, I think 16152 should not be the DN, since
1) my second try of "/usr/local/hadoop/bin/hadoop-daemon.sh start datanode"
says 16117 (i.e. I ran start datanode twice), and
2) ps axu | grep 16117, I got
root 16117 0.0 0.0 17004 904 pts/2S21:34 0
Could you also check what 16152 is? The jsvc is a launcher process,
not the JVM itself.
As I mentioned, JPS is pretty reliable, just wont' show the name of
the JVM launched by a custom wrapper - and will show just PID.
On Mon, Nov 26, 2012 at 7:35 PM, a...@hsk.hk wrote:
> Hi,
>
> Thanks for your
You could also run $sudo service --status-all
-Original Message-
From: a...@hsk.hk [mailto:a...@hsk.hk]
Sent: Monday, November 26, 2012 8:41 AM
To: user@hadoop.apache.org
Cc: a...@hsk.hk
Subject: Re: Datanode: "Cannot start secure cluster without privileged
resources"
Hi,
Hi,
I think you are right!
1) $ jps
16152
16500 Jps
2) ps axu | grep 16152
hduser 16152 0.1 1.4 1834900 116760 ? Sl 21:34 0:06 jsvc.exec
-Dproc_datanode -outfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.out
-errfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.err -pidfile
ubject: Re: Datanode: "Cannot start secure cluster without privileged
resources"
Hi,
I think you are right!
1) $ jps
16152
16500 Jps
2) ps axu | grep 16152
hduser 16152 0.1 1.4 1834900 116760 ? Sl 21:34 0:06 jsvc.exec
-Dproc_datanode -outfile /usr/local/hadoop-1.0.4/libe
12 matches
Mail list logo
