[Lustre-discuss] lustre showing inactive devices
I installed 1 MDS , 2 OSS/OST and 2 Lustre Client. My MDS shows: [code] [root@MDS ~]# lctl list_nids 10.94.214.185@tcp [root@MDS ~]# [/code] On Lustre Client1: [code] [root@lustreclient1 lustre]# lfs df -h UUID bytesUsed Available Use% Mounted on lustre-MDT_UUID 4.5G 274.3M3.9G 6% /mnt/lustre[MDT:0] lustre-OST_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:0] lustre-OST0001_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:1] lustre-OST0002_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:2] lustre-OST0003_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:3] lustre-OST0004_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:4] lustre-OST0005_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:5] lustre-OST0006_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:6] lustre-OST0007_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:7] lustre-OST0008_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:8] lustre-OST0009_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:9] lustre-OST000a_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:10] lustre-OST000b_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:11] filesystem summary:70.9G3.2G 64.0G 5% /mnt/lustre [/code] But Lustre Client2 is displaying it as: [code] [root@alpha ~]# lfs df -h UUID bytesUsed Available Use% Mounted on lustre-MDT_UUID 4.5G 274.3M3.9G 6% in in lustre-OST_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:0] lustre-OST0001_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:1] lustre-OST0002_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:2] lustre-OST0003_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:3] lustre-OST0004_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:4] lustre-OST0005_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:5] OST0006 : inactive device OST0007 : inactive device OST0008 : inactive device OST0009 : inactive device OST000a : inactive device OST000b : inactive device filesystem summary:35.4G1.6G 32.0G 5% /mnt/lustre [/code] Why is it showing inactive device on one machine while not in another? ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Need help on lustre filesystem setup..
Hi,
I am trying to run Apache Hadoop project on parallel filesystem like
lustre. I have 1 MDS, 2 OSS/OST and 1 Lustre Client.
My lustre client shows:
Code:
[root@lustreclient1 ~]# lfs df -h
UUID bytesUsed Available Use% Mounted on
lustre-MDT_UUID 4.5G 274.3M3.9G 6% /mnt/lustre[MDT:0]
lustre-OST_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:0]
lustre-OST0001_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:1]
lustre-OST0002_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:2]
lustre-OST0003_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:3]
lustre-OST0004_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:4]
lustre-OST0005_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:5]
lustre-OST0006_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:6]
lustre-OST0007_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:7]
lustre-OST0008_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:8]
lustre-OST0009_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:9]
lustre-OST000a_UUID 5.9G 276.1M5.3G 5%
/mnt/lustre[OST:10]
lustre-OST000b_UUID 5.9G 276.1M5.3G 5%
/mnt/lustre[OST:11]
filesystem summary:70.9G3.2G 64.0G 5% /mnt/lustre
As I was unsure about which machine I need to install Hadoop
softwareon, I decided to go ahead with installing Hadoop on
LustreClient1.
I configured LustreClient1 with JAVA_HOME and HADOOP parameter with
the following files entry:
File: conf/core-site.xml
Code:
property
namefs.default.name/name
valuefile:///mnt/lustre/value
/property
property
namemapred.system.dir/name
value${fs.default.name}/hadoop_tmp/mapred/system/value
descriptionThe shared directory where MapReduce stores control
files.
/description
/property
I dint make changes in mapred-site.xml.
Now when I start 'bin/start-mapred.sh' which tried to ssh to my own
local machine. I am not sure if I am doing right.
Doubt Do I need to have two Lustre Client for this to work?
Then I tried running wordcount program shown below:
Code:
bin/hadoop jar hadoop-examples-1.1.1.jar wordcount /tmp/rahul
/tmp/rahul/rahul-output
ied 0 time(s); retry policy is
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1
SECONDS)
13/03/14 18:12:29 INFO ipc.Client: Retrying connect to server:
10.94.214.188/10.94.214.188:54311. Already tried 1 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
sleepTime=1 SECONDS)
13/03/14 18:12:30 INFO ipc.Client: Retrying connect to server:
10.94.214.188/10.94.214.188:54311. Already tried 2 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
sleepTime=1 SECONDS)
13/03/14 18:12:31 INFO ipc.Client: Retrying connect to server:
10.94.214.188/10.94.214.188:54311. Already tried 3 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
sleepTime=1 SECONDS)
13/03/14 18:12:32 INFO ipc.Client: Retrying connect to server:
10.94.214.188/10.94.214.188:54311. Already tried 4 time(s); retry
policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
sleepTime=1 SECONDS)
Question:1. As I have been comparing HDFS and Lustre for Hadoop, what
would be the right number of hardware nodes to compare?Say, I have 1
MDS, 2 OSS and 1 Lustre Client, on the other hand, 1 Namenode and 3
datanodes? How can I compare both FS?
Question:2. Do I really need 2 lustre client to setup Hadoop over
Lustre? if it is possible, how can I use OSS and MDS too for Hadoop
setup?
Question:3. As I read regarding the wordcount example, we need to
insert data into HDFS filesystem, do we need to do same for Lustre
too? Whats the command?
Question:4. What are the steps to confirm if HAdoop is actually using lustre FS?
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Understanding lustre setup ..
Hi Colin,
Thanks for the response. So you mean to say I need to install Hadoop
on just one lustre client.
Is it enough to setup Hadoop? But how come I will start
datanode.Where will it run?
Shall I need to keep OSS and MDS untouched for Hadoop?
I have wordcount application testing to be done. Can you suggest me
the steps to configure Hadoop under this setup?
On Tue, Mar 12, 2013 at 12:23 AM, Colin Faber colin_fa...@xyratex.com wrote:
Hi,
In general the clients are isolated nodes which only act as single client
nodes accessing the greater file system (MGS, MDS, and OSS (OSTS))
You can run a client mount on any of these nodes however it's not
recommended as it can lead to deadlock and memory contention problems.
Ideally your hadoop data source from lustre would be a lustre client. This
can consist of a network boot machine which then mounts Lustre FS via it's
interconnect.
So think of it like this:
(LUSTRE: [MGS - MDS] - [OSS { OST OST OST }]) - Client A (mount
/mnt/lustre)
|__^
Then Client A acts as data store exporting /mnt/lustre to your hadoop
cluster.
I hope this makes sense =)
-cf
On 03/11/2013 11:33 AM, linux freaker wrote:
Hello,
I have been reading
http://wiki.lustre.org/images/1/1b/Hadoop_wp_v0.4.2.pdf for setting up
Hadoop over lustre.
Generally in hadoop setup, we have 1 Namenode and various number of
datanodes.
If I want to setup the same keeping Lustre as backend, in the document
it is mentioned that:
.Our experiments run on cluster with 8 nodes in total,
one is mds/namenode, the rest are
OSS/DataNode.
I wonder where does the Lustre Client fit here?
For Hadoop to work , we mention filesystem parameter esp /lustre here.
We dont have /lustre on OSS.
How is it possible?
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Understanding lustre setup ..
Can you please suggest?
On Mon, Mar 18, 2013 at 1:26 PM, linux freaker linuxfrea...@gmail.com wrote:
Hi Colin,
Thanks for the response. So you mean to say I need to install Hadoop
on just one lustre client.
Is it enough to setup Hadoop? But how come I will start
datanode.Where will it run?
Shall I need to keep OSS and MDS untouched for Hadoop?
I have wordcount application testing to be done. Can you suggest me
the steps to configure Hadoop under this setup?
On Tue, Mar 12, 2013 at 12:23 AM, Colin Faber colin_fa...@xyratex.com wrote:
Hi,
In general the clients are isolated nodes which only act as single client
nodes accessing the greater file system (MGS, MDS, and OSS (OSTS))
You can run a client mount on any of these nodes however it's not
recommended as it can lead to deadlock and memory contention problems.
Ideally your hadoop data source from lustre would be a lustre client. This
can consist of a network boot machine which then mounts Lustre FS via it's
interconnect.
So think of it like this:
(LUSTRE: [MGS - MDS] - [OSS { OST OST OST }]) - Client A (mount
/mnt/lustre)
|__^
Then Client A acts as data store exporting /mnt/lustre to your hadoop
cluster.
I hope this makes sense =)
-cf
On 03/11/2013 11:33 AM, linux freaker wrote:
Hello,
I have been reading
http://wiki.lustre.org/images/1/1b/Hadoop_wp_v0.4.2.pdf for setting up
Hadoop over lustre.
Generally in hadoop setup, we have 1 Namenode and various number of
datanodes.
If I want to setup the same keeping Lustre as backend, in the document
it is mentioned that:
.Our experiments run on cluster with 8 nodes in total,
one is mds/namenode, the rest are
OSS/DataNode.
I wonder where does the Lustre Client fit here?
For Hadoop to work , we mention filesystem parameter esp /lustre here.
We dont have /lustre on OSS.
How is it possible?
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] lustre showing inactive devices
Hi, each client must maintain access to each OST / MDT individually. Likely that client 2 is having some connectivity issues to the OSS's hosting those OST's, and client 1 is not. Without detailed logging it's impossible to determine why. -cf On 03/18/2013 01:41 AM, linux freaker wrote: I installed 1 MDS , 2 OSS/OST and 2 Lustre Client. My MDS shows: [code] [root@MDS ~]# lctl list_nids 10.94.214.185@tcp [root@MDS ~]# [/code] On Lustre Client1: [code] [root@lustreclient1 lustre]# lfs df -h UUID bytesUsed Available Use% Mounted on lustre-MDT_UUID 4.5G 274.3M3.9G 6% /mnt/lustre[MDT:0] lustre-OST_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:0] lustre-OST0001_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:1] lustre-OST0002_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:2] lustre-OST0003_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:3] lustre-OST0004_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:4] lustre-OST0005_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:5] lustre-OST0006_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:6] lustre-OST0007_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:7] lustre-OST0008_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:8] lustre-OST0009_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:9] lustre-OST000a_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:10] lustre-OST000b_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:11] filesystem summary:70.9G3.2G 64.0G 5% /mnt/lustre [/code] But Lustre Client2 is displaying it as: [code] [root@alpha ~]# lfs df -h UUID bytesUsed Available Use% Mounted on lustre-MDT_UUID 4.5G 274.3M3.9G 6% in in lustre-OST_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:0] lustre-OST0001_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:1] lustre-OST0002_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:2] lustre-OST0003_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:3] lustre-OST0004_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:4] lustre-OST0005_UUID 5.9G 276.1M5.3G 5% /mnt/lustre[OST:5] OST0006 : inactive device OST0007 : inactive device OST0008 : inactive device OST0009 : inactive device OST000a : inactive device OST000b : inactive device filesystem summary:35.4G1.6G 32.0G 5% /mnt/lustre [/code] Why is it showing inactive device on one machine while not in another? ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Fwd: Understanding lustre setup ..
-- Forwarded message -- From: linux freaker Date: Monday, March 18, 2013 Subject: Understanding lustre setup .. To: Colin Faber colin_fa...@xyratex.com On Monday, March 18, 2013, Colin Faber colin_fa...@xyratex.com wrote: Hi, On 03/15/2013 10:56 AM, linux freaker wrote: Let me explain in brief the clear picture I would need to understand: Aim: Comparing Hadoop over HDFS Vs Hadoop over Lustre Things I tried: I took 5 machines: 1 MDS, 2 OSS/OST and 2 Lustre Client.(each of 4GB RAM and 700 GB hard disk). I created around 6 OST on each OSS for size 6GB each through LVM. Using LVM based OST's can slow things down a lot. Really for best performance you should be using unpartitioned raw disk for ldiskfs formating. On Wed, Mar 13, 2013 at 8:01 AM, linux freaker linuxfrea...@gmail.commailto: linuxfrea...@gmail.com wrote: On Wednesday, March 13, 2013, Colin Faber colin_fa...@xyratex.commailto: colin_fa...@xyratex.com wrote: Hi, I'm sorry, It was a busy day for me. I will try and respond appropriately to your questions tomorrow. -cf On 03/12/2013 07:45 PM, linux freaker wrote: On Tuesday, March 12, 2013, linux freaker linuxfrea...@gmail.commailto: linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com wrote: On Tuesday, March 12, 2013, linux freaker linuxfrea...@gmail.commailto: linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com wrote: On Tuesday, March 12, 2013, Colin Faber colin_fa...@xyratex.commailto: colin_fa...@xyratex.com mailto:colin_fa...@xyratex.com mailto:colin_fa...@xyratex.com wrote: Hi, On 03/11/2013 08:13 PM, linux freaker wrote: On Tuesday, March 12, 2013, linux freaker linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com mailto:linuxfrea...@gmail.com mailto:Thanks for response. I got what you suggested. so i will have 1 mds, 2 oss and 2 lustre client.I will install hadoop on both the lustre client. May I know what are the steps to configure hadoop over lustre? I tried to configure core-site.xml but no idea what other files chinch I need. Can you share the steps? Also, i am trying to run wordcount example over lustre.Please suggest. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
