---------- Forwarded message ---------- From: linux freaker Date: Monday, March 18, 2013 Subject: Understanding lustre setup .. To: Colin Faber <colin_fa...@xyratex.com>
On Monday, March 18, 2013, Colin Faber <colin_fa...@xyratex.com> wrote: > Hi, > > On 03/15/2013 10:56 AM, linux freaker wrote: >> >> Let me explain in brief the clear picture I would need to understand: >> >> >> Aim: Comparing Hadoop over HDFS Vs Hadoop over Lustre >> >> Things I tried: >> >> I took 5 machines: 1 MDS, 2 OSS/OST and 2 Lustre Client.(each of 4GB RAM and 700 GB hard disk). >> I created around 6 OST on each OSS for size 6GB each through LVM. > > Using LVM based OST's can slow things down a lot. Really for best performance you should be using unpartitioned raw disk for ldiskfs formating. > >> >> >> >> On Wed, Mar 13, 2013 at 8:01 AM, linux freaker >> <linuxfrea...@gmail.com<mailto: linuxfrea...@gmail.com>> wrote: >> > >> > >> > On Wednesday, March 13, 2013, Colin Faber <colin_fa...@xyratex.com<mailto: colin_fa...@xyratex.com>> wrote: >> >> Hi, >> >> >> >> I'm sorry, It was a busy day for me. I will try and respond appropriately >> >> to your questions tomorrow. >> >> >> >> -cf >> >> >> >> On 03/12/2013 07:45 PM, linux freaker wrote: >> >>> >> >>> >> >>> On Tuesday, March 12, 2013, linux freaker <linuxfrea...@gmail.com<mailto: linuxfrea...@gmail.com> >> >>> <mailto:linuxfrea...@gmail.com <mailto:linuxfrea...@gmail.com>>> wrote: >> >>> > >> >>> > >> >>> > On Tuesday, March 12, 2013, linux freaker >> >>> > <linuxfrea...@gmail.com<mailto: linuxfrea...@gmail.com> >> >>> > <mailto:linuxfrea...@gmail.com <mailto:linuxfrea...@gmail.com>>> wrote: >> >>> >> >> >>> >> >> >>> >> On Tuesday, March 12, 2013, Colin Faber >> >>> >> <colin_fa...@xyratex.com<mailto: colin_fa...@xyratex.com> >> >>> >> <mailto:colin_fa...@xyratex.com <mailto:colin_fa...@xyratex.com>>> wrote: >> >>> >>> Hi, >> >>> >>> >> >>> >>> On 03/11/2013 08:13 PM, linux freaker wrote: >> >>> >>>> >> >>> >>>> >> >>> >>>> On Tuesday, March 12, 2013, linux freaker < linuxfrea...@gmail.com <mailto:linuxfrea...@gmail.com> >> >>> >>>> <mailto:linuxfrea...@gmail.com <mailto:linuxfrea...@gmail.com>> <mailto:Thanks for response. I got what you suggested. so i will have 1 mds, 2 oss and 2 lustre client.I will install hadoop on both the lustre client. May I know what are the steps to configure hadoop over lustre? I tried to configure core-site.xml but no idea what other files chinch I need. Can you share the steps? Also, i am trying to run wordcount example over lustre.Please suggest.
_______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
