[Lustre-discuss] Lustre User Group 2010 - registration now open and call for presentations
*LUG 2010 - Eighth Annual Lustre User Group conference* When: April 14-16, 2010 Where: Seascape Beach Resort, Monterey Bay, CA Registration: http://LUG2010.org Registration is now open for LUG 2010, the premier event to learn new technical information about Lustre, acquire best practices, and share knowledge about Lustre technology. LUG 2010 is a unique opportunity for users to get advice and answers about specific Lustre implementations from Lustre developers, community engineers and users. Any and all people using or interested in Lustre are welcome to attend. Conference attendees will have access to experts and peers sharing their real-world experiences using Lustre. LUG 2010 will include updates on community development projects, Birds of a Feather sessions, demos and tutorials, and offer many opportunities to meet with the Lustre development team and discuss upcoming enhancements and capabilities. LUG 2010 will be held at the beautiful Seascape beach resort and conference center on Monterey Bay, California (http://seascaperesort.com). *Call for Presentations - LUG 2010* Lustre users are invited to submit an abstract to be considered for presentation at LUG 2010. Presentations are generally 30 minutes in length, but can be 1 hour if needed. It is not necessary to submit a technical paper, just an abstract of your proposed talk and then slides for the talk itself. This is your opportunity to contribute to the Lustre community by sharing your knowledge and lessons learned from Lustre deployments. The deadline to submit presentation abstracts is February 15, 2010. For questions or to submit a presentation abstract, contact Dan Ferber (daniel.fer...@sun.com). *Lustre Advanced User Seminar* The Lustre Advanced User Seminar will be offered on April 14, 2010, before the User Group meeting. This seminar is designed for senior systems administrators, engineers and integrators needing more comprehensive knowledge of Lustre Administration and Troubleshooting techniques. To receive maximum value from this seminar, completion of Administering Lustre-based Clusters (CL-400) and/or prior experience administering Lustre is strongly recommended. Space is limited and registration fee discounts will be offered for LUG attendees. To register for LUG 2010 and the Lustre Advanced User Seminar, visit our registration page at http://LUG2010.org See you at LUG 2010! ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] 'Rename' kernel build result
On Mon, 2009-11-30 at 13:21 +0100, Frank Heckes wrote: > Hello Brian, Hi Frank, > first of all thanks for your patience and answers so far. NP. > No, no I like the idea a lot, but unfortunately the kernel didn't boot > on the hardware we trying to install them (IBM pizzaboxes), even after > rebuilding the initrd, Systemmap... to extend the (initial) driver list. Hrm. Does the Suse-supplied kernel boot those machines or does it have the same problems? I just want to isolate whether this is a problem unique to our kernel build or whether it's a problem with the upstream (Suse supplied) kernel. > So I decided to built the kernel from source. Here the system boots as > it should after installing it to one of the nodes of the (test) > cluster. What do you do differently than is being done by our build? Do you change anything in the source-tree before you build? Probably it would be great to have a complete transcript of what you do once you have downloaded the kernel-lustre-source package from our download site. > And of course there's also a small piece of (childlike) curiosity, too. :-) > I fully agree. I would to the ready made RPM's if it would be possible. > (see above). Indeed. In fact rather than copying what could be some very copious data to this list, can you open a bug in our bugzilla and report back here for me (and the rest of the readers) what the ticket number is? > Here're the Makefile's first 10 lines: Unfortunately it was not enough. Once you have opened the ticket as per above, please attach the complete Makefile. b. signature.asc Description: This is a digitally signed message part ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] I/O on cluster with lustre
Hello, Recently the cluster I am using for computing has been updated to the VMware with the Lustre file system. Cluster uses: Oscar 6.0.3, Sun Grid Engine 6.2u3, Nagios, Ganglia, InfiniBand 10 Gb/s. Nodes access the file system using Ethernet via the Lustre InfiniBand/Ethernet router. I used to run one type of jobs as: --- #$ -N name #$ -o namesys.out #$ -e namesys.err #$ -pe mpi 2 #$ -cwd #$ -v LD_LIBRARY_PATH mpirun -machinefile $TMPDIR/machines -np $NSLOTS /path/.../code.x << EOF name.in name.out EOF --- This is for an open source package (written in Fortran plus some C utilities) and a such way of running was recommended by authors. It was working on the previous version of the cluster, but it does not run on a new lustre filesystem. It starts, but then stays in the queue forever. Is it possible to run this type of jobs on lustre ? Thank you. Best wishes, Goranka ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Anyone built 1.8 or 1.6 on Fedora 12's 2.6.31 yet?
I've got this morning's b1_8 branch to compile on Fedora 12. For those
interested, here are the steps.
(Disclaimer: I am not a programmer and the below changes were hacks I made
based on far too little knowledge. If I've done something horribly wrong,
please reply saying so).
0) Just remembereed - I happened to symlink /usr/src/linux to
kernels/2.6.31.5-127.fc12.x86_64 - Just me being lazy when running configure
too many times.
1) /usr/src/kernels/2.6.31.5-127.fc12.x86_64/include/linux/posix_acl.h needs
#include adding to it.
I don't know why and this is probably my hackiest hack, but if not, it fails to
compile in the configure and causes problems later on.
2) checkout the b1_8 tree and patch it with the attached.
3) sh autogen.sh
4) ./configure --disable-server
5) make rpms CFLAGS=-fno-strict-aliasing
I thought no-strict-aliasing was supposed to be picked up by default for kernel
related builds, but I needed to add this.
Once they're built and installed, strangely, the mgc module will not load
automatically. I had to manually modprobe mgc before I could then successfully
mount a Lustre FS. If anyone knows why or if there's a fix, please let me know.
Hope it helps...
- "Andreas Dilger" wrote:
> On 2009-11-19, at 01:48, Stephen Willey wrote:
> > https://bugzilla.lustre.org/show_bug.cgi?id=21113 seems to suggest
>
> > that Sun/Lustre will be building it as F12 should be the base for
> > RH6 (please correct me if I'm wrong) but I was just wondering if
> > anyone was way ahead and had tried this already?
>
> Some of our other customers are also interested in porting to FC12 as
>
> a stepping stone to RHEL6. We are getting closer on the client side,
>
> but the server still needs work. If anyone is interested to port the
>
> server code, then I'd be interested to hear, because I'd prefer to
> spend any development effort to remove the server patches entirely.
>
> Instead of reworking the patches yet again, this needs some effort in
>
> the Lustre and ldiskfs code to allow it to use newer kernel APIs and
>
> then (hopefully) we'll never need to patch for Lustre again.
>
> > https://bugzilla.lustre.org/show_bug.cgi?id=21340 suggests I can use
>
> > the 2.6.27 patches (are they rolled into 1.8.1.1??) plus the patch
>
> > in that thread to get it working. Again, I'll give it a go but I
> > just figured I'd ask before possibly redoing work that's already
> > been done.
>
>
> The 2.6.27 support (both client and server) should be in 1.8.1.1
> AFAIK, because it runs on SLES11.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
--
Stephen Willey
Senior Systems Engineer
Framestore
19-23 Wells Street, London W1T 3PQ
+44 207 344 8000
www.framestore.com
--- lustre/llite/lloop-old.c 2009-11-26 10:34:02.393003227 +
+++ lustre/llite/lloop.c 2009-11-26 10:35:03.590056455 +
@@ -500,7 +500,7 @@ static int loop_set_fd(struct lloop_devi
lo->lo_queue->unplug_fn = loop_unplug;
/* queue parameters */
-blk_queue_hardsect_size(lo->lo_queue, CFS_PAGE_SIZE);
+blk_queue_logical_block_size(lo->lo_queue, CFS_PAGE_SIZE);
blk_queue_max_sectors(lo->lo_queue,
LLOOP_MAX_SEGMENTS << (CFS_PAGE_SHIFT - 9));
blk_queue_max_phys_segments(lo->lo_queue, LLOOP_MAX_SEGMENTS);
--- lustre/llite/file-old.c 2009-11-26 10:37:51.401943992 +
+++ lustre/llite/file.c 2009-11-26 10:58:28.326208286 +
@@ -47,6 +47,7 @@
#include
#include "llite_internal.h"
#include
+#include
/* also used by llite/special.c:ll_special_open() */
struct ll_file_data *ll_file_data_get(void)
@@ -1423,7 +1424,7 @@ static ssize_t ll_file_aio_read(struct k
count = ll_file_get_iov_count(iov, &nr_segs);
CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p),size="LPSZ",offset=%Ld\n",
- inode->i_ino, inode->i_generation, inode, count, *ppos);
+ inode->i_ino, inode->i_generation, inode, (unsigned int)count, *ppos);
/* "If nbyte is 0, read() will return 0 and have no other results."
* -- Single Unix Spec */
if (count == 0)
@@ -1593,7 +1594,7 @@ repeat:
chunk = end - *ppos + 1;
CDEBUG(D_INODE,"Read ino %lu, "LPSZ" bytes, offset %lld, i_size %llu\n",
- inode->i_ino, chunk, *ppos, i_size_read(inode));
+ inode->i_ino, (unsigned int)chunk, *ppos, i_size_read(inode));
/* turn off the kernel's read-ahead */
if (ltd.lock_style != LL_LOCK_STYLE_NOLOCK) {
@@ -1773,7 +1774,7 @@ static ssize_t ll_file_aio_write(struct
count = ll_file_get_iov_count(iov, &nr_segs);
CDEBUG(D_VFSTRACE, "VFS Op:inode=%lu/%u(%p),size="LPSZ",offset=%Ld\n",
- inode->i_ino, inode->i_generation, inode, count, *ppos);
+ inode->i_ino, inode->i_generation, inode, (unsigned int)count, *ppos);
[Lustre-discuss] quota with vanilla kernel
Dear list, has anybody a running setup with Lustre servers 1.8 on a patched vanilla 2.6.22 kernel using quota? The system works fine otherwise, just quota don't work: I have configured a filesystem on 1.8.1.1 using the parameters ost.quota_type=ug and mdt.quota_type=ug, run 'lfs quotacheck' on a client, but when I run 'lfs quotaon' I get the response: error: quotaon failed (Device or resource busy) and a syslog line: kernel: [ 6243.093049] LustreError: 3766:0:(quota_ctl.c:381:client_quota_ctl()) ptlrpc_queue_wait failed, rc: -16 The client is a patchless 2.6.30.9/Lustre 1_8b. TIA, Ralf -- Ralf Utermann _ Universität Augsburg, Institut für Physik -- EDV-Betreuer Universitätsstr.1 D-86135 Augsburg Phone: +49-821-598-3231 SMTP: ralf.uterm...@physik.uni-augsburg.de Fax: -3411 ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] OSS server
On Mon, 2009-11-30 at 11:23 +0100, Stefano Elmopi wrote: > Hi, Hi, > - one server with MGS/MDS > - one server with OSS > - the OST is on SAN > > > If the server OSS has a hardware failure > and I must physically change the machine. If the machine boots from the network then the new system is a drop-in replacement for the broken machine and you need to do nothing, right? Or if the system boots from it's own disk you would either a) put the disk from the broken OSS into the new hardware or b) re-image the OSS and configure it to be identical (IP address, etc.) as the machine that died, or c) restore the system image from a backup or d) some other reasonable facsimile/combination of the above options that results in the new machine being a like-replacement for the broken machine. None of this is really all that particular to Lustre though and it just typical syadmin practises that one would follow for replacing any machine that dies. When your mail server dies, what do you do to replace it such that it's an identical (to the users) replacement? Same situation for an OSS. > With the new machines, how can I retrieve data on the OST, located on > SAN ? Once the new machine is configured identical to the one it's replacing, it will just access that OST as if nothing happened, no? > I hope to be clear enough !! I think you are. I guess the not-really-particular-to-Lustre nature of the question had me somewhat thinking that there must be something I was missing. I guess there was not. b. signature.asc Description: This is a digitally signed message part ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] MDS performance
Hi list, I know this question has been asked zillion times, but I'll ask it again, because as most lustre-builder wannabe, I'm worried about MDS being a bottleneck. I'm planning to build a 60TB lustre install separated in two pools : - one big / slow / cheap 50 TB archive based on Dell MD3000i+MD1000 RAID 5 OST with two cheap OSS in front of it - one beefier (=smaller / faster / more expensive) 10 TB based on 2 servers being both OSS+OST each composed of 8 cores, 12 GB RAM, 8 NL SAS spindles RAID 10. the big / slow archive is not really a problem but i'm asking myself about the smaller one, given the fact that this storage (if good enough in terms of performance) could double pretty soon by adding more OSS+OST I've been planning to have one redundant MDS over 2 Dell R610 sharing a common sas-attached MD3000 RAID 10 MDT. I've noticed that this array is not the best on the market for perf. but is rather good at $$... I plan on connecting all this on DDR infiniband (along with the main clients -4 of them-). So now for my questions about MDS : - Should I do better having two lustre installs instead of a single install with two pools ? - Should I consider buying a better array for MDT ? - Should I be better using even beefier MDS with internal storage and DRBD ? and about OSS : - should I take "smaller" OSS+OST to improve perf ? - should I split my OST storage "inside" the OSSes for the 10GB storage pool ? I'll be sooo grateful if someone could answer those, I'll be glad to provide any other details one would need to help me out. Thanks. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] 'Rename' kernel build result
Hello Brian, first of all thanks for your patience and answers so far. > On Thu, 2009-11-26 at 15:37 +0100, Frank Heckes wrote: > > Hello Brian, > > Hi, > > > Well, I downloaded two 'source' files: > > Hrm. Why are you dealing with source files? Does our already-built > binary kernel RPMs not suit your environment for some reason? > No, no I like the idea a lot, but unfortunately the kernel didn't boot on the hardware we trying to install them (IBM pizzaboxes), even after rebuilding the initrd, Systemmap... to extend the (initial) driver list. So I decided to built the kernel from source. Here the system boots as it should after installing it to one of the nodes of the (test) cluster. And of course there's also a small piece of (childlike) curiosity, too. > > -1- kernel-lustre-source-2.6.27.29-0.1_lustre.1.8.1.1.src.rpm > > -2- kernel-lustre-source-2.6.27.29-0.1_lustre.1.8.1.1.x86_64.rpm > > > > I used the second for the compilation since the first only install stuff > > in /usr/src/packages/SOURCE and some more complicated procedure has to > > be used for compilation. > > The first is a source rpm which is built using the rpmbuild command. > > > Entries for /usr/src/linux-2.6.27.29-0.1_lustre.1.8.1.1 > > Yes. > > > and /lib/modules-2.6.27.29-0.1_lustre.1.8.1.1 > > Well, just the /lib/modules/2.6.27.29-0.1_lustre.1.8.1.1-default/source > symlink so this is really quite irrelevant. It's > the /usr/src/linux-2.6.27.29-0.1_lustre.1.8.1.1 tree that you are > interested in. > > > will be created when installing rpm '-2-'. > > > Running 'make rpm' leads to > > an RPM with the specified install pathes inside the RPM. > > Can you tell me what the first 10 lines > of /usr/src/linux-2.6.27.29-0.1_lustre.1.8.1.1-obj/x86_64/default/Makefile > say? You might need to adjust some of those lines to make the version say > what you want it to say. > > But most definitely, things are a lot easier if you can use our binary > RPM packages. I'd be interested in knowing why they are not suitable > for you. > I fully agree. I would to the ready made RPM's if it would be possible. (see above). Here're the Makefile's first 10 lines: node006:~ # head -10 /usr/src/linux-2.6.27.29-0.1_lustre.1.8.1.1-obj/x86_64/default/Makefile # Automatically generated by ../../../linux-2.6.27.29-0.1_lustre.1.8.1.1/scripts/mkmakefile: don't edit VERSION = 2 PATCHLEVEL = 6 lastword = $(word $(words $(1)),$(1)) makedir := $(dir $(call lastword,$(MAKEFILE_LIST))) MAKEARGS := -C ../../../linux-2.6.27.29-0.1_lustre.1.8.1.1 MAKEARGS += O=$(if $(patsubst /%,,$(makedir)),$(CURDIR)/)$(patsubst %/, %,$(makedir)) > > So kernel '-2-' shouldn't be used for compilation? > > That should be fine. > Okay. That's the one I used. > b. > Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] OSS server
Hi, I apologize for my lack of clarity, try to be clearer. The scenario is: - one server with MGS/MDS - one server with OSS - the OST is on SAN If the server OSS has a hardware failure and I must physically change the machine. With the new machines, how can I retrieve data on the OST, located on SAN ? I hope to be clear enough !! Thanks Ing. Stefano Elmopi Gruppo Darco - Resp. ICT Sistemi Via Ostiense 131/L Corpo B, 00154 Roma cell. 3466147165 tel. 0657060500 email:stefano.elm...@sociale.it Il giorno 20/nov/09, alle ore 20:00, lustre-discuss-requ...@lists.lustre.org ha scritto: Date: Fri, 20 Nov 2009 10:53:15 -0500 From: "Brian J. Murrell" Subject: Re: [Lustre-discuss] OSS server To: lustre-discuss@lists.lustre.org Message-ID: <1258732395.21860.30.ca...@pc.interlinx.bc.ca> Content-Type: text/plain; charset="utf-8" On Fri, 2009-11-20 at 14:37 +0100, Stefano Elmopi wrote: If I have a simple scenario with one MGS/MDS and one OSS, without failover, and the server OSS crash, I'm afraid "crash" is an ambiguous term in this context. Do you mean the O/S panics or some equivalent event? The solution would be to simply reboot the machine, no? Or do you mean that the hardware itself has broken such that you need to replace it? Wouldn't you just move the disks in the old machine to the new one and boot that so that as far as the rest of the network is concerned, the machine was simply rebooted? Simply replace the server root with one that has the same IP ?? Yes, I think so. Although your scenario is not terribly clear for me. b. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091120/bbdcf19b/attachment-0001.bin ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
