[Lustre-discuss] Proposed LNET config

2009-10-13 Thread Erik Froese
Hello List, I wanted to outline our proposed configuration and hope for some feedback from the list on whether the LNET config is sound. We're planning a site-wide (HPC clusters) Lustre filesystem at NYU to be installed in the next few weeks. Lustre version 1.8.1. The MDS, OSS, and routers will b

Re: [Lustre-discuss] Memory (?) problem with 1.8.1

2009-10-13 Thread Andreas Dilger
On 13-Oct-09, at 04:58, Brian J. Murrell wrote: > On Mon, 2009-10-12 at 17:06 -0700, David Simas wrote: >> During this operation, we notice the value of "buffers" >> reported in '/proc/meminfo' on the OSSs involved increasing >> monotonically >> until it apparently take up all the system's memory

Re: [Lustre-discuss] Lustre-discuss Digest, Vol 45, Issue 27

2009-10-13 Thread Yujun Wu
Hello Brian, Thanks for your e-mail. I believe our situation may be the later case---I need find out. Regards, Yujun On Tue, 13 Oct 2009 lustre-discuss-requ...@lists.lustre.org wrote: >5. Re: mounting lustre client behind firewall (Brian J. Murrell) > Message: 5 > Date: Tue, 13 Oct 2009 12:0

Re: [Lustre-discuss] Moving MGS to separate device

2009-10-13 Thread Brian J. Murrell
On Sun, 2009-10-11 at 19:05 +0200, Bernd Schubert wrote: > Bernd, > I already did this several times, here are the steps I so far used: > > 1) Remove MGS from MDT-device > > tunefs.lustre --nogms /dec/mdt_device > > 2) Create new MGS > > mkfs.lustre --mgs /dev/mgs_device > > 3) Make sure OS

Re: [Lustre-discuss] Setup mail cluster

2009-10-13 Thread Bernd Schubert
On Monday 12 October 2009, Michael Schwartzkopff wrote: > Am Montag, 12. Oktober 2009 15:54:04 schrieb Vadym: > > Hello > > I'm do a schema of mail service so I have only one question: > > Can Lustre provide me full automatic failover solution? > > No. See the lustre manual for this. You need a cl

Re: [Lustre-discuss] Is there a way to set lru_size and have it stick?

2009-10-13 Thread Bernd Schubert
On Saturday 10 October 2009, Andreas Dilger wrote: > On 8-Oct-09, at 22:28, Lundgren, Andrew wrote: > > Is there a way to set the lru_size to a fixed value and have it stay > > that way across mounts? > > > > I know it can be set using: > > $ lctl set_param ldlm.namespaces.*osc*.lru_size=$((NR_CPU*

Re: [Lustre-discuss] lfsck

2009-10-13 Thread Andreas Dilger
On 13-Oct-09, at 06:33, Wojciech Turek wrote: > I am running lfscks on my file systems right now. Once they finished > I would like to re run lfscks to make sure that all problems were > cleaned. Do you know if I need to rebuild mdsdb and ostdbs for the > second lfsck run? I can see that lfsc

Re: [Lustre-discuss] mounting lustre client behind firewall

2009-10-13 Thread Brian J. Murrell
On Tue, 2009-10-13 at 11:53 -0400, Yujun Wu wrote: > Hello Aaron, > > Thanks for your info. Does this mean the client side have to open > both inbound and outbound port on 988 all the way between servers > and clients? No. As Aaron said, the connection would be initiated from a source port < 102

Re: [Lustre-discuss] mounting lustre client behind firewall

2009-10-13 Thread Yujun Wu
Hello Aaron, Thanks for your info. Does this mean the client side have to open both inbound and outbound port on 988 all the way between servers and clients? Regards, Yujun On Mon, 12 Oct 2009, Aaron Knister wrote: > I believe port tcp port 988 is used by default for the tcp lnet > module. I'm

[Lustre-discuss] Slow read performance across OSSes

2009-10-13 Thread James Robnett
I've put together a small test Lustre system which is giving confusing (at least to me) results. All nodes are running fully patched 64bit RHEL 5.3 with the premade Lustre 1.8.1 x86_64 RPMs. The nodes are a bit cobbled together from what I had handy. One MDS: 8 core 2.5 GHz nihalem 8GB RA

Re: [Lustre-discuss] Memory (?) problem with 1.8.1

2009-10-13 Thread Nirmal Seenu
We face this problem on the Lustre servers on our cluster with GigE network. We found that increasing the following value in /etc/sysctl.conf forces the kswapd to kick in a lot earlier and prevent the scenario that you are talking about. Our servers have only 8GB memory, you might want to bump

Re: [Lustre-discuss] Is there a way to set lru_size and have it stick?

2009-10-13 Thread Lundgren, Andrew
Thanks Bernd. For a work around, we are doing a cron every 5 minutes for now to force it down after unmount/remounts. -- Andrew -Original Message- From: Bernd Schubert [mailto:bs_li...@aakef.fastmail.fm] Sent: Tuesday, October 13, 2009 4:15 AM To: Andreas Dilger Cc: Lundgren, Andrew; l

Re: [Lustre-discuss] Memory (?) problem with 1.8.1

2009-10-13 Thread Lundgren, Andrew
This sounds very much like a problem we saw before we changed the lru_size to a fixed size from dynamic. -- Andrew -Original Message- From: lustre-discuss-boun...@lists.lustre.org [mailto:lustre-discuss-boun...@lists.lustre.org] On Behalf Of David Simas Sent: Monday, October 12, 2009 6:

[Lustre-discuss] lfsck

2009-10-13 Thread Wojciech Turek
I am running lfscks on my file systems right now. Once they finished I would like to re run lfscks to make sure that all problems were cleaned. Do you know if I need to rebuild mdsdb and ostdbs for the second lfsck run? I can see that lfsck change timestamps on db files so maybe I don't have to reb

Re: [Lustre-discuss] lustre 1.8.1 on SLES11 64bit in HP blades using Flex-10

2009-10-13 Thread Brian J. Murrell
On Wed, 2009-10-07 at 14:51 +0200, Guillaume Demillecamps wrote: > Hello all, Hi, > kernel: LustreError: 8578:0:(filter_io_26.c:402:filter_do_bio()) Can't > allocate bio 256*4 = 1024 pages > > Any idea where to look at ? Memory. b. signature.asc Description: This is a digitally signed mes

Re: [Lustre-discuss] Memory (?) problem with 1.8.1

2009-10-13 Thread Brian J. Murrell
On Mon, 2009-10-12 at 17:06 -0700, David Simas wrote: > Hello, Hi, > During this operation, we notice the value of "buffers" > reported in '/proc/meminfo' on the OSSs involved increasing monotonically > until it apparently take up all the system's memory - 32 GB. This would likely be OSS read ca

Re: [Lustre-discuss] Is there a way to set lru_size and have it stick?

2009-10-13 Thread Bernd Schubert
On Tuesday 13 October 2009, Andreas Dilger wrote: > On 12-Oct-09, at 12:11, Lundgren, Andrew wrote: > > I have tried using: > > > > # lctl conf_param content-MDT.osc.lru_size=800 > > > > Seen this in the log: > > Oct 12 18:35:36 abcd0202 kernel: Lustre: Modifying parameter content- > > MDT-

Re: [Lustre-discuss] Is there a way to set lru_size and have it stick?

2009-10-13 Thread Andreas Dilger
On 12-Oct-09, at 12:11, Lundgren, Andrew wrote: > I have tried using: > > # lctl conf_param content-MDT.osc.lru_size=800 > > Seen this in the log: > Oct 12 18:35:36 abcd0202 kernel: Lustre: Modifying parameter content- > MDT-mdc.osc.lru_size in log content-client > Oct 12 18:35:36 abcd0202