Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-22 Thread Chris Samuel
On Thursday, 22 November 2018 9:26:13 PM AEDT Christoph Brüning wrote: > Hi Chris, Hi Christoph! [...] > I was wondering if constantly making and deleting XFS projects has a > considerable impact on performance and stability. So I'd be glad if you > could share some of your experience with that

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-22 Thread Christoph Brüning
Hi Chris, I actually had the same idea concerning XFS project quotas when I came across that private-tmp SPANK plugin recently. Haven't had the opportunity to make a test implementation yet, though. I was wondering if constantly making and deleting XFS projects has a considerable impact on p

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-22 Thread Bjørn-Helge Mevik
Shenglong Wang writes: > Hope users do not have absolute path /tmp inside their scripts. We avoid that by giving each job a job-local "/tmp" (and "/var/tmp" for good measure) with a Spank plugin. The plugin then removes the folders afterwards. There are several versions of such spank plugins f

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Christopher Samuel
On 22/11/18 12:38 am, Douglas Duckworth wrote: We are setting TmpFS=/scratchLocal in /etc/slurm/slurm.conf on nodes and controller. However $TMPDIR value seems to be /tmp not /scratchLocal. As a result users are writing to /tmp which we do not want. Our solution to that was to use a plugin th

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Jeffrey T Frey
f a bug has >> crept in?I spoke to the SchedMD guys at SC18 last week and they were not >> aware of a bug but since more than one person is having this difficulty >> something must be wrong somewhere. >> >> -Roger >> >> From: slurm-users [mailt

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Michael Gutteridge
> > *From:* slurm-users [mailto:slurm-users-boun...@lists.schedmd.com > ] *On Behalf Of *Douglas Duckworth > *Sent:* Wednesday, November 21, 2018 7:38 AM > *To:* slurm-users@lists.schedmd.com > *Subject:* [slurm-users] $TMPDIR does not honor "TmpFS" > > Hi > > We ar

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Roberts, John E.
Community List Date: Wednesday, November 21, 2018 at 8:55 AM To: Slurm User Community List Subject: Re: [slurm-users] $TMPDIR does not honor "TmpFS" We have TMPDIR setup inside prolog file. Hope users do not have absolute path /tmp inside their scripts. #!/bin/bash SLURM_BIN="

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Shenglong Wang
to:slurm-users-boun...@lists.schedmd.com] On Behalf Of > Douglas Duckworth > Sent: Wednesday, November 21, 2018 7:38 AM > To: slurm-users@lists.schedmd.com > Subject: [slurm-users] $TMPDIR does not honor "TmpFS" > > Hi > > We are setting TmpFS=/scratchLocal i

Re: [slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Roger Moye
[mailto:slurm-users-boun...@lists.schedmd.com] On Behalf Of Douglas Duckworth Sent: Wednesday, November 21, 2018 7:38 AM To: slurm-users@lists.schedmd.com Subject: [slurm-users] $TMPDIR does not honor "TmpFS" Hi We are setting TmpFS=/scratchLocal in /etc/slurm/slurm.conf on nodes and controller

[slurm-users] $TMPDIR does not honor "TmpFS"

2018-11-21 Thread Douglas Duckworth
Hi We are setting TmpFS=/scratchLocal in /etc/slurm/slurm.conf on nodes and controller. However $TMPDIR value seems to be /tmp not /scratchLocal. As a result users are writing to /tmp which we do not want. We are not setting $TMPDIR anywhere else such as /etc/profile.d nor do users have it d