Swapping caused by very large (regular) file size

[Ian West]

Hello, I have noticed while benchmarking a system with a fair bit of ram
(3G usable of 4G installed) that when using a very large file (3G
upwards) in a simple benchmark it will cause the system to swap, even
though the actual process does not show in top to be using a lot of
memory, as soon as the swapping starts the throughput degrades
dramatically. The 'inactive' ram shown in top increases rapidly and
'free' ram reduces, this seems fair and sensible, but allowing it to
then page to possibly the same spindle/array seems like a bad idea ?

I have tested this on a 4.11 system with 512M of ram as well as a
RELENG-6 system with an areca raid controller, both behave in the same
way, once the file gets to a certain size the system starts paging. Is
there any way to tune this behaviour ?

The test I have been doing is just generating a big file full of nulls,
but bonnie++ causes the same behaviour with very large file sizes.

dd if=/dev/zero bs=32768 of=junkfile count=10 seems to do it quite
reliably on all the boxes I have tested ?

Using cp to copy the file doesnt appear to cause the problem.

Any thoughts or suggestions would be much appreciated ?


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: kernel panic on 6.2-RC2 with GENERIC.

[Ian West]

On Sun, Jan 07, 2007 at 02:25:02PM -0500, Mike Tancsa wrote:
 At 11:43 AM 1/7/2007, Craig Rodrigues wrote:
 On Fri, Jan 05, 2007 at 06:59:10PM +0200, Nikolay Pavlov wrote:
  Hello folks.
  I have kernel panic on GENERIC kernel while executing postmark.
 
 The sequence of steps that Nikolay used to produce this panic was:
 
 - install benchmarks/postmark from ports
 
 root# postmark
 PostMark v1.5 : 3/27/01
 pmset number=1
 pmset transactions=1
 pmset subdirectories=1
 pmshow
 pmrun
 
 I am able to do this on an AMD64 on a AREAC RAID6 file system and on 
 a plain old ata drive on i386 without issue.
 
 the i386 is a few weeks old but I will cvsup and re-try to confirm on 
 both today
 
 [tyan-1u]# postmark
 PostMark v1.5 : 3/27/01
 pmset number=1
 pmset transactions=1
 pmset subdirectories=1
 pmset location /tmp
 pmrun
 Creating subdirectories...Done
 Creating files...Done
 Performing transactions..Done
 Deleting files...Done
 Deleting subdirectories...Done
 Time:
 481 seconds total
 233 seconds of transactions (42 per second)
 
 Files:
 15027 created (31 per second)
 Creation alone: 1 files (62 per second)
 Mixed with transactions: 5027 files (21 per second)
 4990 read (21 per second)
 5009 appended (21 per second)
 15027 deleted (31 per second)
 Deletion alone: 10054 files (115 per second)
 Mixed with transactions: 4973 files (21 per second)
 
 Data:
 27.14 megabytes read (57.78 kilobytes per second)
 85.08 megabytes written (181.13 kilobytes per second)
 pmquit
 [tyan-1u]# uname -a
 FreeBSD tyan-1u.sentex.ca 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #1: 
 Mon Dec 11 17:45:45 EST 
 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/tyan  i386
 [tyan-1u]#
 
 
 and amd64
 
 pmshow
 Current configuration is:
 The base number of files is 1
 Transactions: 1
 Files range between 500 bytes and 9.77 kilobytes in size
 Working directory:
 /mnt (weight=1)
 1 subdirectories will be used
 Block sizes are: read=512 bytes, write=512 bytes
 Biases are: read/append=5, create/delete=5
 Using Unix buffered file I/O
 Random number generator seed is 42
 Report format is verbose.
 pmrun
 Creating subdirectories...Done
 Creating files...Done
 Performing transactions..Done
 Deleting files...Done
 Deleting subdirectories...Done
 Time:
 310 seconds total
 155 seconds of transactions (64 per second)
 
 Files:
 15027 created (48 per second)
 Creation alone: 1 files (103 per second)
 Mixed with transactions: 5027 files (32 per second)
 4990 read (32 per second)
 5009 appended (32 per second)
 15027 deleted (48 per second)
 Deletion alone: 10054 files (173 per second)
 Mixed with transactions: 4973 files (32 per second)
 
 Data:
 27.14 megabytes read (89.65 kilobytes per second)
 85.08 megabytes written (281.04 kilobytes per second)
 pmquit
 [r2-releng6-64]# uname -a
 FreeBSD r2-releng6-64.sentex.ca 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE 
 #0: Thu Dec 28 23:13:18 EST 
 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/router  amd64
 [r2-releng6-64]#
 
 
 both file systems have normal newfs options and fairly standard 
 kernels with default /etc/make.conf and both are SMP

I have seen this identical fault with the new areca driver, my machine
is opteron hardware, but running a regular i386/SMP kernel/world. With
everything at 6.2RC2 (as of 29th of December) except the areca driver
the machine is rock solid, with the 29th of december version of the
areca driver the box will crash on extract of a large tar file, removal
of a large directory structure, or pretty much anything that does a lot
of disk io to different files/locations. There is no error log prior to
seeing the following messages..

Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=433078272, 
length=8192)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=433111040, 
length=16384)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=433209344, 
length=16384)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=433242112, 
length=32768)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=437612544, 
length=4096)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=437616640, 
length=12288)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=437633024, 
length=6144)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=437639168, 
length=2048)]error = 5
Dec 29 14:26:44 aleph kernel: g_vfs_done():da0s1g[WRITE(offset=437641216, 
length=6144)]error = 5

There are a string of these, followed by a crash and reboot. The file system
state can be left very dirty to the point where background fsck seems unable
to recover