I'm running virtualbox 3.2.12_1 if that has anything to do with it. sysctl vfs.zfs.arc_max: 6200000000
While I'm trying to scp, kstat.zfs.misc.arcstats.size is hovering right around that value, sometimes above, sometimes below (that's as it should be, right?). I don't think that it dies when crossing over arc_max. I can run the same scp 10 times and it might fail 1-3 times, with no correlation to the arcstats.size being above/below arc_max that I can see. Scott On Jul 5, 2011, at 3:00 AM, Peter Ross wrote: > Hi all, > > just as an addition: an upgrade to last Friday's FreeBSD-Stable and to > VirtualBox 4.0.8 does not fix the problem. > > I will experiment a bit more tomorrow after hours and grab some statistics. > > Regards > Peter > > Quoting "Peter Ross" <peter.r...@bogen.in-berlin.de>: > >> Hi all, >> >> I noticed a similar problem last week. It is also very similar to one >> reported last year: >> >> http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058708.html >> >> My server is a Dell T410 server with the same bge card (the same pciconf >> -lvc output as described by Mahlon: >> >> http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/058711.html >> >> Yours, Scott, is a em(4).. >> >> Another similarity: In all cases we are using VirtualBox. I just want to >> mention it, in case it matters. I am still running VirtualBox 3.2. >> >> Most of the time kstat.zfs.misc.arcstats.size was reaching vfs.zfs.arc_max >> then, but I could catch one or two cases then the value was still below. >> >> I added vfs.zfs.prefetch_disable=1 to sysctl.conf but it does not help. >> >> BTW: It looks as ARC only gives back the memory when I destroy the ZFS (a >> cloned snapshot containing virtual machines). Even if nothing happens for >> hours the buffer isn't released.. >> >> My machine was still running 8.2-PRERELEASE so I am upgrading. >> >> I am happy to give information gathered on old/new kernel if it helps. >> >> Regards >> Peter >> >> Quoting "Scott Sipe" <csco...@gmail.com>: >> >>> >>> On Jul 2, 2011, at 12:54 AM, jhell wrote: >>> >>>> On Fri, Jul 01, 2011 at 03:22:32PM -0700, Jeremy Chadwick wrote: >>>>> On Fri, Jul 01, 2011 at 03:13:17PM -0400, Scott Sipe wrote: >>>>>> I'm running 8.2-RELEASE and am having new problems with scp. When scping >>>>>> files to a ZFS directory on the FreeBSD server -- most notably large >>>>>> files >>>>>> -- the transfer frequently dies after just a few seconds. In my last >>>>>> test, I >>>>>> tried to scp an 800mb file to the FreeBSD system and the transfer died >>>>>> after >>>>>> 200mb. It completely copied the next 4 times I tried, and then died >>>>>> again on >>>>>> the next attempt. >>>>>> >>>>>> On the client side: >>>>>> >>>>>> "Connection to home closed by remote host. >>>>>> lost connection" >>>>>> >>>>>> In /var/log/auth.log: >>>>>> >>>>>> Jul 1 14:54:42 freebsd sshd[18955]: fatal: Write failed: Cannot allocate >>>>>> memory >>>>>> >>>>>> I've never seen this before and have used scp before to transfer large >>>>>> files >>>>>> without problems. This computer has been used in production for months >>>>>> and >>>>>> has a current uptime of 36 days. I have not been able to notice any >>>>>> problems >>>>>> copying files to the server via samba or netatalk, or any problems in >>>>>> apache. >>>>>> >>>>>> Uname: >>>>>> >>>>>> FreeBSD xeon 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Sat Feb 19 01:02:54 EST >>>>>> 2011 root@xeon:/usr/obj/usr/src/sys/GENERIC amd64 >>>>>> >>>>>> I've attached my dmesg and output of vmstat -z. >>>>>> >>>>>> I have not restarted the sshd daemon or rebooted the computer. >>>>>> >>>>>> Am glad to provide any other information or test anything else. >>>>>> >>>>>> {snip vmstat -z and dmesg} >>>>> >>>>> You didn't provide details about your networking setup (rc.conf, >>>>> ifconfig -a, etc.). netstat -m would be useful too. >>>>> >>>>> Next, please see this thread circa September 2010, titled "Network >>>>> memory allocation failures": >>>>> >>>>> http://lists.freebsd.org/pipermail/freebsd-stable/2010-September/thread.html#58708 >>>>> >>>>> The user in that thread is using rsync, which relies on scp by default. >>>>> I believe this problem is similar, if not identical, to yours. >>>>> >>>> >>>> Please also provide your output of ( /usr/bin/limits -a ) for the server >>>> end and the client. >>>> >>>> I am not quite sure I agree with the need for ifconfig -a but some >>>> information about the networking driver your using for the interface >>>> would be helpful, uptime of the boxes. And configuration of the pool. >>>> e.g. ( zpool status -a ;zfs get all <poolname> ) You should probably >>>> prop this information up somewhere so you can reference by URL whenever >>>> needed. >>>> >>>> rsync(1) does not rely on scp(1) whatsoever but rsync(1) can be made to >>>> use ssh(1) instead of rsh(1) and I believe that is what Jeremy is >>>> stating here but correct me if I am wrong. It does use ssh(1) by >>>> default. >>>> >>>> Its a possiblity as well that if using tmpfs(5) or mdmfs(8) for /tmp >>>> type filesystems that rsync(1) may be just filling up your temp ram area >>>> and causing the connection abort which would be expected. ( df -h ) would >>>> help here. >>> >>> Hello, >>> >>> I'm not using tmpfs/mdmfs at all. The clients yesterday were 3 different >>> OSX computers (over gigabit). The FreeBSD server has 12gb of ram and no bce >>> adapter. For what it's worth, the server is backed up remotely every night >>> with rsync (remote FreeBSD uses rsync to pull) to an offsite (slow cable >>> connection) FreeBSD computer, and I have not seen any errors in the nightly >>> rsync. >>> >>> Sorry for the omission of networking info, here's the output of the >>> requested commands and some that popped up in the other thread: >>> >>> http://www.cap-press.com/misc/ >>> >>> In rc.conf: ifconfig_em1="inet 10.1.1.1 netmask 255.255.0.0" >>> >>> Scott >>> >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" >>> >> >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" >> > > _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
