Re: 8.1-STABLE: zfs and sendfile: problem still exists
on 31/10/2010 02:37 Kostik Belousov said the following:
On Sat, Oct 30, 2010 at 05:43:54PM +0300, Andriy Gapon wrote:
on 30/10/2010 14:25 Artemiev Igor said the following:
On Sat, Oct 30, 2010 at 01:33:00PM +0300, Andriy Gapon wrote:
on 30/10/2010 13:12 Artemiev Igor said the following:
On Sat, Oct 30, 2010 at 12:52:54PM +0300, Andriy Gapon wrote:
Heh, next try.
Got a panic, vm_page_unwire: invalid wire count: 0
Oh, thank you for testing - forgot another piece (VM_ALLOC_WIRE for
vm_page_alloc):
Yep, it work. But VM_ALLOC_WIRE not exists in RELENG_8, therefore i
slightly modified your patch:
I apologize for my haste, it should have been VM_ALLOC_WIRED.
Here is a corrected patch:
Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
===
--- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
(revision 214318)
+++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
(working copy)
@@ -67,6 +67,7 @@
#include sys/sf_buf.h
#include sys/sched.h
#include sys/acl.h
+#include vm/vm_pageout.h
/*
* Programming rules.
@@ -464,7 +465,7 @@
uiomove_fromphys(m, off, bytes, uio);
VM_OBJECT_LOCK(obj);
vm_page_wakeup(m);
-} else if (m != NULL uio-uio_segflg == UIO_NOCOPY) {
+} else if (uio-uio_segflg == UIO_NOCOPY) {
/*
* The code below is here to make sendfile(2) work
* correctly with ZFS. As pointed out by ups@
@@ -474,9 +475,23 @@
*/
KASSERT(off == 0,
(unexpected offset in mappedread for sendfile));
-if (vm_page_sleep_if_busy(m, FALSE, zfsmrb))
+if (m != NULL vm_page_sleep_if_busy(m, FALSE,
zfsmrb))
goto again;
-vm_page_busy(m);
+if (m == NULL) {
+m = vm_page_alloc(obj, OFF_TO_IDX(start),
+VM_ALLOC_NOBUSY | VM_ALLOC_WIRED |
VM_ALLOC_NORMAL);
+if (m == NULL) {
+VM_OBJECT_UNLOCK(obj);
+VM_WAIT;
+VM_OBJECT_LOCK(obj);
+goto again;
+}
+} else {
+vm_page_lock_queues();
+vm_page_wire(m);
+vm_page_unlock_queues();
+}
+vm_page_io_start(m);
Why wiring the page if it is busied ?
Eh? Because it is not?
--
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-STABLE: zfs and sendfile: problem still exists
on 30/10/2010 22:01 Artemiev Igor said the following: On Sat, Oct 30, 2010 at 05:43:54PM +0300, Andriy Gapon wrote: I apologize for my haste, it should have been VM_ALLOC_WIRED. Ok, applied and tested under some load(~1200 active connections, outgoing ~80MB/s). Patch work as expected and i has noted no side effects. Just one question - should grow Active memory counter, if some pages is hot(during multiple sendfile on one file)? Pages used by sendfile are marked as Inactive for faster reclamation on demand. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
RE: 8.1-STABLE: zfs and sendfile: problem still exists
I apologize for my haste, it should have been VM_ALLOC_WIRED. Ok, applied and tested under some load(~1200 active connections, outgoing ~80MB/s). Patch work as expected and i has noted no side effects. Just one question - should grow Active memory counter, if some pages is hot(during multiple sendfile on one file)? Pages used by sendfile are marked as Inactive for faster reclamation on demand. I have a question. When we transfer a file via sendfile, then current code allocates a memory, marked inactive. For example, if the file has size 100 MB, then 100 MB of memory will be allocated. If we have to transfer this file again later, then this memory will used as cache, and no disk io will be required. The memory will be freed if file will be deleted or operating system will need an additional memory. I have correctly understood? If it so, the i continue... Such behaviour is good if we have files with relatively small size. Suppose we have to transfer file with large size (for example, greater than amount of physical memory). While transfering, the inactive memory will grow, pressing the ARC. When size of the ARC will fall to its minimum (vfs.zfs.arc_min), then inactive memory will be reused. So, when transfer is complete, we have: 1. No free memory 2. Size of the ARC has minimal size (it is bad) 3. Inactive memory contains the _tail_ of the file only (it is bad too) Now if we have to transfer this file again, then 1. there is no (or few) file's data in ARC (ARC too small) 2. The inactive memory doesn't contain a head part of the file So the file's data will read from a disk again and again... Also i've noticed that inactive memory frees relatively slowly, so if there is a frequent access to large files, then system will run at very unoptimal conditions. It's imho... Can you comment this? -- Alexander Zagrebin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-STABLE: zfs and sendfile: problem still exists
On Sun, 31 Oct 2010 10:02:44 +0100, Alexander Zagrebin al...@visp.ru wrote: I apologize for my haste, it should have been VM_ALLOC_WIRED. Ok, applied and tested under some load(~1200 active connections, outgoing ~80MB/s). Patch work as expected and i has noted no side effects. Just one question - should grow Active memory counter, if some pages is hot(during multiple sendfile on one file)? Pages used by sendfile are marked as Inactive for faster reclamation on demand. I have a question. When we transfer a file via sendfile, then current code allocates a memory, marked inactive. For example, if the file has size 100 MB, then 100 MB of memory will be allocated. If we have to transfer this file again later, then this memory will used as cache, and no disk io will be required. The memory will be freed if file will be deleted or operating system will need an additional memory. I have correctly understood? If it so, the i continue... Such behaviour is good if we have files with relatively small size. Suppose we have to transfer file with large size (for example, greater than amount of physical memory). While transfering, the inactive memory will grow, pressing the ARC. When size of the ARC will fall to its minimum (vfs.zfs.arc_min), then inactive memory will be reused. So, when transfer is complete, we have: 1. No free memory 2. Size of the ARC has minimal size (it is bad) 3. Inactive memory contains the _tail_ of the file only (it is bad too) Now if we have to transfer this file again, then 1. there is no (or few) file's data in ARC (ARC too small) 2. The inactive memory doesn't contain a head part of the file So the file's data will read from a disk again and again... Also i've noticed that inactive memory frees relatively slowly, so if there is a frequent access to large files, then system will run at very unoptimal conditions. It's imho... Can you comment this? Add more RAM? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
option DDB broken between r214547 - r214596
On one of my machines, I've been having some issues such that I adjusted the kernel config to include some debugging options: options KDB options KDB_TRACE options KDB_UNATTENDED options DDB options DDB_NUMSYM options GDB options DEBUG_REDZONE Now, I actually build for the system in question on my build machine (which builds a GENERIC kernel for itself, as well as kernels for the above-mentioned machine and another). This morning, it was running FreeBSD 8.1-STABLE r214547, building 8.1-STABLE r214596. The GENERIC build succeeded, and went on to the kernel ALBERT. That failed, thus: ... stage 3.2: building everything ... cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /usr/src/sys/net/if_debug.c /usr/src/sys/net/if_debug.c: In function 'if_show_ifnet': /usr/src/sys/net/if_debug.c:65: error: 'struct ifnet' has no member named 'if_index_reserved' *** Error code 1 Stop in /common/S1/obj/usr/src/sys/ALBERT. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. Commenting out the DDB option in the config avoided the failure. Peace, david -- David H. Wolfskill da...@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgp0qqvgzn1vg.pgp Description: PGP signature
Re: option DDB broken between r214547 - r214596
On Sun, 31 Oct 2010, David Wolfskill wrote: Hi, On one of my machines, I've been having some issues such that I adjusted the kernel config to include some debugging options: options KDB options KDB_TRACE options KDB_UNATTENDED options DDB options DDB_NUMSYM options GDB options DEBUG_REDZONE Now, I actually build for the system in question on my build machine (which builds a GENERIC kernel for itself, as well as kernels for the above-mentioned machine and another). This morning, it was running FreeBSD 8.1-STABLE r214547, building 8.1-STABLE r214596. The GENERIC build succeeded, and went on to the kernel ALBERT. That failed, thus: ... stage 3.2: building everything ... cc -c -O -pipe -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -fstack-protector -Werror /usr/src/sys/net/if_debug.c /usr/src/sys/net/if_debug.c: In function 'if_show_ifnet': /usr/src/sys/net/if_debug.c:65: error: 'struct ifnet' has no member named 'if_index_reserved' *** Error code 1 That would be mine though a universe had successfully completed with the merge. I'll go and look. /bz -- Bjoern A. Zeeb Welcome a new stage of life. ks Going to jail sucks -- bz All my daemons like it! http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: option DDB broken between r214547 - r214596
On Sun, 31 Oct 2010, Bjoern A. Zeeb wrote: Hi David, Now, I actually build for the system in question on my build machine (which builds a GENERIC kernel for itself, as well as kernels for the above-mentioned machine and another). This morning, it was running FreeBSD 8.1-STABLE r214547, building 8.1-STABLE r214596. The GENERIC build succeeded, and went on to the kernel ALBERT. That failed, thus: ... /usr/src/sys/net/if_debug.c /usr/src/sys/net/if_debug.c: In function 'if_show_ifnet': /usr/src/sys/net/if_debug.c:65: error: 'struct ifnet' has no member named 'if_index_reserved' *** Error code 1 That would be mine though a universe had successfully completed with the merge. I'll go and look. I still had the stable/8 LINT + 4 more different LINT configs all including DDB build logs form yesterday on ref8; none had an error though it was obviously there. I have no clue but that a KERNFAST build might not have cought it if this wasn't my first change to build test. It should be fixed with r214602. Thanks a lot for the report. /bz -- Bjoern A. Zeeb Welcome a new stage of life. ks Going to jail sucks -- bz All my daemons like it! http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: option DDB broken between r214547 - r214596
On Sun, Oct 31, 2010 at 04:57:42PM +, Bjoern A. Zeeb wrote: ... ... /usr/src/sys/net/if_debug.c /usr/src/sys/net/if_debug.c: In function 'if_show_ifnet': /usr/src/sys/net/if_debug.c:65: error: 'struct ifnet' has no member named 'if_index_reserved' *** Error code 1 That would be mine though a universe had successfully completed with the merge. I'll go and look. I still had the stable/8 LINT + 4 more different LINT configs all including DDB build logs form yesterday on ref8; none had an error though it was obviously there. I have no clue but that a KERNFAST build might not have cought it if this wasn't my first change to build test. It should be fixed with r214602. Thanks a lot for the report. Thanks -- sorry for the hassle. And that I was so preessed for time that I wasn't able to research it a bit first. In any case, I've reverted the change to the kernel config in question, applied the change from r214602, and rebuilt the kernels -- GENERIC, ALBERT, and JANUS -- that tyhe build machine normally builds, and all were successful. Thanks! Peace, david -- David H. Wolfskill da...@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgpIANd3Y6Lmz.pgp Description: PGP signature
Re: Degraded zpool cannot detach old/bad drive
Hi Artem, everyone, Here's the latest update on my case. I did upgrade the system to the latest stable: 8.1-STABLE FreeBSD 8.1-STABLE #0: Sun Oct 31 11:44:06 PDT 2010 After that I did zpool upgrade and zfs upgrade -r all the filesystems. Currently I am running zpool 15 and zfs 4. Everything went fine with the upgrade but unfortunately my problem still persists. There's no difference in this aspect. I still have mfid devices. I also tried chmod-ing as you suggested /dev/mfid devices but zfs/zpool didn't seem to care and imported the array regardless. So at this point since no one else seems to have any ideas and we seem to be stuck I am almost ready to declare defeat on this one. Although the pool is usable I couldn't bring it back to exactly the same state as it was before the disk replacements took place. Disappointing indeed, although not a complete show stopper. I still think that if there's a way to edit the cache file and change the devices that might do the trick. Thanks for all the help, Rumen Telbizov On Fri, Oct 29, 2010 at 5:01 PM, Artem Belevich fbsdl...@src.cx wrote: On Fri, Oct 29, 2010 at 4:42 PM, Rumen Telbizov telbi...@gmail.com wrote: FreeBSD 8.1-STABLE #0: Sun Sep 5 00:22:45 PDT 2010 That's when I csuped and rebuilt world/kernel. There were a lot of ZFS-related MFCs since then. I'd suggest updating to the most recent -stable and try again. I've got another idea that may or may not work. Assuming that GPT labels disappear because zpool opens one of the /dev/mfid* devices, you can try to do chmod a-rw /dev/mfid* on them and then try importing the pool again. --Artem -- Rumen Telbizov http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
HEADS UP: FreeBSD 6.4 and 8.0 EoLs coming soon
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello Everyone, On November 30th, FreeBSD 6.4 and FreeBSD 8.0 will have reached their End of Life and will no longer be supported by the FreeBSD Security Team. Since FreeBSD 6.4 is the last remaining supported release from the FreeBSD 6.x stable branch, support for the FreeBSD 6.x stable branch will also cease at the same point. Users of either of these FreeBSD releases are strongly encouraged to upgrade to either FreeBSD 7.3 or FreeBSD 8.1 before that date. The FreeBSD Ports Management Team wishes to remind users that November 30 is also the end of support for the Ports Collection for both FreeBSD 6.4 RELEASE and the FreeBSD 6.x STABLE branch. Neither the infrastructure nor individual ports are guaranteed to work on these FreeBSD versions after that date. A CVS tag will be created for users who cannot upgrade for some reason, at which time these users are advised to stop tracking the latest ports CVS repository and use the RELEASE_6_EOL tag instead. The current supported branches and expected EoL dates are: +-+ | Branch | Release | Type | Release date | Estimated EoL | |---+++-+-| |RELENG_6 |n/a |n/a |n/a |November 30, 2010| |-| |RELENG_6_4 |6.4-RELEASE |Extended|November 18, 2008|November 30, 2010| |-| |RELENG_7 |n/a |n/a |n/a |last release + 2y| |---+++-+-| |RELENG_7_1 |7.1-RELEASE |Extended|January 4, 2009 |January 31, 2011 | |---+++-+-| |RELENG_7_3 |7.3-RELEASE |Extended|March 23, 2010 |March 31, 2012 | |---+++-+-| |RELENG_8 |n/a |n/a |n/a |last release + 2y| |---+++-+-| |RELENG_8_0 |8.0-RELEASE |Normal |November 25, 2009|November 30, 2010| |---+++-+-| |RELENG_8_1 |8.1-RELEASE |Extended|July 23, 2010|July 31, 2012| +-+ - -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkzONUAACgkQOM7KaQxqam6AGgCcCsVMApQTN0x0fS4yZDfvzKNS 1T4AoJp/mS24RZF6DHrLWssplNNveGcb =L3fZ -END PGP SIGNATURE- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
