Make vnd(4) work on sparse files

2011-06-24 Thread J. Hannken-Illjes
With the attached diff vnd(4) devices will work on sparse files. - Make the strategy decision a device flag and set to VNF_USE_VN_RDWR for files known to be sparse. - Change handle_with_rdwr() to use POSIX_FADV_NOREUSE advise to disable read ahead. - Keep the amount of mapped pages below 1 MByt

Re: Make vnd(4) work on sparse files

2011-06-24 Thread J. Hannken-Illjes
... this time with a diff attached: With the attached diff vnd(4) devices will work on sparse files. - Make the strategy decision a device flag and set to VNF_USE_VN_RDWR for files known to be sparse. - Change handle_with_rdwr() to use POSIX_FADV_NOREUSE advise to disable read ahead. - Keep the

Re: LOCKDEBUG panic: lockdebug_alloc: already initialized

2011-07-02 Thread J. Hannken-Illjes
Looks like the mutex linux_futex.c::futex_lock is missing a mutex_destroy() on module unload to make LOCKDEBUG happy. -- Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) On Jul 2, 2011, at 6:29 PM, dieter roelants wrote: > On Fri, 1 Jul 2011 13:53:01 +0200 > dieter ro

Make VOP_BWRITE() work with layered fs

2011-07-07 Thread J. Hannken-Illjes
With the attached diff VOP_BWRITE() arguments get changed as - VOP_BWRITE(struct buf *) + VOP_BWRITE(struct vnode *, struct buf *) and all calls become VOP_BWRITE(bp->b_vp, bp). With this change layered file systems no longer treat it special as it is now possible to descend on the v

fcntl() F_GETLK semantics vs. test t_vnops.c

2011-07-22 Thread J. Hannken-Illjes
The test fcntl_getlock_pids() from fs/vfs/t_vnops.c assumes fcntl(fd, F_GETLK, &lock) returns the blocking lock with the lowest start offset. Our documentation and POSIX.1 document it returning the "first lock that blocks" but doesn't call for any specific order. Should I fix fcntl() to return th

Re: fcntl() F_GETLK semantics vs. test t_vnops.c

2011-07-22 Thread J. Hannken-Illjes
On Jul 22, 2011, at 12:13 PM, Alexander Nasonov wrote: > J. Hannken-Illjes wrote: >> The test fcntl_getlock_pids() from fs/vfs/t_vnops.c assumes >> fcntl(fd, F_GETLK, &lock) returns the blocking lock with the >> lowest start offset. >> >> Our documentation

Re: pmap_kenter_pa: mapping already present

2011-08-09 Thread J. Hannken-Illjes
It is a bug that should finally get its PR :-) pmap_pdp_ctor() abuses pmap_kenter_pa() to change the protection of an already existing mapping. According to rmind this is because on x86 pmap_protect() does not work for pmap_kernel(). -- Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braun

Re: panic with 5.99.55 starting postgres

2011-08-26 Thread J. Hannken-Illjes
On Aug 25, 2011, at 11:18 PM, Darren Reed wrote: > starting postgres panic's my system every time with this panic: > > panic: kernel diagnostic assertion "uvm_page_locked_p(pg > e " ../../../../arch/x86/x86/pmap.c, line 3214 > > > From the pmap.c that was used to compile the kernel is this > ex

May pool_cache_get() ever return NULL?

2011-10-06 Thread J. Hannken-Illjes
A short question: Given a pool initialized with pool = pool_cache_init(SIZE, 0, 0, 0, "desc", NULL, IPL_NONE, NULL, NULL, NULL); will pool_cache_get(pool, PR_WAITOK) ever return NULL? -- Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

VOP_GETATTR: locking protocol change proposal

2011-10-09 Thread J . Hannken-Illjes
The vnode locking requirement currently allows to call VOP_GETATTR() on an unlocked vnode. This is orthogonal to all other operations that read data or metadata and want at least a shared lock. It also asks for trouble as the attributes may change while the operation is in progress. With the att

Re: VOP_GETATTR: locking protocol change proposal

2011-10-09 Thread J. Hannken-Illjes
On Oct 9, 2011, at 5:02 PM, Emmanuel Dreyfus wrote: > J. Hannken-Illjes wrote: > >> With the attached diff the locking protocol requests at least a shared >> lock and all calls to VOP_GETATTR() outside of file systems respect it. > > Please look at pn_sizemtx in src/

Re: VOP_GETATTR: locking protocol change proposal

2011-11-22 Thread J. Hannken-Illjes
On Nov 22, 2011, at 2:17 AM, YAMAMOTO Takashi wrote: > hi, > >> On Fri, Nov 18, 2011 at 06:31:21AM +, YAMAMOTO Takashi wrote: > postgresql assumes instant lseek(SEEK_END) to get the size of > their heap files. > > http://rhaas.blogspot.com/2011/11/linux-lseek-scalability.h

bdev_strategy() allowed to sleep?

2012-01-14 Thread J. Hannken-Illjes
Just analyzing PR kern/45829 I have a hopefully simple question: Is bdev_strategy() (vndstrategy in my case) allowed to sleep? Limiting the number of entries in the vnd bufq would solve this PR. -- Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: PUFFS deadlock

2012-02-03 Thread J. Hannken-Illjes
On Feb 3, 2012, at 5:45 PM, Emmanuel Dreyfus wrote: > Emmanuel Dreyfus wrote: > >> What about adding a timeout in struct puffs_msgpark and use it for >> inactive operations? Returning EAGAIN from puffs_vnop_inactive seems an >> easy way to work around this deadlock. > > I reply to myself. This

Re: PUFFS deadlock

2012-02-03 Thread J. Hannken-Illjes
On Feb 3, 2012, at 7:23 PM, Emmanuel Dreyfus wrote: > J. Hannken-Illjes wrote: > >> So this is on 5.x ? > > Yes, this is on netbsd-5 > >> For -current the getnewvnode()->getcleanvnode() path has been replaced >> with a cleaner thread so getnewvnode(

Re: PUFFS deadlock

2012-02-03 Thread J. Hannken-Illjes
On Feb 3, 2012, at 8:44 PM, Emmanuel Dreyfus wrote: > J. Hannken-Illjes wrote: > >> kern/vfs_vnode.c rev. 1.12 and 1.13 should be sufficient -- but it needs some >> testing on netbsd-5. > > kern/vfs_vnode.c does not even exist on netbsd-5, I guess it means it is > n

Re: Snapshots in tmpfs

2012-02-23 Thread J. Hannken-Illjes
On Feb 22, 2012, at 1:52 PM, Martin Husemann wrote: > Note that we already have file system snapshots for ffs file systems, > see fss(4). They are used for backup purposes (atomically create a snapshot, > while the file system is busy, then backup the now quiet snapshot) - among > others. Right -

Re: Should sys_unmount() FOLLOW?

2014-11-23 Thread J. Hannken-Illjes
#x27;t work even with root being a directory because the namei() call would hang in VOP_LOOKUP(). Does it work these days? > 2) Desfine a new MNT_RAW mount flag. If umount(8) is called with -R, > pass that flag to unmount(8), and in sys_unmount(), do not use FOLLOW if > MNT_RAW is set. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: race condition between (specfs) device open and close

2014-12-30 Thread J. Hannken-Illjes
* Open is in progress, but not complete, must wait > + * for it to complete. > + */ > + cv_wait(&specopenclose_cv, &device_lock); > + } > if (sn->sn_opencnt != 0) { > sd->sd_opencnt -= (sn->sn_opencnt - 1); > sn->sn_opencnt = 1; Will this work if we revoke a device whose cdev_open() blocks, a TTY waiting for carrier for example? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: NTFS: node leak

2014-12-31 Thread J. Hannken-Illjes
o me, didn't test it either. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: A brelse() in FFS...

2015-02-13 Thread J. Hannken-Illjes
s read with the same size). > I don't understand what the BC_INVAL flag means, and its definition > in sys/buf.h is not very enlightening: > > #define BC_INVAL0x2000 /* Does not contain valid info. > */ > > Which branch do I keep? Both. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Vnode API change: add global vnode cache

2015-03-09 Thread J. Hannken-Illjes
ialised and has link count zero. A diff showing the implementation is here: http://www.netbsd.org/~hannken/vnode-pass7-1a.diff and a diff showing its usage for ffs is here: http://www.netbsd.org/~hannken/vnode-pass7-1b.diff Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: tracking P->V for unmanaged device pages

2015-03-26 Thread J. Hannken-Illjes
if (i++ > 1) kpause("psrlz", false, 1, NULL); mutex_spin_enter(&psz_lock); } while (!kcpuset_iszero(psz->psz_target)); to become faster in the usual "two activity switches" case. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: bread()'s cred

2015-03-27 Thread J. Hannken-Illjes
ooks ok to me. Please take a look at breadn() too. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: fss(4) behaviour if backup fs full

2015-04-28 Thread J. Hannken-Illjes
, no further copy-on-write and reading gives ENXIO (Device not configured). The snapshot device stays configured. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Removal of miscfs/syncfs

2015-05-01 Thread J. Hannken-Illjes
VFS_SYNC as appropriate. - use an API for mount points similiar to the API for vnodes. Diff at http://www.netbsd.org/~hannken/rm-miscfs-syncfs-1.diff Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Removal of miscfs/syncfs

2015-05-02 Thread J . Hannken-Illjes
On 01 May 2015, at 14:16, Christos Zoulas wrote: > In article , > J. Hannken-Illjes wrote: >> Our miscfs/syncfs originating from the softdep import is a pseudo >> file system with one VOP. Its vnodes get used as a kind of marker >> on the syncer worklist so the syncer m

Re: Removal of miscfs/syncfs

2015-05-02 Thread J. Hannken-Illjes
On 01 May 2015, at 20:02, David Holland wrote: > On Fri, May 01, 2015 at 10:17:05AM +0200, J. Hannken-Illjes wrote: >> Our miscfs/syncfs originating from the softdep import is a pseudo >> file system with one VOP. Its vnodes get used as a kind of marker >> on the syncer w

Re: vnd locking

2015-05-22 Thread J. Hannken-Illjes
to sc_flags etc. are protected by mutex. - locking is no longer interruptible (no PCATCH / cv_wait_sig), not sure if this is a problem. - keep the first blank line for functions without locals. - do the white space change with a separate commit. Otherwise looks like a good starting point. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: VOP_PUTPAGE ignores mount_nfs -o soft,intr

2015-06-22 Thread J. Hannken-Illjes
nction nfs_node.c:nfs_gop_write(). Falling back to genfs_gop_write() ignores nfs timeouts. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Very slow transfers to/from micro SD card on a RPi B+

2015-08-18 Thread J. Hannken-Illjes
me to all names in the directory so far leading to 0.5*n*(n-1) calls to memcmp. And our memcmp is slw ... -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Very slow transfers to/from micro SD card on a RPi B+

2015-08-18 Thread J. Hannken-Illjes
On 18 Aug 2015, at 15:06, Joerg Sonnenberger wrote: > On Tue, Aug 18, 2015 at 01:06:02PM +0200, J. Hannken-Illjes wrote: >> And our memcmp is slw ... > > Do you have any thing substancial to justify this? This topic has already been discussed last year, see http://mail-i

Re: NFS related panics and hangs

2015-11-05 Thread J. Hannken-Illjes
e syssrc > tarball, this looks wrong. It also immediately causes the suspicion > that the list isn't in fact protected at all. This file (fs/nfs/client/nfs_clvnops.c) is part of a second (dead) nfs implementation from FreeBSD. It is not part of any kernel. Our nfs lives in sy

Re: issues to compile a kernel

2015-12-04 Thread J. Hannken-Illjes
^ Looks like your sources got mixed from 7.0 and -current. On 7.0 file usr.sbin/makefs/ffs/buf.h is revision 1.9 and has: int bread(struct vnode *, daddr_t, int, struct kauth_cred *, int, struct buf **); while on -current its revision is 1.10 and has: int bread(struct vnode *, daddr_t, int, int, struct buf **); -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: struct file reference at VFS level

2016-04-28 Thread J. Hannken-Illjes
pts to write to the locked part the lock (+) would return EAGAIN ... Wouldn't it be better to put the mandatory locking at the file->vnode level, operations vn_read() and vn_write()? Could you explain the proposed semantics in detail? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: struct file reference at VFS level

2016-04-29 Thread J. Hannken-Illjes
es make the claimed mutual exclusion between lock and mmap similarly unreliable. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: struct file at VFS level (round 1.5)

2016-05-03 Thread J. Hannken-Illjes
if the flags (the last argument) contain "F_POSIX" and "struct file *" otherwise. Fcntl(2) uses F_POSIX where flock(2) does not. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Vnode life cycle: add vnode state

2016-05-20 Thread J . Hannken-Illjes
Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Vnode life cycle: add vnode state

2016-05-21 Thread J. Hannken-Illjes
> On 20 May 2016, at 18:31, Taylor R Campbell > wrote: > > Date: Fri, 20 May 2016 16:38:22 +0200 > From: J. Hannken-Illjes > > 1) Objects vnode and vcache_node always exist together so it makes sense > to merge them into one object. This can be done in tw

Re: Locking strategy for device deletion (also see PR kern/48536)

2016-06-08 Thread J. Hannken-Illjes
ce, but the > close() routine is only called once. So every iteration leaves the refcount > with a net increment of one. > > I cannot figure out why/how the open routine is called twice. IIRC the device open() operation gets called on every open while the device close() operation gets called on last close only. See misfs/specfs/spec_vnops.c::spec_close(). -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Vnode namespace - split sys/vnode.h

2016-10-26 Thread J. Hannken-Illjes
? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Change vnode free lists to lru lists

2016-12-11 Thread J. Hannken-Illjes
sts, merge vrele_thread() into vdrain_thread() and adapt vfs_drainvnodes(). Diffs are here: https://www.netbsd.org/~hannken/lrulist/ Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: vrele vs. syncer deadlock

2016-12-11 Thread J. Hannken-Illjes
aiting for locked parent vnode from the rename Do you have full backtrace? Where is the syncer waiting for the parent? Which file system? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: vrele vs. syncer deadlock

2016-12-12 Thread J. Hannken-Illjes
> On 11 Dec 2016, at 22:33, Nick Hudson wrote: > > On 12/11/16 21:05, J. Hannken-Illjes wrote: >>> On 11 Dec 2016, at 21:01, David Holland wrote: >>> >>> On a low-memory machine Nick ran into the following deadlock: >>> >>> (a) rena

Re: vrele vs. syncer deadlock

2016-12-22 Thread J. Hannken-Illjes
, but we're ostensibly working > towards it.) > > Does anyone have any suggestions for a patch to address this problem > without first rewriting the syncer? While the latter is probably a > good idea, it's not entirely trivial to do and get right, making it > e.g. a poor

Remove vget from global namespace

2016-12-28 Thread J. Hannken-Illjes
a stable state and free the vnode if this hold was the last reference to a reclaimed vnode. 3) Add some "v_usecount == 1" assertions. Diffs are here: https://www.netbsd.org/~hannken/vget/ Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Br

Remount read-only and fstrans

2017-02-19 Thread J. Hannken-Illjes
not scale well to support it for all file systems. Plan is to move fstrans_start()/fstrans_done() into vnode_if.c and enable vfs_suspend() on all file systems. Diffs are here: https://www.netbsd.org/~hannken/vnode_if/ Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de

Re: Remount read-only and fstrans

2017-02-19 Thread J. Hannken-Illjes
> On 19 Feb 2017, at 18:08, Christos Zoulas wrote: > > In article , > J. Hannken-Illjes wrote: >> >> Comments or objections anyone? > > Will that detect open for write file descriptors to removed files? This is the job of vflush(). Since Rev. 1.46 of vfs_m

Add a mountlist iterator

2017-03-30 Thread J. Hannken-Illjes
gress. - struct mount *_mountlist_next(struct mount *) a lockless variant to be used from DDB only. Diffs are here: https://www.netbsd.org/~hannken/ml_iterator/ Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Add a mountlist iterator

2017-03-31 Thread J. Hannken-Illjes
> On 30. Mar 2017, at 17:20, Christos Zoulas wrote: > > In article <534552fb-af29-4219-8390-7514a2665...@eis.cs.tu-bs.de>, > J. Hannken-Illjes wrote: >> Currently vfs_busy() / vfs_unbusy() get used to >> >> - Enter/leave a critical section against unmoun

Re: locking from VOP_INACTIVE to VOP_RECLAIM

2017-04-02 Thread J. Hannken-Illjes
: Functions that VOP_RECLAIM calls would be able to assert, > rather than vaguely assume, exclusive ownership of the vnode lock. > Currently various code paths, e.g. ffs_update, require exclusive > access to the vnode in question, which can't be asserted right now > because of the

Re: Add a mountlist iterator

2017-04-02 Thread J. Hannken-Illjes
> On 1. Apr 2017, at 23:03, Taylor R Campbell > wrote: > >> Date: Thu, 30 Mar 2017 11:21:41 +0200 >> From: "J. Hannken-Illjes" >> >> Plan is to untangle the mountlist processing from vfs_busy() / vfs_unbusy() >> and add an iterator for the mo

Re: locking from VOP_INACTIVE to VOP_RECLAIM

2017-04-02 Thread J. Hannken-Illjes
> On 2. Apr 2017, at 16:05, Taylor R Campbell > wrote: > >> Date: Sun, 2 Apr 2017 10:47:42 +0200 >> From: "J. Hannken-Illjes" >> >>> On 1. Apr 2017, at 20:41, Taylor R Campbell >>> wrote: >>> VOP_RECLAIM then *destroys* the

Re: Add a mountlist iterator

2017-04-02 Thread J. Hannken-Illjes
> On 2. Apr 2017, at 16:34, Taylor R Campbell > wrote: > >> Date: Sun, 2 Apr 2017 11:09:49 +0200 >> From: "J. Hannken-Illjes" >> >>> On 1. Apr 2017, at 23:03, Taylor R Campbell >>> wrote: >>> >>> Any particul

Re: locking from VOP_INACTIVE to VOP_RECLAIM

2017-04-02 Thread J. Hannken-Illjes
> On 2. Apr 2017, at 18:19, Taylor R Campbell > wrote: > >> Date: Sun, 2 Apr 2017 16:34:20 +0200 >> From: "J. Hannken-Illjes" >> >> Looks like your proposal needs some clarification regarding "vnode lock" >> and "the lock&qu

Re: Add a mountlist iterator (round 2)

2017-04-04 Thread J. Hannken-Illjes
erator_destroy(iter); Diffs are here: https://www.netbsd.org/~hannken/ml_iterator2/ Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Add a mountlist iterator (round 2)

2017-04-06 Thread J. Hannken-Illjes
Changed "vfs_unmountall1()" to retrieve mounts in descending generation order which has the additional benefit that we don't need reverse traversals any more. Diffs are here: https://www.netbsd.org/~hannken/ml_iterator2a/ I prefer this API, opinions? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Add a mountlist iterator (round 2)

2017-04-09 Thread J. Hannken-Illjes
> On 6. Apr 2017, at 11:44, J. Hannken-Illjes wrote: > >> >> On 5. Apr 2017, at 05:14, Chuck Silvers wrote: >> >> have you considered a callback-based interface where the loop >> is inside the iteration API rather than in the caller? >> the v

Re: Add a mountlist iterator (round 2)

2017-04-10 Thread J. Hannken-Illjes
> On 9. Apr 2017, at 19:16, Taylor R Campbell > wrote: > >> Date: Sun, 9 Apr 2017 18:47:25 +0200 >> From: "J. Hannken-Illjes" >> >>> On 6. Apr 2017, at 11:44, J. Hannken-Illjes wrote: >>> Good hint. Prepared a partial implementation

Restructure vfs_busy and friends

2017-04-14 Thread J. Hannken-Illjes
renamed to vfs_rele(mp). Operation vfs_trybusy(mp) gets added when needed. Comments or objections? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Problem with lots of (perhaps too much?) memory

2017-05-16 Thread J. Hannken-Illjes
ems to > describe this situation. Unfortunately, it seems that yamt never finished > working on the problem. :( Two questions: - What is the size of your vnode cache (sysctl kern.maxvnodes)? - Is the vdrain thread using CPU too? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Problem with lots of (perhaps too much?) memory

2017-05-17 Thread J. Hannken-Illjes
> On 17. May 2017, at 08:59, Paul Goyette wrote: > > On Wed, 17 May 2017, J. Hannken-Illjes wrote: > >>>> Chances are very high that you are hitting the free page lock contention >>>> bug. >>> >>> Further observation: >>>

Re: Performance of VSTATE_ASSERT_UNLOCKED

2017-09-17 Thread J. Hannken-Illjes
to critical > cases and try to avoid it for common cases. > > Thoughts? > > Joerg > Looks good to me, cannot test anything as I am on vacation. Please commit and request pullup to -8. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: CVS commit: src

2012-03-14 Thread J. Hannken-Illjes
On Mar 14, 2012, at 10:55 AM, Martin Husemann wrote: > On Tue, Mar 13, 2012 at 06:41:18PM +, Elad Efrat wrote: >> Log Message: >> Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with >> something meaningful. All relevant documentation has been updated or >> written. >> >> Most

Re: CVS commit: src

2012-03-16 Thread J. Hannken-Illjes
On Mar 14, 2012, at 10:55 AM, Martin Husemann wrote: > On Tue, Mar 13, 2012 at 06:41:18PM +, Elad Efrat wrote: >> Log Message: >> Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with >> something meaningful. All relevant documentation has been updated or >> written. >> >> Most

Re: VFS reclaims are scarce on netbsd-6

2012-04-13 Thread J. Hannken-Illjes
On Apr 13, 2012, at 1:41 PM, Emmanuel Dreyfus wrote: > I run my perfuse/glusterfs tests on netbsd-6 branch, and I noticed that > I reclaims operations can lag for a long time. After running for a > while, perfused knows about 25217 nodes, most of them idle for hours. > Then kernel will start sendi

Re: selectively disabling atime updates?

2012-05-31 Thread J. Hannken-Illjes
A null mount should do the job and this fstab entry should work: /path/to/orig /path/to/ovarlay null ro,noatime -- Juergen Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) On May 31, 2012, at 2:14 PM, Edgar Fuß wrote: > Walking a directory tree (e.g. during a backup

Re: selectively disabling atime updates?

2012-06-11 Thread J. Hannken-Illjes
On Jun 11, 2012, at 3:45 PM, Edgar Fuß wrote: >> This makes me think that /etc/daily should take similar steps, >> whatever they turn out to be. > Yes, allthough my RAIDframe performance test results show that the effect > decreases with larger block sizes. > > I can think of two ways to acheive

Re: fss status (was: selectively disabling atime updates?)

2012-06-11 Thread J. Hannken-Illjes
On Jun 11, 2012, at 5:17 PM, Edgar Fuß wrote: >> No, snapshots are supported in 6.0. > Ah, great! > Someone should adjust the ffsconfig(8) man page, then: > > $ man fssconfig | tail -5 > BUGS > The fss(4) driver is experimental. Be sure you have a backup before you > use it. > > NetBSD

Re: selectively disabling atime updates?

2012-06-11 Thread J. Hannken-Illjes
On Jun 11, 2012, at 8:25 PM, Manuel Bouyer wrote: > On Mon, Jun 11, 2012 at 01:18:17PM -0400, Greg Troxel wrote: >> >> Thor Lancelot Simon writes: >> >>> On Mon, Jun 11, 2012 at 05:52:27PM +0200, Edgar Fu? wrote: > Yes, but I have to question whether and why it would improve performance >>>

Re: fss: Inappropriate ioctl for device (on dk*)

2012-07-28 Thread J. Hannken-Illjes
On Jul 28, 2012, at 1:08 AM, Edgar Fuß wrote: > Now, additionally to fssconfig -c not working at all for me > (it does work without "-c"), it now looks that it doesn't work > on wedges: > That's on 6.0_BETA2/amd64. Should work on wedges for -current, try attached patch for NetBSD-5 or -6. For m

Re: fss: Inappropriate ioctl for device (on dk*)

2012-07-28 Thread J. Hannken-Illjes
On Jul 28, 2012, at 5:21 PM, Edgar Fuß wrote: >> For me (on -current/i386) `fssconfig -c' works > Since it's uninitialized memory, that may well be true. The `-c' flag just sets an (initialized) pointer to a function and has nothing to do with setting fss.fss_flags. Please ktrace without `-c'.

Re: fss: Inappropriate ioctl for device (on dk*)

2012-07-28 Thread J. Hannken-Illjes
On Jul 28, 2012, at 5:34 PM, Edgar Fuß wrote: >> The `-c' flag just sets an (initialized) pointer to a function and has >> nothing to do with setting fss.fss_flags. > Not directly. But it moves memory. > >> Please ktrace without `-c'. > I already did that a week ago and posted to tech-userlevel.

Re: fss: Inappropriate ioctl for device (on dk*)

2012-07-28 Thread J. Hannken-Illjes
On Jul 28, 2012, at 5:51 PM, Edgar Fuß wrote: >> try attached patch for NetBSD-5 or -6. > Ah, thanks! I applied the first chunk (I already applied something > equivalent to the second myself) and now it works on wedges. > > Can this be pulled up to 6, please? Requested pullup to NetBSD-5 and N

Re: NFS panic

2012-10-25 Thread J. Hannken-Illjes
On Oct 24, 2012, at 5:44 PM, Manuel Bouyer wrote: > On Wed, Oct 24, 2012 at 04:07:34PM +0200, Manuel Bouyer wrote: >> Hello, >> I just got this panic on a NFS server: >> uvm_fault(0xfe9069ecf468, 0x0, 1) -> e >> fatal page fault in supervisor mode >> trap type 6 code 0 rip 804bd391 cs

Re: NFS panic

2012-10-31 Thread J. Hannken-Illjes
On Oct 31, 2012, at 12:04 PM, Edgar Fuß wrote: >> I think I debugged (and fixed) something similar in netbsd-5 ... > But which version did you get the panic on, then? > From reading PR#41147, my impression is that this should be fixed in 6, no? No -- NetBSD 6 does it in a different way and one ca

Re: NFS panic

2012-11-08 Thread J. Hannken-Illjes
and cleaning is right.Please try the attached diff that brings back the fix. --J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) vget.diff Description: Binary data

Re: NFS panic

2012-11-10 Thread J. Hannken-Illjes
On Nov 10, 2012, at 12:20 PM, Manuel Bouyer wrote: > On Thu, Nov 08, 2012 at 12:45:55PM +0100, J. Hannken-Illjes wrote: >> To make it short, Manuels fix was right, me removing it and trying to get it >> done in vn_lock() is wrong. >> >> While vget() vs. cleanvnode

Re: tstile lockup

2012-11-19 Thread J. Hannken-Illjes
bl rwlock and some vnode rwlock. This is ok as long as it doesn't deadlock. Do you get a deadlock or will the system come back to work after some time? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: tstile lockup

2012-11-19 Thread J. Hannken-Illjes
this in the > svn case). Try running `svn ...' as `lockstat -T rwlock svn ...'. By chance we get more information on lock congestion. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Making forced unmounts work

2012-11-27 Thread J. Hannken-Illjes
On Nov 28, 2012, at 3:59 AM, David Young wrote: > On Mon, Nov 26, 2012 at 03:06:34PM +0100, J. Hannken-Illjes wrote: >> Comments or objections? > > I'm wondering if this will fix the bugs in 'mount -u -r /xyz' where a > FFS is mounted read-write at /xyz? S

Re: Problem identified: WAPL/RAIDframe performance problems

2012-11-28 Thread J. Hannken-Illjes
dd fsbsize sized buffer to struct wapbl and teach wapbl_write() to collect data until the buffers start or end touches a fsbsize boundary. As long as the writes don't cross the logs end they already come ordered. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Problem identified: WAPL/RAIDframe performance problems

2012-11-28 Thread J. Hannken-Illjes
On Nov 28, 2012, at 9:20 PM, Brian Buhrow wrote: > Hello. If running 5.1 or 5.2 is acceptable for you, you could run > ffs+softdep since it has all the namei fixes in it. I suppose running fsck on a 6 TByte file system will take hours and softdep needs this after a crash. -- J. H

Re: Problem identified: WAPL/RAIDframe performance problems

2012-11-28 Thread J. Hannken-Illjes
e physical block. I think > the only way to improve that is to copy everything first into a > large buffer. Not very efficient. Needing to copy say 8 Mbytes of data and writing it in big chunks will be much faster than writing it many smaller unaligned segments. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Problem identified: WAPL/RAIDframe performance problems

2012-11-28 Thread J. Hannken-Illjes
g it. Flushing is just writing to the log. These writes have sizes between 512 bytes and the file system block size. Problem is these writes are not multiples of and are not aligned to file system block size. Collecting the data and writing MAXPHYS bytes aligned to MAXPHYS should improve wapbl on

Re: Making forced unmounts work

2012-11-29 Thread J. Hannken-Illjes
On Nov 29, 2012, at 5:17 PM, David Holland wrote: > On Mon, Nov 26, 2012 at 03:06:34PM +0100, J. Hannken-Illjes wrote: >> In short the attached diff: >> >> - Adds a new kernel-internal errno ERESTARTVOP and changes VCALL() to >> restart a vnode operation

Re: Making forced unmounts work

2012-12-02 Thread J . Hannken-Illjes
On Dec 2, 2012, at 5:00 AM, David Holland wrote: > On Thu, Nov 29, 2012 at 06:19:37PM +0100, J. Hannken-Illjes wrote: >>>> In short the attached diff: >>>> >>>> - Adds a new kernel-internal errno ERESTARTVOP and changes VCALL() to >>>> re

Re: Making forced unmounts work

2012-12-04 Thread J. Hannken-Illjes
On Dec 4, 2012, at 4:44 PM, David Holland wrote: > On Sun, Dec 02, 2012 at 05:29:01PM +0100, J. Hannken-Illjes wrote: >>>>> Also I wonder if there's any way to accomplish this that doesn't >>>>> require adding fstrans calls to every operation in every

wapbl_flush() speedup

2012-12-04 Thread J. Hannken-Illjes
get a speed increase of factor 3-4. Comments or objections anyone? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) Index: vfs_wapbl.c === RCS file: /cvsroot/src/sys/kern/vfs_wapbl.c,v retrieving revision

Re: wapbl_flush() speedup

2012-12-04 Thread J. Hannken-Illjes
On Dec 4, 2012, at 8:10 PM, Michael van Elst wrote: > hann...@eis.cs.tu-bs.de ("J. Hannken-Illjes") writes: > >> The attached diff tries to coalesce writes to the journal in MAXPHYS >> sized and aligned blocks. > [...] >> Comments or objections an

Re: wapbl_flush() speedup

2012-12-04 Thread J. Hannken-Illjes
On Dec 4, 2012, at 10:11 PM, David Laight wrote: > On Tue, Dec 04, 2012 at 09:53:11PM +0100, J. Hannken-Illjes wrote: >> >> On Dec 4, 2012, at 8:10 PM, Michael van Elst wrote: >> >>> hann...@eis.cs.tu-bs.de ("J. Hannken-Illjes") writes: >>> &

Re: wapbl_flush() speedup

2012-12-05 Thread J. Hannken-Illjes
wl->wl_devvp, wl->wl_buffer_addr, B_WRITE); > I suppose it's on purpose not to use wapbl_write() here. Yes, wapbl_buffered_write is a helper like wapbl_write and there is no need to go through wapbl_write. An updated diff (names and comments) is attached. -- J. Hannken-Illjes - hann...@

Re: Making forced unmounts work

2012-12-05 Thread J. Hannken-Illjes
. David wants forced unmounts to work even if a thread gets stuck permanently in a vnode operation. I don't see a way to safely reclaim a vnode from a file system when this vnode is in use by another thread. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Making forced unmounts work

2012-12-06 Thread J. Hannken-Illjes
On Dec 6, 2012, at 8:32 AM, Michael van Elst wrote: > hann...@eis.cs.tu-bs.de ("J. Hannken-Illjes") writes: > >> David wants forced unmounts to work even if a thread gets stuck >> permanently in a vnode operation. > > How can it get stuck (short of bugs) ? H

Re: Making forced unmounts work

2012-12-06 Thread J. Hannken-Illjes
possible to add fstrans to the nfs file system making it possible to suspend it. I did not dig deeper but as the nfs client already has timeout mechanisms it should be possible to cover this kind of problem. -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Re: Making forced unmounts work

2012-12-08 Thread J. Hannken-Illjes
Ejecting a removable device currently mounted. In this case the disk driver should error out and all threads should come back with an I/O-error. If this is not the case the driver is buggy. - Refcount and other bugs ... should be fixed ... -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany)

Cleanup bread/breadn API

2012-12-09 Thread J . Hannken-Illjes
changes bread() and breadn() to not return a buffer in case of error. Changing ~160 calls of bread() to not brelse() on error is not included in the diff. Comments or objections? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) Index: vfs_bio.c

Fstrans rewrite without rwlock

2013-01-13 Thread J . Hannken-Illjes
-- comments? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) vfs_trans.c Description: Binary data

Re: Fstrans rewrite without rwlock

2013-01-13 Thread J. Hannken-Illjes
On Jan 13, 2013, at 7:24 PM, Greg Troxel wrote: > > J. Hannken-Illjes writes: > >> Attached is a complete rewrite based on passive serialization and >> condvars. The fast paths (fstrans_start/fstrans_done on a file system >> not suspended or suspending and fscow_ru

Making spec_node table opaque

2013-02-08 Thread J. Hannken-Illjes
diagnostic checks "vp == ... sd_bdevvp" will not fire. Otherwise any vnode matching the criteria gets returned. Comments or objections? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) spec_dev.diff Description: Binary data

  1   2   3   >