Editing (new) quota for a new user
I just started to play around with the new quota system. I tried to set quotas for a user that didn't own any files on the file system in question. With an "interactive" edquota, I got edquota: /export/test (ufs/ffs quota v2): : bad format With a edquota -h ... -s ..., I got a zero exit status, a following "interactive" edquota showing what I entered, but repquota not showing anything for that user. When I allocate a file for that user, edquota -h ... -s ... works, but an "interactive" edquota throws the same error and disables that user's quota. Is this expected behaviour?
Re: RAIDframe performance vs. stripe size
On Sat, 12 May 2012, Edgar Fu? wrote: > > In general it won't access just one filesystem block. > > It will try to readahead 64KB > Oh, so this declustering seems to make matters even more > complicated^Winteresting. > > Staying with my example of a 16K fsbsize FFS on a 4+1 disc Level 5 > RAIDframe with a stripe size of 4*16k=64k: > > Suppose a process does something that could immediately be satisfied > by reading one fs block (probably it matters whether that's a small file, > a small portion of a large file, a small directory, a portion of a large > directory, inodes, free list or whatever?). Now, if that, as I understand, > always causes FFS to in fact issue a 64k request to RAIDframe, this would > need to read a full stripe and so need all but one disc. So it can't be > parallelised with another process' request, can it? Does this mean I'm better > off with a stripe size of 4*64k if I'm after low latency for concurrent > access? The problem here is NFS, which requires writes to be persistent before returning status to the caller. Under normal operation, ufs will attempt to use the buffer cache in the most efficient manner, doing readahead and delaying writes as much as possible to be able to do maximize the number of clustered operations as it can. Now if NFS does not do similar clustering on writes (I don't know NFS that well, especially V3 and V4 which allegedly have write optimizations) then you get the situation where the underlying ufs will try to cluster reads (satisfying reads out of the buffer cache is much faster than hitting the platters) but write out only single filesystem blocks (to satisfy the NFS consistency requirements.) My understanding is that later versions of NFS (v3+) have a mechanism for the client side to request writes without the consistency guarantee and a separate explicit sync operation. But using those is the responsibility of the NFS client machine. Of course, if all the files are on the order of one filesystem block, clustering won't happen at all. I think you should attempt to characterize your workload here to determine the size of the I/O operations the clients are requesting so you can decide if clustering is a benefit to you, and if not, turn it off. (I think it can be tweaked with tunefs(8).) Eduardo
Re: allocating files with indirect blocks (was: choosing the file system block size)
Date:Mon, 14 May 2012 13:35:54 +0200 From:Edgar =?iso-8859-1?B?RnXf?= Message-ID: <20120514113554.gf12...@trav.math.uni-bonn.de> | > For files large enough to need indirect blocks, | > (a) the size is rounded up to the block size, not the frag size | Why is that so? I'm not going to try and speak for Kirk - ask him if you want a more authoritative answer, but ... Basically the idea is to avoid excessive fragmentation - note that no-one (that I know of) has ever seen the need to implement a defragmenter for ffs (unlike some other notable filesystems). Much of the reason for this is that considerable effort was taken to manage the use of fragments to avoid them gradually cluttering the filesystem and needing to be moved around by some maintenance tool. Part of that is this part - fragments exist at all as a space saving measure (earlier attempts at faster filesystems than the original v7/32v filesystems had simply made bigger blocks, but required much more space). But that really matters only for small files, allocating 8Kb, rather than 1Kb, for a 200 byte file is a huge penalty, and since there can be large numbers of such small files, the overall cost is enormous (or was back in the 1980's, these days discs are so big that it matters much less.) But, once the file is big enough that it needs indirect blocks (for 8Kb blocksize, that's > 88Kb if I remember the number of direct blocks in ffs correctly) wasting on average 4Kb matters much less, it's just a few percent, rather than the several hundred percent it can be for small files. Not allowing fragments to be used there, and simply wasting that space means less fragments used overall, and so less fragmentation issues to deal with. kre
Re: allocating files with indirect blocks (was: choosing the file system block size)
>> For files large enough to need indirect blocks, (a) the size is >> rounded up to the block size, not the frag size > Why is that so? I don't know why the FFS designers made that choice. I've always assumed it was for some kind of efficiency reason, but that is speculation on my part. If you mean, where in the code this is done, look at sys/ufs/ffs/ffs_balloc.c; search for NDADDR. I see two relevant tests per FFS version (two in ffs_balloc_ufs1 and two in ffs_balloc_ufs2, or only two total when I look at pre-FFSv2 code). /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
allocating files with indirect blocks (was: choosing the file system block size)
> For files large enough to need indirect blocks, > (a) the size is rounded up to the block size, not the frag size Why is that so? I've tried to find something in the 4.4BSD Book I last read years ago (when I implemented the client-side NFS locking), but couldn't find anything about it.
Re: Crashes in uaudio
Hi,
> So, it seems that we are overwriting
> the end of usb_audio_streaming_type1_descriptor for every descriptor where
> the number of rates is more than 2. The problem is that we only notice when
> we come to read. So, I think that we should set:
>
> uByte tSamFreq[3*AUFMT_MAX_FREQUENCIES];
>
> in the usb_audio_streaming_type1_descriptor definition (uaudioreg.h), make
> sure that we only copy the correct maximum number of bytes when setting it
> up, and remove the test at line 1852 of uaudio.c. We probably should also
> set:
>
> auf->frequency_type = AUFMT_MAX_FREQUENCIES;
>
> in the test at line 1846, just in case we do meet a device which advertises
> more frequencies than we can handle (if we don't, we could end up reading
> memory after the end of the usb_audio_streaming_type1_descriptor).
Looking at the initialisation of the audio descriptors in uaudio_process_as(),
the descripters are pointers into a larger buffer, and we already check the
length against the buffer size, for example:
http://nxr.netbsd.org/source/xref/src/sys/dev/usb/uaudio.c#1578
so the attached patch increases the space for frequencies at the end of
usb_audio_streaming_type1_descriptor (to AUFMT_MAX_FREQUENCIES), and sets
auf->frequency_type to AUFMT_MAX_FREQUENCIES if the hardware reports more
than we can handle. Because of the change to the descriptor definition,
we need an extra include in umidi.c and umidi_quirks.c, but I think that
this is simpler than splitting out the definition of AUFMT_MAX_FREQUENCIES
into a separate uaudiovar.h.
Comments?
Thanks,
J
PS. Patch is against the jmcneill-usbmp branch, but the bug is in current
too.
--
My other computer also runs NetBSD/Sailing at Newbiggin
http://www.netbsd.org// http://www.newbigginsailingclub.org/
cvs diff: Diffing .
Index: uaudio.c
===
RCS file: /cvsroot/src/sys/dev/usb/uaudio.c,v
retrieving revision 1.124.2.5
diff -u -r1.124.2.5 uaudio.c
--- uaudio.c29 Apr 2012 23:05:01 - 1.124.2.5
+++ uaudio.c14 May 2012 09:51:16 -
@@ -1847,12 +1847,8 @@
aprint_error("%s: please increase "
"AUFMT_MAX_FREQUENCIES to %d\n",
__func__, t1desc->bSamFreqType);
- break;
- }
- if (j >= 2) {
- aprint_error("%s: too much tSamFreq: "
- "%d\n",
- __func__, t1desc->bSamFreqType);
+ auf->frequency_type =
+ AUFMT_MAX_FREQUENCIES;
break;
}
auf->frequency[j] = UA_GETSAMP(t1desc, j);
Index: uaudioreg.h
===
RCS file: /cvsroot/src/sys/dev/usb/uaudioreg.h,v
retrieving revision 1.15
diff -u -r1.15 uaudioreg.h
--- uaudioreg.h 28 Apr 2008 20:23:59 - 1.15
+++ uaudioreg.h 14 May 2012 09:51:16 -
@@ -113,7 +113,7 @@
uByte bBitResolution;
uByte bSamFreqType;
#define UA_SAMP_CONTNUOUS 0
- uByte tSamFreq[3*2]; /* room for low and high */
+ uByte tSamFreq[3*AUFMT_MAX_FREQUENCIES];
#define UA_GETSAMP(p, n) ((p)->tSamFreq[(n)*3+0] | ((p)->tSamFreq[(n)*3+1] <<
8) | ((p)->tSamFreq[(n)*3+2] << 16))
#define UA_SAMP_LO(p) UA_GETSAMP(p, 0)
#define UA_SAMP_HI(p) UA_GETSAMP(p, 1)
Index: umidi.c
===
RCS file: /cvsroot/src/sys/dev/usb/umidi.c,v
retrieving revision 1.53.2.5
diff -u -r1.53.2.5 umidi.c
--- umidi.c 25 Feb 2012 10:26:24 - 1.53.2.5
+++ umidi.c 14 May 2012 09:51:17 -
@@ -52,6 +52,7 @@
#include
#include
+#include
#include
#include
#include
Index: umidi_quirks.c
===
RCS file: /cvsroot/src/sys/dev/usb/umidi_quirks.c,v
retrieving revision 1.16.32.1
diff -u -r1.16.32.1 umidi_quirks.c
--- umidi_quirks.c 18 Feb 2012 07:35:10 - 1.16.32.1
+++ umidi_quirks.c 14 May 2012 09:51:17 -
@@ -49,6 +49,7 @@
#include
#include
+#include
#include
#include
#include
