Re: language bindings (fs-independent quotas)

[Manuel Bouyer]

On Fri, Nov 18, 2011 at 04:27:53PM +0200, Alan Barrett wrote:
> >what are you trying to do ?
> 
> I am just trying to enable quotas so that I can test some of the
> quota-related commands.
> 
> >quotaon won't do anything if / doesn't have the userquota or
> >groupquota keyword in the fstab, and you have to run quotacheck
> >before quotaon.  This is for ufs-quota1.
> 
> I don't see that in the quotaon(8) man page.
> 
> "The filesystems specified must have entries in /etc/fstab and be
> mounted."
> 
> I have that.
> 
> "quotaon expects each filesystem to have quota files named
> quota.user and quota.group which are located at the root of the
> associated file system.  These defaults may be overridden in
> /etc/fstab.  By default both user and group quotas are enabled."
> 
> I interpreted that as "by default, quotaon will just work".
> 
> Anyway, when I run quotacheck, it complains:
> 
> $ sudo quotacheck /
> quotacheck: / not found in /etc/fstab
> 
> I do have an entry for "/" in /etc/fstab:
> 
> from_mount/   ffs rw,log  1 0
> 
> This is in a chroot, and the actual device name is /dev/cgd1a,
> but fstab doesn't know that.

quotacheck also needs to have userquota and/or groupquota option in the
fstab (otherwise it doens't know what to check).

I agree the man pages are not clear on this (although if you read the whole
quotacheck man page you get hits about it, like:
 -u  Only user quotas listed in /etc/fstab are to be checked.  See
  also -g.

which I interpret as "quotas are listed in fstab in some way").

I didn't change the behavior nor the man pages for these commands, so
I'm not to blame for this :)

> 
> >For ufs-quota2, quotas are enabled at newfs time, or with tunefs (with
> >the later this has to be done on a read-only mounted filesystem, and you
> >have to run fsck before mounting R/W). quotaon won''t do anything
> >for ufs-quota2.
> 
> The quotaon(8) man page does not say that it's only for some file
> system types, and does not refer to newfs, tunefs, or fsck.

I documented is in fstab(5), which is where userquota and groupquota
are documented. quotaon(8) and quotacheck(8) could probably be improved.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

Re: language bindings (fs-independent quotas)

[Alan Barrett]

On Fri, 18 Nov 2011, Manuel Bouyer wrote:
Assuming that there's no need to handle fields with embedded 
spaces, perl's split() function will DTRT.


No, it does not because there are fields that can be empty.


The common way of dealing with that is to have a placehloder like 
"-" for empty fields.


By the way, I still haven't figured out how to test any of 
this quota stuff.  "quotaon /" followed by "edquota -f /" does 
nothing (no error message, and no useful result).  Using the 
device name "/dev/cgd1a" instead of the file system name "/" 
does not help.


what are you trying to do ?


I am just trying to enable quotas so that I can test some of the
quota-related commands.

quotaon won't do anything if / doesn't have the userquota or 
groupquota keyword in the fstab, and you have to run quotacheck 
before quotaon.  This is for ufs-quota1.


I don't see that in the quotaon(8) man page.

"The filesystems specified must have entries in /etc/fstab and be 
mounted."


I have that.

"quotaon expects each filesystem to have quota files named 
quota.user and quota.group which are located at the root of the 
associated file system.  These defaults may be overridden in 
/etc/fstab.  By default both user and group quotas are enabled."


I interpreted that as "by default, quotaon will just work".

Anyway, when I run quotacheck, it complains:

$ sudo quotacheck /
quotacheck: / not found in /etc/fstab

I do have an entry for "/" in /etc/fstab:

from_mount  /   ffs rw,log  1 0

This is in a chroot, and the actual device name is /dev/cgd1a,
but fstab doesn't know that.


For ufs-quota2, quotas are enabled at newfs time, or with tunefs (with
the later this has to be done on a read-only mounted filesystem, and you
have to run fsck before mounting R/W). quotaon won''t do anything
for ufs-quota2.


The quotaon(8) man page does not say that it's only for some file
system types, and does not refer to newfs, tunefs, or fsck.

--apb (Alan Barrett)

Re: pool_cache_invalidate(9) (wrong?) semantic -- enable xcall invalidation

[Jean-Yves Migeon]

On Thu, 17 Nov 2011 23:08:28 -0600, David Young wrote:

On Fri, Nov 18, 2011 at 01:09:49AM +0100, Jean-Yves Migeon wrote:

- force all xcall(9) API consumers to pass dynamically allocated
arguments, a bit like workqueue(9) enqueues works. Scheduling
xcall(9) is now managed by a SIMPLEQ() of requests.

- extends softint(9) API so we can pass arguments to it as well as
the targetted CPU(s) (optional argument).

The last two points make me think that the softint(9), workqueue(9)
and xcall(9) APIs have a potential for unification; all of these are
somewhat redundant, they all schedule/signal/dispatch stuff to other
threads, albeit under different conditions though.


See "Kernel Continuations" in 
.

IIRC, the document does not contemplate putting xcall(9) under
the proposed kernel-continuations framework, but it does mention
both softints and workqueues as candidates for unification under
continuations.


Interesting reading; and yes, it seems to be case. Cross calls and 
softints are functionally close: they dispatch works to other threads, 
but softint(9) cannot pass arguments whereas xcall(9) do. From an 
implementation PoV, xcall and sofint are CPU-bound threads nowadays, 
with additional strong requirements for softints (cannot sleep, 
lightweight, mapped to interrupt level priorities). xcall threads are 
just plain kernel threads.


The last part of the page is also worth a look. I remember a PoC on L4 
kernels for "single/merge kernel stack" implementations, and matt's idea 
is close to this too.


Lots of ideas, so little time for me :(

--
Jean-Yves Migeon
jeanyves.mig...@free.fr

Re: language bindings (fs-independent quotas)

[Manuel Bouyer]

On Fri, Nov 18, 2011 at 07:46:21AM +0200, Alan Barrett wrote:
> On Fri, 18 Nov 2011, David Holland wrote:
> >The proposed standard format for quotas is an ordinary columnar
> >text file. The reason language bindings came up is that Manuel was
> >complaining, somewhat oddly, that it's hard to handle these in
> >Perl.
> 
> Assuming that there's no need to handle fields with embedded spaces,
> perl's split() function will DTRT.

No, it does not because there are fields that can be empty.

> 
> >And actually, language bindings are probably a good thing anyway;
> >if you have an installation with 50,000 users and you want to frob
> >their quotas from a Perl script, forking 50,000 edquota processes
> >is probably not the best approach.
> 
> Oh my, I missed the part of the edquota man page where it says "a
> temporary file is created for each user".  Why can't it just create
> a single temporary file with a text table of all quotas?
> 
> By the way, I still haven't figured out how to test any of this
> quota stuff.  "quotaon /" followed by "edquota -f /" does nothing
> (no error message, and no useful result).  Using the device name
> "/dev/cgd1a" instead of the file system name "/" does not help.

what are you trying to do ?
quotaon won't do anything if / doesn't have the userquota or groupquota
keyword in the fstab, and you have to run quotacheck before quotaon.
This is for ufs-quota1.

For ufs-quota2, quotas are enabled at newfs time, or with tunefs (with
the later this has to be done on a read-only mounted filesystem, and you
have to run fsck before mounting R/W). quotaon won''t do anything
for ufs-quota2.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

Re: fs-independent quotas

[Manuel Bouyer]

On Fri, Nov 18, 2011 at 12:35:34AM +, David Holland wrote:
>  > With a proplib format, the kernel knows it didn't get the right
>  > argument (it didn't find a key "quotafile" with a string value in the
>  > dictionary). Of course you can still do quotaon /boot if you really
>  > wants to, but then it has been done on purpose, not just because
>  > you gave the wrong pointer to quotactl().
> 
> This is about type-safety. It has nothing to do with compatibility,
> forwards or backwards.

I agree. using proplib is a win for both type-safety and backward
compatibility.

> 
> Unless we split quotactl(2) into multiple system calls, one per
> operation, it can't be fully typesafe; C has no strictly type-safe way
> to be polymorphic.
> 
> It can be made better than quotactl(path, int, int, void *), but only
> a little. The important thing, I think, is that in my proposed world
> quotactl() is no longer meant to be user-callable. (Maybe it should be
> _quotactl() or __quotactl() so nobody calls it by accident.) Only
> libquota calls it, and if you're that concerned about mismatched types
> we can audit the libquota code.
> 
> And, btw, using proplib does not make the code type-safe. To the
> contrary, in fact, even excluding the dynamic typing behavior, because
> proplib's API is so borked. Look in its header files to see how many
> of its typedefs are really void pointers, and be scared...

I'm not. The void pointers are used only internally by proplib,
public functions are type-safe (exept for functions that work on
any data type, i.e  the *data* functions, and opaque pointers where their
type doesn't matters) and you can't use a function with the wrong pointer
type: prop_dictionary_get_int8() takes a int8_t*, prop_dictionary_get_cstring()
takes a char **, and so on. You cannot convert a value from/to the
wrong pointer type.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

Re: fs-independent quotas

[Manuel Bouyer]

On Fri, Nov 18, 2011 at 12:55:32AM +, David Holland wrote:
> On Thu, Nov 17, 2011 at 05:29:22PM +0100, Manuel Bouyer wrote:
>  > I still believe proplib is better. For example, you can go from a
>  > 32bit to a 64bit uid_t/gid_t without versionning (in the data structure
>  > it's still ).
> 
> NO YOU CAN'T. Sorry for shouting, but this really annoys me.
> 
> Proplib handles all integers internally as intmax_t, so *it* won't get
> confused. (Unless intmax_t gets changed, at which point all
> proplib-using syscalls and ioctls need to be versioned, with no clear
> way to provide compat logic; that won't be pretty.)

And it will also error out if you try to store an integer larger than
the type allow. See prop_dictionary_get_int in prop_dictionary_util.c

> 
> But what happens if you do this and then create a user with e.g. uid
> 2^40, and try to read its quota information out with the 32-bit quota
> utilities? Well, lookee here:
> 
>   showuid((uid_t)atoi(*argv));
> 
> (quota.c line 179)

this has nothing to do with the quotactl interface. Yes, there's still
room for improvements in the tools. History is heavy here.
But you can't use this to shoot down the quotactl interface; these are
2 independant things.

> 
> boom, silent truncation to 32 bits. If you're lucky. atoi() is allowed
> to send you nasal demons in this case, too.
> 
> ok, so maybe that's just a bug. What happens if instead you dump all
> quota information?
> 
>   errx(1, "can't find id in quota entry");
> 
> (repquota.c line 294)
> 
> Not very helpful, ISTM.
> 
> Granted, this is probably not markedly worse than the results if doing
> compat the traditional way; but it's no better, either.

You missed the point.
The point is, on a system where uids still fit in 32bits, I can install
a kernel with a 64bits uid_t and the 32bits quota tools will still work,
without any special backward compat handling in the quotactl interface.
Or I can use quota tools with 64bits uid_t on a kernel with 32bits uid_t,
it will work as well.
Of course, if you start using uids that don't fit in 32bits, you need
a kernel and userland with 64bits. But if you don't *need* 64bits
uid_t, you don't have to update userland because you have a 64bits uid_t
kernel, or vice-versa.

> 
> Sure, you could update all the tools to anticipate this case, and
> dozens or hundreds of other unlikely but potentially possible changes
> like this; or you could just send back structs the traditional way,
> and when the time comes, if it ever does, version the syscall or the
> command codes or whatever in the traditional way. Much less effort
> overall.

No, it's more efforts. With a binary struct, you have to version the syscall
(and handle conversion from old to new format) a whole class of changes,
which are handled automagically by proplib.

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

Re: Patch to add support for crazy Realtek 8139-based cards

[Izumi Tsutsui]

> Very recently I have managed to get my hands on a pretty odd RTL8139
> card which does not identify with the system well, because it reports
> 0x0001 as vendor ID. On Linux it worked because Linux already detects
> such cards, while NetBSD doesn't.

- Can you show actual board vendor name of that card?
  (to denote quirks in sources)
- Is that card recognized properly on "supported OS," i.e. MS Windows?
- If yes, does it work with Windows default driver or vendor's driver?
- If it works on Windows, what does the device manager show,
  especially PCI vendor and product IDs?

If the board vendor intentionally sets its PCI vendor ID to 0x0001,
we can add such odd ID into rtk_pci_devs[] in if_rtk_pci.c.

But if it's caused by broken EEPROM or bad drivers which badly
overwrite EEPROM data, we should not accept random ID numbers.

---
Izumi Tsutsui